HTTP Services & REST APIs

http.Handler is any type with a ServeHTTP(w, r) method; http.HandlerFunc turns a plain function into one; http.ServeMux is a handler that routes each request to another handler by its pattern. http.ListenAndServe(addr, mux) then serves it. HandlerFunc is type HandlerFunc func(ResponseWriter, *Request) with a ServeHTTP method that just calls the function. A bare function has no methods, so mux.Handle("/x", fn) does not compile, while mux.HandleFunc does the conversion for you. When a handler needs dependencies such as a database, make it a struct with ServeHTTP, or a method on a struct that holds them. http.HandleFunc and ListenAndServe(addr, nil) use the global http.DefaultServeMux, where any imported package can register routes; net/http/pprof does exactly that. Create your own mux with http.NewServeMux(). ListenAndServe sets no timeouts; in production, configure an http.Server instead.

Path values come from r.PathValue("id"), query parameters from r.URL.Query().Get("q"), headers from r.Header.Get(...), and the body is a stream you decode with json.NewDecoder(r.Body). To reply, set Content-Type, then the status, then encode the body. Everything arrives as a string: parse and check it yourself (strconv.Atoi) and answer 400 when it is wrong. Query().Get returns "" for a missing key; use r.URL.Query().Has("q") when "absent" and "empty" mean different things. Header names are case-insensitive: Get canonicalises the key, so x-request-id finds X-Request-Id. The server closes r.Body for you; the handler just reads it, once. r.FormValue is for HTML forms (application/x-www-form-urlencoded), not for JSON bodies.

The client gets 200 OK with Content-Type: text/plain; charset=utf-8. The first Write sends the status line and headers, so the later WriteHeader and Header().Set change nothing. A response is written in a fixed order: set w.Header() fields, call WriteHeader(status), then write the body. Once a byte of body is written, the headers are on the wire. Write without an earlier WriteHeader calls WriteHeader(200) itself. With no Content-Type, net/http guesses one from the first 512 bytes with http.DetectContentType, and JSON is detected as text/plain. The late call returns nothing you can check; the server only logs http: superfluous response.WriteHeader call. Fix: set the header, call w.WriteHeader(http.StatusCreated), then encode.

Safe methods (GET, HEAD, OPTIONS) do not change server state; idempotent methods have the same effect whether sent once or ten times: the safe ones plus PUT and DELETE. POST and PATCH are neither by default, so retrying them is risky. **PUT replaces the whole resource with the body; fields you leave out are reset. PATCH changes only the fields you send (JSON Merge Patch, RFC 7396, is the common format). Success**: 200 OK with a body, 201 Created plus a Location header after POST, 204 No Content when there is nothing to return, 202 Accepted when the work is queued for later. Client errors: 400 malformed request, 422 well-formed but invalid, 404 no such resource, 409 conflict with the current state (duplicate email, stale version), 429 rate limited. **401 vs 403**: 401 means "who are you?": credentials missing or invalid. 403 means "I know who you are, and you may not". **5xx** means the server failed. Answering 500 for bad input invites retries and fires your error alerts for the client's mistake.

Patterns can now include a method and wildcards, as in GET /items/{id}, and the handler reads the value with r.PathValue("id"). When several patterns match, the most specific one wins; two overlapping patterns where neither is more specific make registration panic. {id} matches one path segment, {path...} the rest of the path, and {$} anchors the end: / alone still matches every path, /{$} only the root. GET also serves HEAD. A path that matches with the wrong method gets 405 Method Not Allowed with an Allow header, written by the mux itself. "More specific" means matching a strict subset of requests: /items/new beats /items/{id}, and GET /items beats /items. /items/{id} and /{kind}/latest both match /items/latest, so registering both panics at startup instead of misrouting later. Wildcards are untyped ({id} matches abc too), so parse and validate in the handler. Still missing: route groups and per-route middleware. Wrap single handlers at registration, or use chi. GODEBUG=httpmuxgo121=1 restores the old matcher during a migration.

The server runs each connection in its own goroutine, and with HTTP/2 each request stream gets its own goroutine too, so one handler runs concurrently for many requests. Anything handlers share must be safe for concurrent use. A field on your server struct or a package-level map written from a handler is a data race. Guard it with a sync.Mutex, use atomic.Int64 for counters, or keep the state in something already safe for concurrent use, such as the *sql.DB pool or Redis. Two requests writing one plain map at once crash the process with fatal error: concurrent map writes. It is not a panic, so no recovery middleware catches it. Request data belongs to the request: keep it in local variables or r.Context(), never in a shared "current user" field. The server does not cap concurrency. A burst of slow requests becomes thousands of goroutines and database calls; limit it with a semaphore middleware, netutil.LimitListener or the pool size. net/http recovers a panic in the handler's goroutine, logs it and closes that connection. Goroutines your handler starts get no such protection. go test -race with concurrent requests finds these races before production does.

Middleware is a function func(http.Handler) http.Handler that wraps the next handler and runs code before and after it. Logging the status needs a wrapper around ResponseWriter, and the request ID travels down the chain in r.Context(). ResponseWriter has no getter for the status, so wrap it and record what WriteHeader receives; if the handler never calls it, the status is 200. Give the wrapper an Unwrap() http.ResponseWriter method so http.NewResponseController can still reach Flush and the write deadlines of the real writer. Use an unexported key type such as type ctxKey struct{} for context.WithValue, so no other package can collide with your key. Echo the ID in a response header and forward it on outbound calls: one ID then links the client, your logs and the services downstream. Trust an incoming X-Request-Id only when it comes from your own proxy; from the internet it is just user input.

The outermost wrapper runs first on the way in and last on the way out: in a(b(c(mux))), a sees the request before b and c. So every layer must sit outside the layers that depend on it. Request ID first: the logger and every error response below it need the ID already in the context. Logging, then recovery: recovery turns a panic into a 500. With logging outside it, the access log records that 500; with logging inside it, the log line is never written. CORS before auth: browsers block a page from reading another origin's responses unless the server allows it with Access-Control-Allow-Origin. Before a request with a JSON body or an Authorization header, the browser sends a preflight OPTIONS without credentials, so an auth layer in front of CORS answers it 401. The Go 1.22 mux would answer it 405 for a GET route, so CORS must answer the preflight before the mux, too. Auth, then a per-user rate limit: a per-user limit needs to know the user; a per-IP limit can sit further out and shed load before auth does any work. Code after next.ServeHTTP runs on the way back, after the inner layers are done.

The middleware sends http.Error even though the handler already wrote part of the body, so the 500 cannot replace the committed 200: the server logs superfluous response.WriteHeader call and internal error is appended to the half-written JSON. Worse than the garbage text: the response still ends cleanly, so a client sees a normal 200 with a truncated body and may treat it as complete. The cause is that the middleware does not know whether anything was already written; it needs a thin ResponseWriter wrapper that records the first Write or WriteHeader. The fix: if nothing was written, reply 500 as before; if something was, re-panic with http.ErrAbortHandler, so the server drops the connection quietly and the client gets a transport error, not a fake success. Always re-panic http.ErrAbortHandler itself. Catch it with a test: a handler that writes a prefix and then panics, served through httptest.NewServer, and assert the client read fails instead of returning a 200 body.

io.ReadAll buffers whatever the client sends, so one huge body can exhaust memory, and the ignored error hides a broken upload. Cap the body with http.MaxBytesReader, decode with json.NewDecoder, and turn each kind of failure into the right 4xx. net/http sets no body limit; Server.MaxHeaderBytes covers headers only. Past the limit, MaxBytesReader returns *http.MaxBytesError (answer 413) and tells the server to close the connection instead of reading the rest. io.EOF from the first Decode means an empty body; *json.SyntaxError and *json.UnmarshalTypeError mean bad JSON: answer 400 with the byte offset or the field name. dec.DisallowUnknownFields() turns a misspelt field into an error instead of a silently ignored key. Decode reads one JSON value, so {"a":1} {"b":2} passes; a second Decode must return io.EOF. Use a pointer field such as *int when "absent" and "zero" must be told apart.

Return a stable JSON error body (RFC 9457 problem details, application/problem+json, is the standard shape) with a status that says whose fault it is. http.Error writes text/plain, and with err.Error() it leaks internals such as SQL and table names. http.Error sets Content-Type: text/plain; charset=utf-8 and X-Content-Type-Options: nosniff, so a JSON client cannot parse the failure. Problem details has type, title, status and detail, plus extension members such as a per-field errors map that a form can show next to each input. Validate after decoding, collect every failing field instead of stopping at the first, and answer 422 (or 400; pick one and use it everywhere). Hand-written checks are fine for a few fields; go-playground/validator adds struct tags such as validate:"required,email". Map domain errors to statuses in one function (errors.Is(err, ErrNotFound) gives 404), log the cause of every 5xx with the request ID, and send the client only a safe message.

Handlers deal only with HTTP, a service layer holds the business rules, and a repository hides the database. Dependencies point inward and are passed in as struct fields built once in main, with no package-level globals, so each test can build its own copy. Handler: decode, validate, call the service, turn the result or the error into a status. No SQL here. Service: business rules and transactions, with no net/http types, so the same logic can serve HTTP, gRPC or a CLI. Repository: SQL and mapping rows to domain types. Its interface is small, declared by the consumer (the service), and the Postgres type satisfies it implicitly. Handlers are methods on a struct that holds the dependencies. A global var db *sql.DB hides what a handler needs, depends on init() order, and makes parallel tests race over one value. Keep all routes in one routes() method, so a test can run the real router, middleware and handlers through httptest. For a small CRUD service three layers can be ceremony; split out a service when real business logic appears.

Model nouns as resources (/posts, /posts/{id}, /posts/{id}/comments), let the HTTP method carry the action, return honest status codes, paginate every list, and version the API so existing clients never break. Resources and methods GET /posts lists, POST /posts creates and answers 201 with Location: /posts/42, GET /posts/{id} reads, PATCH /posts/{id} updates, DELETE /posts/{id} answers 204. No verbs in paths such as /getPosts. An action that is not CRUD becomes a sub-resource: POST /posts/{id}/publish. Filtering, sorting and search go in the query string: GET /posts?author=7&sort=-created_at. Pagination ?limit=20&offset=40 is simple, but deep pages get slow because the database still reads and throws away the skipped rows, and items are skipped or repeated when rows are inserted while the user pages. Cursor (keyset) pagination continues from the last row seen: WHERE (created_at, id) < ($1, $2) ORDER BY created_at DESC, id DESC LIMIT 20. It uses an index, stays stable under writes, and returns an opaque next_cursor, for example those two values base64-encoded. Always cap limit on the server. Versioning and evolution Additive changes, such as a new field or endpoint, need no new version as long as clients ignore unknown fields. A breaking change goes to /v2/... (or a version header), and both versions run until clients have moved. Use one error format everywhere, and RFC 3339 timestamps in UTC.

Since Go 1.22 the standard mux handles methods and path parameters, so plain net/http, or chi when you also want route groups and per-route middleware, covers most services. Gin and echo add their own context type and conveniences; fiber is fast on paper but runs on fasthttp, outside the net/http ecosystem. **net/http: no dependency, a stable API, and every middleware and tool works with it. Missing: route groups, per-route middleware, request binding. chi**: a router that stays pure http.Handler: route groups, r.With(mw), sub-routers and a ready middleware set. The cheapest step up from the standard mux. gin / echo: handlers take *gin.Context or echo.Context, with binding, validation and rendering built in. Large ecosystems, but your handlers are tied to the framework, and standard middleware needs adapters. fiber: an Express-like API on fasthttp: no HTTP/2, and net/http middleware or OpenTelemetry integrations need adapters. fasthttp reuses request objects, so values taken from fiber.Ctx (*fiber.Ctx in v2) must not be kept after the handler returns unless you copy them. Routing is rarely the bottleneck (the database and JSON encoding are), so fit with the ecosystem matters more than router benchmarks.

Keep a token bucket per client (golang.org/x/time/rate provides one) and check it in middleware before the handler runs. Over the limit, answer 429 Too Many Requests with a Retry-After header and do no other work. Token bucket: rate.NewLimiter(10, 20) refills 10 tokens a second and allows bursts of 20; Allow() takes a token or says no without blocking. Key: the user ID after auth, or the client IP. Behind a load balancer r.RemoteAddr is the balancer, so take the IP from the proxy's header, and only from a proxy you trust. Memory: a map of limiters grows with every new IP; evict idle entries on a timer or use an LRU. Several replicas: in-process limits multiply by the number of replicas. For a global limit, keep the counters in Redis (a sliding-window script) or enforce it at the API gateway. A fixed window is the cheapest, but lets through up to twice the rate around the window boundary; a sliding window or a token bucket smooths that out.

http.ListenAndServe builds a server with every timeout at zero, so a client that sends its headers one byte a minute (Slowloris) holds a connection and a goroutine forever. Build your own http.Server and set at least ReadHeaderTimeout. **ReadHeaderTimeout: until the request headers are read. Headers are small, so 5–10 seconds is safe; this is the Slowloris fix. ReadTimeout**: the headers plus the whole body. A low value breaks large uploads, so keep it generous and cap the size with http.MaxBytesReader. **WriteTimeout: on HTTP/1.x it starts once the headers are read, so it is really the budget for the whole handler, and it kills long streams unless they extend it per write. IdleTimeout**: how long a keep-alive connection may wait for its next request. At zero it falls back to ReadTimeout, and with both at zero idle connections never expire. These timeouts close the connection but do not stop your handler. Give handlers a budget with context.WithTimeout, or with http.TimeoutHandler, which answers 503 when time runs out; it buffers the response, so keep it off streaming routes.

Catch the signal with signal.NotifyContext, then call srv.Shutdown(ctx) with a fresh timeout context: it stops accepting, closes idle connections and waits for active requests to finish. srv.Close() cuts live connections instead, which is what resets in-flight requests. ListenAndServe returns http.ErrServerClosed as soon as Shutdown starts. That is not a failure, and main must keep waiting until Shutdown itself returns. Give Shutdown a new context.WithTimeout(context.Background(), ...), because the signal context is already cancelled. If that timeout expires, call Close to cut the rest. Shutdown does not wait for hijacked connections such as WebSockets, or for goroutines your handlers started; track those with RegisterOnShutdown and a sync.WaitGroup. Close the database pool and flush telemetry only after Shutdown returns, or the requests still draining fail on their way out. In Kubernetes, fail the readiness probe first and wait a few seconds so the load balancer stops sending traffic, and keep the whole sequence under terminationGracePeriodSeconds (30 s by default).

The goroutine outlives the request: r.Context() is cancelled as soon as the handler returns, so the export usually stops with context canceled, and writing to w after that point is invalid. A panic inside the goroutine also crashes the whole server. A ResponseWriter must not be used after ServeHTTP returns. The 202 has already gone out, the connection may be serving the next request, and the late Write races with it; go test -race reports it. net/http recovers panics in the handler's own goroutine, not in goroutines the handler starts; each of those needs its own defer recover(). Copy what you need from r before starting the goroutine, and detach the context with context.WithoutCancel(r.Context()) (Go 1.21+) plus a timeout of its own: it keeps values such as the trace ID but drops the cancellation. Track the goroutine with a sync.WaitGroup so graceful shutdown waits for it. Work that must survive a restart belongs in a queue or a jobs table that a worker processes. Answer with a job ID and Location: /exports/{id} that the client can poll; the result is delivered there, not on this response.

Build one *http.Client at startup with a tuned Transport and reuse it everywhere, because the connection pool lives in the Transport. Give each call a deadline through the request context, derived from r.Context(), so a client hanging up cancels the upstream call too. http.DefaultClient and http.Get have no timeout at all: one stuck upstream holds goroutines and sockets forever. A new Client or Transport per call throws the pool away, so every call pays a TCP and TLS handshake. Both types are safe for concurrent use. Client.Timeout caps the whole exchange (dial, TLS, redirects and reading the body) for every request. Treat it as a backstop; the per-call budget is context.WithTimeout plus http.NewRequestWithContext. Errors come wrapped in *url.Error: errors.Is(err, context.DeadlineExceeded) means your deadline passed (answer 504), context.Canceled means your own caller left. Start from http.DefaultTransport.(*http.Transport).Clone() to keep proxy support, dial timeouts and HTTP/2.

Write events in the text/event-stream format and flush each one with http.NewResponseController(w).Flush(). Events that arrive in one lump are stuck in a buffer (net/http's or a proxy's), and the stream dies at WriteTimeout because on HTTP/1.x it covers the whole response. Buffering The response goes through a 4 KB bufio.Writer; without Flush, events wait until it fills or the handler returns. A middleware that wraps ResponseWriter without Unwrap() http.ResponseWriter hides Flush, and rc.Flush() then returns an error matching http.ErrNotSupported. Gzip middleware and http.TimeoutHandler buffer too, so keep them off streaming routes. Proxies buffer as well: send X-Accel-Buffering: no for nginx, and set FlushInterval: -1 on a Go httputil.ReverseProxy. Timeouts and lifetime Extend the deadline before each event with rc.SetWriteDeadline(...) instead of disabling WriteTimeout for the whole server. Send a : ping comment every 15–30 seconds so idle proxies and load balancers keep the connection, and stop when r.Context().Done() fires: the client has gone. SSE or WebSocket SSE is one-way, server to client, over plain HTTP; the browser's EventSource reconnects by itself and resumes with Last-Event-ID. WebSocket is two-way and needs a library such as github.com/coder/websocket or gorilla/websocket; choose it when the client also streams messages.

On the error path the function closes the body without reading it. The Transport puts an HTTP/1.1 connection back into the pool only when the body was read to EOF and closed, so every early Close throws the connection away and the next call dials again. The Transport cannot tell where the next response starts in a half-read stream, so closing early means closing the connection. With HTTP/2 only the stream is reset and the connection survives. Drain before closing, with a bound: io.Copy(io.Discard, io.LimitReader(resp.Body, 64<<10)). Reading 64 KB is cheaper than a new handshake; a much larger body is worth abandoning. Read the error body anyway: the upstream usually explains the failure there. json.Decoder stops after one value and can leave bytes unread, so drain on the success path too. Symptoms: a handshake per request in traces, sockets piling up in TIME_WAIT, and finally connect: cannot assign requested address when the ephemeral ports run out.

Raise MaxIdleConnsPerHost: it defaults to 2, so under concurrency almost every connection is closed after use instead of being pooled, and each request pays a new TCP and TLS handshake. Start from a clone of http.DefaultTransport and size the pool to your steady concurrency. The HTTP/1.1 pool One connection carries one request at a time, so 200 concurrent requests need 200 connections, and by default only 2 per host may stay idle afterwards. MaxIdleConns (100 in DefaultTransport) caps idle connections across all hosts; keep it above the per-host value. MaxConnsPerHost caps all connections to a host. Past it, requests wait for a free connection instead of failing, so every call needs a context deadline. Keep IdleConnTimeout below the upstream's own keep-alive timeout, or you reuse a connection the server is just closing and get intermittent EOF or connection reset. HTTP/2 One connection multiplexes many concurrent streams (the server sets the limit, often 100–250), so the idle-pool settings matter much less. A bare &http.Transport{TLSClientConfig: cfg} quietly stays on HTTP/1.1: a custom TLS config disables HTTP/2 unless ForceAttemptHTTP2 is true, which a clone of DefaultTransport keeps. HTTP/2 removes head-of-line blocking between requests, but one lost TCP packet still stalls every stream on the connection; HTTP/3 over QUIC removes that as well, but needs a third-party library such as quic-go; net/http does not speak it. Verify, do not guess httptrace.ClientTrace reports Reused in GotConn; count it next to ConnectStart to see the real reuse rate.

Send one Idempotency-Key per operation, reused by every retry, and have the server store the result under that key, so a repeat returns the saved response instead of charging again. Transport also replays some requests by itself when a pooled connection dies. What net/http replays by itself Only on a reused idle connection that fails before any response arrives: the server may have closed it just as the request went out. Never on a fresh dial, so it cannot loop. The request must be rewindable: no body, or GetBody set. http.NewRequest sets it for *bytes.Reader, *bytes.Buffer and *strings.Reader. And it must be idempotent: GET, HEAD, OPTIONS, TRACE, or any method carrying an Idempotency-Key or X-Idempotency-Key header. A plain POST is replayed only when the error proves nothing was written. A safe retry loop A fresh *http.Request for every attempt, the same key every time, and the previous response body drained and closed. Retry network errors, 429, 502, 503 and 504 with exponential backoff and jitter, honour Retry-After, and stop when the caller's context is done. Never retry 400 or 422: the answer will not change. A 409 from the in-flight rule is worth one retry after a short delay; a 409 for a real conflict is not. The server side Store key, status and response body under a unique constraint, in the same transaction as the charge. A duplicate that arrives while the first attempt is still running gets 409; keys expire after a day or so.