Goroutines & Channels

A goroutine is a function running concurrently under the Go runtime's own scheduler: it starts with a small stack that grows on demand, and the runtime multiplexes many goroutines onto a few OS threads, so starting one is cheap and needs no system call. Start: go f(x): the arguments are evaluated at once and the return value is discarded. Stack: a thread has a fixed stack of several MB; a goroutine starts small and grows on demand. Scheduling: threads are switched by the kernel; goroutines are switched by the Go scheduler in user space (the G-M-P model). At most GOMAXPROCS threads run Go code at the same time. Identity: a goroutine has no public ID or priority and cannot be killed from outside. It stops only by returning, so you design how it is told to stop.

Concurrency is structuring a program as tasks that make progress independently; parallelism is executing several of them at the same instant on different cores. Concurrency is about design, parallelism about execution. A correct concurrent Go program also runs on one core: with GOMAXPROCS=1 goroutines interleave but never run simultaneously. Parallelism needs several cores and GOMAXPROCS above 1. The default is the number of usable CPUs. Concurrency pays off without parallelism when tasks mostly wait: a thousand HTTP calls in flight on one core finish far sooner than one after another. CPU-bound work gets faster only with parallelism.

When main returns the process exits and the runtime cuts off every other goroutine mid-flight without running its deferred calls; to wait for them use a sync.WaitGroup (or a channel). go f(x) returns at once and promises nothing about when f runs; without a wait, main can exit before f has even started. The standard wait is sync.WaitGroup: wg.Add(1) before go, defer wg.Done() inside, wg.Wait() before returning. Since Go 1.25 wg.Go(f) replaces the Add/go/Done trio with one call. When you also need results or errors, wait on a channel or use errgroup instead. time.Sleep in main is not a wait: it only hides the race until the machine is slower.

Hand the result back through something the caller waits on: a channel, a pre-sized slice plus a sync.WaitGroup, or an errgroup.Group when errors matter too. Channel: the goroutine sends, the caller receives; the receive is both the wait and the handoff. For a single result use make(chan T, 1), so the sender can finish even if nobody reads any more. **Slice + WaitGroup**: each goroutine writes only its own element results[i], so no lock is needed and the order is kept. Read the slice only after wg.Wait(). **errgroup**: the same slice pattern, plus the first error from g.Wait(). Writing to a shared variable without a wait or a lock is a data race, even when it seems to work in tests.

An unbuffered channel (make(chan T)) is a handoff: a send blocks until another goroutine receives, and a receive until someone sends. A buffered one (make(chan T, n)) blocks a send only when its n slots are full and a receive only when it is empty. A completed send on an unbuffered channel also tells the sender that the receiver has the value; it is a synchronisation point. A buffer decouples sender and receiver by up to n items. It smooths bursts but does not make a slow consumer faster. len(ch) is the number of items buffered right now and cap(ch) the buffer size. len is stale the moment you read it, so never use it to decide whether a send will block. If every goroutine ends up blocked, the runtime stops the program with fatal error: all goroutines are asleep - deadlock!.

On a closed channel a receive returns the remaining buffered values and then the zero value forever, while a send or a second close panics. On a nil channel a send or a receive blocks forever, and close panics. v, ok := <-ch tells a real zero apart from "closed": ok is false only once the channel is closed and drained. for v := range ch makes that check for you and stops. close throws nothing away: buffered values are still delivered before receivers start getting zero values. A closed channel is always ready, so a bare v := <-ch in an endless for spins a CPU core after the close. A nil channel is never ready, which is useful: setting a channel variable to nil switches its case off inside a select. The panic messages are send on closed channel, close of closed channel and close of nil channel.

select waits until one of its channel operations can proceed and runs that case, choosing at random if several are ready; default makes it non-blocking, and a timeout is just one more case <-time.After(d) or case <-ctx.Done(). With no case ready and no default, the goroutine blocks. With a default, select never blocks; that is how you write a non-blocking send or receive. For a timeout add case <-time.After(d):, or better case <-ctx.Done():, so the caller controls the deadline. Every channel expression and every value to send is evaluated once on entry, even for the cases that are not chosen. select {} with no cases blocks forever. If one case must win (stop before taking more jobs), check it first in its own select with a default. Usually closing the jobs channel is the cleaner fix.

It prints 1, 2, 3, then dies with fatal error: all goroutines are asleep - deadlock!: range waits for a fourth value because nobody closed ch, and no other goroutine could send one. The detector fires only when every goroutine is blocked, so a live server never trips it. The fix is close(ch) after the sending loop: range then drains the buffer, ends, and done is printed. In a server the listener, a ticker or a signal handler is always alive, so a real deadlock shows up as hung requests and a growing goroutine count, not as a crash. To find it in production, take a goroutine dump: /debug/pprof/goroutine?debug=2 prints every stack with its wait state and duration, for example [chan receive, 12 minutes]. kill -QUIT prints the same but also kills the process. The usual channel deadlocks: an unbuffered send with no receiver, range over a channel nobody closes, two goroutines each waiting to send to the other.

The sender closes (the goroutine that knows no more values are coming) and receivers never do. Directional types let the compiler enforce that: you cannot send on or close a <-chan T, and you cannot receive from a chan<- T. Close only when receivers must learn that the stream has ended (range, ok == false). A channel nobody closes is simply garbage-collected. With several senders none of them closes: a separate goroutine closes after wg.Wait(), otherwise some sender panics on a closed channel. A plain chan T converts to either direction implicitly when passed, so the narrowing costs nothing at run time. Closing is also a broadcast: every goroutine waiting on a done channel wakes at once. That is the standard way to stop many goroutines, and ctx.Done() is exactly such a channel. The usual stage signature: take in <-chan T, create out inside, defer close(out) in the goroutine that sends, return it as <-chan T.

A number below 1000 that changes from run to run: count++ is a load, an add and a store, so two goroutines can read the same value and one increment is lost. Data race: both goroutines load n and both store n+1. go run -race reports WARNING: DATA RACE on the count++ line. Fixes: atomic.Int64 for a single number, a sync.Mutex when several fields change together, or one goroutine that sums values received from a channel. The race detector finds it; the output alone does not, because a run can print 1000 by luck.

wg.Add(1) runs inside the new goroutine, so wg.Wait() can see a zero counter before any goroutine has started and return at once; fetchAll then hands back a slice the workers are still filling. go func() only schedules the goroutine. The loop usually finishes and reaches Wait before the first Add runs, so most entries usually come back empty. It is also a data race: the caller reads bodies while workers write it, and go run -race reports it. The fix is to call wg.Add(1) in the parent before go, so the counter is already right when Wait runs. Since Go 1.25 wg.Go(func() { ... }) does Add and Done for you and removes the trap. The closure capturing i and u is fine: since Go 1.22 each iteration has its own loop variables. go vet in Go 1.25 catches it with the waitgroup analyzer: "WaitGroup.Add called from inside new goroutine".

A channel is a pointer to a runtime hchan struct: a lock, an optional ring buffer, and two queues of parked goroutines, sendq and recvq. A blocked sender is parked in sendq, and its OS thread moves on to other goroutines. A receiver is already waiting in recvq: the value is copied straight onto its stack and it is made runnable; the buffer is not touched. This is the whole of an unbuffered handoff. There is room in the buffer: the value is copied into buf[sendx], sendx moves around the ring and qcount goes up. Neither: the sender wraps itself in a sudog (goroutine plus a pointer to the value), joins sendq and calls gopark. It costs no CPU while it waits. A receive mirrors this. When senders are waiting on a full buffer, it takes the oldest value and moves the first waiting sender's value into the freed slot, so FIFO order holds. close sets the closed flag and wakes every parked goroutine: receivers get zero values, senders panic. Every operation takes the channel's lock, which is why a channel costs more than a bare sync.Mutex and why one hot channel shared by many cores becomes a point of contention. make returns a pointer, so passing a channel to a function copies a reference, not the queue.

Use a channel when ownership of data moves between goroutines or when you must wait on several events at once; use a mutex when goroutines share state with a short critical section. The proverb is about handing data over, not about guarding a field. Channel fits: a job to a worker, a result back, a completion or cancellation signal, a select over several sources with a timeout. A mutex has no equivalent of select. Mutex fits: a cache, a counter, a map of sessions. An "owner goroutine" serving the same cache over channels pays a reply-channel allocation and two scheduler handoffs per Get, and serialises readers that sync.RWMutex would let run together; it is measurably slower. Hidden costs of channels: every operation takes a lock and may park a goroutine, and the ownership rules (who closes, who drains) bring leak and panic risks a mutex does not have. A single number is cheaper still with sync/atomic. The standard library itself is full of mutexes, and every channel has a lock inside.

A goroutine leak is a goroutine that never returns because it waits for something that will never happen; it keeps its stack and everything it references alive, so memory and the goroutine count grow with traffic. Blocked send: a worker sends its result on an unbuffered channel after the caller timed out and left. For one result use make(chan T, 1); for a stream, select every send against ctx.Done(). Blocked receive: range over a channel nobody closes, or a wait on a channel nobody will ever send on. No way to stop: a loop on a ticker with no ctx.Done() case, or a network call made without a context or timeout. In production: chart runtime.NumGoroutine(); the goroutine profile (/debug/pprof/goroutine?debug=1) groups identical stacks, so thousands parked on one chan send line stand out. Go 1.26 adds an experimental goroutineleak profile (GOEXPERIMENT=goroutineleakprofile) that reports goroutines blocked on channels or locks no running code can reach. In tests: defer goleak.VerifyNone(t) from go.uber.org/goleak fails a test that leaves goroutines behind. -race does not see leaks.

time.After(d) returns a channel that fires once; NewTimer does the same but can be stopped and reset; NewTicker fires repeatedly; AfterFunc runs a function in its own goroutine instead. Since Go 1.23 forgotten timers no longer leak. Before Go 1.23 a timer stayed alive until it fired, so case <-time.After(5 * time.Minute) in a busy loop piled up five minutes of timers, and a time.Tick ticker was never freed. Since Go 1.23 timers and tickers nothing references are garbage-collected even if never stopped, and timer channels are unbuffered, so after Stop or Reset returns no stale tick can arrive. The old drain-before-Reset idiom is no longer needed. The behaviour follows the go line in the main module's go.mod, not the toolchain: a module still on go 1.22 keeps the old semantics. GODEBUG=asynctimerchan=1 also brings them back. In a hot loop one NewTimer with Reset per pass still beats time.After: one allocation instead of one per message. defer t.Stop() on tickers anyway; AfterFunc's Stop returns false when the function has already started.

Start N goroutines that range over one jobs channel and send to one results channel; the producer closes jobs when it is done, and a separate goroutine closes results after wg.Wait(), which ends the caller's range over results. Size N by the bottleneck, not by the input: the database pool size for inserts, runtime.GOMAXPROCS(0) for CPU work. One goroutine per row is what makes Postgres answer sorry, too many clients already. An unbuffered jobs channel gives backpressure for free: the producer blocks while all workers are busy, so the input is never all in memory. close(results) cannot go after the range over results (that line is never reached) and no single worker may close it while others still send. For cancellation every blocking send also selects on ctx.Done(), so a cancelled caller does not leave workers stuck. If you know exactly how many results come back, receive that many and skip closing results entirely.

Use a buffered channel as a counting semaphore (sem := make(chan struct{}, 8), send before starting the work, receive when it ends) or errgroup.Group with SetLimit(8). Unlike a pool, each item still gets its own short-lived goroutine; the limit only caps how many exist at once. Acquire in the loop, before go: the loop itself waits, and at most eight goroutines exist. Acquiring inside the goroutine still creates one per item; they just queue on the semaphore. Release with defer, so an early return or a panic does not lose a slot for good. A bare send cannot be cancelled. Wrap it in a select with ctx.Done(), or use golang.org/x/sync/semaphore, whose Acquire(ctx, n) takes a context and supports weighted slots. Semaphore fits when work already arrives as a loop of independent items. Worker pool fits when workers hold expensive state (a connection, a buffer) or when you want a fixed set of long-lived goroutines.

Start one goroutine per input that copies its values to a shared output, plus one goroutine that closes the output once a WaitGroup says every copier is done. If an input never closes, the output never closes either, so the merge must also stop on ctx.Done(). Each copier selects both its receive and its send against ctx.Done(); otherwise a stuck input or a consumer that stopped reading leaves copiers blocked forever. Only the closer goroutine closes out: the copiers are several senders and must not. Order between inputs is not preserved; order within one input is. For exactly two inputs a single select loop works too: when an input closes, set its variable to nil; a nil channel is never ready, so its case drops out. Without that, the closed channel is ready on every pass and floods out with zero values.

Give each goroutine its own unbuffered channel as a turn token: it waits for its token, prints, and passes the token to the other. Only one goroutine holds the turn at any moment, so the output is strictly 1, 2, 3 and so on. The unbuffered channel is the handoff: a send completes only when the other goroutine is ready to take its turn. Whoever prints the last number must not pass the turn: nobody is left to receive, and that send would block forever. main starts the game by handing out the first token and waits with a WaitGroup; without the wait it exits before anything is printed. The same can be done with a sync.Mutex, a sync.Cond and a shared turn variable, but the channel version is shorter and shares no state.

Use errgroup.WithContext from golang.org/x/sync/errgroup: every task started with g.Go uses the shared context, the first non-nil error cancels it, and g.Wait() waits for every task and returns that first error. Cancellation is cooperative: the context is cancelled, but a task that ignores ctx runs to the end, and Wait waits for it. Wait returns only the first error. If you need all of them, collect them yourself under a mutex and return errors.Join(errs...). g.SetLimit(n) caps concurrency: g.Go blocks while n tasks run. Call it once before the first Go; changing it while tasks are active panics. g.TryGo is the non-blocking variant. Write results into a pre-sized slice by index; distinct elements need no lock. The derived ctx is cancelled once Wait returns, so do not use it for work after the group. A panic in a task is not caught by the group: it crashes the process, as in any goroutine. recover inside the task if you need to turn it into an error.

Use a token bucket: tokens refill at a fixed rate up to a burst size, each request takes one, and an empty bucket means wait or reject. In real code that is rate.NewLimiter from golang.org/x/time/rate, which is already goroutine-safe. The state (tokens and the time of the last refill) lives in one struct behind a sync.Mutex. Refill lazily on each call from the elapsed time instead of running a goroutine per limiter. Allow() answers at once; for an HTTP API that means 429 Too Many Requests, ideally with Retry-After. Wait(ctx) blocks until a token is free and fails if the context ends first. A ticker filling a buffered channel is a quick live-coding answer, but it spends a goroutine per limiter and cannot "reject now" without extra code.

Keep a map of entries with an expiry time behind a sync.RWMutex, treat expired entries as missing on read, and delete them with one janitor goroutine per cache that sweeps on a ticker and stops on Close. A time.AfterFunc or a goroutine per key costs a timer and a closure per entry, so memory grows with writes rather than with live keys. Checking the expiry on read keeps results correct even when the janitor is late; the janitor only frees memory. The janitor needs a stop path (Close or ctx.Done()), or every cache created in tests leaks a goroutine. Get can stay on RLock because it deletes nothing; deletion needs the write lock, so it is left to the janitor. Cache stampede: on a miss use singleflight.Group.Do(key, fn) so one goroutine loads; see the Redis cache question for the context trap.

Each stage takes an input channel, starts a goroutine that reads it, and returns an output channel it closes when done. Normal shutdown flows downstream through close; early shutdown must flow upstream through a shared ctx.Done() that every send selects on. Downstream: close When the source runs out it closes its output; the next stage's range ends, its defer close(out) fires, and the chain unwinds stage by stage. Each stage closes only the channel it owns. Upstream: cancellation A channel carries no signal back to its sender. If the consumer breaks after ten values, the last stage parks forever in its send, the stage before it parks too: one leaked goroutine per stage. So every send is select { case out <- v: case <-ctx.Done(): return }, and the consumer calls cancel() (deferred) when it stops. Errors travel with the data (struct{ v T; err error }) or through an errgroup, which also cancels the other stages. Large inputs For a 1 TB file, stream it: read chunks with a bufio.Reader, hand chunks between stages, and keep channels small. Memory is then bounded by stages × buffer × chunk size, not by the file. Speed up the slow stage by running several goroutines in it and merging their outputs; if order matters, tag items with their position first. A test that reads everything proves nothing about leaks: test an early break with goleak.VerifyNone. -race does not detect blocked goroutines.

Backpressure is the slow consumer slowing the producer down: a blocking send on a full channel makes the producer wait, so work never piles up faster than it is processed. A buffer only absorbs short bursts; it does not change throughput. What a buffer buys It decouples two stages whose speeds vary around the same average, so neither stalls on every hiccup. Size it to the burst or batch you expect, not "large to be safe". If the consumer is permanently slower, any buffer fills and you are back to blocking, just later, with more memory and more latency per item. A huge buffer hides the problem: latency grows quietly, and a restart loses everything queued in memory. When the consumer is permanently slower Let it block and push the pressure upstream: to the HTTP client (timeouts, 503), to the queue consumer's lag, to the producer's own rate. Add consumers to the slow stage, as long as the bottleneck is not one shared resource. Batch or coalesce: insert 500 rows per statement instead of one, or keep only the latest value per key. Shed load you can afford to lose (metrics, debug logs) with a non-blocking send: select { case ch <- v: default: dropped.Add(1) }. Export the drop counter, or the data lies silently. Spill to durable storage (a message queue, disk) when every item matters and bursts are long. Never put a blocking send to an optional sink such as metrics on the request path: when the collector stalls, every handler stalls with it.

Fix each item's position before the work fans out: a single dispatcher gives every item its own one-slot result channel and pushes those channels onto a FIFO queue in input order; the consumer reads the queue and waits on each result in turn. One dispatcher walks the input, so the queue order is the input order. Per item it makes res := make(chan Result, 1), sends res onto a chan chan Result, and hands the item plus res to a worker. The consumer is for res := range queue { out <- <-res }. It waits on item k while later items are already finished in their own buffers, so the parallelism is untouched. The one-slot buffer keeps the worker free: unbuffered, it would wait for a consumer that is behind. The queue's capacity is the memory bound: when it is full the dispatcher blocks, so only about cap(queue) results (plus the ones in flight) are held, not the whole file. The alternative is to tag items with a sequence number and keep a reorder buffer (a map or a min-heap) that emits the next expected number; it also needs a window limit, or one slow item lets the buffer grow without bound.