Concurrency

Field	Value
Type	Skill Resource
Source	~/.copilot/skills/backend/references/concurrency.md
Description	Not specified

Source Content

Concurrency

Two rules carry the weight: no raw go keyword in service code, and -race on every test run. The first prevents leaked, panic-swallowing goroutines; the second turns data races from production mysteries into failed builds.

Use sourcegraph/conc, not raw goroutines

A bare go f() swallows panics (crashing the whole process), has no built-in error propagation, and is trivially leaked. sourcegraph/conc fixes all three: it recovers panics in child goroutines and re-raises them in the parent, waits cleanly, and bounds parallelism.

Fire-and-forget with a wait

import "github.com/sourcegraph/conc"

var wg conc.WaitGroup
wg.Go(func() { sendWelcomeEmail(ctx, user) })
wg.Go(func() { warmCache(ctx, user.ID) })
wg.Wait() // panics in either child are recovered and re-panicked here

Fan-out with results and errors

import "github.com/sourcegraph/conc/pool"

p := pool.NewWithResults[Invoice]().WithContext(ctx).WithMaxGoroutines(8)
for _, id := range ids {
  id := id
  p.Go(func(ctx context.Context) (Invoice, error) {
    return s.repo.ByID(ctx, id)
  })
}
invoices, err := p.Wait() // first error cancels the context for the rest
if err != nil {
  return nil, fmt.Errorf("batch load invoices: %w", err)
}

WithMaxGoroutines bounds parallelism so a 10k-item batch does not open 10k DB connections. WithContext cancels siblings on the first error.

Ordered output

When results must come back in input order, use conc/stream — it runs work concurrently but emits callbacks in submission order.

Every blocking call takes a context

context.Context is always the first parameter and is honored, not ignored.

func (s *InvoiceService) Finalize(ctx context.Context, id domain.InvoiceID) (domain.Invoice, error) {
  // ctx flows into the repo, which flows it into pgx, which cancels the query on timeout
  inv, err := s.repo.ByID(ctx, id)
  ...
}

• Thread the request context from the handler (c.Context() in Fiber) down through service and repository so a client disconnect or deadline cancels the whole chain.
• Never store a context in a struct — pass it.
• Never pass context.TODO() in production paths; it signals an unfinished wiring.

-race is non-negotiable

task test runs with the race detector every time, locally and in CI:

go test -race ./...

• A detected race is a failed build, not a flaky test to retry.
• The race detector only catches races that actually execute, so it depends on test coverage exercising the concurrent paths — another reason concurrent code earns a test.
• Race builds are slower and use more memory; that cost is paid in CI so it is never paid in production.

Shared state needs a guard

If two goroutines touch the same memory and at least one writes, you need synchronization — a mutex, a channel, or sync/atomic. Prefer not sharing: give each goroutine its own data and combine results at the join (the pool.NewWithResults pattern above does this for you). Share memory by communicating, do not communicate by sharing memory.

Concurrent map access can crash the whole process

A plain Go map is the sharpest case of “shared state needs a guard” above: an unsynchronized concurrent read+write on a map is not merely a data race the race detector flags — it is a fatal runtime error, fatal error: concurrent map read and map write, that recover() cannot catch. It takes down the entire process, even when the offending write happens inside a goroutine supervised by conc.

// before: two conc goroutines writing the same map with no guard — a crash waiting to happen
var counts = map[string]int{}

var wg conc.WaitGroup
wg.Go(func() { counts["a"]++ })
wg.Go(func() { counts["b"]++ })
wg.Wait()

// after: guarded with a mutex (or use sync.Map if the access pattern is read-heavy)
var (
  mu     sync.Mutex
  counts = map[string]int{}
)

var wg conc.WaitGroup
wg.Go(func() {
  mu.Lock()
  counts["a"]++
  mu.Unlock()
})
wg.Go(func() {
  mu.Lock()
  counts["b"]++
  mu.Unlock()
})
wg.Wait()

context.WithoutCancel for detached background work

Work that must outlive the request that spawned it — a fire-and-forget audit log write kicked off from a handler that should not be cancelled just because the client disconnected — should derive from context.WithoutCancel(ctx), not context.Background().

wg.Go(func() {
  detached := context.WithoutCancel(ctx) // survives the parent's cancellation
  if err := s.audit.Record(detached, event); err != nil {
    log.Error("audit write failed", zap.Error(err))
  }
})

WithoutCancel preserves every value carried on ctx — including the trace context, so the detached write still shows up correlated in traces (references/observability.md) — while detaching cancellation and deadline. context.Background() loses all of that, including trace_id, orphaning the write from the request that caused it.

Nested deadlines: the shortest one wins

A child context.WithTimeout derived from a parent that already carries a shorter deadline still expires when the parent does — a child asking for more time than its parent has left never gets it. The effective deadline is always the earliest one in the chain.

parentCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()

// child asks for 10s, but parentCtx already expires in 2s — the child still expires in 2s
childCtx, cancel := context.WithTimeout(parentCtx, 10*time.Second)
defer cancel()

This matters when composing service calls that each set their own timeout: a repository call’s 10-second timeout is silently capped the moment it is derived from a handler context that only had 2 seconds left.

goleak in tests

Add go.uber.org/goleak’s goleak.VerifyTestMain(m) to a package’s TestMain so a leaked goroutine — one blocked forever on an unbuffered channel because a conc pool was never drained or waited on — fails the test instead of silently piling up.

func TestMain(m *testing.M) {
  goleak.VerifyTestMain(m)
}

This plugs into the existing test suite described in references/testing.md: it runs alongside the table-driven and coverage gates, not instead of them.