The first question on every new Go service with a database: standard library database/sql? sqlx? pgx directly? An ORM like GORM? sqlc for code generation? The honest answer is that the right choice depends on the access pattern, the team’s SQL proficiency, and how much you value type safety at the cost of verbosity. Here’s what each option looks like in practice. ...
The first feature flag I implemented was a boolean in a config file. Toggle it to true, deploy, feature is on. Toggle to false, deploy, feature is off. Simple. The fifteenth feature flag was a percentage rollout with user cohort targeting, metric-based automatic rollback, and a kill-switch that worked without a deployment. The gap between those two implementations is what this post is about. ...
Go has a strong house style that experienced practitioners converge on. Some of it is enforced by gofmt and golint. The rest is transmitted through code reviews, the standard library, and writing enough Go to feel the natural grain of the language. After several years of Go, here are the patterns that mark idiomatic code and why they work. ...
Unstructured logging is a debugging tool. Structured logging is an observability tool. The difference: unstructured logs tell a human what happened; structured logs tell a machine what happened, and the machine can then tell a human efficiently. Unstructured: 2022-11-23 10:58:34 ERROR failed to process order ORD-12345 for user usr_abc: connection refused Structured: {"time":"2022-11-23T10:58:34Z","level":"ERROR","msg":"failed to process order", "order_id":"ORD-12345","user_id":"usr_abc","error":"connection refused", "service":"order-service","version":"1.2.3"} The structured log can be indexed, filtered, aggregated, and alerted on. order_id:ORD-12345 returns all logs for that order across all services. The unstructured log requires grep and hope. ...
We had an incident that took down pricing for 23 minutes during the London open. High severity, real monetary impact, humbling root cause: a configuration value that worked in staging silently didn’t apply in production due to an environment variable naming collision. The postmortem process that followed was one of the better-run ones I’ve participated in. Here’s what made it useful. ...
This is the JVM false-sharing problem in a different language. The rules differ slightly, the tooling differs, but the underlying hardware constraint — cache lines are 64 bytes and sharing one across goroutines is expensive — is identical. ...
On-call has a bad reputation in software engineering, and often deservedly so. Being paged at 3am for an alert that didn’t need to wake anyone is demoralising. Being on-call for systems you didn’t build, don’t understand, and can’t fix is terrifying. Being paged multiple times a night for weeks is a health risk. But on-call done well is a powerful practice. It creates direct feedback between the reliability of what you build and the experience of carrying it. When engineers are responsible for their own systems, they ship more reliable systems. Here’s the version that worked at the European fintech firm. ...
Before OpenTelemetry, distributed tracing was a vendor-specific integration. You chose Jaeger or Zipkin or Datadog, added their SDK, and traced against their API. Switching vendors meant rewriting instrumentation. Adding a library that used a different tracing SDK meant two tracing systems running in parallel. OpenTelemetry solved this with a vendor-neutral API and a pluggable exporter model. The API stays the same; you swap exporters. Most major observability vendors now accept OTel format natively. Here’s what a solid Go integration looks like. ...
The fintech startup adopted Kafka early — we were processing market events at rates that would have overwhelmed any request-response queue. Two years in, with a five-broker cluster handling 200k messages/sec at peak, the operational experience was significantly different from what I’d expected based on the documentation and conference talks. ...
The team I joined was called “platform engineering.” The mandate was loosely: make other engineering teams faster. The first three months were spent understanding what “faster” meant, which was more contentious than expected. ...