logoalt Hacker News

Wide logging: Stripe's canonical log line pattern

14 pointsby alcazartoday at 3:48 PM5 commentsview on HN

Comments

bradleybudatoday at 5:35 PM

This is all good advice but it's not what I would traditionally think of as "logging" - it's something more like what the industry calls APM or tracing.

> For a web service, that usually means one log event at the end of every request.

The point of a log, as I have seen it used, is to understand what's going on during a request. Yes, some of that is timing / success / failure information, but you also need:

- Logical tracing - at a high level, what decisions did we make (authz, business logic) when serving this request

- Partial failures or warnings - the request may have succeeded, but we may have failed or timed out a call to a dependent service and delivered a partially degraded response as a result

- Tracepoints to support active development - how often do we hit a certain codepath (to help developers roll out or deprecate features)

It's useful to distinguish between "context" (things we know about a request), timing, success, and plain old logging (using human language to describe to an operator what the system is doing). Context should be included with every line - I strongly agree with OP's proposal of wide events, don't make operators ever JOIN if you can avoid it - but it won't always be the same because we learn things about the request as we process it. Yes, if you wait until the end of a request, you will have maximal context, but at the loss of fine-grained detail during processing.

show 1 reply
deskamesstoday at 5:30 PM

Are there any restrictions on how short the error_slug should be? The meat of some of my errors can be pretty long (for example an ffmpeg error). There are also many phases to a job - call them tasks. Can a canonical log line be a collection of task log lines?

show 1 reply
formerly_proventoday at 4:49 PM

It's strange how undercooked logging is (not just in standard libraries, but in general) given everyone and every app does it.

show 1 reply