[ADP-607] Event Tracing
Overview
In distributed systems, tracing the flow and processing of events is crucial for understanding system behavior, diagnosing issues, and optimizing performance. This ADP aims to define a standard method for implementing event tracing using W3C TraceContext and CloudEvents distributed tracing extension.
Guidelines
MUST use the W3C TraceContext standard for cross-service distributed tracing.
SHOULD use the CloudEvents distributed tracing extension to include tracing information in events.
Event producers MUST add tracing context when generating events.
Event consumers MUST propagate tracing context when processing events.
Tracing implementation SHOULD be transparent to application code, minimizing intrusiveness.
W3C TraceContext
W3C TraceContext defines two HTTP headers:
traceparent
: Contains trace identifier and parent identifiertracestate
: Contains vendor-specific tracing information
Example:
traceparent: 00-0af7651916cd43dd8448eb211c80319c-b9c7c989f97918e1-01
tracestate: rojo=00f067aa0ba902b7,congo=t61rcWkgMzE
CloudEvents Distributed Tracing Extension
CloudEvents defines two extension attributes to support distributed tracing:
traceparent
: Corresponds to W3C TraceContext'straceparent
tracestate
: Corresponds to W3C TraceContext'stracestate
Example CloudEvent:
{
"specversion" : "1.0",
"type" : "com.example.someevent",
"source" : "/mycontext",
"id" : "C234-1234-1234",
"time" : "2018-04-05T17:31:00Z",
"traceparent" : "00-0af7651916cd43dd8448eb211c80319c-b9c7c989f97918e1-01",
"tracestate" : "rojo=00f067aa0ba902b7,congo=t61rcWkgMzE",
"data" : {
"message" : "Hello, World!"
}
}
Implementation Recommendations
Use tracing libraries or frameworks that support W3C TraceContext.
Ensure correct propagation of
traceparent
andtracestate
when passing events between services.Include trace IDs in log entries to facilitate correlation between logs and traces.
Consider implementing sampling strategies to control the volume of trace data.
Use distributed tracing visualization tools to analyze and present trace data.
Security Considerations
Ensure that tracing information does not contain sensitive data.
Implement appropriate access controls to limit access to trace data.
Consider using sampling in production environments to reduce the volume of trace data, thereby lowering potential security risks.