multimind.observability.tracing records runs (root traces) and nested spans for LLM, chain, and tool calls, writes them to a local SQLite file, and optionally exports them to a self-hosted MultiMind platform over HTTP. It ships in the core install (its only dependency is httpx, already a core requirement).
Privacy first: by default no prompt or response text is stored or exported. Each run keeps only the character count and the SHA-256 hash of its inputs and outputs, which is enough to spot duplicates and payload drift without retaining the text.
Quick start
generate, chat, generate_stream, chat_stream, generate_sync, and chat_sync call on the wrapped model records one llm run with start and end times, token usage, and cost. Any other attribute (for example model_name) is proxied to the wrapped model, so trace_model composes with guard() and track_costs():
Runs and spans
RunTracer.run() returns a context manager; Run.span() opens a child. A run that is still open when its parent finishes is closed automatically, and a finished root run is dispatched to the sinks with all of its spans.
run_type must be one of llm, chain, or tool; anything else raises ValueError. Leaving a with block through an exception marks the run status="error" and stores "ExceptionType: message" in error.
Calls made on a traced model inside an open run are nested under it automatically (ambient context via contextvars), so the LLM span lands beneath the chain that made it:
traced decorator records one run per call of a sync or async function:
What a run contains
Each run and span is a dict with the same fields, which is also the row layout of the SQLiteruns table and the body sent to the platform:
Privacy defaults and store_content
The default tracer never writes prompt or completion text anywhere: inputs and outputs are null in memory, in SQLite, and in the export payload. The tests in tests/test_tracing.py assert that a prompt containing an SSN appears neither in the HTTP body nor in the SQLite file bytes.
Pass store_content=True to keep inputs and outputs verbatim. This makes traces far easier to debug, but it means prompts and completions, including any PII they contain, are written to the SQLite file and sent to the platform. Pair it with the PII guard if you need it in a regulated environment.
SQLite sink
sqlite_path creates the file (and parent directories) on construction with a single runs table whose columns are the fields above; tokens and tags are stored as JSON text. Root runs and spans are rows in the same table linked by parent_id.
HTTP export
POST {export_url}/api/v1/runs with an X-API-Key header and the JSON body {"run": {...}, "spans": [{...}, ...]}. A failed request (connection error or HTTP 4xx/5xx) is logged and the payload stays buffered; tracer.pending_exports reports how many are waiting and tracer.flush() retries them. flush() is also registered with atexit so buffered runs are retried at interpreter shutdown.
Default tracer and environment variables
trace_model(model) without an explicit tracer uses a process-wide tracer built from the environment on first use:
get_default_tracer() returns that instance and reset_default_tracer() discards it (useful in tests after changing the environment). store_content cannot be enabled through the environment; it must be passed explicitly.
API summary
Source: multimind/observability/tracing.py. Tests: tests/test_tracing.py.