dprovenance.dev / integrations / langchain

LangChain tracing
that fails the PR.

DProvenanceKit ships a drop-in callback handler for LangChain and LangGraph: pass it via config={"callbacks": [...]} and every chain or agent run is recorded as a queryable, diffable trace. Then diff a candidate run against a golden baseline and gate CI when the reasoning drifts — a tool called in a different order, a retrieval step skipped, a changed execution path. Agent regression testing, a couple lines of setup.

Works with langchain-core (and LangGraph) via the standard callbacks interface — the same stream LangChain hands every tracer.

# 01 · install

Install

shell
$ pip install "dprovenancekit[langchain]"

Python 3.9+ · zero third-party deps in the core · the extra pulls in langchain-core.

# 02 · pass the callback

Pass the callback

DProvenanceTracer.trace() yields a callback handler. Pass it to any chain / graph invocation and the run is recorded — no other code changes.

trace_chain.pypython
from dprovenancekit import SQLiteTraceStore
from dprovenancekit.integrations.langchain import DProvenanceTracer, LangChainTraceEvent

store = SQLiteTraceStore(LangChainTraceEvent, "traces.sqlite")
tracer = DProvenanceTracer(store)

with tracer.trace(context_id="customer-42") as cb:
    answer = chain.invoke(question, config={"callbacks": [cb]})

# The run is now recorded — query it, diff it against a known-good run, or
# compare run fingerprints to detect when the agent took a different path.

Full reference: the LangChain section of the README.

# 03 · what gets recorded

What gets recorded

DProvenanceCallbackHandler translates LangChain's callback stream into a trace:

  • Each on_llm_start / on_tool_start / on_retriever_start / on_chain_start (and its completion) becomes a typed event in execution order.
  • LangChain's run_id / parent_run_id become the trace's span tree.
  • The active model / tool / retriever becomes the event's engine.
  • Lifecycle provenance edges are emitted: DERIVED_FROM (start → completion) and INFORMED (parent → child).
  • Options: capture_payloads (prompt/completion/IO previews), link_lifecycle (edges), record_chains (LCEL / LangGraph chain noise).

A run's fingerprint is the structural identity of its execution path, so two runs that diverge produce different fingerprints — a cheap regression signal.

# 04 · gate in ci

Then gate it in CI

Once runs are recorded, diff any run against a golden baseline and fail the PR when the reasoning drifts. The server-less gate CLI exits non-zero on regression, and the DProvenanceKit Regression Gate GitHub Action drops it into any workflow — see the full gate story on the main page.