Configuration: the flag variant a step ran under
When a team changes how an agent behaves with a feature flag (a new prompt, a wider tool set, a different model route), the first question after something shifts is which variant was live. Provantix keeps the flags a step ran under, as the source reported them, and will not compare two evaluation windows that ran under different ones.
In the product
Product capture, 15–16 November 2026. Open the image for full size. Counts and dates are examples from the captured workspace.
Configuration is read beside steps in a run when the source supplies flag information. No flag variant is visible in this captured run. Do not infer that no flag applied: the screenshot is an example of a run without that reported context.
What is recorded
If a span your agent sends carries OpenTelemetry feature flag attributes, or a feature_flag.evaluation event with them, the record keeps the flag and the variant served:
| Attribute | Used for |
|---|---|
feature_flag.key |
The flag. Required, or nothing is kept. |
feature_flag.result.variant (older exporters: feature_flag.variant) |
The variant served, such as v2 or on. |
feature_flag.provider.name (older: feature_flag.provider_name) |
Who evaluated the flag. Shown on hover. |
A flag whose only value is a plain true or false is kept as that. Any other flag value is not read, because a value can be large or sensitive; the flag is kept with no variant. At most ten flags are kept per step.
The run view shows each flag as flag=variant beside the step. It is the source's statement: Provantix does not know whether the flag changed anything, and a step with no flag shown means the source did not send one, not that no flag applied.
Evaluation results
An outside evaluator can say which configuration it ran under, by sending configuration with its results: { "flag": "new-prompt", "variant": "v2" }. Both are required if you send it. See Evaluations API.
The configuration is part of what makes two windows the same series. If the window before ran under new-prompt=v1 and the recent window under new-prompt=v2, the cohort reads not comparable and says the configuration changed, rather than reporting a fall or a rise. A result with no configuration does not match one that has it: not stated is its own value. The model comparison refuses in the same way.
What it does not do
- It does not say that a flag caused a change in quality, cost or errors. It keeps the two facts side by side.
- It does not read the flag system. It reads what your instrumentation sent. A change your control system makes, such as a rollback, can be sent as well, and Provantix then checks whether the same cohort recovered: see Recovery after a change.
- It does not yet filter Investigate by variant, or limit an expected step to one variant.
Flags are kept in the erasable part of the record, so they are not part of the sealed commitments and exports do not change.
