Compare behavior without pretending every difference is causal.

Analysis sequence

Increase claim strength one design at a time.

1. 01
   Extract

   Map normalized events into explicit analysis signals and preserve missingness.

2. 02
   Pair

   Declare what baseline and candidate executions actually share.

3. 03
   Compare

   Find sustained changes using signal-specific tolerance and persistence.

4. 04
   Replicate

   Quantify a prespecified scalar effect across matched independent pairs.

5. 05
   Perturb

   Test a mechanism with controlled contrasts and meaningful-effect margins.

6. 06
   Correct

   Control multiplicity when many hypotheses or adaptive searches are involved.

Trace extraction

Make the analysis input explicit.

Extraction rejects mixed runs, duplicate event IDs, nonnumeric values, missing payload paths, and ambiguous within-step multiplicity. Empty steps remain visible so missing evidence cannot become a zero.

from iso_obs.trace import EventSignalSpec, extract_trace_steps
from iso_obs_schemas import EventType

extraction = extract_trace_steps(
    events,
    signals=[
        EventSignalSpec(
            signal="gripper_force_n",
            event_type=EventType.METRIC_RECORDED,
            value_path="value",
            where={"name": "gripper_force_n"},
        )
    ],
)

Behavioral divergence

Separate difference from meaningful divergence.

Pairing quality is part of the result. Exact replay, seed matching, configuration matching, and unpaired data do not support the same interpretation.

from iso_obs.divergence import PairingAssessment, SignalSpec, compare_traces

pairing = PairingAssessment.assess(
    same_scenario_version=True,
    same_environment_version=True,
    same_seed=True,
    same_perturbation_realization=True,
)

report = compare_traces(
    baseline_steps,
    candidate_steps,
    signals=[
        SignalSpec(
            path="gripper_force_n",
            category="control",
            absolute_tolerance=0.5,
            relative_tolerance=0.05,
            persistence_steps=3,
        )
    ],
    pairing=pairing,
)

Paired effects

Quantify a repeated difference and its uncertainty.

The paired report includes mean and median difference, a bootstrap confidence interval, and a paired sign-flip randomization p-value. Weak pairing is rejected rather than silently treated as independent evidence.

from iso_obs.replication import PairedObservation, analyze_paired_effect

report = analyze_paired_effect([
    PairedObservation(
        pair_id=f"seed-{seed}",
        baseline=baseline_results[seed],
        candidate=candidate_results[seed],
        pairing_quality=pairing.quality,
    )
    for seed in matched_seeds
])

Controlled perturbations

Ask a causal question only after controlling the contrast.

A perturbation plan declares the interventions, outcome, direction, and material-change margin before observations are assessed. The resulting campaign is governed by its weakest contrast.

from iso_obs.causal_perturbations import assess_causal_perturbations

report = assess_causal_perturbations(
    perturbation_plan,
    paired_perturbation_observations,
)

Relevant API

Classes and capabilities in this workflow.