Proofs

What relflow has learned in controlled experiments, and where information is lost.

Can a model find a matching identity, recover a hidden subtotal, or preserve enough information through a nested summary? These experiments isolate one relationship at a time. Controls remove the signal, change the pairing, or deliberately discard information so that a useful prediction has a clear explanation.

Each scenario opens with a short Insights summary: what the proof shows, why it matters, and where the conclusion stops. A model tree, contrasting records, and expandable code sections explain the experiment together. Each page comes from the same Python file you can download and run. The collection includes retention, adaptation, reset, deactivation, and deletion after schema mutations.

The capability experiments P071–P078 test unfamiliar instruction combinations, rules inferred from examples, frozen representation transfer, incomplete evidence, larger and two-hop relations, contextual inconsistency, numerical representation tradeoffs, and hidden-group recovery. Their controls distinguish a successful local task from broader reuse of a learned relationship.

Read The Status

Statuses summarize recorded evidence in proofs/results.yaml. Each experiment has a stable ID, such as P014, independent of its title or script name. Some experiments are designed to confirm a limitation, such as losing count when averaging identical values. Their expected outcome is useful evidence about model design. Each page provides commands to reproduce the experiment. Acceptance thresholds are labeled as gates. Each page shows the latest full run’s measurements, with behavioral checks collapsed by default. Expand them to inspect the individual criteria. Short smoke runs never replace full experiment results.

Status What the evidence establishes
Passing The reported experiment clears its gates, usually for one deterministic seed. The result is provisional: broader stability or generalization remains to be established.
Limited The experiment successfully reproduces an expected loss of information or unsuccessful modeling route. Its checks are met when that limitation appears; this is not a failed experiment.
Partial Some behavior is supported, while a broader claim remains untested or inconsistent across recorded conditions and seeds.
Failing The latest full run completed but did not meet every stated behavioral criterion.
Error The latest full run could not complete. Its error is recorded separately from model quality.
Missing No full experiment run has been recorded.

These are empirical checks of learned behavior. Training speed and memory use require separate measurements. Learned arithmetic is approximate; use exact preprocessing for fixed business rules that must be exact.

Explore The Proofs

Sort by ID, Proof, Family, or Status, using the column headings or the order menu. Filter by an ID, task, family, or status, then select any part of a proof’s row to explore its insights, examples, and evidence.

ID Proof Family Status
P001 Sums within the trained length range Cardinality generalization Passing
P002 Sums beyond trained collection lengths Cardinality generalization Passing
P003 A mean that survives bag duplication Cardinality generalization Passing
P004 Restoring sums with a visible count Cardinality generalization Passing
P005 Extending a visible-count sum Cardinality generalization Passing
P006 Covariance from aligned pairs Distribution statistics Passing
P007 Variance from raw values Distribution statistics Passing
P008 Averaging supplied centered products Distribution statistics Passing
P009 Averaging supplied squared deviations Distribution statistics Passing
P010 Weighted sums for a requested group Grouped weighted aggregation Passing
P011 Selecting and summing supplied contributions Grouped weighted aggregation Passing
P012 Requesting a value by rank Order statistics Passing
P013 When Mean loses contribution count Weighted aggregation Limited
P014 Weighted sums from raw pairs Weighted aggregation Passing
P015 Summing supplied contributions Weighted aggregation Passing
P016 Weighted means across varying lengths Weighted aggregation Passing
P017 Signal and Noise Calibration Passing
P018 Cluster Commitment with Hidden Regimes Clustering Partial
P019 A Plain Cluster Input Stays Dormant Clustering Limited
P020 Cluster Commitment with Stable Labels Clustering Partial
P021 Equality for Unseen Identities Identity Passing
P022 Retrieve Through Learned Summaries Argmax retrieval Passing
P023 Retrieve the Winning Payload Argmax retrieval Passing
P024 Copying by Position Associative recall Passing
P025 Recall by an Unseen Key Associative recall Passing
P026 Average the Requested Group Category-conditioned reduction Passing
P027 Choose a Group and an Operation Category-conditioned reduction Passing
P028 The Collection Overlap Boundary Collection overlap Limited
P029 Compare Two Unseen Identities Collection overlap Passing
P030 Keep Each Subtotal with Its Item Item alignment Passing
P031 Choose a Numeric Operation Operation-conditioned reduction Passing
P032 Compare Each Item with Its Group Peer-relative inference Passing
P033 Share One Summary Across Grouped Targets Peer-relative inference Passing
P034 Subtract a Supplied Peer Mean Peer-relative inference Passing
P035 Compare Each Item with Its Peers Peer-relative inference Passing
P036 Transfer Values by Category Identity Sibling entity transfer Passing
P037 Transfer Values by an Unseen Identity Sibling entity transfer Passing
P038 Null Is Not Zero Value state Passing
P039 Learning a Fixed-Length Sum Hierarchical statistics Passing
P040 Preserving Counts Across a Hierarchy Hierarchical statistics Passing
P041 An Average Does Not Preserve Count Hierarchical statistics Limited
P042 Combining Weekday and Hour Calendar reasoning Passing
P043 Resolving the Leap-Day Boundary Calendar reasoning Limited
P044 Learning Month from Annual Phase Calendar reasoning Passing
P045 Day-of-Year Cannot Identify Weekday Calendar reasoning Limited
P046 Learned Predictions Survive Neutral Edits Mutation retention Passing
P047 Learn an Added Target Without Losing Existing Tasks Mutation adaptation Failing
P048 Reset One Learned Target and Teach It Again Mutation reset Passing
P049 Deactivate a Learned Input and Restore Its Contribution Mutation ablation Passing
P050 Delete a Learned Input and Adapt to the Remaining Information Mutation ablation Passing
P051 Learning from sampled reconstruction masks Dynamic masking Passing
P052 Reconstructing selected nested values Dynamic masking Passing
P053 Recovering from dynamically hidden branches Dynamic masking Passing
P054 Prefix-only prediction and sequence rollout Dynamic masking Passing
P055 Unknown categories are not nulls Vocabulary and OOV Limited
P056 High accuracy can hide unknown target labels Vocabulary and OOV Limited
P057 Vocabulary growth preserves labels across admission orders Vocabulary and OOV Limited
P058 Unknown Set identities and an untrained fallback Vocabulary and OOV Limited
P059 Admitting a label is not learning to predict it Vocabulary and OOV Passing
P060 Categorical Prior Probabilities Calibration Passing
P061 Cold-Start Confidence Vocabulary Limited
P062 Pretraining Coverage and Reconstruction Vocabulary Limited
P063 Set probabilities and partial-OOV supervision Vocabulary Passing
P064 Learning an unavailable Set fallback Vocabulary Passing
P065 Cluster Pretraining Coverage Vocabulary Failing
P066 Category admission during staged pretraining Vocabulary Failing
P067 Regression Through Percentiles Quantile Passing
P068 Quantile Predictions Saturate at Learned Support Quantile Limited
P069 Learning a closed set of classes Enum Passing
P070 Declaring a class does not teach its meaning Enum Passing
P071 Familiar Instructions in Unseen Combinations Compositional generalization Failing
P072 Infer an Affine Rule From Context Examples Learning from context Partial
P073 Transferring reconstruction embeddings to a small-label task Representation Partial
P074 Combining noisy and incomplete evidence Representation Failing
P075 Identity Retrieval with Distractors and Two Hops Sibling entity transfer Failing
P076 Detecting Contextual Inconsistency Reconstruction Passing
P077 Percentiles, Magnitudes, and Noisy Targets Quantile Passing
P078 Recovering Hidden Behavioral Regimes Clustering Failing
No matching items

What Strengthens A Result

The shared protocol calls for three paired data/model seeds before promoting a core claim, plus at least ten seeds to calibrate a new threshold. Held-out splits, matched baselines, and controls must continue to support the claimed relationship. Each page identifies the additional work for that scenario.

Regression errors are generally reported as nRMSE: model RMSE divided by the RMSE of a baseline that predicts the training mean. Lower is better; 1 matches that baseline. Other baselines, such as an oracle with information removed, are identified on the relevant page. Classification pages report accuracy or ROC AUC, together with the appropriate chance or corruption control.

Run uv run python proofs/run.py --list to list proof IDs, or uv run python proofs/run.py P014 --accelerator gpu to run one on a GPU. Each script uses seeded synthetic records and records its metrics, controls, seed, environment, and source fingerprint. make proofs runs all scripts serially.

Complete results