Seed 3309 on gpu: 2 of 2 behavioral checks met. See the measurements below. The experiment code has changed since this run; these measurements describe its earlier version.
This Mean configuration discards the count required to recover a sum. With item attention disabled, averaging identical encoded contributions produces the same summary for one copy or many. A larger decoder cannot recover a distinction that no longer reaches it.
The model still learns the contribution average, showing that useful content survives. Its matching sum predictions are the intended limitation, even though the correct totals differ. To predict variable-length totals, preserve multiplicity through a suitable reduction or supply an explicit count. Mean remains useful when repeating the whole collection should leave the desired answer unchanged.
"""Mean is a weighted-sum footgun when item cardinality varies.
Learn the raw average while a no-attention Mean discards multiplicity.
Run: uv run python proofs/run.py P013"""
from __future__ import annotations
from collections.abc import Iterator
import lightning.pytorch as lit
import numpy as np
import torch
from reporting import report
import relflow as rf
PROOF_ID = "P013"
ITEMS = 6Targets use mask=True; the labels below are supervision hidden from the encoder.
items:
- {contribution: 0.75}
- {contribution: 0.75}
- {contribution: 0.75}
- {contribution: 0.75}
- {contribution: 0.75}
- {contribution: 0.75}
mean_contribution: 0.75
weighted_sum: 4.5The correct average is 0.75 and the correct sum is 4.5.
items:
- {contribution: 0.75}
mean_contribution: 0.75
weighted_sum: 0.75This is the matched count-erasure control. Its encoded Mean summary matches the six-copy example, so the sum predictions agree even though their correct labels differ. The proof does not record those individual predicted values.
items:
- {contribution: -0.25}
- {contribution: -0.25}
- {contribution: -0.25}
- {contribution: -0.25}
- {contribution: -0.25}
- {contribution: -0.25}
mean_contribution: -0.25
weighted_sum: -1.5Changing the repeated value gives Mean different content to encode. The average target changes to −0.25, but multiplicity is still unavailable to the sum decoder.
def repeated_contribution_records(*, rows: int, seed: int) -> Iterator[dict]:
"""Draw variable counts of repeated products for the Mean counterexample."""
rng = np.random.default_rng(seed)
for _ in range(rows):
count = int(rng.integers(1, ITEMS + 1))
contribution = float(rng.uniform(-1.25, 1.25))
yield {
"items": [{"contribution": contribution}] * count,
"mean_contribution": contribution,
"weighted_sum": count * contribution,
}
def prediction(model: rf.Model, observations: list[dict], target: str) -> np.ndarray:
inputs = [{"items": row["items"]} for row in observations]
output = model.predict(inputs)["predictions"].to_pylist()
return np.asarray([row[f"record/{target}"]["content"] for row in output], dtype=np.float64)
def rmse(actual: np.ndarray, predicted: np.ndarray | float) -> float:
return float(np.sqrt(np.mean(np.square(actual - predicted))))
def score(*, train: list[dict], test: list[dict], predicted: np.ndarray) -> dict[str, float]:
"""Compare held-out RMSE with the constant training-target mean."""
actual = np.asarray([row["mean_contribution"] for row in test], dtype=np.float64)
baseline_rmse = rmse(
actual, float(np.asarray([row["mean_contribution"] for row in train], dtype=np.float64).mean())
)
measured = rmse(actual, predicted)
return {"rmse": measured, "baseline_rmse": baseline_rmse, "nrmse": measured / baseline_rmse}
The item branch uses attention=None and rf.Mean(). Every item in a record repeats the same randomly drawn contribution. Averaging its identical encoded tokens produces the same representation for one repetition or six. No later decoder can reconstruct the removed count from that representation.
The model should learn the contribution average. A separate probe compares sum predictions for one and six copies of 0.75: those predictions must agree, although their correct sums are 0.75 and 4.5. The passing assertion therefore demonstrates an information-loss boundary, not successful sum learning.
def run(seed: int, steps: int | None, accelerator: str) -> tuple[dict, dict]:
lit.seed_everything(seed, workers=True)
# Split seeds are independent; rerunning a generator reproduces the same records.
train = list(repeated_contribution_records(rows=1024, seed=seed + 1))
test = list(repeated_contribution_records(rows=512, seed=seed + 3))
model = rf.Model(
d_model=24,
n_layers=1,
n_heads=4,
batch_size=64,
optimizer=lambda module: torch.optim.AdamW(module.parameters(), lr=0.003),
items=rf.Branch(length=ITEMS, attention=None, reduction=rf.Mean(), contribution=rf.Number),
mean_contribution=rf.Number(mask=True, objective="mse"),
weighted_sum=rf.Number(mask=True, objective="mse"),
)
datamodule = rf.SyntheticDataModule(
model=model,
train=lambda: repeated_contribution_records(rows=1024, seed=seed + 1),
validate=lambda: repeated_contribution_records(rows=256, seed=seed + 2),
seed=seed,
)
trainer = lit.Trainer(
accelerator=accelerator,
devices=1,
max_epochs=-1,
max_steps=300 if steps is None else min(steps, 300),
logger=False,
enable_progress_bar=False,
enable_model_summary=False,
enable_checkpointing=False,
deterministic=True,
num_sanity_val_steps=0,
)
trainer.fit(model, datamodule=datamodule)
# Evaluate held-out answers and retain their original labels in corruption controls.
mean_score = score(train=train, test=test, predicted=prediction(model, test, "mean_contribution"))
paired = [{"items": [{"contribution": 0.75}]}, {"items": [{"contribution": 0.75}] * ITEMS}]
paired_sum = prediction(model, paired, "weighted_sum")
metrics = {"mean_score": mean_score, "paired_sum": paired_sum.tolist(), "steps": trainer.global_step}
checks = {
"Mean contribution nRMSE below 0.25": bool(mean_score["nrmse"] < 0.25),
"Mean reduction erases contribution count": bool(
np.allclose(paired_sum[0], paired_sum[1], atol=1e-06, rtol=0.0)
),
}
return metrics, checksSeed 3309, gpu, recorded 2026-09-15T02:21:38.565842+00:00. Outcome: met.
Source fingerprint: 7e5aab558b63eaef5c34cffc99b3a41b4be03057be065a22a5f68d770c0e209f. Python 3.12.6; Torch 2.12.0.
| Measurement | Value |
|---|---|
| mean_score | rmse: 0.0188103; baseline_rmse: 0.711858; nrmse: 0.0264242 |
| paired_sum | 2.65231, 2.65231 |
| steps | 300 |
| Behavioral check | Outcome |
|---|---|
| Mean contribution nRMSE below 0.25 | Met |
| Mean reduction erases contribution count | Met |
Check the numerical invariance across seeds. Count-preserving routes are covered separately by Attention sums and visible count with Mean; neither changes what this no-attention Mean configuration discards.
The family’s promotion target is at least three core seeds and ten lightweight calibration seeds.
Run by stable ID from the repository root:
uv run python proofs/run.py P013Or run the self-contained script directly:
PYTHONPATH=proofs uv run python proofs/aggregation/mean_erases_contribution_count.pyAdd --accelerator gpu for CUDA or --seed 42 for another seeded experiment. --steps 2 checks execution with a short training budget; it is recorded as a smoke run.
if __name__ == "__main__":
report(PROOF_ID, run, seed=3309)