Cluster Commitment with Stable Labels

Repeated identities and a reconstruction objective can move adaptive cluster commitment toward the generating group count.

Repeated merchant identities each have a stable label drawn from five groups. The model predicts that label while also reconstructing masked identity representations. The experiment checks how many clusters become committed and how broadly they are used.

Seed 0 on gpu: 4 of 4 behavioral checks met. See the measurements below.

Insights

The right number of clusters does not mean the right identities were grouped together. Repeated identities provide consistent label evidence, and the Cluster reconstruction loss updates usage and commitment.

The recorded run ends near five groups. However, committed count is derived by rounding and clamping usage perplexity: these are coupled diagnostics, not independent confirmations of recovery. Incorrect groups can satisfy both gates.

Training and validation reuse observations, with no held-out skill or assignment-agreement requirement. Treat this as evidence about mechanism activation. Demonstrating useful clustering still requires independent predictions, partition recovery, and controls without genuine generating groups.

Setup

"""P020: track adaptive clusters across five repeated-identity labels.

Training and validation deliberately reuse observations. Count and usage
perplexity diagnose the mechanism; they do not prove partition recovery or
held-out prediction quality.
"""

from collections.abc import Iterator
from functools import partial

import lightning.pytorch as lit
import torch
from reporting import report

import relflow as rf
from relflow.tensorfields.extensions.cluster import Embedder

PROOF_ID = "P020"

Examples

Each generating identity appears 20 times with a stable hidden label. The Cluster field is an input with a reconstructing mask; its displayed identity is not always visible during training.

One identity in the first group

merchant_id: c0-id3
label: L0

Repeated observations give this identity consistent evidence for label L0.

A peer identity in the same group

merchant_id: c0-id9
label: L0

The identifier differs, but the behavior matches. These two identities belong together in the generator’s partition.

A different group

merchant_id: c4-id8
label: L4

This identity belongs to a different generating group. The five groups have ten identities each, for 1,000 total observations. Their label relationship is ground truth; the present proof checks cluster count and usage, not whether these particular identities received the correct learned assignments.

Synthetic data and controls

def records(seed: int) -> Iterator[dict]:
    """Repeat each of 50 identities twenty times with one stable regime label."""
    rows = [
        {"merchant_id": f"c{cluster}-id{identity}", "label": f"L{cluster}"}
        for cluster in range(5)
        for identity in range(10)
        for _ in range(20)
    ]
    order = torch.randperm(len(rows), generator=torch.Generator().manual_seed(seed)).tolist()
    for index in order:
        yield rows[index]


class Trajectory(lit.Callback):
    """Inspect Cluster internals; these diagnostics do not establish partition accuracy."""

    def __init__(self, address: rf.Address) -> None:
        self.address = address
        self.rows: list[dict] = []

    def on_train_epoch_end(self, trainer: lit.Trainer, pl_module: lit.LightningModule) -> None:
        embedder = pl_module.nodes[self.address].embedder
        if not isinstance(embedder, Embedder):
            raise TypeError(f"{self.address} requires a Cluster embedder, got {type(embedder).__name__}")
        usage = embedder.usage_ema.detach()
        probabilities = usage / usage.sum().clamp_min(1e-12)
        bounded = probabilities.clamp_min(1e-12)
        entropy = -(bounded * bounded.log()).sum()
        self.rows.append(
            {
                "epoch": trainer.current_epoch,
                "n_committed": int(embedder.committed.sum().item()),
                "perplexity": float(torch.exp(entropy).item()),
                "adherence": float(embedder.adherence_ema.item()),
            }
        )

Model tree

Figure 1: Merchant ID uses a 50% training mask with reconstruction enabled. Its label is always hidden from model inputs.

How it works

The merchant field uses rf.Cluster(mask=rf.Mask(rate=0.5, reconstruct=True), ...). Its reconstruction loss updates adaptive usage and commitment state. Repetition lets evidence accumulate for an identity. label=rf.Category(mask=True, ...) supplies the supervised task without exposing its answer as an input.

Assignments belong to stable identities. Row-varying sibling context does not directly condition the Cluster reconstruction query. The plain-input control shows what happens when the Cluster reconstruction objective is absent.

The model trains for 30 deterministic epochs. A callback inspects internal Cluster state at each epoch; the gates examine its final five entries.

Training and evaluation

def run(seed: int, steps: int | None, accelerator: str) -> tuple[dict, dict]:
    lit.seed_everything(seed, workers=True)
    model = rf.Model(
        name="event",
        d_model=32,
        n_layers=1,
        n_heads=4,
        batch_size=64,
        merchant_id=rf.Cluster(capacity=64, n_clusters=(3, 15), mask=rf.Mask(rate=0.5, reconstruct=True)),
        label=rf.Category(mask=True, size=5, p_unavailable=0.0),
    )
    model.optimizer = lambda module: torch.optim.AdamW(module.parameters(), lr=3e-3)
    source = partial(records, seed=seed)
    data = rf.SyntheticDataModule(model=model, train=source, validate=source, seed=seed)
    trajectory = Trajectory(rf.Address("event", "merchant_id"))
    trainer = lit.Trainer(
        accelerator=accelerator,
        max_epochs=30,
        max_steps=steps if steps is not None else -1,
        callbacks=[trajectory],
        logger=False,
        enable_progress_bar=False,
        enable_model_summary=False,
        enable_checkpointing=False,
        deterministic=True,
    )
    trainer.fit(model=model, datamodule=data)

    tail = trajectory.rows[-5:]
    committed = [row["n_committed"] for row in tail]
    perplexity = [row["perplexity"] for row in tail]
    return {
        "trajectory": trajectory.rows,
        "final_committed": committed,
        "final_perplexity": perplexity,
    }, {
        "Final committed counts remain between 4 and 7": all(4 <= value <= 7 for value in committed),
        "Final perplexity remains within 1.5 of five": all(abs(value - 5) <= 1.5 for value in perplexity),
        "Terminal commitment is above the lower bound": committed[-1] != 3,
        "Terminal commitment is below the upper bound": committed[-1] != 15,
    }

Evidence

Latest full run

Seed 0, gpu, recorded 2026-09-15T02:24:00.576894+00:00. Outcome: met.

Source fingerprint: ad145610d04bc8af8d40fb104042fd9aa3abb73ef86d4fade1cf1001bc933df5. Python 3.12.6; Torch 2.12.0.

Measurement	Value
trajectory	30 values; final 8: epoch: 22; n_committed: 5; perplexity: 5.16149; adherence: 0.00289054, epoch: 23; n_committed: 5; perplexity: 5.14678; adherence: 0.00270216, epoch: 24; n_committed: 5; perplexity: 5.13371; adherence: 0.00252678, epoch: 25; n_committed: 5; perplexity: 5.12076; adherence: 0.00236039, epoch: 26; n_committed: 5; perplexity: 5.11023; adherence: 0.00220868, epoch: 27; n_committed: 5; perplexity: 5.10123; adherence: 0.00206638, epoch: 28; n_committed: 5; perplexity: 5.09319; adherence: 0.0019342, epoch: 29; n_committed: 5; perplexity: 5.08391; adherence: 0.00181166
final_committed	5, 5, 5, 5, 5
final_perplexity	5.12076, 5.11023, 5.10123, 5.09319, 5.08391

Behavioral checks

Behavioral check	Outcome
Final committed counts remain between 4 and 7	Met
Final perplexity remains within 1.5 of five	Met
Terminal commitment is above the lower bound	Met
Terminal commitment is below the upper bound	Met

Recorded results.

Remaining work

Use independent observations for each split, add held-out downstream skill against a marginal baseline, and require adjusted Rand index (ARI) ≥ 0.80 for partition recovery. Add unique-identity and no-regime controls. Replace internal-state inspection with a public diagnostic when available.

The family requires three passing core seeds and at least ten calibration seeds before its terminal-window thresholds can be promoted.

Reproduce

Run by stable ID from the repository root:

uv run python proofs/run.py P020

Or run the self-contained script directly:

PYTHONPATH=proofs uv run python proofs/cluster/reconstructing_category_labels.py

Add --accelerator gpu for CUDA or --seed 42 for another seeded experiment. --steps 2 checks execution with a short training budget; it is recorded as a smoke run.

Download the complete proof.

if __name__ == "__main__":
    report(PROOF_ID, run, seed=0)