Cluster

Labels that share learned latent groups.

Cluster learns shared groups for labels, such as merchants. Use it when you want to test whether a learned grouping provides useful context for payment prediction, as well as keeping a persistent merchant vocabulary. Category is the simpler choice for independent label embeddings; Hash compares IDs without keeping a vocabulary.

import relflow as rf

model = rf.Model.xs(
    amount=rf.Number,
    merchant=rf.Cluster(
        bounds=64,
        mask=rf.Mask(rate=0.15, reconstruct=True),
    ),
    disputed=rf.Boolean(mask=True),
)

One labeled payment:

amount: 84.50
merchant: North Market
disputed: false

The reconstructing mask hides some merchant labels during training and asks the model to recover them. That objective trains the cluster assignments; dispute supervision alone does not. The groups need not match human categories.

Input and options

Accepts scalar Boolean, integer, floating-point, string, or binary labels, with one compatible Arrow family per field. Training grows the vocabulary; validation, test, and prediction reuse it. Unknown labels remain valued and use a shared unavailable assignment. Nulls have a separate state.

See Online vocabulary for automatic growth, admission timing with workers/DDP, supported training strategies, and changes during fine-tuning.

Option	Default	Meaning
bounds	required	Integer K fixes the cluster count; (lower, upper) enables adaptive committed clusters with 0 < lower <= upper. Stored as n_clusters.
p_unavailable	0.01	Training probability of replacing known input labels with unavailable content. Reconstruction answers remain intact.
ema_decay	0.99	Smoothing for usage and range-adherence statistics, in [0, 1].
revive_temperature	10.0	Nonnegative revival schedule control; zero disables inactive-cluster revival.

The upper bound fixes physical cluster width. Ranged bounds allow callbacks to revive inactive columns or merge similar active columns. They do not guarantee equal use or human-interpretable groups. Vocabulary growth expands label rows without changing these cluster bounds.

Learning and inspection

The training process binds batch-local label dictionaries before forward. Growth preserves existing IDs, learned assignment rows, exposure counters, and supported optimizer state, including the shared unavailable assignment.

A reconstructing mask is required to train the cluster head and adaptive selection. A plain visible Cluster can influence other targets, but creates no cluster reconstruction loss of its own. See shared leaf options.

Reconstruction combines unweighted state classification with unweighted label cross entropy through clusters, normalized over populated labels only. Unallocated label rows and the unavailable input assignment are not output candidates. The decoder uses ancestor memory but opts out of direct sibling query conditioning.

Unknown target identities contribute neither label reconstruction nor balancing updates. Usage EMA, committed-cluster selection, and adherence use known reconstruction targets only. An epoch with no known targets also skips revival and merging. Known-input augmentation does not remove that target’s supervision. The auxiliary loss.cluster still encourages balanced use of latent clusters; this is not a guarantee of calibrated label probabilities.

Metrics separate vocabulary coverage from reconstruction quality:

Metric	Meaning
targets.known, targets.unavailable	Known and unknown selected valued target counts.
coverage.content	Known / all selected valued targets.
accuracy.content, nll.content, loss.content	Known-label accuracy and unweighted cross entropy over populated labels.
accuracy.all	Correct known predictions / all selected valued targets, counting unknown answers as incorrect.

Training scores describe the latest batch; validation and test aggregate counts and sums across batches and DDP ranks. With no known targets, conditional reconstruction metrics are NaN; state supervision remains valid. coverage.content and the target counts replace the old training-only cluster.sentinel_share diagnostic. See pretraining evaluation.

For an application-supplied trained model:

labels = rf.Cluster.vocabulary(model, rf.Address("merchant"))
assignments = rf.Cluster.assignments(model, rf.Address("merchant"))
status = rf.Cluster.status(model, rf.Address("merchant"))

labels is an index-ordered tuple. Each assignment contains a cluster ID and probabilities across all configured columns. Status exposes committed column IDs and usage EMA; these inspection calls read local model state under DDP.

An idle model also supports rf.Cluster.assign(model, address, label, assignment) for persistent changes and rf.Cluster.override(model, address, assignments) as a temporary context manager. An assignment is a cluster ID or a nonnegative probability vector of upper-bound width summing to one. These imperative APIs grow per-label storage before writing when necessary, preserving earlier rows, the unavailable assignment, and cluster width. They cannot run while the model is active or vocabulary state is shared with workers. Overrides restore labels and assignments on exit; any expanded allocation remains available. Saving or rebuilding during an open override is rejected.

Prediction

cluster contains the best cluster’s integer value and probability; content contains the best populated vocabulary label and its probability. Label probabilities are normalized over populated labels, excluding unused capacity and the unavailable row. Labels use large_string; an empty vocabulary yields a null label and zero probability.

Cluster confidence and vocabulary-label confidence answer different questions. Neither estimates the probability that an input belongs to the known vocabulary. Evaluate cold-input populations separately.

See the prediction envelope for shared state probabilities, inferred, and repeated shapes.

The reconstructing Cluster proof reports internal convergence for repeated identities. Its page distinguishes that evidence from partition recovery and held-out prediction. The plain-input control shows why downstream supervision alone leaves adaptive Cluster state dormant. The regime recovery proof tests the stronger claim directly: assignment agreement with hidden generating groups and prediction quality on independent observations, against a no-regime control.