Learning Modes And Embeddings

relflow learns by reconstructing selected values. Supervised learning hides an answer on every observation; self-supervised learning hides some values from otherwise unlabeled records. embed=True separately requests a representation for downstream use. It does not create a learning objective.

Pretraining And Fine-Tuning shows how to transfer learned context from reconstruction to a labeled task.

Learn Without A Label

This schema reconstructs sampled purchase amounts and product identities. The application supplies its own customer observations and training splits.

import relflow as rf

model = rf.Model(
    name="customer",
    d_model=128,
    n_layers=3,
    n_heads=8,
    embed=True,
    region=rf.Category(size=64),
    purchases=rf.Branch(
        length=64,
        embed=True,
        days_ago=rf.Number,
        amount=rf.Number(mask=rf.Mask(rate=0.15, reconstruct=True)),
        items=rf.Branch(
            length=24,
            product=rf.Category(
                size=8192,
                mask=rf.Mask(rate=0.10, reconstruct=True),
            ),
            quantity=rf.Number,
        ),
    ),
)

Training requires an optimizer and at least one reconstruction objective; see Training. A model with only embed=True has no objective to learn from. To combine supervised and self-supervised learning, add an always-hidden target alongside sampled reconstruction fields. Each leaf’s weight scales its loss contribution.

Choose Where To Export

embed=True location	Exported representation
Root	The root’s reduced context.
Branch	The branch’s reduced context.
Leaf	The decoder’s pooled contextual representation before its datatype output head.

A leaf embedding is not its raw input embedding. It can include ancestor context and aligned sibling conditioning. A branch embedding summarizes the context controlled by that branch’s reduction.

With default one-output reductions, the example exports one customer vector and one purchase-history vector per customer. It does not export one vector per purchase. If items also had embed=True, it would export one item-list summary per purchase because the outer purchase coordinates remain.

rf.Attention(n_outputs=...) retains multiple reduction outputs. reduction=None retains routed child slots, whose width depends on branch length and child outputs. Choose reduction for modeling needs; it also changes embedding output shape. See Branch.

Read The Output

After training, observations is an application-supplied Arrow table:

import pyarrow.compute as pc

predictions = model.predict(observations)["predictions"]
customer_embedding = pc.struct_field(
    pc.struct_field(predictions, "customer"),
    "embedding",
)
purchase_history_embedding = pc.struct_field(
    pc.struct_field(predictions, "customer/purchases"),
    "embedding",
)

Exported vectors are float32, L2-normalized along their d_model axis; zero vectors remain zero. Repeated outer coordinates and multiple reduction outputs appear as fixed-size Arrow list axes. Sampled reconstruction fields do not produce ordinary decoded prediction output unless a deterministic reconstruction policy also applies.

Evaluate an embedding against its intended downstream task. Root context, branch context, and leaf context answer different questions and need not be interchangeable. Preserve the schema and trained state with the model artifact.