Masking And Reconstruction

A mask selects node occurrences, hides their inputs, and optionally makes them reconstruction targets. The same mechanism supports supervised labels, self-supervised learning, input dropout, and ablation.

For a complete workflow that changes these objectives between training stages, see Pretraining And Fine-Tuning.

Choose The Intended Effect

Goal	Field configuration
Ordinary input	`mask=False`
Train-only learned-mask dropout	`mask=0.15`
Train-only structural dropout	`mask=rf.Mask(rate=0.15, skip=True)`
Reconstruct sampled values	`mask=rf.Mask(rate=0.15, reconstruct=True)`
Supervised target, never embedded	`mask=True`
Remove an input in every stage without a target	`mask=rf.Mask(skip=True, dropout=False)`

mask=True is shorthand for rf.Mask(skip=True, dropout=False, reconstruct=True). It differs from mask=1.0, which hides every input using learned-mask dropout during training and creates no reconstruction objective.

Without skip, a selected value remains a present coordinate represented by a learned mask. With skip=True, its input content is not tensorized or embedded and its coordinate is absent from encoder attention. Reconstruction targets still retain their pristine values for the loss.

A mask selects whole values: one Number, Text value, or Set, for example. It does not select individual tokens inside Text or members inside Set.

Configure A Policy

Option	Meaning
`query`	Boolean selector in the input; omitted means every retained owner occurrence is eligible.
`rate`	Fraction of eligible occurrences to sample; omitted means select all eligible occurrences.
`skip`	Omit selected input representations instead of using a learned mask.
`reconstruct`	Add an objective for selected observed values, or request predictions.
`dropout`	Apply only during training; defaults to `not reconstruct`.

dropout=True and reconstruct=True cannot be combined. Rates must be between zero and one. A zero rate removes the policy; a rate of one normalizes to no rate, making selection deterministic.

The normalized policy determines when selection runs:

Policy	Train	Validate/test	Predict
`dropout=True`	Active	Off	Off
`dropout=False`, no rate	Active	Active	Active
`dropout=False`, fractional rate	Sampled	Sampled with epoch fixed at zero	Off

Validation and test use different random streams. Sampling depends on seed, stage, owner address, epoch, and record layout; do not assume that changing batching preserves exactly the same selections.

Select Within Nested Data

A selector is an ordinary Boolean value. It can arrive in the input or be created by a preprocessor. This is one record:

merchant: grocery
transactions:
  - amount: 42.75
    days_ago: 3
    routing:
      hide_amount: true
  - amount: 18.50
    days_ago: 1
    routing:
      hide_amount: false

The leaf’s mask query starts from its containing transaction:

import relflow as rf

transactions = rf.Branch(
    length=64,
    amount=rf.Number(
        mask=rf.Mask(query="routing.hide_amount", reconstruct=True),
    ),
    days_ago=rf.Number,
)

Selected amounts are hidden and reconstructed. routing need not be a model field. The selector must provide exactly one non-null Boolean per retained owner record; integers, lists, missing paths, and null selectors are rejected. A node’s value query selects its source value; Mask.query independently selects which owner occurrences receive the policy.

Place a policy on the branch when all descendants should share a decision:

transactions = rf.Branch(
    length=64,
    mask=rf.Mask(rate=0.10, skip=True),
    amount=rf.Number,
    days_ago=rf.Number,
)

Each selected transaction hides both fields and any nested descendants. A root mask similarly broadcasts through an observation. Reconstructing branch or root policies create objectives on active descendant leaves, whose extensions must support reconstruction.

Derive A Selector

For a root-level amount field, a processor can derive a selector from an application verification flag. Keep this Boolean derivation outside the model:

import polars as pl

@rf.preprocess
def select_unverified(frame: pl.DataFrame) -> pl.DataFrame:
    return frame.with_columns(
        pl.col("amount_verified").fill_null(False).not_().alias("predict_amount")
    )

amount = rf.Number(
    mask=rf.Mask(query="predict_amount", reconstruct=True),
)

Include amount in the model and pass preprocessor=select_unverified to its data module. For direct prediction pass preprocess=select_unverified to model.predict(...). The source retains the amount field and verification flag; the processor adds a non-null Boolean selector before mask resolution.

Combine Policies

Pass a tuple or list of rf.Mask objects to combine selections:

amount = rf.Number(
    mask=(
        rf.Mask(rate=0.10),
        rf.Mask(query="routing.hide_amount", skip=True, reconstruct=True),
    ),
)

Ancestor and local selections accumulate. A skip wins over a learned mask; reconstruction selections accumulate independently. A child policy cannot restore an input hidden by an ancestor. Duplicate policies are normalized away.

Targets During Training And Prediction

Fitting needs observed target values. Explicit nulls can contribute value-state objectives for nullable fields; structurally absent positions do not invent labels. A fractional reconstruction policy trains its decoder but does not request ordinary prediction output. To request deterministic reconstruction later, update the policy.

A directly bound, unconditional target such as label=rf.Boolean(mask=True) may be omitted from prediction input. For a repeated target, its containing branch records must remain to establish output positions. Targets using an explicit value query or conditional reconstruction selector still need their source structure and selector to be bindable.

Conditional predictions retain the schema’s fixed shape. Use each coordinate’s inferred flag to identify requested outputs. See Prediction Output for the Arrow contract.