Text

Free-form strings encoded by a frozen text model.

A payment description can carry detail beyond the merchant label, such as whether the charge is a delivery order. Text encodes those free-form strings with a frozen Hugging Face model. RelFlow learns to combine their representations with the amount and other fields. It does not fine-tune that encoder or generate text.

Install the optional text dependency:

uv add "relflow[text]"

import relflow as rf

model = rf.Model.xs(
    amount=rf.Number,
    merchant=rf.Category,
    memo=rf.Text,
    disputed=rf.Boolean(mask=True),
)

One labeled payment:

amount: 84.50
merchant: North Market
memo: "North Market delivery order"
disputed: false

Input and options

The Arrow leaf must contain strings. An empty string is valued; a null has a separate state. Tokenization pads and truncates at max_length.

Option	Default	Meaning
model	"google/bert_uncased_L-2_H-128_A-2"	Hugging Face identifier or local model path.
max_length	128	Positive tokenization length limit.
tokenizer_batch_size	4096	Maximum strings materialized per tokenizer call.
encoder_batch_size	32	Maximum flattened text values per frozen-encoder call.
encoder_pooling	"cls"	"cls", "mean", or "pooler"; the last requires pooler_output.
cache	False	Reuse pooled frozen-encoder outputs and token rows: True keeps every distinct embedding; a positive integer caps retained entries.
objective	"l2"	Embedding reconstruction loss: "l1" or "l2".
jitter	rf.Jitter()	Training noise on the pooled frozen embedding before learned projection.

cls and mean require last_hidden_state. Text fields using the same model reuse its frozen encoder within the process. Right-padded values are grouped by length and cropped per encoder batch; left-padded values retain their full width. max_length still controls truncation and tensor storage.

Repeated text

Repeated descriptions can reuse frozen-encoder outputs across forward passes:

memo = rf.Text(cache=10_000)

cache=False disables this reuse. cache=True retains every distinct tokenized input encountered; a positive integer retains at most that many entries. The bounded cache evicts the least frequently reused resident entry, breaking ties by least recent use. It favors frequent reuse rather than tracking a global ranking of every description ever seen. Texts that tokenize identically, including after truncation, can share an entry.

Entries are pristine pooled embeddings stored in CPU memory, separately for each text field in each process. Encoder device and precision distinguish entries. The same setting also keeps token rows in a separate least-recently-used cache: 4,096 rows for True, or at most the configured entries for a positive integer, bounded further by a 64 MiB payload limit and invalidated when tokenizer settings change. Duplicate strings within a batch are tokenized once either way. The trainable projection and training jitter still run on every forward pass, so caching preserves their gradients and fresh noise while avoiding repeated tokenizer and frozen-encoder work.

Checkpoints retain the cache setting but omit cached embeddings and token rows; a loaded or copied model starts cold. Under DDP, each process builds its own caches. Use a finite cap when inputs contain many distinct descriptions: cache=True has no embedding limit and can keep growing in host memory.

Learning and output

A reconstructing mask trains the decoder against the pristine frozen embedding, using state loss and the configured content objective on valued targets. Jitter affects only visible continuous inputs, not strings, token IDs, or target embeddings. Both settings of jitter.normalize act at the same pooled-embedding boundary.

Text has no decoded public state/content payload. embed=True exports a contextual relflow embedding, distinct from the frozen encoder output; see shared leaf options. Use Vector when the source already supplies an embedding.

External model assets

The first use of a remote model identifier may download tokenizer and encoder assets. relflow checkpoints save the request and learned relflow weights, but do not bundle those external assets. Every loading or serving process must be able to resolve the same identifier or local path. For reproducible deployment, prepare a pinned encoder directory and ship it with the artifact.