Theoretical Background¶

SeqTrees is based on sequential synthesis of tabular data. Instead of learning one model for the whole table at once, it decomposes the joint distribution into an ordered sequence of conditional distributions.

For variables X1, X2, ..., Xd, the model uses the chain rule:

P(X1, X2, ..., Xd)
=
P(X1) P(X2 | X1) P(X3 | X1, X2) ... P(Xd | X1, ..., Xd-1)

In SeqTrees, the first variable is sampled from an empirical marginal distribution. Every later variable is generated by routing the already generated partial row through a conditional tree and sampling from the reached leaf.

A Small Example¶

Suppose the preprocessed table has four variables:

age, sex_code, income, risk_code

where:

• age and income are continuous floats;
• sex_code and risk_code are discrete integer category codes.

If we choose this variable order:

variable_order = ["age", "sex_code", "income", "risk_code"]

then SeqTrees models the table as:

P(age, sex_code, income, risk_code)
=
P(age)
P(sex_code | age)
P(income | age, sex_code)
P(risk_code | age, sex_code, income)

The corresponding SeqTrees code is:

from seqtrees import SequentialTreeSynthesizer

model = SequentialTreeSynthesizer(
    variable_order=["age", "sex_code", "income", "risk_code"],
    continuous_columns=["age", "income"],
    discrete_columns=["sex_code", "risk_code"],
    min_samples_leaf=5,
    random_state=42,
)

model.fit(data)
synthetic = model.sample(1000)

What Happens During Fitting¶

Using the order above, fitting proceeds as follows:

1. Learn empirical distribution for age.
2. Fit a conditional tree for sex_code using age as predictor.
3. Fit a conditional tree for income using age and sex_code as predictors.
4. Fit a conditional tree for risk_code using age, sex_code, and income as predictors.

Each leaf stores the observed target values that reached that leaf during training. For discrete variables, synthetic values are sampled from those observed integer codes. For continuous variables, SeqTrees can either sample observed values or interpolate within a leaf, depending on continuous_strategy.

What Happens During Sampling¶

Sampling mirrors fitting:

1. Sample age from P(age).
2. Use sampled age to route through the sex_code tree.
3. Sample sex_code from the reached leaf.
4. Use sampled age and sex_code to route through the income tree.
5. Sample income from the reached leaf.
6. Use sampled age, sex_code, and income to route through the risk_code tree.
7. Sample risk_code from the reached leaf.

At the end, SeqTrees has generated one complete synthetic row.

Choosing Variable Order¶

The variable order matters because each later variable is conditioned on all earlier variables, a known issue in sequential-tree synthesis ¹. SeqTrees supports three options.

Use Input Column Order¶

If variable_order is not supplied and optimize_order=False, SeqTrees uses the input table's column order.

model = SequentialTreeSynthesizer(
    continuous_columns=["age", "income"],
    discrete_columns=["sex_code", "risk_code"],
)

Specify The Order Manually¶

Use variable_order when domain knowledge suggests a natural order.

model = SequentialTreeSynthesizer(
    variable_order=["age", "sex_code", "income", "risk_code"],
    continuous_columns=["age", "income"],
    discrete_columns=["sex_code", "risk_code"],
)

This order says:

age first
sex_code depends on age
income depends on age and sex_code
risk_code depends on age, sex_code, and income

Optimize The Order¶

Use optimize_order=True when you want SeqTrees to choose a greedy order from the data.

model = SequentialTreeSynthesizer(
    optimize_order=True,
    continuous_columns=["age", "income"],
    discrete_columns=["sex_code", "risk_code"],
)

The optimizer starts with a low-entropy variable and repeatedly chooses the next variable with the lowest tree-estimated conditional entropy given the variables already selected. This combines entropy-based tree modeling ² with a greedy information-theoretic forward-selection principle ³.

Let the table contain variables

and let \(S_t\) be the set of variables already selected after \(t\) steps. SeqTrees first chooses the variable with the smallest empirical entropy:

where the empirical entropy is

Here, \(\mathcal{V}_j\) is the set of observed values of \(X_j\), and \(\widehat{p}_j(x)\) is the empirical frequency of value \(x\) in the training data.

After the first variable, SeqTrees uses a greedy forward selection rule. For each remaining candidate variable \(X_c \notin S_t\), it fits a shallow conditional tree

using the variables in \(S_t\) as predictors and \(X_c\) as the target. The candidate is scored by the average entropy of the leaf distribution reached by each training row:

where \(s_i\) is row \(i\) restricted to the already selected variables \(S_t\), and \(L_T(s_i)\) is the leaf reached by that row in the fitted candidate tree. Equivalently, if the tree leaves are \(\mathcal{L}(T_c)\), this is the leaf-size-weighted entropy:

SeqTrees then appends the candidate with the smallest score:

This is called greedy because the choice at step \(t+1\) only minimizes the current conditional-entropy score. It does not search over all \(d!\) possible variable orders. In the implementation, these scoring trees are capped at depth \(\min(\texttt{max\_depth}, 3)\), so the heuristic remains cheaper than fitting full trees for every possible ordering.

After fitting, inspect the order:

model.fit(data)
print(model.get_variable_order())

Continuous And Discrete Variables¶

For pandas DataFrame input, SeqTrees uses ifcfill to define variable meaning before the tree model is fitted. Original categorical columns are label-encoded and restored after sampling; original integer columns are treated as numeric integer variables. List-based input must already follow these conventions:

• continuous variables must be floats;
• discrete variables must be integer category codes;
• binary variables are discrete integer category codes;
• null values are not accepted;
• raw categorical strings are accepted only through pandas DataFrame input;
• one-hot encoded inputs are not the intended workflow.

For example:

model = SequentialTreeSynthesizer(
    continuous_columns=["age", "bmi"],
    discrete_columns=["sex_code", "income_bin", "risk_code"],
    continuous_strategy="interpolate",
)

With continuous_strategy="empirical", numeric variables are sampled from observed training values in the reached leaf. With continuous_strategy="interpolate", SeqTrees may generate interpolated values inside the reached leaf for columns declared continuous and for integer columns detected from DataFrame input.