➕ Cross Features

Cross Features in KDP

Capture powerful interactions between features to uncover hidden patterns in your data.

📋 Overview

Cross features model the interactions between input features, unlocking patterns that individual features alone might miss. They're especially powerful for capturing relationships like "product category × user location" or "day of week × hour of day" that drive important outcomes in your data.

🔗

Feature Interaction

Capture complex relationships between features

🎯

Pattern Discovery

Uncover hidden correlations in your data

⚡

Efficient Processing

Optimized for large-scale feature crosses

🎛️

Fixed Width

Each cross adds one column, whatever the cardinality

🧠 How Cross Features Work

KDP crosses two columns by hashing the pair of raw values into a fixed number of bins, and appends that bin index to the output as a single column.

🔄

Feature Combination

Merging values from different features

#️⃣

Hashing

Mapping each pair into one of nr_bins buckets

🧮

One Extra Column

The bin index, appended to the categorical block

🔍

Pattern Discovery

Finding non-linear relationships between features

📝 Basic Usage

from kdp import PreprocessingModel, FeatureType

# Define your features. Both sides of a cross must be categorical: the pair of
# raw values is hashed, so the columns have to be strings or integers.
features = {
    "product_category": FeatureType.STRING_CATEGORICAL,
    "user_country": FeatureType.STRING_CATEGORICAL,
    "age_group": FeatureType.STRING_CATEGORICAL
}

# Create a preprocessor with cross features
preprocessor = PreprocessingModel(
    path_data="customer_data.csv",
    features_specs=features,

    # Define crosses as (feature1, feature2, nr_bins)
    feature_crosses=[
        ("product_category", "user_country", 32),  # pairs hashed into 32 bins
        ("age_group", "user_country", 16)          # pairs hashed into 16 bins
    ]
)

# Each cross adds exactly one column to the output, holding the bin index of
# the (feature1, feature2) pair -- a value in [0, nr_bins).

⚙️ Key Configuration Parameters

Parameter	Description	Default	Suggested Range
`feature1`	First feature to cross. Must be declared in `features_specs` and be a string or integer column	-	Any categorical feature name
`feature2`	Second feature to cross, under the same rules	-	Any categorical feature name
`nr_bins`	Number of hash buckets the pair is mapped into. Bigger means fewer collisions between distinct pairs	-	Around the number of pairs you expect to see

🛠️ Cross Feature Types

Categorical × Categorical

The most common type, capturing relationships between discrete features:

from kdp import FeatureType, PreprocessingModel

# Creating categorical crosses
preprocessor = PreprocessingModel(
    features_specs={
        "product_category": FeatureType.STRING_CATEGORICAL,
        "user_country": FeatureType.STRING_CATEGORICAL
    },
    feature_crosses=[
        ("product_category", "user_country", 32)
    ]
)

Categorical × Bucketed Numerical

A numeric column cannot be crossed directly -- hashing needs discrete values, and a float column is refused when the preprocessor is built. Bucket it into a categorical column of your own first:

import pandas as pd
from kdp import FeatureType, PreprocessingModel

# Turn the numeric column into bands, then cross the bands
frame = pd.read_csv("products.csv")
frame["price_band"] = pd.cut(
    frame["price"],
    bins=[0, 10, 50, 200, float("inf")],
    labels=["budget", "standard", "premium", "luxury"],
).astype(str)
frame.to_csv("products_banded.csv", index=False)

preprocessor = PreprocessingModel(
    path_data="products_banded.csv",
    features_specs={
        "product_category": FeatureType.STRING_CATEGORICAL,
        "price_band": FeatureType.STRING_CATEGORICAL,
    },
    feature_crosses=[
        ("product_category", "price_band", 32)
    ]
)

Date Crosses

A DateFeature is one column that expands into cyclical encodings inside the model; there are no separate <name>_hour or <name>_day_of_week features to cross. Derive the components you want to cross as their own categorical columns:

import pandas as pd
from kdp import FeatureType, PreprocessingModel

frame = pd.read_csv("transactions.csv")
stamps = pd.to_datetime(frame["transaction_time"])
frame["transaction_day_of_week"] = stamps.dt.day_name()
frame["transaction_hour"] = stamps.dt.hour.astype(str)
frame.to_csv("transactions_parts.csv", index=False)

preprocessor = PreprocessingModel(
    path_data="transactions_parts.csv",
    features_specs={
        "transaction_time": FeatureType.DATE,
        "transaction_day_of_week": FeatureType.STRING_CATEGORICAL,
        "transaction_hour": FeatureType.STRING_CATEGORICAL,
    },
    # Cross day of week with hour of day
    feature_crosses=[
        ("transaction_day_of_week", "transaction_hour", 16)
    ]
)

Multiple Crosses

Combine multiple cross features to capture complex interactions:

from kdp import FeatureType, PreprocessingModel

# Creating multiple crosses
preprocessor = PreprocessingModel(
    features_specs={
        "product_category": FeatureType.STRING_CATEGORICAL,
        "user_country": FeatureType.STRING_CATEGORICAL,
        "device_type": FeatureType.STRING_CATEGORICAL,
        "age_group": FeatureType.STRING_CATEGORICAL
    },
    # Define multiple crosses to capture different interactions
    feature_crosses=[
        ("product_category", "user_country", 32),
        ("device_type", "user_country", 16),
        ("product_category", "age_group", 24)
    ]
)

💡 Advanced Cross Feature Techniques

🔍 Attention Over Crosses

Crossed columns join the feature set, so tabular attention weighs them alongside everything else:

# Attention runs over the whole feature set, crosses included
from kdp import PreprocessingModel, FeatureType

preprocessor = PreprocessingModel(
    path_data="data.csv",
    features_specs={
        "product_id": FeatureType.STRING_CATEGORICAL,
        "user_id": FeatureType.STRING_CATEGORICAL,
    },
    feature_crosses=[("product_id", "user_id", 32)],
    tabular_attention=True,
    tabular_attention_heads=4,
    tabular_attention_placement="all_features"
)

🧠 Three-Way Interactions

feature_crosses takes pairs. Cover a three-way interaction with its pairs:

from kdp import FeatureType, PreprocessingModel

# Each cross is a pair. For three-way interactions, cross every pair and let
# the model combine them -- a cross cannot be crossed again.
preprocessor = PreprocessingModel(
    path_data="data.csv",
    features_specs={
        "product_category": FeatureType.STRING_CATEGORICAL,
        "user_location": FeatureType.STRING_CATEGORICAL,
        "time_of_day": FeatureType.STRING_CATEGORICAL,
    },
    feature_crosses=[
        ("product_category", "user_location", 32),
        ("product_category", "time_of_day", 32),
        ("user_location", "time_of_day", 32),
    ]
)

🔧 Real-World Examples

E-commerce Recommendations

# Cross features for e-commerce recommendations
from kdp import PreprocessingModel, FeatureType
from kdp.features import CategoricalFeature, DateFeature

preprocessor = PreprocessingModel(
    path_data="ecommerce_data.csv",
    features_specs={
        # User features
        "user_segment": FeatureType.STRING_CATEGORICAL,
        "user_device": FeatureType.STRING_CATEGORICAL,

        # Product features
        "product_category": CategoricalFeature(
            name="product_category",
            feature_type=FeatureType.STRING_CATEGORICAL,
            embedding_size=32
        ),
        "product_price_range": FeatureType.STRING_CATEGORICAL,

        # Temporal features. The date column feeds the model its cyclical
        # encodings; the two categorical columns beside it are what the crosses
        # use, because a cross needs discrete values.
        "browse_time": DateFeature(
            name="browse_time"
        ),
        "browse_is_weekend": FeatureType.STRING_CATEGORICAL,
        "browse_hour": FeatureType.STRING_CATEGORICAL
    },

    # Define crosses for recommendation patterns
    feature_crosses=[
        # User segment × product category (what segments like what categories)
        ("user_segment", "product_category", 48),

        # Device × price range (mobile users prefer different price points)
        ("user_device", "product_price_range", 16),

        # Temporal × product (weekend browsing patterns)
        ("browse_is_weekend", "product_category", 32),

        # Time of day × product (morning vs evening preferences)
        ("browse_hour", "product_category", 32)
    ]
)

Fraud Detection

# Cross features for fraud detection
from kdp import PreprocessingModel, FeatureType
from kdp.features import NumericalFeature, DateFeature

preprocessor = PreprocessingModel(
    path_data="transactions.csv",
    features_specs={
        # Transaction features
        "transaction_amount": NumericalFeature(
            name="transaction_amount",
            feature_type=FeatureType.FLOAT_RESCALED,
            use_distribution_aware=True
        ),
        "merchant_category": FeatureType.STRING_CATEGORICAL,
        "payment_method": FeatureType.STRING_CATEGORICAL,

        # User features
        "user_country": FeatureType.STRING_CATEGORICAL,
        "account_age_days": FeatureType.FLOAT_NORMALIZED,

        # Time features, plus the discrete columns the crosses need: an hour
        # band and an amount band derived from the raw columns above.
        "transaction_time": DateFeature(
            name="transaction_time"
        ),
        "transaction_hour": FeatureType.STRING_CATEGORICAL,
        "amount_band": FeatureType.STRING_CATEGORICAL
    },

    # Cross features for fraud patterns
    feature_crosses=[
        # Country × merchant (unusual combinations)
        ("user_country", "merchant_category", 32),

        # Payment method × amount band (unusual methods for large amounts)
        ("payment_method", "amount_band", 24),

        # Hour × amount band (unusual times for large transactions)
        ("transaction_hour", "amount_band", 24),

        # Country × time (transactions from unusual locations at odd hours)
        ("user_country", "transaction_hour", 32)
    ],

    # Enable tabular attention for additional interaction discovery
    tabular_attention=True
)

📊 Model Architecture

graph TD A1[Feature 1] --> C[Pair the raw values] A2[Feature 2] --> C C --> D[Hash into nr_bins buckets] D --> E[Cast the bin index to float32] E --> F[One extra output column] style A1 fill:#e3f2fd,stroke:#64b5f6,stroke-width:2px style A2 fill:#e3f2fd,stroke:#64b5f6,stroke-width:2px style C fill:#e8f5e9,stroke:#66bb6a,stroke-width:2px style D fill:#fff8e1,stroke:#ffd54f,stroke-width:2px style E fill:#f3e5f5,stroke:#ce93d8,stroke-width:2px style F fill:#e8eaf6,stroke:#7986cb,stroke-width:2px

KDP pairs the two raw values, hashes the pair into one of nr_bins buckets, and appends that bin index to the output as a single float column alongside the categorical features.

💎 Pro Tips

🎯 Choose Meaningful Crosses

Focus on feature pairs with likely interactions based on domain knowledge:

Product × location (regional preferences)
Time × event (temporal patterns)
User × item (personalization)
Price × category (price sensitivity)

⚠️ Beware of Sparsity

Crosses between high-cardinality features produce many distinct pairs, and nr_bins decides how many of them share a bucket:

Too few bins and unrelated pairs collide into one value
Too many and most bins are never seen by the model
The columns feeding a cross can themselves use category_encoding="hashing" when they have many values

📏 Choosing nr_bins

The third element of the tuple is the number of hash buckets, not an embedding size:

Start near the number of pairs you actually expect to see
Small crosses (a handful of categories each): 8-32 bins
Larger crosses: a few times the distinct pair count, to keep collisions rare
The output width is one column per cross whatever you choose

🔄 Alternative Approaches

Consider other interaction modeling techniques alongside crosses:

Enable tabular_attention=True to automatically discover interactions
Use transfo_nr_blocks for more sophisticated feature relationships
Bucket a numeric column into bands to bring it into a cross

🔄 Comparing With Alternatives

Approach	Pros	Cons	When to Use
Cross Features	Explicit modeling of specific interactions	Need to specify each interaction	When you know which interactions matter
Tabular Attention	Automatic discovery of interactions	Less control	When you're unsure which interactions matter
Transformer Blocks	Most powerful interaction modeling	Most computationally expensive	For complex interaction patterns
Feature MoE	Adaptive feature processing	Higher complexity	For heterogeneous feature sets

← Date Features Distribution-Aware Encoding →

➕ Cross Features

Cross Features in KDP

📋 Overview

Feature Interaction

Pattern Discovery

Efficient Processing

Fixed Width

🧠 How Cross Features Work

Feature Combination

Hashing

One Extra Column

Pattern Discovery

📝 Basic Usage

⚙️ Key Configuration Parameters

🛠️ Cross Feature Types

Categorical × Categorical

Categorical × Bucketed Numerical

Date Crosses

Multiple Crosses

💡 Advanced Cross Feature Techniques

🔍 Attention Over Crosses

🧠 Three-Way Interactions

🔧 Real-World Examples

E-commerce Recommendations

Fraud Detection

📊 Model Architecture

💎 Pro Tips

🎯 Choose Meaningful Crosses

⚠️ Beware of Sparsity

📏 Choosing nr_bins

🔄 Alternative Approaches

🔄 Comparing With Alternatives

🔗 Related Topics