🏷️ Categorical Features

Categorical Features in KDP

Turn labels and IDs into dense vectors, one-hot columns, or hashed buckets.

📋 Overview

KDP learns the vocabulary of a categorical column during the statistics pass, then encodes it one of three ways: a learned embedding, a one-hot vector, or a hash into a fixed number of buckets. Hashing is the only one that needs no vocabulary, so it is also the only one that works without a data pass.

🚀 The Two Categorical Feature Types

Feature type	Column dtype	Use for
`FeatureType.STRING_CATEGORICAL`	string	City names, product codes, channels — anything written as text.
`FeatureType.INTEGER_CATEGORICAL`	integer	IDs and codes that are numbers but have no ordering.

MULTI_CATEGORICAL and STRING_HASHED do not exist

Earlier documentation listed these as feature types, along with a multi-value workflow using separator and multi_hot. FeatureType has exactly eleven members and neither of these is among them, so FeatureType.MULTI_CATEGORICAL raises AttributeError. Hashing is not a separate feature type either — it is the category_encoding option below. To encode a multi-value column, split it into columns yourself or use a custom pipeline.

📝 Basic Usage

from kdp import PreprocessingModel, FeatureType

preprocessor = PreprocessingModel(
    path_data="data.csv",
    features_specs={
        "city": FeatureType.STRING_CATEGORICAL,
        "store_id": FeatureType.INTEGER_CATEGORICAL,
    },
)
preprocessor.build_preprocessor()

⚙️ Configuration Parameters

Parameter	Type	Default	Description
`category_encoding`	str	`"EMBEDDING"`	`"EMBEDDING"`, `"ONE_HOT_ENCODING"` or `"HASHING"`.
`embedding_size`	int	derived from vocabulary size	Width of the learned embedding. Also used when hashing with an embedding on top (default 8 there).
`hash_bucket_size`	int	derived from vocabulary size	Number of hash buckets. Hashing only. Setting it explicitly removes the need for a statistics pass.
`salt`	int	`None`	Seed for the hash, so two columns hash differently. Hashing only.
`hash_with_embedding`	bool	`False`	Put a learned embedding on top of the hash instead of a multi-hot vector. Hashing only.

Options that are silently ignored

embedding_dim (the real name is embedding_size), vocabulary_size, max_vocabulary_size, use_embedding, unknown_token, oov_buckets, multi_hot, separator, pretrained_embeddings, multi_hash and num_hash_functions are not read by KDP. CategoricalFeature accepts any keyword argument, so passing them looks like it works and changes nothing. Out-of-vocabulary values are always routed to a single reserved slot; that is not configurable.

🎛️ The Three Encodings

Embedding (default)

from kdp.features import CategoricalFeature, CategoryEncodingOptions, FeatureType

CategoricalFeature(
    name="city",
    feature_type=FeatureType.STRING_CATEGORICAL,
    category_encoding=CategoryEncodingOptions.EMBEDDING,
    embedding_size=16,
)

Leave embedding_size unset and KDP derives it from the vocabulary size.

One-hot

CategoricalFeature(
    name="status",
    feature_type=FeatureType.STRING_CATEGORICAL,
    category_encoding=CategoryEncodingOptions.ONE_HOT_ENCODING,
)

Output width is the vocabulary size. Good for a handful of categories, wasteful beyond a few dozen.

Hashing

CategoricalFeature(
    name="user_id",
    feature_type=FeatureType.STRING_CATEGORICAL,
    category_encoding=CategoryEncodingOptions.HASHING,
    hash_bucket_size=1024,
    salt=42,                    # optional, decorrelates two hashed columns
    hash_with_embedding=True,   # embedding instead of multi-hot
    embedding_size=16,          # width of that embedding
)

Hashing maps values into a fixed bucket count, so it handles unbounded cardinality and unseen values without growing. Collisions are the trade-off.

Hashing can skip the statistics pass entirely

When every feature is a hashing categorical with an explicit hash_bucket_size, nothing has to be learned from your data, so path_data is not required and build_preprocessor() runs immediately. Omit hash_bucket_size and the bucket count is derived from the vocabulary, which does require a data pass.

from kdp import CategoricalFeature, CategoryEncodingOptions, FeatureType, PreprocessingModel

# Builds with no dataset at all
preprocessor = PreprocessingModel(
    features_specs={
        "user_id": CategoricalFeature(
            name="user_id",
            feature_type=FeatureType.STRING_CATEGORICAL,
            category_encoding=CategoryEncodingOptions.HASHING,
            hash_bucket_size=32,
        ),
    },
)
preprocessor.build_preprocessor()

📐 Output Widths

Encoding	Output width
Embedding	`embedding_size` (derived if unset)
One-hot	vocabulary size
Hashing, multi-hot (default)	`hash_bucket_size`
Hashing with embedding	`embedding_size` (default 8)

🔗 Combining With Other Features

Crossing two categoricals

from kdp import FeatureType, PreprocessingModel

preprocessor = PreprocessingModel(
    path_data="data.csv",
    features_specs={
        "city": FeatureType.STRING_CATEGORICAL,
        "channel": FeatureType.STRING_CATEGORICAL,
    },
    feature_crosses=[("city", "channel", 10)],
)

Feature selection and attention

from kdp import FeatureType, PreprocessingModel

preprocessor = PreprocessingModel(
    path_data="data.csv",
    features_specs={"city": FeatureType.STRING_CATEGORICAL},
    feature_selection_placement="categorical",   # or "all_features"
    tabular_attention=True,
    tabular_attention_placement="categorical",
    transfo_nr_blocks=2,                         # transformer over categoricals
    transfo_placement="categorical",
)

💎 Practical Notes

Reach for hashing on high cardinality

User and session IDs blow up a vocabulary. A fixed bucket count keeps the model the same size no matter how many values appear.

One-hot only for small vocabularies

Width equals vocabulary size, so it grows linearly with the number of categories.

Salt when you hash two columns

Without different salts, the same value in two columns lands in the same bucket and the model cannot tell them apart.

Integer IDs are not numbers

Use INTEGER_CATEGORICAL, not a float type — otherwise the model reads store 7 as greater than store 3.

🏷️ Categorical Features