What Is a Feature Store? A Practical Guide With Feast

A feature store is the layer that defines each machine learning feature once and serves it two ways: as point-in-time correct history for building training sets (the offline store) and as the latest values at low latency for live inference (the online store). Its job is to remove training-serving skew and data leakage, and to let several models reuse the same features. You need one when multiple models share features or when features must be fresh at request time. Before that, a tested shared feature module and a warehouse table do the same job with less to operate.

This guide explains the problem in concrete terms, shows point-in-time joins in plain SQL, walks through a working Feast setup, and covers the checks that tell you whether the whole thing is actually consistent in production.

Key takeaways

• The core problem is consistency, not storage. A model must see the same feature values in training as in serving, computed from information available at decision time.
• Point-in-time joins prevent leakage. Each training row gets the feature values that existed at its timestamp, never later ones.
• Use the time a value became available, not the time it describes. A nightly job's "as of midnight" value is not usable until the job lands.
• The online store is a cache. It must be rebuildable from the offline store at any time.
• Prove consistency with a skew check. Log what serving used, recompute it offline, and compare.
• Adopt one when several models share features or freshness is measured in seconds. Not before.

The problem a feature store solves

A data scientist builds features in a notebook: joins, 30-day rolling averages, encodings. The model looks good. Then an engineer reimplements the same logic in the serving path, against live sources, in a different language or library. The two versions differ in small ways: an inner join instead of a left join, a window that includes today instead of ending yesterday, a null filled with zero instead of the mean. No error is raised. The model simply performs worse than its evaluation promised, and nobody can say why.

The second failure is leakage. Consider a churn model trained on label events and a daily feature table.

A correct join gives the March label orders_30d = 4 and the April label orders_30d = 1. A naive join on "latest value per customer" gives both rows 0, a value from after the customer had already churned. The model learns that zero recent orders predicts churn perfectly, which is true and useless: by the time the feature reads zero, the customer is gone.

A feature store addresses both failures with one definition per feature, point-in-time correct retrieval for training, and the same definition feeding the online values used at inference.

The components

Note what is not on the list: the feature store rarely computes the hard features itself. Most teams keep transformations in their existing data stack (see the modern data engineering stack) and use the feature store for definition, retrieval and serving.

Point-in-time joins in plain SQL

You can get point-in-time correctness without any feature store, and it is worth understanding the query before handing it to a tool. DuckDB has a native as-of join:

SELECT
  l.customer_id,
  l.event_timestamp,
  l.churned,
  f.orders_30d,
  f.avg_order_value_30d
FROM labels AS l
ASOF LEFT JOIN customer_features AS f
  ON l.customer_id = f.customer_id
 AND l.event_timestamp >= f.feature_timestamp;

In warehouses without as-of joins, a window function does the same thing. The lower bound plays the role of a time-to-live: a feature value older than that is treated as missing rather than current.

SELECT customer_id, event_timestamp, churned, orders_30d, avg_order_value_30d
FROM (
  SELECT
    l.customer_id,
    l.event_timestamp,
    l.churned,
    f.orders_30d,
    f.avg_order_value_30d,
    ROW_NUMBER() OVER (
      PARTITION BY l.customer_id, l.event_timestamp
      ORDER BY f.feature_timestamp DESC
    ) AS rn
  FROM labels AS l
  LEFT JOIN customer_features AS f
    ON f.customer_id = l.customer_id
   AND f.feature_timestamp <= l.event_timestamp
   AND f.feature_timestamp > l.event_timestamp - INTERVAL '2 days'
) AS ranked
WHERE rn = 1;

Interval syntax differs slightly between warehouses; the logic does not. One subtlety matters more than the syntax: feature_timestamp must be when the value became available to the serving system, not the period it describes. If the nightly job that computes "orders in the 30 days to midnight" lands at 02:00, a decision made at 01:00 could not have used it. Stamp feature rows with the landing time, or shift them by the job's worst-case delay.

A worked Feast example

Feast is a widely adopted open-source feature store. It provides the registry, point-in-time retrieval and online serving, and plugs into storage you already run. The configuration below uses local files and Redis; in production the offline store is usually your warehouse.

project: churn
provider: local
registry: data/registry.db
offline_store:
  type: file
online_store:
  type: redis
  connection_string: "localhost:6379"

Definitions are Python, reviewed and versioned like any other code:

"""feature_repo/definitions.py"""
from datetime import timedelta
 
from feast import Entity, FeatureService, FeatureView, Field, FileSource
from feast.types import Float32, Int64
 
customer = Entity(name="customer", join_keys=["customer_id"])
 
order_stats_source = FileSource(
    name="customer_order_stats",
    path="data/customer_order_stats.parquet",
    timestamp_field="feature_timestamp",     # when the value became available
    created_timestamp_column="created_at",   # tie-breaker for duplicate rows
)
 
customer_order_stats = FeatureView(
    name="customer_order_stats",
    entities=[customer],
    ttl=timedelta(days=2),                   # training joins skip values older than this
    schema=[
        Field(name="orders_30d", dtype=Int64),
        Field(name="avg_order_value_30d", dtype=Float32),
        Field(name="days_since_last_order", dtype=Int64),
    ],
    source=order_stats_source,
    online=True,
    tags={"owner": "growth-ml", "refresh": "daily"},
)
 
churn_v1 = FeatureService(name="churn_v1", features=[customer_order_stats])

The FeatureService matters more than it looks. It pins the exact feature set a model version was trained on. When the model changes its features, create churn_v2 rather than editing churn_v1, so the model in production keeps receiving exactly what it expects.

Register the definitions and load the latest values into the online store:

feast apply
feast materialize-incremental $(date -u +"%Y-%m-%dT%H:%M:%S")

In production, materialization runs on a schedule from your orchestrator, and its success or failure is monitored like any other job.

To build a training set, pass the entities and timestamps you want features for. Label columns pass straight through.

import pandas as pd
from feast import FeatureStore
 
store = FeatureStore(repo_path="feature_repo")
 
entity_df = pd.DataFrame({
    "customer_id": [1001, 1001, 1002],
    "event_timestamp": pd.to_datetime(["2026-03-01", "2026-04-01", "2026-04-01"], utc=True),
    "churned": [0, 1, 0],
})
 
training_df = store.get_historical_features(
    entity_df=entity_df,
    features=store.get_feature_service("churn_v1"),
).to_df()

At inference, the same feature service returns the latest values:

online = store.get_online_features(
    features=store.get_feature_service("churn_v1"),
    entity_rows=[{"customer_id": 1001}],
).to_dict()

Both paths read the same definitions. That is the whole point: there is no second implementation to drift. The training and serving code around these calls is covered in our guide to building production AI pipelines in Python.

Do you need a feature store?

The cost of a feature store is operational: another registry to keep in sync, materialization jobs to monitor, an online database to run. Pay it when the duplication and skew it removes cost more.

Choosing the online store

The online store is a cache of the latest feature values, keyed by entity. Pick it on latency, operability and what you already run.

Whatever you choose, it must be rebuildable by rematerializing from the offline store. If losing the online store would lose data, it has quietly become a primary database. For the broader trade-offs between these systems, see our database selection guide.

Streaming features and freshness

Some features only matter if they are fresh: transactions in the last five minutes, items in the current session. For those, a streaming job computes windowed aggregates and writes them to the online store directly (Feast calls this a push source), while also writing them to the offline store so training sees the same values.

Three things decide whether streaming features are trustworthy:

1. A freshness target per feature. Write down how old a value may be at read time, and alert when the online store falls behind it.
2. Late and out-of-order events. Decide how long the job waits (its watermark) and what happens to events that arrive after it. Training must apply the same rule, or skew returns through the back door.
3. Identical window semantics offline. A sliding five-minute window in Flink and a tumbling one in the backfill job are different features with the same name.

Proving there is no skew

Trust, but verify. Log the feature values the service actually used with each prediction, then recompute them offline for the same entity and timestamp and compare.

import pandas as pd
 
 
def skew_report(served: pd.DataFrame, store, service: str, tol: float = 1e-6) -> pd.DataFrame:
    """Compare feature values used at serving time with offline recomputation."""
    offline = store.get_historical_features(
        entity_df=served[["customer_id", "event_timestamp"]],
        features=store.get_feature_service(service),
    ).to_df()
    merged = served.merge(offline, on=["customer_id", "event_timestamp"],
                          suffixes=("_served", "_offline"))
    rows = []
    for name in [c[: -len("_served")] for c in merged.columns if c.endswith("_served")]:
        a, b = merged[f"{name}_served"], merged[f"{name}_offline"]
        mismatch = ~((a - b).abs() <= tol) & ~(a.isna() & b.isna())
        rows.append({"feature": name, "rows": len(merged), "mismatch_rate": float(mismatch.mean())})
    return pd.DataFrame(rows).sort_values("mismatch_rate", ascending=False)

Run it nightly on a sample. Some mismatch is expected for features that changed between serving and the next materialization; a feature that mismatches far more than its neighbors is a bug, and this report is usually how it is found.

What to monitor

• Materialization runs: success, duration, rows written. A failed run does not error at serving time; the online store keeps serving the last values it received.
• Freshness: the newest feature timestamp per feature view against its freshness target.
• Null and default rate at serving: a sudden rise means missing keys, expired values or a broken source.
• Distribution drift: the same per-feature checks you run on model inputs.
• Online store health: read latency, memory, evictions, connection errors.

These belong on the same dashboards as the model's own metrics. The broader MLOps picture, including how feature monitoring fits with model monitoring, is in our introduction to MLOps.

What goes wrong in production

• Materialization quietly stopped. Lookups kept returning values that grew older every hour, or nulls once the online store expired them, and the model scored on stale features or defaults. Nothing errored. Alert on the job, on feature age at serving time and on the serving null rate.
• Event time was used instead of availability time. Training rows got values that the live system would only see hours later. Offline metrics were better than anything production could reach.
• Entity key types did not match. Training used integer IDs, the service sent strings. Every online lookup missed and returned nulls. Validate key types at the service boundary.
• Timezone-naive timestamps. A few hours of offset is enough to leak or to miss the latest value. Store everything in UTC with explicit zones.
• A feature definition was edited in place. Last month's training sets could no longer be rebuilt, and the production model received a feature with a familiar name and a new meaning. Version features and feature services instead.
• A backfill overwrote history. The offline store must be append-only, or point-in-time correctness is gone.
• Nobody owned the features. Three versions of "customer lifetime value" with different windows, no descriptions, and no one to ask. Require an owner and a description in the registry.

Frequently Asked Questions

What is a feature store in machine learning?

A feature store is a system that defines machine learning features once and serves them consistently for both training and inference. It keeps historical feature values for building point-in-time correct training sets and the latest values in a low-latency store for live predictions. A registry records what each feature means, where it comes from and who owns it.

What is the difference between an offline and an online feature store?

The offline store holds the full history of feature values, usually in a data warehouse or lake, and is queried in bulk to build training sets as of past timestamps. The online store holds only the latest value per entity in a key-value database such as Redis or DynamoDB, and is read one entity at a time during inference. Materialization jobs copy values from the offline side to the online side.

Do I need a feature store?

Usually not for the first model. A shared, tested feature module and warehouse tables solve skew for one or two models with much less to operate. A feature store pays off when several models reuse the same features, when features must be fresh at request time, or when teams keep rebuilding features that already exist elsewhere.

What is training-serving skew?

Training-serving skew is a difference between the feature values a model saw in training and the values it receives in production for the same situation. It usually comes from two implementations of the same feature, different data sources, or timing differences, and it degrades the model without any error being raised. The fix is a single feature definition used by both paths plus a check that compares served values with offline recomputation.

What is a point-in-time join?

A point-in-time join attaches to each training example the feature values that were available at that example's timestamp, and nothing later. It prevents data leakage, where a model trains on information it could not have had when the real decision was made. Feature stores do this automatically, and in SQL it is an as-of join or a window function that picks the latest feature row at or before each event.

What are the main feature store options?

Feast is a widely used open-source feature store and plugs into existing warehouses and key-value stores. Hopsworks is a dedicated feature store platform, and Databricks, Google Vertex AI and Amazon SageMaker each include a feature store in their ML platforms. If you already run one of those platforms, try its built-in store before adding another system.

Need help with ML data infrastructure?

We design feature pipelines and stores sized to the models you actually run, and we would rather talk you out of one you do not need yet than operate one that adds nothing. When we build it, we stay on the pager for it.

• AI & Machine Learning practice
• Cloud Architecture practice
• Talk to an engineer