Skip to main content

Warehouse Native Experimentation

Overview​

Warehouse Native enables experimentation workflows, from targeting and assignment to analysis, and provides a statistical engine for analyzing existing experiments with measurement tools like CUPED in Harness Feature Management & Experimentation (FME).

How Warehouse Native works​

Warehouse Native runs experimentation jobs directly in your data warehouseA centralized repository for storing and managing large volumes of structured and semi-structured data. Examples include Snowflake, BigQuery, Redshift, and Databricks. by using your existing data to calculate metrics and enrich experiment analyses.

The data model is designed around two two primary types of data: assignment data and performance/behavioral data, which power the FME statistical engine in your warehouse.

Key components include:

  • Assignment data: Tracks user or entity assignments to experiments. This includes metadata about the experiment.
  • Performance and behavioral data: Captures metrics, events, and user behavior relevant to the experiment.
  • Experiment metadata: Contains definitions for experiments, including the experiment ID, name, start/end dates, traffic allocation, and grouping logic.
  • Metric definitions: Defines how metrics are computed in the warehouse, including aggregation logic, denominators, and optional CUPED adjustments. These definitions ensure analyses are standardized across experiments.

Cloud Experimentation​

Cloud ExperimentsA method of running feature management experiments using a cloud-based service provided by Harness FME, which handles data collection, analysis, and reporting. are executed and analyzed within Harness FME, which collects feature flag impressions and performance data from your application and integrations. For more information, see the Cloud Experimentation documentation.

Warehouse Native​

Warehouse Native ExperimentsA method of running feature management experiments directly within your data warehouse, leveraging its processing power and existing data infrastructure. are executed directly in your data warehouse, leveraging assignment and behavioral data from Harness FME to calculate metrics and run statistical analyses at scale.

Get started​

To start using Warehouse Native Experimentation:

  1. Connect Harness FME to your data warehouse.
  2. Prepare your assignment and metric source tables in your data warehouse.
  3. Configure your assignment and metric sources in Harness FME.
  4. Define your metrics and create experiments in Harness FME.

Once you've created metric definitions and started running experiments in your data warehouse, you can access analyses in Harness FME.