# Data Formulator - Deep Dive

> **TL;DR**: Data Formulator is Microsoft Research's AI-powered tool that lets you create complex data visualizations by combining drag-and-drop UI with natural language. Instead of writing code to transform data, you describe what you want and the AI generates the transformations. Think "Tableau meets ChatGPT" - but smarter and free.

---

## The Problem: Why Data Visualization Is Hard

Creating visualizations sounds simple: you have data, you want a chart. But in reality, analysts hit walls:

### The Data Transformation Barrier

Most data isn't "visualization-ready." Before you can chart anything, you need to:

| Task | What It Means | Example |
|------|---------------|---------|
| **Aggregate** | Combine rows | Sum sales by region |
| **Filter** | Remove rows | Only show 2024 data |
| **Derive** | Calculate new columns | Profit = Revenue - Cost |
| **Join** | Combine tables | Add customer names to orders |
| **Pivot** | Reshape data | Turn months into columns |

**The catch**: These transformations require programming (Python, SQL, Excel formulas). Non-technical analysts are stuck.

### The Chat Problem

"Just use ChatGPT!" doesn't work well for visualization because:

| Issue | What Happens |
|-------|--------------|
| **Verbosity** | Describing a chart in words takes forever |
| **No Iteration** | Can't easily tweak and refine |
| **Black Box** | You don't see what the AI did to your data |
| **Disconnected** | AI generates code, but you still need to run it somewhere |

### The Tool Problem

Traditional tools force a choice:

```
Option A: Simple tools (Excel)
  └── Easy to use, but limited transformations

Option B: Powerful tools (Tableau, Power BI)
  └── Can do anything, but steep learning curve

Option C: Code (Python, R)
  └── Maximum flexibility, but requires programming
```

**Data Formulator's insight**: What if you could describe what you want, and AI handles the hard parts?

---

## What Is Data Formulator?

Data Formulator is an **AI-powered visualization tool** from Microsoft Research that:

1. Lets you **describe** visualizations using UI + natural language
2. **Automatically transforms** your data (aggregations, calculations, filtering)
3. **Generates interactive charts** without writing code
4. Supports **iterative exploration** with branching history

### The Key Innovation: Concept Binding

Instead of working with raw data columns, you work with **concepts** - abstract ideas of what you want to visualize:

```
Traditional Approach:
  "I need to GROUP BY region, SUM(sales), then JOIN with targets..."

Data Formulator Approach:
  "I want to see Total Revenue by Region compared to Target"
```

You define the *concept* ("Total Revenue"), and AI figures out *how* to calculate it.

---

## How It Works

### The Three-Step Process

```
┌─────────────────────────────────────────────────────────────────────┐
│                    DATA FORMULATOR WORKFLOW                          │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   STEP 1: Define Your Intent                                        │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  UI: Drag fields to chart axes                              │   │
│   │  NL: "Calculate total sales per quarter"                    │   │
│   │  Combined: Drag "Sales" to Y-axis + "show quarterly totals" │   │
│   └─────────────────────────────────────────────────────────────┘   │
│                              ↓                                      │
│   STEP 2: AI Generates Transformations                              │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  LLM analyzes your intent                                   │   │
│   │  Generates Python/Pandas code                               │   │
│   │  Executes transformations                                   │   │
│   │  Creates Vega-Lite specification                            │   │
│   └─────────────────────────────────────────────────────────────┘   │
│                              ↓                                      │
│   STEP 3: Interactive Visualization                                 │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  Chart rendered in browser                                  │   │
│   │  Hover for details                                          │   │
│   │  Inspect transformed data                                   │   │
│   │  View generated code                                        │   │
│   └─────────────────────────────────────────────────────────────┘   │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

### The Interface

Data Formulator has four main panels:

| Panel | Purpose |
|-------|---------|
| **Concept Shelf** | Define new data concepts (calculated fields) |
| **Chart Builder** | Drag-and-drop to specify visualization |
| **Table View** | Inspect the transformed data |
| **Visualization** | See the final interactive chart |

### Example Workflow

**Goal**: "Show top 5 products by revenue with year-over-year growth"

1. **Load data**: Upload sales CSV
2. **Define concept**: Click "+", type "Total Revenue = sum of sales amount"
3. **Define concept**: Click "+", type "YoY Growth = percentage change from previous year"
4. **Build chart**:
   - Drag "Product" to X-axis
   - Drag "Total Revenue" to Y-axis
   - Drag "YoY Growth" to Color
5. **Add filter**: Type "top 5 by revenue"
6. **AI generates**: Python code for aggregation, growth calculation, ranking
7. **View result**: Interactive bar chart with growth coloring

**Time**: ~3 minutes (vs. 30+ minutes coding manually)

---

## Key Features

### 1. Hybrid UI + Natural Language

You don't have to choose between clicking and typing:

| Approach | Best For | Example |
|----------|----------|---------|
| **UI Only** | Simple mappings | Drag "Sales" to Y-axis |
| **NL Only** | Complex calculations | "Calculate compound annual growth rate" |
| **Combined** | Precise intent | Drag "Date" to X-axis + "aggregate by quarter" |

**Research finding**: Users average only **7.28 words** per concept description. The UI handles the rest.

### 2. Data Threads (Iteration History)

Unlike chat interfaces where history is linear, Data Formulator supports **branching exploration**:

```
Start
  │
  ├── Chart 1: Bar chart of sales by region
  │     │
  │     ├── Branch A: Add trend line
  │     │     └── Final: Bar + trend
  │     │
  │     └── Branch B: Convert to pie chart
  │           └── Final: Pie chart
  │
  └── Chart 2: Scatter plot of price vs quantity
        └── Final: Scatter with regression
```

**Benefits**:
- **Backtrack** without losing work
- **Compare** different approaches
- **Reuse** previous designs as templates
- **Document** exploration history

### 3. Transparency (Not a Black Box)

You can always see what AI did:

| View | What You See |
|------|--------------|
| **Generated Code** | Actual Python/Pandas transformation |
| **Transformed Data** | The resulting dataset |
| **Vega-Lite Spec** | The visualization grammar |

This means you can:
- Verify correctness
- Learn from the code
- Manually edit if needed
- Export for use elsewhere

### 4. Large Dataset Support

Version 0.2+ uses **DuckDB** as a backend:

- Handle **million-row datasets** efficiently
- Only fetch data needed for visualization
- SQL-powered aggregations
- Fast local processing

### 5. Flexible Data Input

| Source | Supported |
|--------|-----------|
| **Files** | CSV, TSV, XLSX |
| **Databases** | MySQL, PostgreSQL, MongoDB |
| **Cloud** | BigQuery, Azure Data Explorer |
| **Storage** | Amazon S3, Azure Blob |
| **Unstructured** | Screenshots, text blocks, web tables |

### 6. Multiple AI Models

Data Formulator uses **LiteLLM** to support:

| Provider | Models |
|----------|--------|
| **OpenAI** | GPT-4o, GPT-4, GPT-3.5 |
| **Anthropic** | Claude 3.5 Sonnet, Claude 3 Opus |
| **Azure** | Azure OpenAI deployments |
| **Local** | Ollama (Mistral, Llama, etc.) |

**Recommended**: GPT-4o or Claude 3.5 Sonnet for best results.

### 7. Chart Types

- **Standard**: Bar, line, scatter, area
- **Advanced**: Treemaps, waterfall, box plots
- **Geographic**: Choropleth maps, US maps
- **Categorical**: Pie charts
- All powered by **Vega-Lite** grammar

### 8. Report Generation

- Select charts from your exploration
- Generate markdown-style reports
- Editable with **Chartifact**
- Share insights with data backing

---

## Installation & Usage

### Quick Start (5 Minutes)

```bash
# Create virtual environment (recommended)
python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install Data Formulator
pip install data_formulator

# Run the application
python -m data_formulator
```

Opens automatically at `http://localhost:5000`

### Alternative: GitHub Codespaces

1. Go to https://github.com/microsoft/data-formulator
2. Click green "Code" button → "Codespaces"
3. Wait 2-5 minutes for setup
4. Pre-configured environment, no local install

### Configuration

Create `api-keys.env` for persistent API keys:

```bash
OPENAI_API_KEY=sk-your-key-here
ANTHROPIC_API_KEY=sk-ant-your-key-here
# Optional
AZURE_API_KEY=your-azure-key
AZURE_ENDPOINT=https://your-endpoint.openai.azure.com/
```

### Basic Usage

1. **Launch**: `python -m data_formulator`
2. **Load Data**: Click "New Project" → Upload CSV/XLSX
3. **Create Concept**: Click "+" in Concept Shelf → Describe calculation
4. **Build Chart**: Drag concepts to X/Y/Color axes
5. **Iterate**: Use data threads to explore alternatives
6. **Export**: Generate report or download chart

---

## Technical Architecture

### System Layers

```
┌─────────────────────────────────────────────────────────────────┐
│                     DATA FORMULATOR ARCHITECTURE                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   FRONTEND (TypeScript + React)                                 │
│   ├── Dataset upload/preview                                    │
│   ├── Concept shelf (drag-and-drop)                             │
│   ├── Chart builder (visualization mapping)                     │
│   ├── Natural language input                                    │
│   └── Visualization display (Vega-Lite)                         │
│                                                                 │
│   BACKEND (Python)                                              │
│   ├── Data loading/preprocessing                                │
│   ├── LLM communication (via LiteLLM)                           │
│   ├── Code generation                                           │
│   ├── Code execution (sandboxed)                                │
│   └── Database connectivity                                     │
│                                                                 │
│   DATABASE LAYER                                                │
│   ├── DuckDB (local, fast SQL)                                  │
│   ├── External DBs (MySQL, PostgreSQL, etc.)                    │
│   └── Cloud storage (S3, Azure Blob)                            │
│                                                                 │
│   LLM LAYER (via LiteLLM)                                       │
│   ├── OpenAI (GPT-4o, GPT-4)                                    │
│   ├── Anthropic (Claude 3.5)                                    │
│   ├── Azure OpenAI                                              │
│   └── Ollama (local models)                                     │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
```

### Code Generation Flow

When you request a visualization, this happens:

```
1. User Intent
   └── UI actions + natural language

2. Prompt Construction
   ├── System prompt: "You are a data scientist..."
   ├── Data context: column names, types, sample rows
   └── User goal: visualization specification

3. LLM Response
   ├── Field specification (JSON)
   │   └── What columns to create, how to compute them
   └── Python code (template-based)
       └── Actual transformation logic

4. Execution
   ├── Run Python code in sandbox
   ├── Validate results
   └── Handle errors (auto-retry with fixes)

5. Visualization
   ├── Generate Vega-Lite spec
   └── Render interactive chart
```

### Example Generated Code

For "Show total sales by region, top 5 only":

```python
import pandas as pd

def transform_data(df):
    # Aggregate sales by region
    result = df.groupby('region').agg({
        'sales': 'sum'
    }).reset_index()

    # Rename for clarity
    result = result.rename(columns={'sales': 'total_sales'})

    # Sort and take top 5
    result = result.nlargest(5, 'total_sales')

    return result
```

---

## Data Formulator vs Other Tools

### vs Tableau

| Aspect | Data Formulator | Tableau |
|--------|-----------------|---------|
| **Price** | Free (open-source) | ~$70/user/month |
| **AI Integration** | Core feature | Limited |
| **Learning Curve** | Low-moderate | Steep |
| **Data Transformation** | AI-automated | Manual/code |
| **Maturity** | Research prototype (2023) | Enterprise (20+ years) |
| **Best For** | Exploration, iteration | Polished dashboards |

### vs Power BI

| Aspect | Data Formulator | Power BI |
|--------|-----------------|----------|
| **Price** | Free | Free-$10/month |
| **Ecosystem** | Standalone | Microsoft integrated |
| **AI Features** | LLM-powered | Growing (Copilot) |
| **Setup Time** | 5 minutes | 30+ minutes |
| **Best For** | Quick exploration | Enterprise reporting |

### vs ChatGPT/Claude for Data

| Aspect | Data Formulator | Chat AI |
|--------|-----------------|---------|
| **Interface** | Purpose-built UI | Chat window |
| **Visualization** | Built-in, interactive | Separate step |
| **Iteration** | Data threads, branching | Linear chat |
| **User Control** | High (see code, data) | Low (black box) |
| **Data Handling** | Million-row support | Context limits |

---

## Research Background

### The Team

**Lead**: Chenglong Wang, Principal Researcher at Microsoft Research Redmond

**Key Contributors**:
- Bongshin Lee
- Steven M. Drucker
- John Thompson
- Dan Marshall
- Jianfeng Gao

### Published Research

| Paper | Venue | Year | Recognition |
|-------|-------|------|-------------|
| Data Formulator: AI-powered Concept-driven Visualization | IEEE VIS | 2023 | Best Paper Honorable Mention |
| Data Formulator 2: Iterative Creation with AI | CHI | 2025 | Top-tier HCI venue |

### User Study Results

**CHI 2025 Study**:
- **Participants**: 8 analysts with varying expertise
- **Tasks**: Recreate professional data scientist work
- **Completion Rate**: 100%
- **Average Time**: 20 minutes per complex visualization
- **Prompt Efficiency**: 7.28 words average per concept
- **Attempts per Concept**: 1.62 (low iteration needed)

---

## How Data Formulator Can Help Us Achieve Our Goals

Looking at our "Software Development in the Age of AI" repository, Data Formulator connects directly to our mission:

### 1. **Demonstrates AI + UI Synergy**

Our repo explores AI-assisted development. Data Formulator shows a mature example of:

| Principle | How Data Formulator Implements It |
|-----------|-----------------------------------|
| Human-AI collaboration | User guides, AI executes |
| Transparency | Generated code is visible |
| Iteration support | Data threads enable branching |
| Reduced friction | 7 words average per concept |

### 2. **Complements Our Other Deep Dives**

| Our Deep Dive | Connection to Data Formulator |
|---------------|-------------------------------|
| **CCPM** | Both use structured workflows for complex tasks |
| **DeepAgents** | Both separate user intent from execution |
| **MCP Servers** | Data Formulator could use MCP for database connections |
| **Vibe Kanban** | Both emphasize visual interfaces for AI orchestration |
| **ML System Design** | Data Formulator enables exploration of ML metrics/results |

### 3. **Practical Tool for Our Community**

Data analysis is part of software development:
- Analyze logs and metrics
- Explore performance data
- Visualize system behavior
- Generate reports for stakeholders

Data Formulator makes this accessible without Python expertise.

### 4. **Validates Our Interaction Philosophy**

Our repo emphasizes effective human-AI interaction. Data Formulator's research validates:
- **Hybrid interfaces** (UI + NL) outperform either alone
- **Iteration support** is essential for complex tasks
- **Transparency** builds user trust
- **Low verbosity** (7 words!) is achievable with good design

### 5. **Connects to Knowledge Base Work**

Our `.claude/knowledge/` system could integrate Data Formulator patterns:
- Document visualization best practices
- Create playbooks for data exploration
- Reference Data Formulator's concept-binding approach

### 6. **Open Source Alignment**

Like all tools we document, Data Formulator is:
- Free and open-source (MIT License)
- Actively maintained (633+ commits)
- Community-driven (contributed data loaders)
- Transparent (research papers, code visible)

---

## Real-World Use Cases

### 1. Software Metrics Analysis

```
Input: CI/CD pipeline logs
Goal: "Show build time trends by service, highlight failures"

Data Formulator:
- Aggregates build times
- Calculates failure rates
- Generates trend line with failure markers
```

### 2. Performance Investigation

```
Input: Application performance metrics
Goal: "Compare response times before and after deployment"

Data Formulator:
- Filters by time ranges
- Calculates percentiles (P50, P95, P99)
- Creates before/after comparison chart
```

### 3. Business Reporting

```
Input: Sales database
Goal: "Show quarterly revenue by product category with targets"

Data Formulator:
- Aggregates sales by quarter and category
- Joins with targets table
- Calculates variance
- Creates grouped bar chart with target lines
```

### 4. Research Data Exploration

```
Input: Experiment results CSV
Goal: "Visualize accuracy vs parameters, color by model type"

Data Formulator:
- Parses parameter strings
- Creates scatter plot
- Adds regression lines per model
```

---

## Quick Reference

### Installation

```bash
pip install data_formulator
python -m data_formulator
```

### Key Commands

| Action | How |
|--------|-----|
| New project | Click "New Project" |
| Load data | Upload CSV/XLSX or connect database |
| Add concept | Click "+" in Concept Shelf |
| Build chart | Drag concepts to axes |
| View code | Click "Code" tab |
| Branch | Create new thread from any point |
| Export | Generate report |

### Supported Chart Types

Bar, Line, Scatter, Area, Pie, Treemap, Waterfall, Box Plot, Choropleth Map

---

## Key Links

| Resource | URL |
|----------|-----|
| GitHub Repository | https://github.com/microsoft/data-formulator |
| Online Demo | https://data-formulator.ai |
| PyPI Package | https://pypi.org/project/data-formulator/ |
| Research Paper (VIS 2023) | https://www.microsoft.com/en-us/research/publication/data-formulator-ai-powered-concept-driven-visualization-authoring/ |
| Research Paper (CHI 2025) | https://www.microsoft.com/en-us/research/publication/data-formulator-2-iteratively-creating-rich-visualizations-with-ai/ |
| Microsoft Research Blog | https://www.microsoft.com/en-us/research/blog/data-formulator-exploring-how-ai-can-help-analysts-create-rich-data-visualizations/ |
| Azure AI Foundry Labs | https://labs.ai.azure.com/projects/data-formulator/ |

---

## Quick Stats

| Metric | Value |
|--------|-------|
| Creator | Microsoft Research |
| Lead Researcher | Chenglong Wang |
| First Release | October 2023 |
| License | MIT (open-source) |
| GitHub Stars | 14,600+ |
| Commits | 633+ |
| Latest Version | 0.5.1 (Dec 2025) |
| Supported LLMs | GPT-4o, Claude 3.5, Ollama, etc. |
| Data Backend | DuckDB (million-row support) |
| Awards | Best Paper Honorable Mention (VIS 2023) |

---

## Bottom Line

Data Formulator represents what happens when researchers deeply understand both **AI capabilities** and **user needs**:

**The insight**: You don't need to choose between "easy but limited" (Excel) and "powerful but complex" (Python). With the right AI integration, you can have both.

**The approach**:
1. Let users express intent through UI + natural language
2. AI handles the data transformation code
3. Keep everything transparent and editable
4. Support iteration through branching history

**The result**: Complex visualizations in 20 minutes instead of hours, accessible to analysts who don't code.

For our repository focused on AI-assisted development, Data Formulator is a masterclass in **human-AI interaction design**. It shows how to combine AI power with user agency - a principle that applies far beyond data visualization.

If you work with data (and who doesn't in software development?), Data Formulator is worth trying. It's free, takes 5 minutes to install, and might change how you think about data exploration.

---

*Document created: January 2026*
*Sources: [GitHub - microsoft/data-formulator](https://github.com/microsoft/data-formulator), [Microsoft Research Publications](https://www.microsoft.com/en-us/research/publication/data-formulator-ai-powered-concept-driven-visualization-authoring/), [CHI 2025 Proceedings](https://dl.acm.org/doi/10.1145/3706598.3713296), [Microsoft Research Blog](https://www.microsoft.com/en-us/research/blog/data-formulator-exploring-how-ai-can-help-analysts-create-rich-data-visualizations/)*
