Canvas Chat — Overview

# Canvas Chat

A visual, non-linear chat interface where conversations are nodes on an infinite canvas.

Try it online · GitHub · Docs

## Quick Start

One command. No installation.

    uvx canvas-chat launch

Your browser opens automatically to the local server.

Or try it instantly at ericmjl--canvas-chat-fastapi-app.modal.run — bring your own API keys.

Core Idea

## Conversations as a Graph

Infinite canvas.
Directed acyclic graph.
Local-first storage.

Pan & zoom Branch anywhere No server data IndexedDB
👤User
Human
"Tell me about..."
🤖AI
AI
Response A
🤖AI
AI
Response B
👤User
Human
"Go deeper on..."
🔄Merged
Summary
Multi-select reply
Drag nodes to move — infinite canvas, pan, zoom, explore

## Core Workflows

Six key workflows. Each slide has an interactive mini-canvas — drag nodes like in the app.

## Chat & Reply

Type a message, press Enter. Click any node and reply to branch the conversation.

👤User
Human
"Tell me about..."
🤖AI
AI
Here’s a quick overview…
Drag nodes to move

## Highlight & Branch

Select text within a node, click Branch to create an excerpt node and continue from there.

Selection
"..."
👤User
Human
Quantum computing uses qubits and superposition.
Excerpt
Selected excerpt
Drag nodes to move

## Multi-Select

Cmd/Ctrl+Click multiple nodes to combine their context into a single reply.

🤖AI
AI
First model’s response…
🤖AI
AI
Second model’s response…
👤User
Human
Compare these and summarize
Drag nodes to move
Thinking

## Auto-Layout

Click the layout button to arrange nodes in a clean left-to-right hierarchy.

👤User
Human
Prompt
🤖AI
AI
Response 1
🤖AI
AI
Response 2
Zoom Layout Export

## Navigate

Drag to pan, scroll to zoom. Semantic zoom: zoomed out shows summaries.

🔄Summary
Summary
Key points from the discussion…
🔄Summary
Summary
Summary of the main ideas…
🔄Summary
Summary
Conclusions and next steps…

## Export / Import

Save sessions as `.canvaschat` files. Share or restore anytime.

👤User
Human
Session
🤖AI
AI
Saved · Load to restore
Save as .canvaschat · Load anytime to restore

Spotlight

## Slash Commands

Type `/` to explore.
Powerful features, one keystroke away.

/search /research /committee /matrix /factcheck /code

Full command reference →

🤖 Research report
⋮⋮ RESEARCH
/research quantum computing

## Key Features

- Web Research — `/research` generates deep reports via Exa API
- Web Search — `/search` finds relevant pages and summarizes them
- LLM Committee — `/committee` consults multiple LLMs and synthesizes
- Matrix Evaluation — `/matrix` builds cross-product comparison tables
- Fact-Check — `/factcheck` verifies claims with web evidence

- Code Execution — `/code` runs Python in-browser with Pyodide
- Image Analysis — paste, drag, or upload images for multimodal AI
- PDF Import — upload PDFs and chat with their contents
- Markdown & Math — full Markdown with KaTeX LaTeX rendering
- Multi-Provider — OpenAI, Anthropic, Google, Groq, Ollama, OpenRouter

## Settings & Data

Bring your own keys. Your data stays in your browser.

### API Keys

Configure via the Settings panel. Supported providers:

OpenAI Anthropic Google AI Groq Ollama OpenRouter Exa

Keys stored in `localStorage` — never sent to Canvas Chat servers.

### Local-First Storage

All conversation data lives in your browser's IndexedDB. Nothing is stored server-side.

- Export sessions as `.canvaschat` files
- Import to restore or share
- Admin mode for enterprise: server-side keys via `config.yaml`

## Tech Stack

Deliberately simple. No frameworks, no build step.

Frontend: SVG canvas + CSS. Backend: Python. Storage: IndexedDB. Deployment: Modal.

## Run Locally

For contributors and local development.

### Prerequisites

- Python 3.11+
- Pixi (recommended) or uv

### Setup

    git clone https://github.com/ericmjl/canvas-chat.git
    cd canvas-chat
    pixi install

### Dev Server

    pixi run dev

Opens at `http://127.0.0.1:7865`. Hot reload is enabled — edit files and the server restarts automatically.

### Tests

    pixi run test      # Python
    pixi run test-js   # JavaScript

## Codebase at a Glance

    canvas-chat/
    ├── src/canvas_chat/
    │   ├── app.py            # FastAPI routes
    │   ├── config.py         # Configuration
    │   ├── plugins/          # Python plugins
    │   └── static/
    │       ├── index.html    # Main page
    │       ├── css/          # Modular CSS (9 files)
    │       └── js/           # ES modules
    │           ├── app.js        # Orchestrator
    │           ├── canvas.js     # SVG canvas
    │           ├── crdt-graph.js # CRDT graph
    │           ├── chat.js       # LLM API
    │           └── feature-*.js  # Plugins
    ├── tests/                # Unit tests
    ├── cypress/              # E2E tests
    ├── docs/                 # Diataxis docs
    └── modal_app.py          # Deployment

### Key modules

- `app.js` — main orchestrator, slash commands, keyboard shortcuts
- `canvas.js` — SVG canvas, pan/zoom, node rendering, semantic zoom
- `crdt-graph.js` — Yjs-backed graph data model and traversal
- `chat.js` — LLM API calls, streaming via SSE
- `graph-types.js` — node/edge type definitions and factories
- `feature-plugin.js` — plugin base class with AppContext DI

### Design principles

- Excalidraw-inspired aesthetic
- No frameworks — vanilla JS + CSS
- Local-first, no server-side user data

## Three-Level Plugin System

Extensibility from simple custom nodes to complex features. Detail →

## Building a Plugin

Extend Canvas Chat with a few lines of JavaScript.

    import { FeaturePlugin } from './feature-plugin.js';

    export class MyFeature extends FeaturePlugin {
      constructor(context) {
        super(context);
        this.graph = context.graph;
        this.canvas = context.canvas;
        this.chat = context.chat;
      }

      getSlashCommands() {
        return [{
          command: '/mycommand',
          description: 'Does something cool',
          placeholder: 'Enter input...',
        }];
      }

      async handleCommand(command, args, context) {
        // Your logic here
      }
    }

Register in `feature-registry.js` or load via `config.yaml` plugins list. See the docs for the full guide.

## Documentation & Contributing

### Diataxis Docs

Documentation follows the Diataxis framework:

- `docs/explanation/` — Design decisions, "why" docs
- `docs/how-to/` — Task-oriented guides
- `docs/reference/` — API and config reference

### Contributing

- All changes via pull request (never push to main)
- Pre-commit hooks for JSDoc linting
- Unit tests (Python + JS) and Cypress E2E tests
- Pixi as task runner (`pixi run dev`, `pixi run test`)

Browse the repo: github.com/ericmjl/canvas-chat

# Thank You

Try it online · GitHub · Docs

    uvx canvas-chat launch

Supplementary slides:

- All slash commands
- Plugin levels in detail
- Keyboard shortcuts

Supplementary

## All Slash Commands

- `/search <query>` — Web search via Exa
- `/research <topic>` — Deep research report via Exa
- `/committee <question>` — Multi-LLM consultation + synthesis
- `/matrix <context>` — Cross-product evaluation table
- `/factcheck` — Verify claims with web evidence

- `/code` — Run Python in-browser (Pyodide)
- `/flashcards` — Generate flashcards from content
- `/git <url>` — Fetch a Git repository
- `/youtube <url>` — Fetch YouTube video transcript
- `/fetch <url>` — Fetch URL content (HTML, PDF)
- `/note` — Create a note node

Type `/` in the input to see the autocomplete menu with all available commands.

Supplementary

## Plugin Levels in Detail

### Level 1: Custom Nodes

- Register via `node-registry.js`
- Custom rendering (HTML/CSS)
- Custom action buttons
- Custom data model
- No slash commands needed

Example: poll node, chart node

### Level 2: Feature Plugins

- Extend `FeaturePlugin` base class
- Slash commands via `getSlashCommands()`
- Multi-step LLM workflows
- AppContext dependency injection
- Canvas event handlers

Example: /committee, /matrix, /research

### Level 3: Extension Hooks

- Subscribe to plugin events
- Intercept operations
- Logging and validation
- Multiple listeners per event
- Enhance, don't replace

Example: smart-fix plugin

Plugins can be JS-only, Python-only, or paired (JS + Python). Configured in `config.yaml`.

Supplementary

## Keyboard Shortcuts

### Navigation

- `Cmd/Ctrl + F` — Search nodes
- `Escape` — Deselect all nodes
- `F` — Fit all nodes in view
- `L` — Auto-layout

### Selection

- `Cmd/Ctrl + Click` — Multi-select
- `Cmd/Ctrl + A` — Select all

### Node Actions

- `E` — Edit selected node
- `C` — Copy node content
- `Delete/Backspace` — Delete selected nodes
- `Cmd/Ctrl + Z` — Undo
- `Cmd/Ctrl + Shift + Z` — Redo

### Other

- `?` — Open help modal
- `N` — New note node

Click a slide to jump · Esc to close