Local News Multiverse Agent Platform

Built from scratch by Alec Meeker, CEO and Editor-in-Chief of Bushwick Daily, as both a learning exercise in distributed systems design and a mission-driven tool to democratize AI workflow development for local news organizations.

Related Open Source Projects

• Email Ingestion Data Warehouse
• Multiverse Agent Platform (this document)

The Mission

Local journalism is dying. Not because people don't care, but because small publications can't compete with the operational efficiency of large media companies. While The New York Times has teams of engineers building AI tools, your local newspaper is still manually sorting through emails and copy-pasting content between platforms.

This platform exists to change that.

The Multiverse Agent Orchestration Platform gives any local publication—regardless of technical resources—the same AI workflow infrastructure that billion-dollar media companies are building internally. And because it's 100% local and open-source, your data never leaves your servers, your workflows are yours forever, and you can share what you build with other publications facing the same challenges.

The Vision: A Cooperative AI Infrastructure for Local News

The database architecture of this platform wasn't designed just for one publication. Every workflow, prompt, and pipeline is hashed and versioned in a portable format. This means:

• Pipeline Sharing: A publication that builds a great "press release classifier" workflow can export it and share it with other users of the platform
• Collective Learning: Anonymized performance metrics can help the community identify which workflow patterns work best
• Reduced Duplication: Instead of every local publication reinventing the wheel, successful patterns can propagate
• Federated Evolution: Improvements made by one publication can benefit all

This is cooperative infrastructure for an industry that desperately needs it.

Why I Built This From Scratch

The Problem with LangGraph Cloud

LangGraph is excellent software. But LangGraph Cloud means:

• Your data goes to external servers
• You're locked into a vendor's pricing model
• You can't deeply customize the persistence layer
• You can't share workflows between organizations easily

For a local news publication handling sensitive source communications, tip submissions, and editorial workflows, cloud dependency is a non-starter.

What This Platform Does Differently

I built this platform to replicate LangGraph's core capabilities—state machines, checkpointing, human-in-the-loop interrupts—but with critical additions:

1. 100% Local: Everything runs on PostgreSQL. No external dependencies. Your data never leaves your infrastructure.
2. Multiverse Exploration: Instead of single-path execution, run up to 5 parallel strategy paths simultaneously and compare results before committing.
3. Complete Version Control: Every prompt, workflow, and agent configuration is immutably versioned. Nothing is ever overwritten. Full rollback capability forever.
4. Built-in Hypothesis Testing: A/B test workflow changes with statistical analysis before deploying to production.
5. Self-Evolving Agents: Automatic performance monitoring with evolution triggers that improve agents based on measured outcomes.
6. Portable Pipelines: Hash-based workflow identification means pipelines can be exported, shared, and imported across installations.

Technical Architecture

Design Principles

This platform is built on several fundamental principles that inform every architectural decision:

1. Immutability Over Mutation

Nothing in this system is ever overwritten. When you change a prompt, a new version is created. When you modify a workflow, a new version is branched. When an agent evolves, the before/after states are both preserved.

Why? Because AI systems fail in subtle ways. A prompt that worked yesterday might fail tomorrow due to model updates, data drift, or edge cases you haven't seen yet. When things break, you need to know exactly what changed and when—and you need to roll back instantly.

2. Complete Observability

Every input to every agent is logged. Every output is captured. Every decision point records the options considered and the choice made. Every execution captures timing, token usage, and error states.

Why? Because you can't improve what you can't measure. And when a user reports that "the AI did something weird last Tuesday," you need to be able to replay exactly what happened.

3. Parallel Exploration Before Commitment

Instead of running a workflow and hoping for the best, this platform can explore up to 5 different strategies simultaneously. Each path runs independently, with its own state. You compare results and select the winner—or merge the best elements from multiple paths.

Why? Because the "best" approach often isn't obvious until you've tried alternatives. And in production, you want to test changes on real data before committing to them.

4. Local-First, Always

Every piece of state lives in PostgreSQL. Checkpoints, key-value storage, workflow definitions, execution history—all local. The platform functions identically whether you're online or offline.

Why? Because vendor lock-in kills small organizations. Because network dependencies cause production failures. Because your data is yours.

5. Clean Architecture with Dependency Injection

The codebase follows strict layer separation: Domain (pure business logic) → Application (orchestration) → Infrastructure (external systems). All dependencies are injected via a central container.

Why? Because testability matters. Because you should be able to swap PostgreSQL for SQLite for testing without changing business logic. Because the codebase should remain maintainable as it grows.

System Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                           PRESENTATION LAYER                                │
│                                                                             │
│  ┌───────────────────────────────────────────────────────────────────┐   │
│  │                    STREAMLIT WORKBENCH (Port 8501)                   │   │
│  │  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐ │   │
│  │  │  Agent Forge │ │  Workflow   │ │   Prompt    │ │    Time    │ │   │
│  │  │              │ │   Builder    │ │   Manager   │ │   Machine  │ │   │
│  │  └──────────────┘ └──────────────┘ └──────────────┘ └─────────────┘ │   │
│  │  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐ │   │
│  │  │  Laboratory  │ │ Exploration │ │   HITL      │ │  Analytics │ │   │
│  │  │   (Testing)  │ │   Canvas    │ │   Review    │ │  Dashboard │ │   │
│  │  └──────────────┘ └──────────────┘ └──────────────┘ └─────────────┘ │   │
│  └───────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌───────────────────────────────────────────────────────────────────┐   │
│  │                      FastAPI SERVICES                                │   │
│  │  Exploration API (8001) | Workflow API (8002) | Prompt API (8003)    │   │
│  └───────────────────────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────────────────────────┘
                                      │
┌─────────────────────────────────────▼──────────────────────────────────────┐
│                           APPLICATION LAYER                                 │
│                                                                             │
│  ORCHESTRATORS: OrchestratorV3, Unified Orchestrator, Exploration Engine   │
│  SERVICES (16): AgentService, PromptService, WorkflowService, etc.         │
└────────────────────────────────────────────────────────────────────────────┘
                                      │
┌─────────────────────────────────────▼──────────────────────────────────────┐
│                            DOMAIN LAYER                                     │
│                                                                             │
│  AGENTS (12): BaseAgent, StrategicPlannerAgent, EAIAWorkbenchAdapter, etc. │
│  TOOLS & CAPABILITIES: RAG_SEARCH, DATA_QUERY, ENTITY_LOOKUP, CALCULATION  │
└────────────────────────────────────────────────────────────────────────────┘
                                      │
┌─────────────────────────────────────▼──────────────────────────────────────┐
│                        INFRASTRUCTURE LAYER                                 │
│                                                                             │
│  DEPENDENCY INJECTION: Container, lazy initialization, config via env      │
│  REPOSITORIES (17): PromptRepo, WorkflowRepo, AgentRepo, ExecutionRepo...  │
│  PERSISTENCE: PostgreSQLCheckpointer, PostgreSQLStore (LangGraph compat)   │
│  DATABASE: PostgreSQL (18 migrations, 30+ tables)                          │
└────────────────────────────────────────────────────────────────────────────┘

Database Architecture

The database is the foundation of the entire system. It's designed around four core subsystems, each implementing specific capabilities that together create a complete MLOps platform for agentic workflows.

Subsystem 1: Prompt Version Control

Tables: prompt_templates, prompt_versions, prompt_execution_logs

• Immutable Versions: The version column is auto-incremented and never updated. When you "edit" a prompt, you create version N+1.
• Single Active Version: Only one version per template can have is_active = TRUE. Switching versions is atomic.
• Complete Execution Logging: Every single LLM call is logged with full input/output in JSONB for debugging, performance comparison, and cost tracking.

Subsystem 2: Multiverse Workflow Engine

Tables: mv_workflow_versions, mv_execution_contexts, mv_exploration_paths, mv_path_execution_history, mv_workflow_audit_trail

• Git-Like Workflow Versioning: The parent_version foreign key creates a directed acyclic graph of workflow history. Branch, experiment, and merge.
• UUID-Based Execution Contexts: Each email processing session gets a UUID-identified context, enabling tracking across service restarts.
• Parallel Path Exploration: Up to 5 mv_exploration_paths can be active per context, each with independent state.
• Node-Level Granularity: mv_path_execution_history captures every single node execution for exact replay capability.

Subsystem 3: Agent Evolution & Intelligence

Tables: mv_intelligent_agents, mv_agent_evolution_history, mv_recognized_patterns, mv_hypothesis_tests, mv_system_capabilities

• Self-Describing Agents: Each agent stores its own schema definitions (input_schema, output_schema) in JSONB for runtime validation.
• Evolution Lineage: Every agent modification creates a record with before/after snapshots, trigger cause, and performance delta.
• Built-in Hypothesis Testing: A/B test workflow changes with statistical analysis before deployment.

Subsystem 4: LangGraph Local Storage

Tables: local_threads, thread_state_history, langgraph_store, thread_interrupts

• LangGraph API Compatibility: Maintains compatibility with LangGraph's thread ID format for easy migration.
• Full Checkpoint History: Stores every checkpoint for time-travel debugging and audit compliance.
• Hierarchical Key-Value Store: Uses namespace TEXT[] with GIN index for hierarchical organization.

The Exploration Engine: Parallel Universe Processing

The exploration engine is the core innovation of this platform. It implements true parallel strategy exploration using Python's asyncio.

class ExplorationSession:
    def __init__(self, ...):
        # Parallel execution management
        self.path_queue: Queue = Queue()
        self.path_tasks: dict[UUID, Task] = {}
        self.max_parallel_paths = 5
        self.exploration_semaphore = Semaphore(self.max_parallel_paths)

        # Data lineage tracking
        self.data_manifest = DataManifest(email_id=email_id)
        self.lineage_tracker = DataLineageTracker()

1. Semaphore-Controlled Parallelism: Up to 5 paths can execute simultaneously, with asyncio semaphore preventing resource exhaustion.
2. Independent State Per Path: Each ExplorationPath has its own path_state JSONB. Modifications to one path don't affect others.
3. Data Lineage Tracking: Every piece of data is tracked from origin through transformations.
4. Intervention Points: The engine supports 10 types of user interventions mid-execution:

   CONTINUE, MODIFY_INPUT, MODIFY_NODE, ADD_NODE, REMOVE_NODE,
   CHANGE_ROUTE, CREATE_BRANCH, PAUSE, ABANDON, MODIFY_WORKFLOW

Why Database-Driven?

This platform is built for journalists, not engineers.

Every agent definition, workflow configuration, prompt template, and tool parameter lives in PostgreSQL—not in Python code. This means:

For Non-Technical Users

• Create agents through a web UI: The Streamlit Workbench provides forms, dropdowns, and visual editors. No code required.
• Edit prompts like documents: Change how your AI behaves by editing text in a browser.
• See your changes instantly: Click save, and your agent uses the new configuration. No deployment, no restart.
• Roll back mistakes: Every change is versioned. One click to restore.

For Technical Users

• Agents are data, not code: Export a working agent as JSON. Import it on another server. Share it.
• Schema validation at runtime: Input/output schemas stored as JSONB are validated when agents execute.
• Full audit trail: Every modification is logged with timestamp, context, and before/after state.

The Streamlit Workbench

The UI was designed with a specific user in mind: a local news editor who is technical enough to understand what AI can do, but doesn't have time to write code.

Components

• Agent Forge: Create and edit intelligent agents with visual schema editors. Define inputs, outputs, capabilities, and model settings without writing code.
• Workflow Builder: Drag-and-drop workflow construction with real-time Graphviz visualization.
• Prompt Manager: Full version control UI. See diffs, roll back, compare performance.
• Exploration Canvas: Watch parallel paths execute in real-time. Pause, modify, branch at any point.
• Tool Manager: Register new tools that agents can use.
• Time Machine: Browse any historical state. Replay executions.
• Laboratory: Test prompts with sample data before deployment.
• Analytics Dashboard: Performance metrics across all agents, workflows, and prompts.

Quick Start

Prerequisites

• Python 3.11+
• PostgreSQL 14+
• OpenAI API key (or Anthropic)

Installation

# Clone the repository
git clone https://github.com/alecmeeeker/local-news-multiverse-agent-platform.git
cd local-news-multiverse-agent-platform

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Copy environment template and configure
cp .env.example .env
# Edit .env with your API keys and database credentials

Database Setup

# Create the database
createdb agent_db

# Run migrations in order
for f in migrations/*.sql; do
    echo "Running $f..."
    psql -d agent_db -f "$f" 2>&1 | grep -v "already exists"
done

The migrations will:

1. Create the email stub table with sample NYC press releases for testing
2. Set up the multiverse schema (agents, workflows, exploration paths)
3. Configure LangGraph-compatible storage
4. Seed 3 starter agents (EmailTriageAgent, PressReleaseAnalyzer, StoryDraftGenerator)
5. Create a sample workflow connecting them

Start the Platform

# Start all services (APIs + UI)
./start_system.sh

# Or start components individually:

# API servers
python run_production_system.py

# Streamlit UI (separate terminal)
streamlit run ui/agents_v3_workbench.py --server.port 8501

Access Points

Technology Stack

Future Possibilities

Pipeline Marketplace for Local News

Because every workflow is hashed and versioned:

• A publication that builds a great "press release classifier" can share it
• Other publications import with one command
• Prompts can be customized for local voice
• Performance improvements can be federated back

Collective Learning

With opt-in anonymized metrics sharing:

• Which workflow patterns perform best across publications?
• What prompt structures have the highest success rates?
• Where are the common failure modes?

Contributing

This is an open-source project built for the local news community. Contributions are welcome:

1. Bug Reports: Open an issue with reproduction steps
2. Feature Requests: Open an issue describing the use case
3. Pull Requests: Fork, branch, and submit PRs against develop
4. Documentation: Help improve docs, examples, tutorials

Related Projects

• Email Ingestion Data Warehouse — Upstream data source for this platform
• NYCNewsScanner Report — The production system this was extracted from

Built with determination in Brooklyn, NYC.