Vendor: stanford-oval
STORM is Stanford's open-source LLM-powered knowledge curation system that autonomously researches topics through multi-perspective question asking, iterative retrieval, and structured report generation with citations, now extended with Co-STORM for human-AI collaborative workflows.
| Repository | stanford-oval/storm |
|---|---|
| GitHub Stars | ★ 31.1k |
| Forks | 2.9k forks |
| Primary Language | Python |
| License | MIT |
| Technical Domain | AGENTS |
$ pip install knowledge-storm
STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking) represents a significant architectural advancement in autonomous research agents. Developed at Stanford's OVAL lab, it addresses the fundamental challenge of generating comprehensive, well-structured long-form reports from open-domain knowledge without human-guided search queries. The system's core innovation lies in its multi-perspective question-asking paradigm: rather than relying on a single query trajectory, STORM decomposes a topic into multiple viewpoints (simulated expert personas), generates targeted questions from each perspective, and uses those questions to drive retrieval. This approach dramatically reduces the information gap between what the LLM knows and what it needs to know, producing richer and more balanced knowledge bases. The architecture separates concerns cleanly into retrieval modules (RM), knowledge curation pipelines, and report generation stages, with a modular design supporting multiple search backends including You.com, Bing, and vector-based retrieval over user-provided documents. The Co-STORM extension introduces a collaborative loop where human users can intervene, correct, and guide the research process, addressing a critical limitation of fully autonomous agents. With 31,000+ GitHub stars and active development including litellm integration for model flexibility, STORM has become a reference implementation for retrieval-augmented long-form content generation, demonstrating that structured prompt orchestration combined with iterative retrieval can produce publication-quality research reports with proper citations.
STORM emerged from Stanford's OVAL (Open-Access Virtual Assistant Lab) research group, driven by the observation that existing RAG (Retrieval-Augmented Generation) systems suffer from a fundamental limitation: they rely on user-provided queries that may not capture the full breadth of information needed for comprehensive topic understanding. The design philosophy centers on the insight that the quality of a research report is bounded by the quality and diversity of questions asked during the retrieval phase. By simulating multiple expert personas—each with distinct perspectives on a topic—STORM generates a diverse set of questions that collectively cover the topic space more thoroughly than any single query trajectory could achieve. This multi-perspective approach is implemented through a structured prompt pipeline where the LLM first identifies relevant expert roles, then generates questions from each role's viewpoint, and finally uses those questions to drive iterative retrieval cycles.
The architectural breakthrough lies in the separation of the knowledge curation phase from the report generation phase, with a well-defined intermediate representation (the knowledge base) that captures retrieved information organized by topic and perspective. This design enables several key properties: the knowledge base can be inspected and modified by humans (as in Co-STORM), the retrieval process can be parallelized across perspectives, and the final report generation benefits from a structured, citation-ready knowledge graph rather than unstructured retrieved chunks. The system's modular retrieval interface (RM) abstracts away search engine specifics, allowing seamless switching between You.com's YouRM, Bing's BingSearchRM, and VectorRM for private document grounding, making the architecture adaptable to different deployment contexts and data access requirements.
The Co-STORM extension, published at EMNLP 2024, addresses the inherent limitations of fully autonomous research by introducing a collaborative loop. In this paradigm, the human user acts as a domain expert who can approve, reject, or modify the questions generated by the AI agent, effectively steering the research direction while the AI handles the mechanical work of retrieval and synthesis. This human-AI collaboration model is architecturally significant because it demonstrates that the multi-perspective question-asking framework can serve as a shared interface between human and machine intelligence, where both parties contribute to shaping the knowledge base that ultimately feeds the report generation stage.
Enterprise knowledge synthesis represents STORM's primary production use case. Organizations can deploy STORM to automatically generate comprehensive market research reports, competitive analyses, and industry briefings by pointing it at a topic and leveraging its multi-perspective retrieval to gather information from diverse web sources. The citation-aware report generation ensures that every claim in the output can be traced back to a source, which is critical for enterprise compliance and fact-checking workflows. The VectorRM integration further enables enterprises to ground research on internal documents, proprietary databases, or curated knowledge bases, making it suitable for scenarios where web search alone is insufficient or where data privacy requires closed-domain retrieval.
Academic and research institutions can leverage STORM as an automated literature review assistant. Given a research topic, STORM's multi-perspective approach naturally maps to the different subfields and methodological approaches that characterize academic disciplines. The system can generate structured topic outlines that serve as starting points for literature surveys, with each section grounded in retrieved sources. The Co-STORM mode is particularly valuable here, as domain experts can guide the AI to focus on specific research questions, methodologies, or theoretical frameworks, effectively using the system as an intelligent research assistant that handles the tedious work of search and organization while the expert provides strategic direction.
Technical documentation and knowledge base generation is another compelling use case. Engineering teams can use STORM to generate comprehensive technical reports on complex subjects—whether explaining a new technology, documenting architectural decisions, or creating onboarding materials. The system's ability to decompose topics into structured outlines with proper citations makes it suitable for producing documentation that meets professional standards. The modular architecture allows teams to integrate STORM into existing CI/CD pipelines or documentation workflows, with the Streamlit-based demo light providing a starting point for custom UI development.
Human-AI collaborative research workflows enabled by Co-STORM open up scenarios where the quality of output depends on domain expertise that cannot be fully encoded in prompts. In legal research, medical literature synthesis, or policy analysis, a domain expert working alongside STORM can ensure that the AI's retrieval covers the right questions while benefiting from the AI's ability to rapidly gather and organize information from hundreds of sources. This collaborative model is architecturally elegant because it treats the question-asking interface as a shared workspace where human and AI contributions are merged into a unified knowledge base.
Installation is straightforward via pip: pip install knowledge-storm. The package includes all core components including the retrieval modules, knowledge curation pipeline, and report generation stages. For model configuration, STORM now supports litellm integration (v1.1.0+), which provides a unified interface to 100+ LLM providers. A minimal configuration involves setting environment variables for your preferred LLM provider (e.g., OPENAI_API_KEY for GPT-4o) and specifying the model in the StormGeneratorLM configuration. The system supports separate model configurations for different stages—question generation, retrieval, and report writing—allowing users to optimize cost-performance tradeoffs by using cheaper models for retrieval and premium models for final report generation.
A basic usage example involves instantiating a StormDataStore for knowledge persistence, configuring a retrieval module (e.g., BingSearchRM with your Bing API key or VectorRM for local documents), and running the run_storm pipeline with a topic string. The pipeline executes through several stages: topic exploration (generating initial questions and perspectives), knowledge base construction (iterative retrieval and organization), and report generation (synthesizing the knowledge base into a structured document with citations). For Co-STORM, the run_co_storm function provides a similar interface but includes human-in-the-loop checkpoints where the user can review and modify questions before retrieval proceeds. The Streamlit demo light (demo_light.py) provides a minimal web interface for interactive exploration without requiring custom frontend development.
For production deployment, users should configure the StormGeneratorLM and StormChatLM classes with appropriate model parameters, set up the retrieval module with proper API credentials, and optionally configure a VectorRM with an embedding model for private document grounding. The system's modular architecture means that each component can be independently configured and tested. Example configuration for Bing search retrieval: from knowledge_storm.rm import BingSearchRM; rm = BingSearchRM(bing_search_api_key='YOUR_KEY', top_k=10). For VectorRM with user documents: from knowledge_storm.rm import VectorRM; rm = VectorRM(embedding_model='text-embedding-3-small', documents=['path/to/docs/']). The knowledge store can be persisted to disk using JsonStormDataStore for stateful multi-session research workflows.
STORM demonstrates strong production readiness with its clean modular architecture, comprehensive test coverage, and active development cadence. The separation of retrieval, knowledge curation, and report generation into independent modules enables straightforward debugging and component-level optimization. The litellm integration (v1.1.0) significantly improves deployment flexibility by allowing organizations to use whatever LLM infrastructure they already have, whether that's OpenAI, Anthropic, self-hosted models via vLLM, or enterprise API gateways. The package-based distribution (knowledge-storm on PyPI) simplifies integration into existing Python-based data pipelines and ML infrastructure. However, users should be aware that the system's multi-stage pipeline can be computationally expensive: a single research run may involve dozens of LLM calls for question generation, retrieval, and synthesis, making token budget management critical for cost-sensitive deployments.
Scalability considerations center on the retrieval phase, which can be parallelized across perspectives but is bounded by search API rate limits and latency. The VectorRM option addresses this for closed-domain scenarios where retrieval is a local vector similarity search rather than an external API call. Latency for a full research report generation typically ranges from 5-20 minutes depending on topic complexity, number of perspectives, and LLM speed, which is acceptable for asynchronous report generation workflows but not suitable for real-time interactive use cases. Security considerations are minimal for the core system since it operates as a data processing pipeline without sandboxing requirements, though organizations should ensure that retrieved web content is appropriately filtered and that API keys for search engines and LLM providers are properly secured. The main caveat is that report quality is inherently dependent on retrieval quality—if the search backend returns irrelevant or low-quality results, the downstream synthesis will reflect those limitations regardless of LLM capability.
Debugging the multi-stage pipeline benefits from the structured intermediate representations: the knowledge base can be inspected at any point to verify that retrieval is producing relevant content, and the topic outline can be reviewed before report generation to ensure proper structure. The Co-STORM mode provides an additional debugging layer by allowing human review of questions before retrieval. For production monitoring, users should track metrics at each pipeline stage: question diversity (number of unique perspectives), retrieval hit rate (relevance of returned documents), knowledge base coverage (breadth of topics covered), and report quality (citation density, structural coherence). The system's MIT license and active community (31,000+ stars) ensure that bugs are rapidly identified and fixed, and the well-documented API surface makes it straightforward to extend or modify individual components for specific deployment requirements.
STORM has achieved significant ecosystem adoption with over 31,000 GitHub stars, making it one of the most popular open-source AI agent projects. The research preview at storm.genie.stanford.edu demonstrates the system's capabilities through an interactive web interface where users can input topics and receive full-length research reports with citations. The system has been featured in academic presentations at NAACL 2024 and its Co-STORM extension was accepted to EMNLP 2024 main conference, validating the research contributions behind the open-source implementation. The rapid iteration cycle—from initial release through refactored codebase, Bing Search support, VectorRM integration, Co-STORM release, and litellm integration—demonstrates a healthy development velocity that keeps the project at the forefront of autonomous research agent capabilities.
The integration with litellm in v1.1.0 represents a significant real-world adoption signal, as it enables organizations using diverse LLM infrastructure to leverage STORM without vendor lock-in. This is particularly important for enterprises that may use self-hosted open-source models (Llama, Mixtral) for cost reasons or regulatory compliance, while still benefiting from STORM's sophisticated multi-perspective research architecture. The Streamlit-based demo light provides a low-friction entry point for developers to experiment with the system, while the modular Python API enables deeper integration into custom applications. The VectorRM addition addresses a critical enterprise requirement: the ability to ground research on proprietary documents rather than relying solely on public web search, making STORM viable for internal knowledge synthesis tasks where data privacy and domain specificity are paramount.
Community contributions and third-party integrations have expanded STORM's applicability beyond its original academic research focus. The modular retrieval interface has enabled community members to implement custom RM backends for specialized search engines, internal knowledge bases, and academic databases. The Co-STORM collaborative mode has inspired similar human-AI collaboration patterns in other agent frameworks, establishing STORM as a reference architecture for retrieval-augmented long-form content generation. The project's clean separation of concerns—retrieval, knowledge curation, and generation—has made it a teaching tool for understanding complex multi-agent architectures, with its design patterns being referenced in discussions of agent system architecture across the AI community.
storm is an open-source AI project developed primarily in Python under the MIT license. STORM is Stanford's open-source LLM-powered knowledge curation system that autonomously researches topics through multi-perspective question asking, iterative retrieval, and structured report generation with citations, now extended with Co-STORM for human-AI collaborative workflows.. STORM emerged from Stanford's OVAL (Open-Access Virtual Assistant Lab) research group, driven by the observation that existing RAG (Retrieval-Augmented Generation) systems suffer from a fundamental limitation: they rely on user-provided queries that may not capture the full breadth of information needed for comprehensive topic understanding. The design philosophy centers on the insight that the quality of a research report is bounded by the quality and diversity of questions asked during the retrieval phase. By simulating multiple expert personas—each with distinct perspectives on a topic—STORM generates a diverse set of questions that collectively cover the topic space more thoroughly than any single query trajectory could achieve. This multi-perspective approach is implemented through a structured prompt pipeline where the LLM first identifies relevant expert roles, then generates questions from each role's viewpoint, and finally uses those questions to drive iterative retrieval cycles. The architectural breakthrough lies in the separation of the knowledge curation phase from the report generation phase, with a well-defined intermediate representation (the knowledge base) that captures retrieved information organized by topic and perspective. This design enables several key properties: the knowledge base can be inspected and modified by humans (as in Co-STORM), the retrieval process can be parallelized across perspectives, and the final report generation benefits from a structured, citation-ready knowledge graph rather than unstructured retrieved chunks. The system's modular retrieval interface (RM) abstracts away search engine specifics, allowing seamless switching between You.com's YouRM, Bing's BingSearchRM, and VectorRM for private document grounding, making the architecture adaptable to different deployment contexts and data access requirements. The Co-STORM extension, published at EMNLP 2024, addresses the inherent limitations of fully autonomous research by introducing a collaborative loop. In this paradigm, the human user acts as a domain expert who can approve, reject, or modify the questions generated by the AI agent, effectively steering the research direction while the AI handles the mechanical work of retrieval and synthesis. This human-AI collaboration model is architecturally significant because it demonstrates that the multi-perspective question-asking framework can serve as a shared interface between human and machine intelligence, where both parties contribute to shaping the knowledge base that ultimately feeds the report generation stage.
Installation is straightforward via pip: pip install knowledge-storm. The package includes all core components including the retrieval modules, knowledge curation pipeline, and report generation stages. For model configuration, STORM now supports litellm integration (v1.1.0+), which provides a unified interface to 100+ LLM providers. A minimal configuration involves setting environment variables for your preferred LLM provider (e.g., OPENAI_API_KEY for GPT-4o) and specifying the model in the StormGeneratorLM configuration. The system supports separate model configurations for different stages—question generation, retrieval, and report writing—allowing users to optimize cost-performance tradeoffs by using cheaper models for retrieval and premium models for final report generation. A basic usage example involves instantiating a StormDataStore for knowledge persistence, configuring a retrieval module (e.g., BingSearchRM with your Bing API key or VectorRM for local documents), and running the run_storm pipeline with a topic string. The pipeline executes through several stages: topic exploration (generating initial questions and perspectives), knowledge base construction (iterative retrieval and organization), and report generation (synthesizing the knowledge base into a structured document with citations). For Co-STORM, the run_co_storm function provides a similar interface but includes human-in-the-loop checkpoints where the user can review and modify questions before retrieval proceeds. The Streamlit demo light (demo_light.py) provides a minimal web interface for interactive exploration without requiring custom frontend development. For production deployment, users should configure the StormGeneratorLM and StormChatLM classes with appropriate model parameters, set up the retrieval module with proper API credentials, and optionally configure a VectorRM with an embedding model for private document grounding. The system's modular architecture means that each component can be independently configured and tested. Example configuration for Bing search retrieval: from knowledge_storm.rm import BingSearchRM; rm = BingSearchRM(bing_search_api_key='YOUR_KEY', top_k=10). For VectorRM with user documents: from knowledge_storm.rm import VectorRM; rm = VectorRM(embedding_model='text-embedding-3-small', documents=['path/to/docs/']). The knowledge store can be persisted to disk using JsonStormDataStore for stateful multi-session research workflows.
storm is well-suited for Enterprise knowledge synthesis: Automatically generating comprehensive market research reports, competitive analyses, and industry briefings from web sources with proper citations, Academic research assistance: Producing structured literature reviews and topic surveys by synthesizing information across multiple academic perspectives, Technical documentation generation: Creating in-depth technical reports on complex subjects by autonomously researching and organizing information from diverse sources, Human-AI collaborative research: Using Co-STORM to pair domain experts with AI agents for iterative knowledge curation where humans guide research direction. With an overall rating of 4.8/5, it offers strong community activity, reliable performance, and easy integration with existing AI pipelines.
STORM demonstrates strong production readiness with its clean modular architecture, comprehensive test coverage, and active development cadence. The separation of retrieval, knowledge curation, and report generation into independent modules enables straightforward debugging and component-level optimization. The litellm integration (v1.1.0) significantly improves deployment flexibility by allowing organizations to use whatever LLM infrastructure they already have, whether that's OpenAI, Anthropic, self-hosted models via vLLM, or enterprise API gateways. The package-based distribution (knowledge-storm on PyPI) simplifies integration into existing Python-based data pipelines and ML infrastructure. However, users should be aware that the system's multi-stage pipeline can be computationally expensive: a single research run may involve dozens of LLM calls for question generation, retrieval, and synthesis, making token budget management critical for cost-sensitive deployments. Scalability considerations center on the retrieval phase, which can be parallelized across perspectives but is bounded by search API rate limits and latency. The VectorRM option addresses this for closed-domain scenarios where retrieval is a local vector similarity search rather than an external API call. Latency for a full research report generation typically ranges from 5-20 minutes depending on topic complexity, number of perspectives, and LLM speed, which is acceptable for asynchronous report generation workflows but not suitable for real-time interactive use cases. Security considerations are minimal for the core system since it operates as a data processing pipeline without sandboxing requirements, though organizations should ensure that retrieved web content is appropriately filtered and that API keys for search engines and LLM providers are properly secured. The main caveat is that report quality is inherently dependent on retrieval quality—if the search backend returns irrelevant or low-quality results, the downstream synthesis will reflect those limitations regardless of LLM capability. Debugging the multi-stage pipeline benefits from the structured intermediate representations: the knowledge base can be inspected at any point to verify that retrieval is producing relevant content, and the topic outline can be reviewed before report generation to ensure proper structure. The Co-STORM mode provides an additional debugging layer by allowing human review of questions before retrieval. For production monitoring, users should track metrics at each pipeline stage: question diversity (number of unique perspectives), retrieval hit rate (relevance of returned documents), knowledge base coverage (breadth of topics covered), and report quality (citation density, structural coherence). The system's MIT license and active community (31,000+ stars) ensure that bugs are rapidly identified and fixed, and the well-documented API surface makes it straightforward to extend or modify individual components for specific deployment requirements.
Minimal tool for running large language models locally