Deep-research
Your AI-Powered Research Assistant. Conduct iterative, deep research using search engines, web scraping, and Gemini LLMs, all within a lightweight and understandable codebase.
This tool uses Firecrawl for efficient web data extraction and Gemini for advanced language understanding and report generation.
The goal of this project is to provide the simplest yet most effective implementation of a deep research agent. It's designed to be easily understood, modified, and extended, aiming for a codebase under 500 lines of code (LoC).
Key Features:
- MCP Integration: Seamlessly integrates as a Model Context Protocol (MCP) tool into AI agent ecosystems.
- Iterative Deep Dive: Explores topics deeply through iterative query refinement and result processing.
- Gemini-Powered Queries: Leverages Gemini LLMs to generate smart, targeted search queries.
- Depth & Breadth Control: Configurable depth and breadth parameters for precise research scope.
- Smart Follow-up Questions: Intelligently generates follow-up questions for query refinement.
- Comprehensive Markdown Reports: Generates detailed, ready-to-use Markdown reports.
- Concurrent Processing for Speed: Maximizes research efficiency with parallel processing.
Workflow Diagram
Persona Agents in open-deep-research
What are Persona Agents?
In
deep-research, we utilize the concept of "persona agents" to guide the behavior of the Gemini language models. Instead of simply prompting the LLM with a task, we imbue it with a specific role, skills, personality, communication style, and values. This approach helps to:- Focus the LLM's Output: By defining a clear persona, we encourage the LLM to generate responses that are aligned with the desired expertise and perspective.
- Improve Consistency: Personas help maintain a consistent tone and style throughout the research process.
- Enhance Task-Specific Performance: Tailoring the persona to the specific task (e.g., query generation, learning extraction, feedback) optimizes the LLM's output for that stage of the research.
Examples of Personas in use:
- Expert Research Strategist & Query Generator: Used for generating search queries, this persona emphasizes strategic thinking, comprehensive coverage, and precision in query formulation.
- Expert Research Assistant & Insight Extractor: When processing web page content, this persona focuses on meticulous analysis, factual accuracy, and extracting key learnings relevant to the research query.
- Expert Research Query Refiner & Strategic Advisor: For generating follow-up questions, this persona embodies strategic thinking, user intent understanding, and the ability to guide users towards clearer and more effective research questions.
- Professional Doctorate Level Researcher (System Prompt): This overarching persona, applied to the main system prompt, sets the tone for the entire research process, emphasizing expert-level analysis, logical structure, and in-depth investigation.
By leveraging persona agents,
deep-research aims to achieve more targeted, consistent, and high-quality research outcomes from the Gemini language models.How It Works
Features
- MCP Integration: Available as a Model Context Protocol tool for seamless integration with AI agents
- Iterative Research: Performs deep research by iteratively generating search queries, processing results, and diving deeper based on findings
- Intelligent Query Generation: Uses Gemini LLMs to generate targeted search queries based on research goals and previous findings
- Depth & Breadth Control: Configurable parameters to control how wide (breadth) and deep (depth) the research goes
- Smart Follow-up: Generates follow-up questions to better understand research needs
- Comprehensive Reports: Produces detailed markdown reports with findings and sources
- Concurrent Processing: Handles multiple searches and result processing in parallel for efficiency
Requirements
- Node.js environment (v22.x recommended)
- API keys for:
Setup
Node.js
- Clone the repository:
- Install dependencies:
- Set up environment variables: Create a
.env.localfile in the project root and add your API keys:
- Build the project:
Usage
As MCP Tool
To run
deep-research as an MCP tool, start the MCP server:You can then invoke the
deep-research tool from any MCP-compatible agent using the following parameters:query(string, required): The research query.
depth(number, optional, 1-5): Research depth (default: moderate).
breadth(number, optional, 1-5): Research breadth (default: moderate).
existingLearnings(string[], optional): Pre-existing research findings to guide research.
Example MCP Tool Invocation (TypeScript):
Standalone CLI Usage
To run
deep-research directly from the command line:Example:
MCP Inspector Testing
For interactive testing and debugging of the MCP server, use the MCP Inspector:
License
MIT License - Free and Open Source. Use it freely!
** Let's dive deep into research!
Recent Improvements (v0.2.0)
Enhanced Research Validation:
- Added academic input/output validation
- Input validation: Minimum 10 characters + 3 words
- Output validation: Citation density (1.5+ per 100 words)
- Recent sources check (3+ post-2019 references)
- Conflict disclosure enforcement
Gemini Integration Upgrades:
- Embedded Gemini analysis in research workflow
- Integrated Gemini Flash 2.0 for faster processing
- Added semantic text splitting for LLM context management
- Improved error handling for API calls
Code Quality Improvements:
- Added concurrent processing pipeline
- Removed redundant academic-validators module
- Enhanced type safety across interfaces
- Optimized dependencies (30% smaller node_modules)
New Features:
- Research metrics tracking (sources/learnings ratio)
- Auto-generated conflict disclosure statements
- Recursive research depth control (1-5 levels)
- Research metrics tracking (sources/learnings ratio)
- MCP tool integration improvements
Performance:
- 30% faster research cycles
- 40% faster initial research cycles
- 60% reduction in API errors
- 25% more efficient token usage
Deep-research
Your AI-Powered Research Assistant. Conduct iterative, deep research using search engines, web scraping, and Gemini LLMs, all within a lightweight and understandable codebase.
This tool uses Firecrawl for efficient web data extraction and Gemini for advanced language understanding and report generation.
The goal of this project is to provide the simplest yet most effective implementation of a deep research agent. It's designed to be easily understood, modified, and extended, aiming for a codebase under 500 lines of code (LoC).
Key Features:
- MCP Integration: Seamlessly integrates as a Model Context Protocol (MCP) tool into AI agent ecosystems.
- Iterative Deep Dive: Explores topics deeply through iterative query refinement and result processing.
- Gemini-Powered Queries: Leverages Gemini LLMs to generate smart, targeted search queries.
- Depth & Breadth Control: Configurable depth and breadth parameters for precise research scope.
- Smart Follow-up Questions: Intelligently generates follow-up questions for query refinement.
- Comprehensive Markdown Reports: Generates detailed, ready-to-use Markdown reports.
- Concurrent Processing for Speed: Maximizes research efficiency with parallel processing.
Workflow Diagram
Persona Agents in open-deep-research
What are Persona Agents?
In
deep-research, we utilize the concept of "persona agents" to guide the behavior of the Gemini language models. Instead of simply prompting the LLM with a task, we imbue it with a specific role, skills, personality, communication style, and values. This approach helps to:- Focus the LLM's Output: By defining a clear persona, we encourage the LLM to generate responses that are aligned with the desired expertise and perspective.
- Improve Consistency: Personas help maintain a consistent tone and style throughout the research process.
- Enhance Task-Specific Performance: Tailoring the persona to the specific task (e.g., query generation, learning extraction, feedback) optimizes the LLM's output for that stage of the research.
Examples of Personas in use:
- Expert Research Strategist & Query Generator: Used for generating search queries, this persona emphasizes strategic thinking, comprehensive coverage, and precision in query formulation.
- Expert Research Assistant & Insight Extractor: When processing web page content, this persona focuses on meticulous analysis, factual accuracy, and extracting key learnings relevant to the research query.
- Expert Research Query Refiner & Strategic Advisor: For generating follow-up questions, this persona embodies strategic thinking, user intent understanding, and the ability to guide users towards clearer and more effective research questions.
- Professional Doctorate Level Researcher (System Prompt): This overarching persona, applied to the main system prompt, sets the tone for the entire research process, emphasizing expert-level analysis, logical structure, and in-depth investigation.
By leveraging persona agents,
deep-research aims to achieve more targeted, consistent, and high-quality research outcomes from the Gemini language models.How It Works
Features
- MCP Integration: Available as a Model Context Protocol tool for seamless integration with AI agents
- Iterative Research: Performs deep research by iteratively generating search queries, processing results, and diving deeper based on findings
- Intelligent Query Generation: Uses Gemini LLMs to generate targeted search queries based on research goals and previous findings
- Depth & Breadth Control: Configurable parameters to control how wide (breadth) and deep (depth) the research goes
- Smart Follow-up: Generates follow-up questions to better understand research needs
- Comprehensive Reports: Produces detailed markdown reports with findings and sources
- Concurrent Processing: Handles multiple searches and result processing in parallel for efficiency
Requirements
- Node.js environment (v22.x recommended)
- API keys for:
Setup
Node.js
- Clone the repository:
- Install dependencies:
- Set up environment variables: Create a
.env.localfile in the project root and add your API keys:
- Build the project:
Usage
As MCP Tool
To run
deep-research as an MCP tool, start the MCP server:You can then invoke the
deep-research tool from any MCP-compatible agent using the following parameters:query(string, required): The research query.
depth(number, optional, 1-5): Research depth (default: moderate).
breadth(number, optional, 1-5): Research breadth (default: moderate).
existingLearnings(string[], optional): Pre-existing research findings to guide research.
Example MCP Tool Invocation (TypeScript):
Standalone CLI Usage
To run
deep-research directly from the command line:Example:
MCP Inspector Testing
For interactive testing and debugging of the MCP server, use the MCP Inspector:
License
MIT License - Free and Open Source. Use it freely!
** Let's dive deep into research!
Recent Improvements (v0.2.0)
Enhanced Research Validation:
- Added academic input/output validation
- Input validation: Minimum 10 characters + 3 words
- Output validation: Citation density (1.5+ per 100 words)
- Recent sources check (3+ post-2019 references)
- Conflict disclosure enforcement
Gemini Integration Upgrades:
- Embedded Gemini analysis in research workflow
- Integrated Gemini Flash 2.0 for faster processing
- Added semantic text splitting for LLM context management
- Improved error handling for API calls
Code Quality Improvements:
- Added concurrent processing pipeline
- Removed redundant academic-validators module
- Enhanced type safety across interfaces
- Optimized dependencies (30% smaller node_modules)
New Features:
- Research metrics tracking (sources/learnings ratio)
- Auto-generated conflict disclosure statements
- Recursive research depth control (1-5 levels)
- Research metrics tracking (sources/learnings ratio)
- MCP tool integration improvements
Performance:
- 30% faster research cycles
- 40% faster initial research cycles
- 60% reduction in API errors
- 25% more efficient token usage