Cloudflare Browser Rendering Experiments & MCP Server
This project demonstrates how to use Cloudflare Browser Rendering to extract web content for LLM context. It includes experiments with the REST API and Workers Binding API, as well as an MCP server implementation that can be used to provide web context to LLMs.
Project Structure
Prerequisites
- Node.js (v16 or later)
- A Cloudflare account with Browser Rendering enabled
- TypeScript
- Wrangler CLI (for deploying the Worker)
Installation
- Clone the repository:
- Install dependencies:
Cloudflare Worker Setup
- Install the Cloudflare Puppeteer package:
- Configure Wrangler:
- Deploy the Worker:
- Test the Worker:
Running the Experiments
Basic REST API Experiment
This experiment demonstrates how to use the Cloudflare Browser Rendering REST API to fetch and process web content:
Puppeteer Binding API Experiment
This experiment demonstrates how to use the Cloudflare Browser Rendering Workers Binding API with Puppeteer for more advanced browser automation:
Content Extraction Experiment
This experiment demonstrates how to extract and process web content specifically for use as context in LLMs:
MCP Server
The MCP server provides tools for fetching and processing web content using Cloudflare Browser Rendering for use as context in LLMs.
Building the MCP Server
Running the MCP Server
Or, for development:
MCP Server Tools
The MCP server provides the following tools:
- `fetch_page` - Fetches and processes a web page for LLM context
- `search_documentation` - Searches Cloudflare documentation and returns relevant content
- `extract_structured_content` - Extracts structured content from a web page using CSS selectors
- `summarize_content` - Summarizes web content for more concise LLM context
Configuration
To use your Cloudflare Browser Rendering endpoint, set the `BROWSER_RENDERING_API` environment variable:
Replace `YOUR_WORKER_URL_HERE` with the URL of your deployed Cloudflare Worker. You'll need to replace this placeholder in several files:
- In test files: `test-puppeteer.js`, `examples/debugging-tools/debug-test.js`, `examples/testing/content-test.js`
- In the MCP server configuration: `cline_mcp_settings.json.example`
- In the browser client: `src/browser-client.ts` (as a fallback if the environment variable is not set)
Integrating with Cline
To integrate the MCP server with Cline, copy the `cline_mcp_settings.json.example` file to the appropriate location:
Or add the configuration to your existing `cline_mcp_settings.json` file.
Key Learnings
- Cloudflare Browser Rendering requires the `@cloudflare/puppeteer` package to interact with the browser binding.
- The correct pattern for using the browser binding is: ```javascript import puppeteer from '@cloudflare/puppeteer'; // Then in your handler: const browser = await puppeteer.launch(env.browser); const page = await browser.newPage(); ```
- When deploying a Worker that uses the Browser Rendering binding, you need to enable the `nodejs_compat` compatibility flag.
- Always close the browser after use to avoid resource leaks.
License
MIT