Website Scraper
A command-line tool and MCP server for scraping websites and converting HTML to Markdown.
Features
- Extracts meaningful content from web pages using Mozilla's [Readability](https://github.com/mozilla/readability) library (the same engine used in Firefox's Reader View)
- Converts clean HTML to high-quality Markdown with TurndownService
- Securely handles HTML by removing potentially harmful script tags
- Works as both a command-line tool and an MCP server
- Supports direct conversion of local HTML files to Markdown
Installation
Usage
CLI Mode
MCP Server Mode
This tool can be used as a Model Context Protocol (MCP) server:
Code Structure
- `src/index.ts` - Core functionality and MCP server implementation
- `src/cli.ts` - Command-line interface implementation
- `src/data_processing.ts` - HTML to Markdown conversion functionality
API
The tool exports the following functions:
License
ISC