🕷️ Crawl For Ai OpenClaw Plugin & Skill | ClawHub
Looking to integrate Crawl For Ai into your AI workflows? This free OpenClaw plugin from ClawHub helps you automate web & frontend development tasks instantly, without having to write custom tools from scratch.
What this skill does
Web scraping using local Crawl4AI instance. Use for fetching full page content with JavaScript rendering. Better than Tavily for complex pages. Unlimited usage.
Install
openclaw skills install @angusthefuzz/crawl-for-ainpx clawhub@latest install crawl-for-aiFull SKILL.md
Open original| name | version | description |
|---|---|---|
| crawl-for-ai | 1.0.1 | Web scraping using local Crawl4AI instance. Use for fetching full page content with JavaScript rendering. Better than Tavily for complex pages. Unlimited usage. |
SKILL.md content below is scrollable.
Crawl4AI Web Scraper
Local Crawl4AI instance for full web page extraction with JavaScript rendering.
Endpoints
Proxy (port 11234) — Clean output, OpenWebUI-compatible
- Returns:
[{page_content, metadata}] - Use for: Simple content extraction
Direct (port 11235) — Full output with all data
- Returns:
{results: [{markdown, html, links, media, ...}]} - Use for: When you need links, media, or other metadata
Usage
# Via script
node {baseDir}/scripts/crawl4ai.js "url"
node {baseDir}/scripts/crawl4ai.js "url" --json
Script options:
--json— Full JSON response
Output: Clean markdown from the page.
Configuration
Required environment variable:
CRAWL4AI_URL— Your Crawl4AI instance URL (e.g.,http://localhost:11235)
Optional:
CRAWL4AI_KEY— API key if your instance requires authentication
Features
- JavaScript rendering — Handles dynamic content
- Unlimited usage — Local instance, no API limits
- Full content — HTML, markdown, links, media, tables
- Better than Tavily for complex pages with JS
API
Uses your local Crawl4AI instance REST API. Auth header only sent if CRAWL4AI_KEY is set.