Skill Seekers
Turns a documentation site, a repository or a PDF into a skill the agent can load, instead of re-reading it every time
What is Skill Seekers?
An agent working against an unfamiliar library reads the same documentation site over and over, a page at a time, paying for it on every run. Skill Seekers converts that source once: point it at a documentation site, a repository, a PDF, a wiki or a video and it produces a structured knowledge asset, which it can then export as an agent skill or into a retrieval pipeline. Eighteen kinds of source go in and it can package for a couple of dozen destinations, so the same extraction serves a coding assistant and a search index. It also checks for conflicts, which is the failure mode when two versions of the same documentation end up loaded at once.
What can you do with Skill Seekers?
- Convert a source once, not on every run — A documentation site becomes a stored asset, so the agent stops paying to re-read the same pages each time it needs them.
- Take in more than web pages — Repositories, PDFs, wikis, notebooks and videos are all accepted, which covers the internal material that never became a documentation site.
- Export to where you actually need it — The same extraction can be packaged as an agent skill or fed into a retrieval pipeline, rather than being locked to one consumer.
- Catch two versions of the same thing — Conflict detection flags overlapping material, which is what goes wrong when last year's documentation is still loaded alongside this year's.
- Drive it from an agent — It exposes its own tooling over MCP, so the conversion can be requested in the same session where the gap in knowledge was noticed.
Before you choose Skill Seekers
- Converting someone else's documentation makes a copy of it. Check the source's licence and terms before packaging and redistributing a vendor's material as a skill.
- A converted skill is a snapshot: the documentation it came from moves on, and nothing in the agent will tell you that the material it is confidently quoting is a year out of date.
Frequently asked questions
Is Skill Seekers free for commercial use?
Skill Seekers is released under the MIT licence — OSI-approved open source, which permits commercial use.
How can Skill Seekers be deployed?
Skill Seekers is available as Runs locally / Self-hosted.
Documentation
Reproduced from the yusufkaraaslan/Skill_Seekers README, published under MIT. Read the original ↗
Skill Seekers
English | 简体中文 | 日本語 | 한국어 | Español | Français | Deutsch | Português | Türkçe | العربية | हिन्दी | Русский
🧠 The data layer for AI systems. Skill Seekers turns documentation sites, GitHub repos, PDFs, videos, notebooks, wikis, and more — 18 source types — into structured knowledge assets, ready to power AI Skills (Claude, Gemini, OpenAI), RAG pipelines (LangChain, LlamaIndex, Pinecone), and AI coding assistants (Cursor, Windsurf, Cline). Prepare once, export to 22 targets.
💛 Sponsors
Launch Partner
Atlas Cloud — A full-modal, OpenAI-compatible AI inference platform. Skill Seekers supports it as a packaging/enhancement target via --target atlas with ATLAS_API_KEY.
Silver Sponsors
Become a sponsor · GitHub Sponsors
🚀 Quick Start
# 1. Install
pip install skill-seekers
# 2. Create a skill from any source
skill-seekers create https://docs.djangoproject.com/
# 3. Package it for your AI platform
skill-seekers package output/django --target claudeYou now have output/django-claude.zip, ready to use.
# Pick a different AI agent for enhancement (default: claude)
skill-seekers create https://docs.djangoproject.com/ --agent kimi
skill-seekers create https://docs.djangoproject.com/ --agent-cmd "my-custom-agent run"🛰️ AI-driven project scan
Point scan at a project and an AI agent reads its manifests, README, Dockerfile/CI and sampled source imports — then emits one config per detected framework, plus a <project>-codebase.json for your own code:
skill-seekers scan ./my-react-app --out ./configs/scanned/
# → react.json, vite.json, tailwind.json, jest.json, my-react-app-codebase.json
skill-seekers create ./configs/scanned/react.jsonIf a detection has no existing preset, the AI generates a fresh config; on exit you can optionally publish it back to the community registry.
All 18 source types
skill-seekers create facebook/react # GitHub repository
skill-seekers create ./my-project # Local codebase
skill-seekers create manual.pdf # PDF
skill-seekers create report.docx # Word
skill-seekers create book.epub # EPUB
skill-seekers create notebook.ipynb # Jupyter
skill-seekers create openapi.yaml # OpenAPI/Swagger
skill-seekers create presentation.pptx # PowerPoint
skill-seekers create guide.adoc # AsciiDoc
skill-seekers create page.html # Local HTML (or a whole dir)
skill-seekers create feed.rss # RSS/Atom
skill-seekers create curl.1 # Man page
# Video (YouTube, Vimeo, or local — needs skill-seekers[video])
skill-seekers create --video-url https://www.youtube.com/watch?v=... --name mytutorial
skill-seekers create --setup # auto-install GPU-aware visual deps
skill-seekers create --space-key TEAM --name wiki # Confluence
skill-seekers create --database-id ... --name docs # Notion
skill-seekers create --chat-export-path ./slack-export --name team-chat # Slack/DiscordSee the Scraping Guide for every source type and its options.
📦 Installation
pip install skill-seekers # Core: scraping, GitHub, PDF, packaging
pip install skill-seekers[all-llms] # + every LLM platform
pip install skill-seekers[mcp] # + MCP server
pip install skill-seekers[all] # EverythingNot sure what you need? Run the wizard: skill-seekers-setup
| Install | Adds |
|---|---|
skill-seekers[gemini] | Google Gemini support |
skill-seekers[openai] | OpenAI ChatGPT support |
skill-seekers[all-llms] | All LLM platforms |
skill-seekers[mcp] | MCP server for Claude Code, Cursor, etc. |
skill-seekers[video] | YouTube/Vimeo transcript & metadata extraction |
skill-seekers[video-full] | + Whisper transcription & visual frame extraction |
skill-seekers[jupyter] | Jupyter Notebook support |
skill-seekers[pptx] | PowerPoint support |
skill-seekers[confluence] | Confluence wiki support |
skill-seekers[notion] | Notion pages support |
skill-seekers[rss] | RSS/Atom feed support |
skill-seekers[chat] | Slack/Discord chat export support |
skill-seekers[asciidoc] | AsciiDoc support |
skill-seekers[all] | Everything |
Video visual deps (GPU-aware): after installing
skill-seekers[video-full], runskill-seekers create --setupto auto-detect your GPU and install the matching PyTorch variant + easyocr.
Prerequisites: Python 3.10+, Git. New here? → Bulletproof Quick Start 🎯
📚 Documentation
| I want to… | Read this |
|---|---|
| Get started quickly | Quick Start — 3 commands to your first skill |
| Understand the concepts | Core Concepts |
| Scrape sources | Scraping Guide — all 18 source types |
| Enhance skills with AI | Enhancement Guide · Enhancement Modes |
| Export skills | Packaging Guide |
| Build workflows | Workflows |
| Look up a command | CLI Reference — all 19 commands |
| Configure | Config Format · Environment Variables |
| Set up MCP | MCP Setup · MCP Reference |
| Integrate with RAG / IDEs | LangChain · RAG Pipelines · Cursor · Windsurf · Cline |
| Handle huge doc sets | Large Documentation — 10K–40K+ pages |
| Understand the architecture | UML Architecture — 14 diagrams |
| Fix a problem | Troubleshooting |
Complete documentation index: docs/README.md
🎯 What you get
| Use case | Output | Powers |
|---|---|---|
| AI Skills | Comprehensive SKILL.md + reference files | Claude Code, Gemini, GPT |
| RAG pipelines | Chunked documents with rich metadata | LangChain, LlamaIndex, Haystack |
| Vector databases | Pre-formatted data ready for upsert | Pinecone, Chroma, Weaviate, FAISS, Qdrant |
| AI coding assistants | Context files your IDE AI reads automatically | Cursor, Windsurf, Cline, Continue.dev |
Export targets (22)
skill-seekers package output/react --target claude # → Claude Skill (ZIP + YAML)
skill-seekers package output/react --target langchain # → LangChain Documents
skill-seekers package output/react --target llama-index # → LlamaIndex TextNodes
skill-seekers package output/react --target ibm-bob # → IBM Bob skill directoryLLM platforms (12): claude · gemini · openai · minimax · opencode · kimi · deepseek · qwen · openrouter · together · fireworks · markdown
RAG & vector (8): langchain · llama-index · haystack · chroma · faiss · weaviate · qdrant · pinecone
Other (2): atlas · ibm-bob
See the Feature Matrix for per-platform support details.
Why it matters
- ⚡ 99% faster — days of manual data prep → 15–45 minutes
- 🎯 Real skill quality — 500+ line
SKILL.mdfiles with examples, patterns, and guides - 📊 RAG-ready chunks — smart chunking preserves code blocks and context
- 🔄 Multi-source — combine docs + GitHub + PDFs + videos into one knowledge asset
- 🌐 One prep, every target — export to 22 targets without re-scraping
- ✅ Battle-tested — 3,900+ tests, 68 workflow presets, production-ready
✨ Key capabilities
Three-layer discovery for JavaScript SPA sites (sitemap.xml → llms.txt → headless browser rendering), automatic llms.txt detection (10× faster when present), smart topic categorization, and a lenient HTML parser fallback so broken markup still scrapes.
→ Scraping Guide · llms.txt Support
Three-stream architecture: code analysis (AST, design patterns, tests), documentation (README, docs/, wiki), and community (issues, PRs, metadata). The C3.x pipeline adds 10 GoF pattern detectors across 9 languages, usage examples extracted from tests, AI-written how-to guides, config extraction, and architecture overviews.
skill-seekers create ./my-project --preset quick # 1–2 min, surface level
skill-seekers create ./my-project --preset standard # balanced (default)
skill-seekers create ./my-project --preset comprehensive # deep, exhaustive→ Pattern Detection · How-To Guides · Test Example Extraction
Every AI call runs through one transport, in API mode (Anthropic, Google Gemini, OpenAI, Moonshot/Kimi, MiniMax) or LOCAL mode (Claude Code, Kimi Code, Codex, Copilot, OpenCode, custom agents — no API costs). Control depth with --enhance-level 0-3 and pick an agent with --agent.
→ Enhancement Guide · Enhancement Modes · Multi-Agent Setup
One config can pull documentation, GitHub, PDFs, videos, and more into a single knowledge asset, with conflict detection and pairwise synthesis across sources.
YouTube, Vimeo, and local files. Three-tier transcript fallback (subtitles → YouTube transcript API → local Whisper), plus optional visual extraction that OCRs on-screen code from sampled frames.
Quality scoring with a gate (skill-seekers quality output/react/ --threshold 7), provisional English readability metrics (informational — they never affect the score), doc-change detection with scheduled re-scrapes and notifications, streaming ingestion for very large doc sets, and incremental updates.
→ Large Documentation · Code Quality
🔌 MCP Integration (40 tools)
Skill Seekers ships an MCP server for Claude Code, Cursor, Windsurf, VS Code + Cline, and IntelliJ IDEA.
# stdio mode (Claude Code, VS Code + Cline)
python -m skill_seekers.mcp.server_fastmcp
# HTTP mode (Cursor, Windsurf, IntelliJ)
python -m skill_seekers.mcp.server_fastmcp --transport http --port 8765Then just ask your assistant: “Package and upload the React skill.”
→ MCP Setup · MCP Reference · HTTP Transport
🤖 Installing to AI agents
Skills install automatically into 19 AI coding agents:
skill-seekers install-agent output/react/ --agent cursor
skill-seekers install-agent output/react/ --agent all # every detected agent
skill-seekers install-agent output/react/ --agent cursor --dry-run| Agent | Path | Scope |
|---|---|---|
| Claude Code | ~/.claude/skills/ | Global |
| Cursor | .cursor/skills/ | Project |
| VS Code / Copilot | .github/skills/ | Project |
| Amp | ~/.amp/skills/ | Global |
| Goose | ~/.config/goose/skills/ | Global |
| OpenCode | ~/.opencode/skills/ | Global |
| Letta | ~/.letta/skills/ | Global |
| Aide | ~/.aide/skills/ | Global |
| Windsurf | ~/.windsurf/skills/ | Global |
| Neovate | ~/.neovate/skills/ | Global |
| Roo Code | .roo/skills/ | Project |
| Cline | .cline/skills/ | Project |
| Aider | ~/.aider/skills/ | Global |
| Bolt | .bolt/skills/ | Project |
| Kilo Code | .kilo/skills/ | Project |
| Continue | ~/.continue/skills/ | Global |
| Kimi Code | ~/.kimi/skills/ | Global |
| IBM Bob | .bob/skills/ | Project |
Uploading to Claude
export ANTHROPIC_API_KEY=sk-ant-...
skill-seekers package output/react/ --upload # package + upload
skill-seekers upload output/react.zip # upload an existing zipNo API key? Package it and upload output/react.zip manually at claude.ai/skills.
⚙️ How it works
graph LR
A[Documentation Website] --> B[Skill Seekers]
B --> C[Scraper]
B --> D[AI Enhancement]
B --> E[Packager]
C --> F[Organized References]
D --> F
F --> E
E --> G[AI Skill .zip]
G --> H[Upload to AI Platform]- Scrape — extract every page (checking
llms.txtfirst) - Categorize — organize content into topics (API, guides, tutorials, …)
- Enhance — AI writes a comprehensive
SKILL.mdwith examples - Package — bundle into a platform-ready artifact
- Upload — ship it to your AI platform (optional)
Architecture
8 core modules + 5 utility modules (~200 classes):
| Module | Purpose |
|---|---|
| CLICore | Git-style command dispatcher, source auto-detection |
| Scrapers | 18 source-type extractors on a shared build layer |
| Adaptors | 22 output platform formats behind one SkillAdaptor ABC |
| Analysis | C3.x codebase pipeline, 10 GoF pattern detectors |
| Enhancement | AI improvement via a single AgentClient transport |
| Packaging | Package, upload, and install skills |
| MCP | FastMCP server (40 tools, 10 tool modules) |
| Sync | Doc change detection and notification |
→ UML Architecture · API Reference · Skill Architecture
🆕 New in v3.9.0
- HTML parser fallback for broken markup (#96) — severely malformed pages no longer scrape as empty; well-formed pages are byte-identical.
- Transient-failure retries — the doc scraper (#97) and MCP
fetch_config(#92) now retry connection blips and 5xx with backoff; 4xx still fails fast. - Whisper transcription fallback (#420) — local videos without subtitles finally get a real transcript.
- MiniMax image OCR + registry-driven multimodal providers (#423) — providers declare their wire protocol and image capability; China-issued keys work against the right endpoint.
- Token-lean GitHub issue defaults (#169) — GitHub skills no longer bundle full closed-issue history by default.
- Env-driven CORS across all three servers (#422, #424) — no more wildcard origins with credentials.
Full history: CHANGELOG.md
📈 Performance
| Documentation size | Time | Output |
|---|---|---|
| Small (< 100 pages) | 5–10 min | ~2 MB |
| Medium (100–500 pages) | 15–30 min | ~10 MB |
| Large (500–2,000 pages) | 30–60 min | ~40 MB |
| Huge (10K–40K+ pages) | Use stream | See Large Documentation |
🐛 Troubleshooting
skill-seekers doctor # diagnose installation & environment
skill-seekers sync-config # detect config driftCommon issues and fixes: Troubleshooting Guide · TROUBLESHOOTING.md
🤝 Contributing
Contributions are welcome — see CONTRIBUTING.md.
- 📋 Development Roadmap & Tasks — pick any task
- 💬 Discussions — questions and ideas
- 🐛 Issues — bugs and feature requests
📝 License
MIT — see LICENSE.
🔒 Security
This README has been shortened. The full version is on GitHub. Read the original ↗
Other options in Tools & Integrations
openclawTools & IntegrationsOpenClaw installs a Gateway on your own machine and makes it the local control plane for sessions, tools, events and channel connections, so the Control UI, CLI, TUI and every connected messaging channel drive the same assistant. Models can be hosted or local, and the project expects new capabilities to arrive as plugins built on the plugin SDK rather than as changes to the core. The constraints are stated plainly in the README: it is designed for a single operator, tools run on the host for the main session unless you configure sandboxing, and DM-capable channels leave the assistant reachable by unknown senders until you approve a pairing with `openclaw pairing approve <channel> <code>`. The README tells you to read the security, exposure and sandboxing guides before connecting other users or exposing the Gateway remotely, which is a fair signal that shared or multi-user deployment is not the case it was built for.
hermes-agentTools & IntegrationsHermes Agent is Nous Research's own agent: a terminal UI plus a single gateway process that carries the same conversation into Telegram, Discord, Slack, WhatsApp, Signal and Email, with a cron scheduler, isolated subagents, and seven terminal backends from local and Docker to Modal and Vercel Sandbox. Its distinguishing piece is a closed learning loop — the agent curates its own memory on periodic nudges, creates skills after complex tasks, and searches past sessions through FTS5 with LLM summarization — which means the state that makes it useful accumulates under `~/.hermes`, outside your version control, and needs occasional pruning. It is an assistant you run and configure rather than a library you build on: the README documents `hermes` subcommands and slash commands, not an embedding API, so it is a poor fit if you wanted an agent loop to call from your own code. Putting a shell-capable agent behind chat platforms also makes the command-approval and DM-pairing settings load-bearing rather than optional.
n8nWorkflowConnects hundreds of services as workflows you draw on screen — each step in the flow is a node — and increasingly puts an AI agent node inside those workflows. The licence is the thing to check first: it is source-available under the Sustainable Use License, not OSI-approved open source. Internal use and self-hosting are fine; reselling it as a hosted service is not.