claude

web-scraping-data-extraction

Web-scraping & data-extraction engineering team — agents (extraction-architect, scraper-implementation-engineer) answering 'how do we acquire this web data legally, robustly, and at the right cost?': API-vs-scrape source choice, a legal/ethical gate (robots.txt, ToS, public-vs-authenticated, rate, PII/copyright) BEFORE any code, fetch strategy (HTTP vs headless, JSON-endpoint-first), resilient parsing (structured data over brittle selectors, validate-to-schema), change-detection & re-crawl scheduling, and the extract→validate→dedup→store pipeline with provenance. skills, a knowledge bank (a Mermaid decision tree + a dated 2026 tooling/legal reference). Distinct from data-orchestration/data-streaming (move data that ALREADY exists) and backend-engineering — this team owns ACQUIRING web data. Legality-first: anti-bot is a stop-signal, never evasion for abuse; ToS/robots specifics volatile + jurisdictional, retrieval-dated + verify-at-use. Needs ravenclaude-core.

9 on GitHubby Matt Corbettv0.1.2MITupdated today
/

Plugin web-scraping-data-extraction from marketplace ravenclaude in mcorbett51090/RavenClaude. Commands read from the repo's manifest — today.

What it installs

Skills (3)

  • crawl-scheduling-and-pipeline

    Design the crawl schedule, change-detection, and the extraction-to-storage pipeline — a re-crawl cadence matched to data volatility, incremental crawls via conditional GET / ETag / sitemap lastmod, a polite rate-limit/backoff budget, and storage with provenance (source URL + fetch timestamp) and ded

  • legal-ethical-and-fetch-strategy

    Run the legality/ethics gate BEFORE any code (robots.txt, ToS, public-vs-authenticated, rate, PII/copyright), prefer an API/feed/export over scraping, then choose the fetch strategy (HTTP + parser vs headless, JSON-endpoint-first). Traverses the top of the web-scraping decision tree. Reach for this

  • resilient-extraction-and-parsing

    Extract web data defensively — prefer structured data (JSON-LD / microdata / __NEXT_DATA__ / JSON API) over brittle CSS/XPath selectors, anchor selectors on stable attributes with fallbacks, and validate every record to a schema so breakage is DETECTED not silently wrong. Traverses the parse branch

Subagents (2)

  • extraction-architect

    Use to DECIDE how to acquire web data — API-vs-scrape, the legal/ethical gate (robots.txt, ToS, auth, PII) BEFORE code, fetch strategy (HTTP vs headless), change-detection/scheduling. Legality-first, never evasion for abuse. NOT moving existing data → data-orchestration; not legal advice.

  • scraper-implementation-engineer

    Use to BUILD web extraction — resilient fetchers/parsers (structured data over selectors, JSON-endpoint-first), the extract→validate→dedup→store pipeline, and polite rate-limiting/backoff. NOT deciding what's legal to scrape → extraction-architect; not generic services → backend-engineering.

View source on GitHub mcorbett51090.github.io

More from mcorbett51090/RavenClaude

All 184 plugins →

Domain-neutral Claude Code foundation: specialist agents, skills, gates, hooks, rules, templates. Capability Grounding Protocol, Structured Output Protocol, Researcher meta-skill, comfort-posture dashboard, opt-in command-review tribunal (the Thing), WebFetch sanitizer, branch-archive escape hatch, brand-extraction (homepage → brand kit), /wireframe (describe anything → validated model + high-fi Artifact + Mermaid), Problems & Resolutions view, declarative-visualization (Vega-Lite/Deneb/SVG, Gate 101 + Gate 103 svg-report-lint, viz-spec-reviewer), and Claude orchestrator escape hatch (/claude-orchestrate). Slash commands: /init-agent-ready, /wrap, /set-posture, /dashboard, /forge, /wireframe, /reset-plugin-cache, /handoff.

70 skills · 14 commands · 18 agents · hooks

9

Power Platform specialist team — agents (incl. power-platform-tester, power-bi-engineer) and skills, with strong ALM/git coverage for solutions, flows, and PBIP. A house-opinions hook flags 8 §3/§4 violations; the knowledge bank carries production decision trees (PA-flow recovery, Dataverse token-acquisition, PCF React surface, PBI deploy/refresh, custom-connector build-path, PBIR Enhanced infinite-spinner debug + full build reference, DAX silent-zero scoring via the `Domain` pattern, sempy.fabric notebook reference, Power BI Copilot report-readiness, Code Apps connector gotchas, PBIP deployment variables + #839, PBIR reference enrichment, PBIP report fast-solve triage router [MCP-optional], main-forms-are-Web-API-authorable grounding) plus a real-engagement scenarios bank. Bundles the community pbix-mcp server (d0nk3yhm/pbix-mcp, MIT) for .pbix/.pbit read/write/DAX-eval (`pip install pbix-mcp`); documents (not bundles) the official Microsoft Dataverse MCP (CLAUDE.md §9a). Extends ravenclaude-core.

24 skills · 8 commands · 11 agents · 1 MCP server · hooks

9

finance

claude

Corporate finance & FP&A specialist team — FP&A analyst (budgets, forecasts, variance commentary), financial modeler (3-statement, DCF, scenarios), controller (close, JEs, reconciliations), treasury analyst (cash, covenants, FX), valuation analyst (DCF + comps + precedent), audit-prep specialist (PBC, walkthroughs, SOC1/2 narratives), and board-pack composer. skills incl. a gold-standard NetSuite close (OAuth2 M2M + SuiteQL BS/IS trial balance, layperson runbook), templates, a knowledge bank (ASC 805 / 718 / 740, accrual-cutoff, WACC, cost accounting, FP&A operating model + decision support, NetSuite integration landscape), advisory hooks (hardcoded rates / plaintext PII / missing Sources / Assumptions; secrets-PII scan). Inherits ravenclaude-core protocols.

23 skills · 7 commands · 7 agents · hooks

9

Web design & build specialist team — web architect (IA, stack, hosting), UX designer (wireframes, flows, conversion), visual designer (brand, typography, tokens), frontend implementer (HTML/CSS/JS, React, Astro, Next), content strategist (copy, microcopy, SEO content), accessibility auditor (WCAG 2.2 AA/AAA), and performance engineer (Core Web Vitals, image/font optimization). skills (the gold-standard-website-pipeline gated build pipeline, static-site build, design-system audit, card/tile UI, accessibility review, Core Web Vitals tuning, technical SEO audit, information architecture, conversion design, content audit, design-tokens scaffolding, third-party hygiene, Fluent UI v9 + React implementation), templates, an advisory hook, and a knowledge bank seeded with a curated 2026 reference set of "cutting edge yet simple" marketing-site donors — the design agents carry compact inline priors plus a pointer to the full brief. Inherits ravenclaude-core protocols.

19 skills · 7 commands · 7 agents · hooks · LSP

9

EdTech-flavored Partner Success Manager team — specialist agents anchored on the PSM lane, with skills, templates, an advisory hook, a knowledge bank with Mermaid decision trees, a engagement bank, and a stdlib psm_calc.py calculator (seat-utilization / renewal-NRR-forecast / time-to-value). Covers QBRs, success plans, health scores, a self-contained Power-BI-style partner-health report, onboarding, touchpoint logs, adoption sequencing, 90-day implementation arcs, train-the-trainer curricula, advocacy/reference pipelines, renewal decision memos, and the integration contract for a centralized CS dashboard (Planhat + Salesforce + Snowflake + Zapier + Granola). Team growth ships as skills + knowledge + templates, not parallel agents. Primary-source citations with retrieval dates. Vertical-explicit but segment-agnostic — fits K-12, higher-ed, corporate L&D, or mixed EdTech books. Inherits ravenclaude-core protocols including the alternate-methods Capability Grounding Protocol.

17 skills · 5 commands · 6 agents · hooks

9

Financial-regulatory & compliance specialist team — function agents (AML/KYC, regulatory-reporting, risk-and-controls, policy & procedure, examination-prep, Bermuda-insurance) PLUS jurisdiction/regulator specialists: BMA financial-institutions (banking, trust, corporate-services, fund-admin, investment-business, money-service, digital-assets — the primary build-out, with deep AML/ATF + supervision/fees + licensing-classification trees), CIMA (Cayman), Bahamas, Channel Islands (Jersey JFSC + Guernsey GFSC), UK PRA, and US federal+state. primary-source-cited regulator knowledge files, best-practice rules, skills, templates, a defensive PreToolUse hook (PII scrub — SSN/IBAN/credit-card/Bermuda TIN/wire instructions). Positioning reflects field experience inside a Tier-1 financial regulator (BMA). Inherits ravenclaude-core protocols.

11 skills · 5 commands · 12 agents · hooks

9

Similar plugins

Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown (with bot detection/CAPTCHA bypass), search Google with structured JSON results, extract data from 40+ websites (Amazon, LinkedIn, Instagram, TikTok, YouTube, and more), orchestrate Bright Data's 60+ MCP tools, built-in best practices for Web Unlocker, SERP API, Web Scraper API, and Browser API, Python SDK best practices for the brightda...

21 skills

anthropics/claude-plugins-official 47,076

Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown (with bot detection/CAPTCHA bypass), search Google with structured JSON results, extract data from 40+ websites (Amazon, LinkedIn, Instagram, TikTok, YouTube, and more), orchestrate Bright Data's 60+ MCP tools, built-in best practices for Web Unlocker, SERP API, Web Scraper API, and Browser API, Python SDK best practices for the brightda...

21 skills

anthropics/knowledge-work-plugins 35,601

A universal scraping skill with intelligent routing, token budget tracking, and quota awareness. Supports Firecrawl (BYOK, free-tier compatible) and local Python extraction via requests/BeautifulSoup/pandas.

1 skill · 1 command · 1 agent

alirezarezvani/claude-skills 29,725

Thirteen slash commands wrapping the designlang CLI: /extract (full design language → DTCG, Tailwind, Figma), /site (crawl a whole site → one canonical design system + consistency grade), /grade (shareable HTML report card + SVG badge), /battle (head-to-head graded comparison), /remix (restyle in 6 vocabularies — brutalist, swiss, art-deco, cyberpunk, soft-ui, editorial), /pack (one downloadable design-system bundle), /theme-swap (OKLCH-correct recolour around a new brand primary), /brand (full editorial brand-guidelines book — 13 chapters, hand-off-ready), /pair (fuse two designs across configurable axes — colours from one site, typography from another), /studio (live token-editor with component preview + export), /verify (rebuild from tokens and pixel-diff vs live for a fidelity score), /fidelity (score a clone vs the original — visual pixel-diff + motion fidelity — with a ranked correction plan), /gallery (build a shareable static gallery of measured clones).

2 skills · 14 commands

Manavarya09/design-extract 5,068

browse

claude

Automate web browser interactions using natural language. Use when the user asks to browse websites, navigate web pages, extract data from websites, take screenshots, fill forms, click buttons, or interact with web applications.

18 skills

browserbase/skills 3,757

skyvern

claude

Browser automation in Claude Code through Skyvern Cloud: open pages, fill forms, extract structured data, take screenshots, and run repeatable workflows. Sign in with OAuth from /mcp. Runs can use Skyvern credits.

1 skill · 1 MCP server

davepoon/buildwithclaude 3,741

Maintain web-scraping-data-extraction?

This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.

claude & codex plugins184 plugins