claude

web-scraping

Web Scraping: the legal and regulatory landscape (CFAA and the hiQ/Bright Data line, contract and ToS exposure, the DMCA §1201 shift, copyright and TDM, GDPR and the EDPB's 2026 scraping guidelines, the EU AI Act), when not to scrape and what to do instead, robots.txt and the emerging crawl-permission layer, how the modern web actually serves data, the tool ladder (requests, curl_cffi, Scrapy, Playwright, managed services), parsing and extraction, bot detection, scale and reliability engineering, data quality, storage and pipelines, operational ethics, scraping for AI and RAG, and building a site that handles crawlers well.

5 on GitHubby Adam Matthew Steinbergerv0.1.0MITupdated todaydata-engineering
/

Plugin web-scraping from marketplace vibey in the-vibey-project/vibey. Commands read from the repo's manifest — today.

What it installs

Skills (5)

  • scraping-ethics-ai-corpora-and-site-defense

    Use when weighing the operational ethics of collection, building a corpus for AI or RAG, or configuring and defending a site: rate limits and being a good guest on someone else's infrastructure, provenance, opt-out signals and legal basis per source for AI training, and running a site that handles c

  • scraping-legal-landscape-and-permissions

    Use when deciding whether and how you are permitted to scrape: the four separate legal questions, computer-access law and the hiQ and Bright Data line under the CFAA, contract and terms-of-service exposure where scrapers actually lose, the DMCA §1201 shift, copyright, database rights and personal da

  • scraping-reference

    Use when checking a scraping anti-pattern, weighing a contested question (is scraping public data ethical, does robots.txt bind you, should sites block AI crawlers, is pay-per-crawl the right model, LLM extraction versus deterministic selectors, managed service versus building it yourself), confirmi

  • scraping-scale-reliability-and-data-quality

    Use when a scraper has to run reliably and produce trustworthy data: scale and reliability engineering (concurrency, rate limiting, retries and backoff, proxies, scheduling), data quality and validation against the silent breakage that produces plausible-looking wrong data for months, and storage, p

  • scraping-tooling-extraction-and-blocking

    Use when building the scraper: how the modern web actually serves data (server-rendered, client-rendered, and the internal JSON API behind the page — the single most valuable technique here), the structured data sites give you for free, pagination and state, the tool ladder from requests and curl_cf

View source on GitHub the-vibey-project.github.io

More from the-vibey-project/vibey

All 157 plugins →

Source-cited deep dives across agriculture, operating systems, cognition, civilization, politics, science, technology, storytelling, and integrated systems.

18 skills

5

Military Science, Rockets, Space, Fusion, and Mars: strategic theory and engineering physics in one reference, at the war-college survey level for the first and the engineering level for the rest. Military science as the one field whose object of study is a reactive adversary, so no principle holds unconditionally — war as a political instrument or nothing, defence as an economic and industrial problem first, institutional learning as the hardest capability to copy; Clausewitz's trinity, friction, centre of gravity, culminating point and defence as the stronger form, Sun Tzu and Jomini and Soviet operational art, Mahan against Corbett, and the air power promise against the post-1945 bombing surveys; the three levels of war and why tactical success does not aggregate into strategic success, the principles as contradictory heuristics, manoeuvre versus attrition, OODA with ORIENT restored, mission command; deterrence by punishment and by denial, Schelling on compellence, the stability-instability paradox and the measurement problem, survivability and the triad, Kahn's escalation thresholds and off-ramps, alliance abandonment versus entrapment, and grand strategy as ends, ways and means — with the near-miss record and the line that deterrence working for decades is compatible with having been repeatedly lucky. Then the readiness/modernization/size trilemma, logistics and the industrial base as the professional's subject, doctrine and its rigidity paradox, intelligence failing analytically rather than collectively, procurement's monopsony structure, modern conflict from the counter-UAS cost-per-engagement problem through cyber, Kessler syndrome and information operations to total defence — and the law of armed conflict in full: jus ad bellum against jus in bello, the equality of application, distinction, proportionality stated correctly rather than as a casualty comparison, military necessity, humanity, and command responsibility. Rocket engineering from Tsiolkovsky and

12 skills

5

Evangelism, Apologetics, Preaching, Teaching and Church Planting: a practitioner reference for Christian ministry that treats evangelism as a communication task and then holds itself to what the evidence on communication actually supports. The persuasion findings the source stands behind -- listening first produces durable change, conversation beats broadcast, moral reframing is difficult and rarely done, relevant evidence is the strongest identified lever, facts alone against identity-linked beliefs persuade less than assumed -- read as tendencies rather than measured effects, with no number supplied because the source supplies none; trust decomposed into competence, reliability and care and why most trust failures are the third mistaken for the first; the four levels of listening, silence as the most underused technique, asking before advising, and recognising when a spiritual question has actually been asked. Six evangelistic approaches -- relational, lifestyle, missional, conversational, proclamation, service -- compared on strengths and limitations, with the limitations column read carefully because half of it is structural fact and half theological judgement held inside the source's tradition; the four common gospel presentations (the Roman Road, the Bridge to Life, the Four Spiritual Laws, Creation-Fall-Redemption-Restoration) and what each compression drops. The ethics of influence carried as load-bearing rather than as an appendix: the disclosure test -- if you told the person exactly what you were doing and why, would this still work -- the structural line between persuasion, manipulation and coercion, power asymmetry where a pastor, parent or employer holds it, exploited states, and the hard cases including altar calls, which the source does not rule on. The cosmological, teleological, moral and ontological arguments as the source gives them with the standard objection each actually faces, kept inside its own framing that the case is cumulative rather tha

10 skills

5

From Autograph to Your Bible -- the biblical languages, translation, textual criticism, archaeology and historical method as one connected reference, because a translator cannot choose a word until a textual critic has decided which words are there, a textual critic weighs manuscripts that archaeologists dug up and palaeographers dated, and historians use all of it and then argue about what it means. Where the source marks a claim settled, majority, contested or as of 2026, the marker stays exactly where the source put it, and contested questions stay contested. Biblical Hebrew and the three layers of a printed Hebrew Bible, the Masorah and ketiv/qere, the Aleppo and Leningrad codices, the linguistic-dating fight, the unsolved verbal system, Biblical Aramaic, and the words that break translators' hearts; Koine Greek, the papyri and Semitisms, verbal aspect and deponency, and the Septuagint's legend and reality. The Vetus Latina, Jerome and the composite Vulgate, the Latin renderings that changed Western history; Gothic, the Heliand, the pre-Luther printed Bibles, Luther's September Testament and Sendbrief, the Lutherbibel 2017; Old English to Wycliffe, Tyndale and Geneva, how the King James Version was actually made, and the modern English spectrum. How committees really translate -- base texts, drafting and votes, the root fallacy, gender language, the divine name -- and how minority-language field translation works in 2026, from comprehension testing and back translation to Paratext and AI drafting. Textual criticism: copying errors, external and internal evidence after Royse, Erasmus to NA28, NA29 and the Editio Critica Maior, how many manuscripts there really are, the CBGM and the initial text, the famous variants, and the Hebrew Bible's plural text with BHQ, HUBP and HBCE. Archaeology: context, stratigraphy and the Harris Matrix, radiocarbon and the Hallstatt plateau, ancient DNA and archaeometallurgy, the inscriptions from the Merneptah Stele to the Gallio ins

9 skills

5

Company and Non-Profit Valuation, Market Research, Buying and Selling: standards, premises and levels of value, the income, market and asset approaches, the size-segmented deal market and its documents, researching deal markets and multiples and how each indicator misleads, normalizing earnings (SDE, adjusted EBITDA, quality of earnings), a regression of multiples, WACC and DCF, discounts and the EV-to-equity bridge, the buyer playbook (debt capacity, LOI structure, SBA financing, LBO sensitivity, diligence, closing), the seller playbook (exit readiness, process, comparing LOIs on after-tax present value, asset versus stock, QSBS, alternatives), deep diligence and fraud, startups, public companies, 409A, ESOP and disputes, and non-profits treated as their own problem (no owners and no equity price, mergers and affiliations, IRC 4958, Form 990 and ratio analysis, stress tests, impact measurement), plus a tested numpy and pandas toolkit. Decision support, not investment, legal, tax, accounting or valuation-opinion advice.

9 skills

5

Developer Experience: the evidence-graded blueprint, in two parts, for making a codebase, CLI, API, documentation set, UI and AI-agent surface easy to use well, and for placing each feature on the right device surface (VS Code, desktop, iOS, Android, web) so non-developers reach a working result too. Short and trustworthy feedback loops, cognitive load, one-command local setup, CLI conventions (streams, exit codes, --json, --dry-run, NO_COLOR), error messages that say what failed and how to fix it, API and SDK usability, Diataxis documentation, TUI and screen-reader accessibility, onboarding and contribution barriers, AGENTS.md, llms.txt and MCP tool design for coding agents, and how to measure DevEx with surveys paired with system data, with every claim graded A, B or C and the weak or contested evidence called out.

9 skills

5

Similar plugins

Delegates dbt and warehouse work to altimate-code, a specialized CLI agent with 100+ purpose-built data tools (SQL analysis, column-level lineage, dbt build/test/run, warehouse profiling, FinOps, connectivity to Snowflake, BigQuery, Redshift, Databricks, Postgres, MySQL, DuckDB).

2 skills · 4 commands · 1 agent · 1 MCP server · hooks

anthropics/claude-plugins-official 47,076

Data engineering, ML, and AI specialists - data pipelines, machine learning, LLM architecture

VoltAgent/awesome-claude-code-subagents 26,513

Agents for data engineering, machine learning, and AI development

12 agents

davepoon/buildwithclaude 3,741

5 live-detection Databricks skills — cost-leak-hunter, cluster-forensics, uc-migration-pilot, streaming-guardian, bundle-medic — backed by the databricks-workspace-mcp server (npm). The v2 rebuild: real workspace detection over static docs.

5 skills · 1 MCP server · hooks

jeremylongshore/tons-of-skills-marketplace 2,917

hamilton

claude

Expert AI assistant for Apache Hamilton framework development - create DAGs, apply decorators, debug dataflows, and optimize pipelines

7 skills

apache/hamilton 2,631

dex

codex

Agent skills for warehouse exploration, dbt transformation and semantic modeling, and schema-drift maintenance.

3 skills

hashgraph-online/awesome-codex-plugins 1,566

Maintain web-scraping?

This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.

claude & codex plugins157 plugins