headless-chrome-crawler

About this project

Headless Chrome Crawler API Examples Tips Code of Conduct Contributing Changelog Distributed crawler powered by Headless Chrome Features Crawlers based on simple requests to HTML files are generally fast. However, it sometimes ends up capturing empty bodies, especially when the websites are built on such modern frontend frameworks as AngularJS, React and Vue.js. Powered by Headless Chrome, the crawler provides simple APIs to crawl these dynamic websites with the following features: Distributed crawling Configure concurrency, delay and retry Support both depth-first search and breadth-first search algorithm Pluggable cache storages such as Redis Support CSV and JSON Lines for…

GitHub

Reviews

Built by

yujiosaka

Imported from GitHub · not yet claimed on GitPalace

View developer pageSign in with GitHub to claim

Maintain yujiosaka/headless-chrome-crawler? Claiming verifies admin access through your GitHub account and gives you control of this listing.

You might also like

freeCodeCamp

Web

freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming, and computer science for free.

★ New455.1K stars

by freeCodeCamp

system-design-primer

Web

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

★ New368.4K stars

by donnemartin

developer-roadmap

Web

Interactive roadmaps, guides and other educational content to help developers grow in their careers.

★ New366.4K stars

by nilbuild

deepseek-harness

Web

DeepSeek Harness: Everything is a Plugin.

★ New214K stars

by deepseek-ai