Skip to content

@scrapy-plugins Scrapy Plugins

All

    Repositories list

    35 repositories

    • scrapy-download-handlers-incubator

      Public
      Additional download handlers for Scrapy
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2•3•7•0•Updated Sep 13, 2026Sep 13, 2026

    • scrapy-playwright

      Public
      🎭 Playwright integration for Scrapy
      pythonpython3scrapy
      pythonpython3scrapyhacktoberfestchrome-headlesspython-asyncioheadless-browserjavascript-rendererfirefox-headlessplaywright
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •161•1.5k•13•9•Updated Sep 7, 2026Sep 7, 2026

    • scrapy-zyte-api

      Public
      Zyte API integration for Scrapy
      plugincrawlerproxy
      plugincrawlerproxyscrapingscrapy
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •24•43•15•22•Updated Sep 1, 2026Sep 1, 2026

    • zyte-spidermon

      Public
      Python
      •1•1•0•1•Updated Aug 11, 2026Aug 11, 2026

    • scrapy-feedexport-warc

      Public
      Python
      •
      Apache License 2.0
      •1•0•0•0•Updated Aug 11, 2026Aug 11, 2026

    • scrapy-zyte-smartproxy

      Public
      Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
      plugincrawlerproxy
      plugincrawlerproxyscrapingscrapycrawler-detection
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •92•364•5•3•Updated May 4, 2026May 4, 2026

    • scrapy-crawl-maps

      Public
      Scrapy plugin that allows defining the logic of a spider using a directed graph defined in JSON format.
      BSD 3-Clause "New" or "Revised" License
      •0•3•0•1•Updated May 21, 2025May 21, 2025

    • scrapy-spider-metadata

      Public
      Python
      •
      MIT License
      •3•8•3•1•Updated Mar 24, 2025Mar 24, 2025

    • scrapy-deltafetch

      Public
      Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
      hacktoberfesthacktoberfest2021
      hacktoberfesthacktoberfest2021
      Python
      •48•275•13•4•Updated Feb 26, 2025Feb 26, 2025

    • scrapy-feedexporter-sftp

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •4•7•0•0•Updated Feb 12, 2025Feb 12, 2025

    • scrapy-splash

      Public
      Scrapy+Splash for JavaScript integration
      scrapyheadless-browsers
      scrapyheadless-browsers
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •456•3.2k•62•7•Updated Feb 11, 2025Feb 11, 2025

    • scrapy-settings-log

      Public
      An extension that allows a user to display all or some of their scrapy spider settings at runtime.
      Python
      •
      GNU General Public License v3.0
      •4•1•0•0•Updated Aug 20, 2024Aug 20, 2024

    • scrapy-feedexporter-dropbox

      Public
      Scrapy feed exporter for Dropbox
      Python
      •3•1•0•0•Updated Jul 11, 2024Jul 11, 2024

    • scrapy-crawlera-fetch

      Public archive
      Scrapy Downloader Middleware for Crawlera Fetch API
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •10•9•5•5•Updated Jun 5, 2024Jun 5, 2024

    • scrapy-feedexporter-azure-storage

      Public
      Python
      •7•4•0•4•Updated May 18, 2024May 18, 2024

    • scrapy-feedexporter-onedrive

      Public
      Export to OneDrive
      Python
      •1•3•0•0•Updated Feb 23, 2024Feb 23, 2024

    • scrapy-djangoitem

      Public
      Scrapy extension to write scraped items using Django models
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •86•501•10•3•Updated Oct 15, 2023Oct 15, 2023

    • scrapy-feedexporter-google-sheets

      Public
      Python
      •2•2•0•1•Updated Aug 15, 2023Aug 15, 2023

    • scrapy-incremental

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1•2•0•0•Updated May 1, 2023May 1, 2023

    • scrapy-feedexporter-google-drive

      Public
      Python
      •3•3•1•0•Updated Jan 4, 2023Jan 4, 2023

    • scrapy-statsd

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •3•7•1•2•Updated Oct 19, 2022Oct 19, 2022

    • scrapy-dotpersistence

      Public
      A scrapy extension to sync `.scrapy` folder to an S3 bucket
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •8•18•2•1•Updated Mar 28, 2022Mar 28, 2022

    • scrapy-magicfields

      Public
      Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•56•0•0•Updated Mar 16, 2022Mar 16, 2022

    • scrapy-pagestorage

      Public
      A scrapy extension to store requests and responses information in storage service
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•27•3•0•Updated Mar 11, 2022Mar 11, 2022

    • scrapy-snowflake-stage-exporter

      Public
      Snowflake database loading utility with Scrapy integration
      etlsnowflakescrapy
      etlsnowflakescrapy
      Python
      •2•6•0•0•Updated Dec 3, 2021Dec 3, 2021

    • scrapy-headless

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •8•29•2•2•Updated Apr 28, 2021Apr 28, 2021

    • scrapy-jsonschema

      Public
      Scrapy schema validation pipeline and Item builder using JSON Schema
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •12•45•1•0•Updated Mar 26, 2021Mar 26, 2021

    • scrapy-jsonrpc

      Public
      Scrapy extension to control spiders using JSON-RPC
      Python
      •71•299•7•2•Updated Aug 26, 2019Aug 26, 2019

    • scrapy-hcf

      Public
      Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•5•1•1•Updated Aug 28, 2018Aug 28, 2018

    • scrapy-monkeylearn

      Public
      A Scrapy pipeline to categorize items using MonkeyLearn
      Python
      •12•38•1•0•Updated Apr 28, 2017Apr 28, 2017
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.