scrapy/scrapy

 scrapy / scrapy

scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python. https://scrapy.org

scrapy0044881212161620202424282832322017-06-222017-06-232017-06-242017-06-252017-06-262017-06-272017-06-282017-06-292017-06-302017-07-012017-07-022017-07-032017-07-042017-07-052017-07-062017-07-072017-07-082017-07-092017-07-102017-07-112017-07-122017-07-132017-07-142017-07-152017-07-162017-07-172017-07-182017-07-192017-07-202017-07-21scrapy169.1153846153881.86141761112017-06-221724.831564986777.53247572522017-06-231940.547745358168.87459195342017-06-241756.263925729477.53247572522017-06-25771.9801061008120.8218945842017-06-261587.696286472186.19035949692017-06-2713103.41246684494.84824326872017-06-2816119.12864721581.86141761112017-06-2913134.84482758694.84824326872017-06-3011150.561007958103.506127042017-07-0115166.27718832986.19035949692017-07-0234181.99336873.940463665162017-07-0323197.70954907251.55882440992017-07-0422213.42572944355.88776629582017-07-0522229.14190981455.88776629582017-07-0611244.858090186103.506127042017-07-0711260.574270557103.506127042017-07-083276.290450928138.1376621282017-07-0911292.0066313103.506127042017-07-1015307.72281167186.19035949692017-07-1119323.43899204268.87459195342017-07-1228339.15517241429.91411498052017-07-1319354.87135278568.87459195342017-07-148370.587533156116.4929526982017-07-158386.303713528116.4929526982017-07-1613402.01989389994.84824326872017-07-1725417.73607427142.90094063812017-07-1823433.45225464251.55882440992017-07-1924449.16843501347.2298825242017-07-2027464.88461538534.24305686632017-07-2139.11538461538138.1376621282017-06-22-124.8315649867155.4534296712017-06-23740.5477453581120.8218945842017-06-24856.2639257294116.4929526982017-06-25271.9801061008142.4666040132017-06-26287.6962864721142.4666040132017-06-274103.412466844133.8087202422017-06-281119.128647215146.7955458992017-06-293134.844827586138.1376621282017-06-300150.561007958151.1244877852017-07-013166.277188329138.1376621282017-07-029181.9933687112.1640108122017-07-030197.709549072151.1244877852017-07-046213.425729443125.150836472017-07-050229.141909814151.1244877852017-07-061244.858090186146.7955458992017-07-077260.574270557120.8218945842017-07-083276.290450928138.1376621282017-07-094292.0066313133.8087202422017-07-10-1307.722811671155.4534296712017-07-11-1323.438992042155.4534296712017-07-125339.155172414129.4797783562017-07-133354.871352785138.1376621282017-07-145370.587533156129.4797783562017-07-152386.303713528142.4666040132017-07-163402.019893899138.1376621282017-07-177417.736074271120.8218945842017-07-187433.452254642120.8218945842017-07-1913449.16843501394.84824326872017-07-201464.884615385146.7955458992017-07-21-29.11538461538159.7823715572017-06-22124.8315649867146.7955458992017-06-23040.5477453581151.1244877852017-06-24056.2639257294151.1244877852017-06-25-271.9801061008159.7823715572017-06-26187.6962864721146.7955458992017-06-270103.412466844151.1244877852017-06-280119.128647215151.1244877852017-06-292134.844827586142.4666040132017-06-301150.561007958146.7955458992017-07-012166.277188329142.4666040132017-07-023181.9933687138.1376621282017-07-03-1197.709549072155.4534296712017-07-042213.425729443142.4666040132017-07-053229.141909814138.1376621282017-07-063244.858090186138.1376621282017-07-071260.574270557146.7955458992017-07-080276.290450928151.1244877852017-07-092292.0066313142.4666040132017-07-10-2307.722811671159.7823715572017-07-110323.438992042151.1244877852017-07-123339.155172414138.1376621282017-07-131354.871352785146.7955458992017-07-142370.587533156142.4666040132017-07-150386.303713528151.1244877852017-07-16-1402.019893899155.4534296712017-07-17-1417.736074271155.4534296712017-07-182433.452254642142.4666040132017-07-190449.168435013151.1244877852017-07-20-1464.884615385155.4534296712017-07-21starforkwatch

 README

Scrapy

PyPI Version Build Status Wheel Status Coverage report Conda Version

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at: http://scrapy.org

Requirements

  • Python 2.7 or Python 3.3+
  • Works on Linux, Windows, Mac OSX, BSD

Install

The quick way:

pip install scrapy

For more details see the install section in the documentation: http://doc.scrapy.org/en/latest/intro/install.html

Releases

You can download the latest stable and development releases from: http://scrapy.org/download/

Documentation

Documentation is available online at http://doc.scrapy.org/ and in the docs directory.

Community (blog, twitter, mail list, IRC)

See http://scrapy.org/community/

Contributing

See http://doc.scrapy.org/en/master/contributing.html

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

See http://scrapy.org/companies/

Commercial Support

See http://scrapy.org/support/