U-Crawler

依赖

gevent,requests,BeautifulSoup,lxml,urlparse,optparse,

使用说明

12345678910111213

Usage: U-Crawler.py [-q] query [--limit] number [-o] filenameOptions: --version show program's version number and exit -h, --help show this help message and exit -q QUERY, --query=QUERY The query of search engine. -l LIMIT, --limit=LIMIT The limit of each search engine. -o NAME, --output=NAME If not use -o,the filename of output is time string. -b, --baseurl The url of writing in file,if it is set,the url will remove path and param.