Metadata-Version: 2.1
Name: crawlframe
Version: 1.0.3
Summary: A Web Crawling and Web Scraping framework (crawlframe) for Python.
Home-page: https://github.com/chenxinqun/crawlframe
Author: ChenXinqun
Author-email: chenxinqun163@163.com
Maintainer: ChenXinqun <chenxinqun163@163.com>
Maintainer-email: chenxinqun163@163.com
License: Apache 2.0
Platform: UNKNOWN
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Development Status :: 5 - Production/Stable
Classifier: Operating System :: POSIX
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Framework :: AsyncIO
Requires-Python: >=3.5.3
Description-Content-Type: text/markdown
Requires-Dist: bs4
Requires-Dist: pytz
Requires-Dist: lxml
Requires-Dist: redis
Requires-Dist: psutil
Requires-Dist: psycopg2
Requires-Dist: requests

CrawlFrame: A Web Crawling framework
==========================
[![PyPI](https://img.shields.io/pypi/v/crawlframe.svg)](https://pypi.org/project/crawlframe/)

##### Behold, the power of CrawlFrame:

``` 
crawlf createapp <appname> -url http://<website>/
crawlf createpro <projectname>
crawlf start app:<name>
crawlf start project
crawlf stop app:<name>
crawlf stop project
crawlf stop <pid>
```

[CrawlFrame](https://github.com/chenxinqun/crawlframe) officially supports python3.5+ .

Installation
------------

To install CrawlFrame, simply use [pip](https://pypi.org/project/crawlframe/):

``` 
$ pip install crawlframe
```


Documentation
-------------
In the construction...


Changed
-------------
[CHANGES.md](https://github.com/chenxinqun/crawlframe/blob/prepack/CHANGES.md)

crawlframe-1.0.3
==========================
下载器并发方式变更



crawlframe-1.0.2
==========================
```
CRAWLFRAME_SURVIVE_SWITCH = True
CRAWLFRAME_SURVIVE_MAX = int
reload spider
```
CRAWLFRAME_SURVIVE_SWITCH = True
CRAWLFRAME_SURVIVE_MAX = int
设置两个参数
发出一定请求数之后, 自动重启爬虫. 避免内存溢出.
修复middle不能加载以及logger不能创建的bug.
BaseLogger改为单例模式.


crawlframe-1.0.1
==========================
``` 
crawlf createapp <appname> -url http://<website>/
crawlf createpro <projectname>
crawlf start app:<name>
crawlf start project
crawlf stop app:<name>
crawlf stop project
crawlf stop <pid>
``` 

实际上以上命令只实现了start 与 stop. 
但是已经能够在项目中使用了.
为了方便自己安装使用, 就先上传到 pypi 与 github 了.

