{
  "name": "Verata",
  "tagline": "Yet another scraper",
  "body": "# Verata\r\nYet another scraper\r\n[![Build Status](https://travis-ci.org/CodersOfTheNight/verata.svg?branch=master)](https://travis-ci.org/CodersOfTheNight/verata)\r\n[![Coverage Status](https://coveralls.io/repos/github/CodersOfTheNight/verata/badge.svg?branch=master)](https://coveralls.io/github/CodersOfTheNight/verata?branch=master)\r\n[![Documentation Status](https://readthedocs.org/projects/verata/badge/?version=latest)](http://verata.readthedocs.io/en/latest/?badge=latest)\r\n\r\nWhy even consider?\r\n------------------\r\nIt works just by providing correct config - no coding is required\r\n\r\nHow to install\r\n--------------\r\n`pip install verata` \r\n\r\nSupported versions\r\n------------------\r\nIt is tested on Python versions:\r\n- 2.7\r\n- 3.4\r\n- 3.5\r\n\r\nUsage\r\n-----\r\n`verata --config=config-file.yml`\r\n\r\nOptionally you can setup environment file:\r\n\r\n`verata --config=config-file.yml --env=.secret-env`\r\n\r\nDocs\r\n----\r\n[http://verata.readthedocs.io](http://verata.readthedocs.io/en/latest/)\r\n\r\nConfig example\r\n--------------\r\n```yaml\r\n---\r\nname: Python Org scrapper\r\ndescription: Just scrape it for testing\r\nsite_root: https://www.python.org\r\nstart_page: /blogs\r\ncookies:\r\n  authToken: abc1234\r\n  remember: true\r\nheaders:\r\n  \"User-Agent\": \"Mozilla/5\"\r\npages:\r\n  - name: Blog\r\n    link_pattern: /blog%\r\n    mappings:\r\n      - name: title\r\n        path: h3[class=\"event-title\"]/a\r\n```\r\n\r\n\r\nIt support login to website as well...\r\n--------------------------------------\r\n```yaml\r\nname: A super secret page\r\ndescription: Only we have access\r\nsite_root: http://page.secret\r\nstart_page: /restricted_area\r\nauth:\r\n  url: /login\r\n  method: POST\r\n  params:\r\n    user: {{ secret_user }}\r\n    password: {{ secret_password }}\r\n```\r\n\r\nLocked web is a big part of the internet, however it is rarely accessed by scrappers.\r\nThis tool gives you possibility to login to some of them (CAPTCHA is a bit pain).\r\n\r\nVariables `secret_user` and `secret_password` are being picket from file `.env`\r\nwhich would look like this:\r\n```bash\r\nsecret_user=demo\r\nsecret_password=demo\r\n```\r\n\r\nIt is done like this, because usually there are some variables we don't want to expose in config and put to any source control system.\r\n",
  "note": "Don't delete this file! It's used internally to help with page regeneration."
}