Metadata-Version: 2.1
Name: requests-tor
Version: 0.6.1
Summary: Multithreading requests via TOR with automatic TOR new identity
Home-page: https://github.com/deedy5/requests_tor
Author: deedy5
Author-email: deedy-ru@ya.ru
License: MIT
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: requests (>=2.25.0)
Requires-Dist: stem (>=1.8.0)

# requests_tor

Multithreading requests via [TOR](https://www.torproject.org) with automatic TOR new identity.

Wrapper of the [requests](https://docs.python-requests.org) and [stem](https://stem.torproject.org) libraries.
Returns [requests.Response](https://docs.python-requests.org/en/latest/api/#requests.Response) object.

Masking as Tor Browser by using its default headers:
``` "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-US,en;q=0.5",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
```

### Install

```
pip install -U requests_tor
```

### Dependencies
Download and start [Tor Browser](https://www.torproject.org/download/) or install [Tor](https://www.torproject.org/docs/installguide.html.en)

---
### Simple usage
```python
from requests_tor import RequestsTor

rt = RequestsTor() #for Tor Browser
rt = RequestsTor(tor_ports=(9050,), tor_cport=9051) #for Tor

url = 'https://foxnews.com'
r = rt.get(url)
print(r.text)

urls = (f'https://foxnews.com' for _ in range(10))
res = rt.get_urls(urls)
for result in res:
    print(result.text)
```

---
### Advanced usage
[Edit torrc file](https://support.torproject.org/tbb/tbb-editing-torrc/):

1. add [socks ports](https://www.torproject.org/docs/tor-manual.html.en#SocksPort),
```
SocksPort 9000 IsolateDestAddr
SocksPort 9001 IsolateDestAddr
SocksPort 9002 IsolateDestAddr
SocksPort 9003 IsolateDestAddr
SocksPort 9004 IsolateDestAddr
```
2. add password for control port [not necessary]:

generate and add in torrc file [HashedControlPassword](https://www.torproject.org/docs/tor-manual.html.en#HashedControlPassword).
```
HashedControlPassword hashed_password
```
---
```python
from requests_tor import RequestsTor

rt = RequestsTor(tor_ports=(9000, 9001, 9002, 9003, 9004), tor_cport=9151, password=None,
                 autochange_id=5, threads=8,)
"""
    tor_ports = specify Tor socks ports tuple (default is (9150,), as the default in Tor Browser),
    if more than one port is set, the requests will be sent sequentially through the each port;
    tor_cport = specify Tor control port (default is 9151 for Tor Browser, for Tor use 9051);
    password = specify Tor control port password (default is None);
    autochange_id = number of requests via a one Tor socks port (default=5) to change TOR identity;
    threads = specify threads to download urls list (default=8),
    """

# check your ip
rt.check_ip()

# new Tor identity. Сalling this function includes time.sleep(3)
rt.new_id()

# test automatic TOR new identity
rt.test()

# get url. TOR new identity is executed after (autochange_id * len(tor_ports)) requests.
# get(url) can accept params, headers and other arguments from requests library
url = 'https://foxnews.com'
r = rt.get(url)
print(r.text) 

# get urls list concurrently. TOR new identity is executed depending on the number of socksports and 
# autochange_id parameter. In case of 5 socksports and autochange_id=5, after downloading 5*5=25 urls
# TOR identity will be changed. It does matter, because calling TOR new identity includes time.sleep(3).
# get_urls(urls) can accept params, headers and other arguments from requests library.
urls = (f'https://api.my-ip.io/ip' for _ in range(10))
results = rt.get_urls(urls)
for result in results:
    print(result.url, result.text) 


 ```
### Example: downloading list of urls concurrently with unique ip for each url
Urls:  https://habr.com/ru/post/1 - https://habr.com/ru/post/50

```python
from requests_tor import RequestsTor

rt = RequestsTor(tor_ports=(9000, 9001, 9002, 9003, 9004), autochange_id=1)

urls = (f'https://habr.com/ru/post/{x}' for x in range(1, 50))
results = rt.get_urls(urls)
for result in results:
    print(result.status_code, result.url)
print(results[-1].text)
```


