# Changelog

## 2.3.0

**15 December 2023**

- add `data` to key_for_request allowing caching of POSTs if desired, thanks for Forest Gregg
- improve tests/packaging issues, thanks to Forest Gregg

## 2.2.0

**18 May 2023**

- allow using `Scraper` as a context manager, thanks to Forest Gregg
- pin urllib3 to avoid version issues, thanks to Hannah Cushman Garland
- fix `mypy` issues, thanks to John Seekins

## 2.1.0

**6 November 2021**

- bugfix: use parameters in caching (https://codeberg.org/jpt/scrapelib/issues/110)

## 2.0.7

**6 July 2022**

- bump Cipherlist dependency

## 2.0.6

**23 June 2021**

- set default values for verify & allow_redirects to True (matching requests & expected behavior)

## 2.0.5

**15 June 2021**

- allow importing SQLiteCache and MemoryCache from scrapelib

## 2.0.4

**13 April 2021**

- bugfix: params was not carried through properly

## 2.0.2

**9 April 2021**

- added py.typed

## 2.0.1

**9 April 2021**

-   Python 3.7 installation fix
-   fixed docker image

## 2.0.0

**9 April 2021**

-   removed response.code backwards-compatibility shim
-   dropped Python 2 shims
-   removed/refactored some internal interfaces
-   added first pass at (experimental for now) mypy typing

## 1.2.0

**13 November 2018**

-   add verify parameter to optionally disable SSL verification

## 1.1.1

**16 April 2018**

-   small tweak to warning on retry
-   public email change

## 1.1.0

**6 June 2017**

- added more resilient retry logic that retries when any type of exception occurs during a request

## 1.0.2

**16 April 2017**

-   add retry on ChunkedEncodingError
-   depend on requests\[security\] for better SSL handling

## 1.0.1

**16 April 2017**

-   broken release

## 1.0.0

**20 March 2015**

- drop deprecated urlopen interface
- documentation thanks to poliquin sqlite cache thanks to poliquin
- fix for SSLError retries pointed out by Eric Mill

## 0.10.1

**22 January 2015**

- SQLite cache backend (thanks Chris Poliquin!)
- test and fix for header merging bug

## 0.10.0

**15 July 2014**

- added kwarg to use last modified headers when using caching, thanks to Kaitlin Devine
- fix for chardet bug when opening large PDFs (& other binary files) with urlretrieve - thanks to Katilin Devine
- deprecation of urlopen in favor of Requests\'s request(), get(), post(), etc.
- removal of robots.txt code
- switch tests to py.test
- addition of wheel for release

## 0.9.1

**28 March 2014**

- support kwargs in request paths -thanks to Drew Vogel
- allow_cookies removal and documentation fix -thanks to Joe Germuska
- add dir param to urlretrieve -thanks to Alison Rowland

## 0.9.0

**22 May 2013**

- replace FTPSession with FTPAdapter
- fixes for latest requests

## 0.8.0

**18 March 2013**

- requests 1.0 compatibility
 - removal of requests pass-throughs
 - deprecation of setting parameters via constructor

## 0.7.4

**20 December 2012**

- bugfix for status_code coming from a cache
- bugfix for setting user-agent from headers
- fix requests version at <1.0

## 0.7.3

**21 June 2012**

- fix for combination of FTP and caching
- drop unnecessary ScrapelibSession
- bytes fix for scrapeshell
- use UTF8 if encoding guess fails

## 0.7.2

**9 May 2012**

- bugfix for user-agent check
- bugfix for cached content with r characters
- bugfix for requests \>= 0.12
- cache_dir deprecation is total

## 0.7.1

**27 April 2012**

- breaking change: no longer accept URLs without a scheme
- deprecation of error_dir & context-manager mode
- addition of overridable accept_response hook
- bugfix: retry on more requests errors
- bugfix: unicode cached content no longer incorrectly encoded
- implement various requests enhancements separately for ease of reuse
- convert more Scraper parameters to properties

## 0.7.0

**23 April 2012**

- rewritten internals to use requests, dropping httplib2
- as a result of rewrite, caching behavior no longer attempts to
  be compliant with the HTTP specification but is much more
  configurable
- added cache_write_only option
- deprecation of accept_cookies, use_cache_first, cache_dir
  parameter
- improved tests
- improved Python 3 support

## 0.6.2

**20 April 2012**

- bugfix for POST-redirects
- drastically improved test coverage
- add encoding to ResultStr

## 0.6.1

**19 April 2012**

- add .bytes attribute to ResultStr
- bugfix related to bytes in urlretrieve

## 0.6.0

**19 April 2012**

- remove urllib2 fallback for HTTP
- rework entire test suite to not rely on Flask
- Unicode & Str unification
- experimental Python 3.2 support

## 0.5.8

**15 February 2012**

- fix to test suite from Alex Chiang

## 0.5.7

**2 February 2012**

- -p, \--postdata parameter
- argv fix for IPython \<= 0.10 from Joe Germuska
- treat FTP 550 errors as HTTP 404s
- use_cache_first improvements

## 0.5.6

**9 November 2011**

- scrapeshell fix for IPython \>= 0.11
- scrapelib.urlopen can take method/body params too

## 0.5.5

**27 September 2011**

- use None for no timeout, never create non-blocking socket
- documentation and owernship changes

## 0.5.4

**7 June 2011**

- actually fix reinstantiation of Http object

## 0.5.3

**7 June 2011**

- bugfix for reinstantiation of Http object

## 0.5.2

**16 May 2011**

- support timeout for urllib2 requests

## 0.5.1

**6 April 2011**

- bugfix for exception handling on retry
- fix a deprecation warning for Python 2.6+

## 0.5.0

**18 March 2011**

- sphinx documentation
- addition of scrapeshell
- addition of retry_on_404 parameter to urlopen
- bugfix to exception handling scope issue
- bugfix within tests to avoid false negative

## 0.4.3

**11 February 2011**

- fix retry on certain httplib2 errors
- add a top-level urlopen function

## 0.4.2

**8 February 2011**

- fix retry on socket errors
- close temporary file handle

## 0.4.1

**7 December 2010**

- support retry of requests that produce socket timeouts
- increased test coverage

## 0.4.0

**8 November 2010**

- bugfix: tests require unittest2 or python 2.7
- configurable retry handling for random failures

## 0.3.0

**5 October 2010**

- bugfixes for cookie handling
- better test suite
- follow redirects even after a POST
- change several configuration variables into properties
- request timeout argument

## 0.2.0

**9 July 2010**

- use_cache_first option to avoid extra HTTP HEAD requests
- raise_errors option to treat HTTP errors as exceptions
- addition of urlretrieve
