Metadata-Version: 2.1
Name: pickledir
Version: 0.1.3
Summary: File-based key-value storage. Keys are strings, values are objects pickle serializable.
Home-page: https://github.com/rtmigo/pickledir_py#readme
Author: Artёm IG
Author-email: ortemeo@gmail.com
License: MIT
Keywords: cache,pickle,file,directory
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Operating System :: POSIX
Classifier: Operating System :: Microsoft :: Windows
Description-Content-Type: text/markdown

File-based key-value storage.

Keys can be any string. Objects must be [pickle](https://docs.python.org/3/library/pickle.html) serializable.

---

Unit-tested with Python 3.6-3.9 on macOS, Ubuntu and Windows.

# Install

``` bash
$ pip3 install pickledir
```

# Use

### Save, read, delete

``` python3
from pickledir import PickleDir

cache = PickleDir('path/to/my_cache_dir')

# saving data to files
cache['a'] = 'hello, user!'
cache['b'] = 1
cache['c'] = [1, 2, 3, 4, 5]

# reading files
print(cache['a'])
print(cache['b'])
print(cache['c'])

# delete item
del cache['b']
```

### Read all values

``` python3
for key, value in cache.items():
    print(key, value)
```

### Set expiration time on writing

The expired items will be removed from the storage.

``` python3    
cache.set('a', 1000, max_age = datetime.timedelta(seconds=1))
print(cache.get('a'))  # 1000
time.sleep(2)     
print(cache.get('a'))  # None (and removed from storage)
```

### Set expiration time on reading

The expired items will not be returned, but kept in the storage.

``` python3
cache['b'] = 1000
time.sleep(2)
cache.get('b' max_age = datetime.timedelta(seconds=1)) # None
cache.get('b' max_age = datetime.timedelta(seconds=9)) # 1000
```

### Set data version 

``` python3 
cache = PickleDir('path/to/dir', version=1)
cache['a'] = 'some_data'
```

You can read all stored data while the `version` value is `1`.

``` python3 
cache = PickleDir('path/to/dir', version=1)
print(cache.get('a'))  # 'some_data'
```

If you decide that all the data in the cache is out of date, just pass the 
constructor a version number that you haven't used before.

``` python3 
cache = PickleDir('path/to/dir', version=2)
print(cache.get('a'))  # None
```

Now all that is saved with version `2` is actual data. Any other version is 
considered obsolete and will be gradually removed.

Do not create the `PickleDir` with an old version number. 
It will make the data unpredictable.

``` python3
cacheV1 = PickleDir('path/to/dir', version=1)  # ok
cacheV1['a'] = 'old A'
cacheV1['b'] = 'old B'

cacheV2 = PickleDir('path/to/dir', version=2)  # ok
cacheV2['a'] = 'new A'

cacheV1 = PickleDir('path/to/dir', version=1)  # don't do this
print(cacheV1.get('b'))  # Schrödinger's data ('old B' or None)
```


# Under the hood

The implementation is deliberately naive. Each file stores a pickled dictionary
(`dict_in_file: Dict`).
`dict_in_file[key]` keeps the value of the item associated with `key`.

With a small number of items, each item is stored in a separate file (in a
dictionary with a single item). With a larger number of items, some files will
store dictionaries with multiple items. There will be a maximum of 4096 files in
total. The string keys of two items generate the same hash, the items are stored
in the same file.

This solution is extremely simple and universally compatible.

However, with a large number or size of items, disadvantages also appear.
Reading and modifying files that contain multiple items is predictably slower:
we will read and save the whole dictionary each time.




