Python: module AIGO.utils.shelve

AIGO.utils.shelve | index /home/dpm/projects/AIGO-0.1.0/AIGO/utils/shelve.py

`Manage shelves of pickled objects.
A "shelf" is a persistent, dictionary-like object. The difference
with dbm databases is that the values (not the keys!) in a shelf can
be essentially arbitrary Python objects -- anything that the "pickle"
module can handle. This includes most class instances, recursive data
types, and objects containing lots of shared sub-objects. The keys
are ordinary strings.
To summarize the interface (key is a string, data is an arbitrary
object):
import shelve
d = shelve.open(filename) # open, with (g)dbm filename -- no suffix
d[key] = data # store data at key (overwrites old data if
# using an existing key)
data = d[key] # retrieve a COPY of the data at key (raise
# KeyError if no such key) -- NOTE that this
# access returns a *copy* of the entry!
del d[key] # delete data stored at key (raises KeyError
# if no such key)
flag = d.has_key(key) # true if the key exists; same as "key in d"
list = d.keys() # a list of all existing keys (slow!)
d.close() # close it
Dependent on the implementation, closing a persistent dictionary may
or may not be necessary to flush changes to disk.
Normally, d[key] returns a COPY of the entry. This needs care when
mutable entries are mutated: for example, if d[key] is a list,
d[key].append(anitem)
does NOT modify the entry d[key] itself, as stored in the persistent
mapping -- it only modifies the copy, which is then immediately
discarded, so that the append has NO effect whatsoever. To append an
item to d[key] in a way that will affect the persistent mapping, use:
data = d[key]
data.append(anitem)
d[key] = data
To avoid the problem with mutable entries, you may pass the keyword
argument writeback=True in the call to shelve.open. When you use:
d = shelve.open(filename, writeback=True)
then d keeps a cache of all entries you access, and writes them all back
to the persistent mapping when you call d.close(). This ensures that
such usage as d[key].append(anitem) works as intended.
However, using keyword argument writeback=True may consume vast amount
of memory for the cache, and it may make d.close() very slow, if you
access many of d's entries after opening it in this way: d has no way to
check which of the entries you access are mutable and/or which ones you
actually mutate, so it must cache, and write back at close, all of the
entries that you access. You can call d.sync() to write back all the
entries in the cache, and empty the cache (d.sync() also synchronizes
the persistent dictionary on disk, if feasible).`

Modules
`` |  |
UserDict |  |  |

Classes
`` |  | UserDict.DictMixin Shelf BsdDbShelf DbfilenameShelf
class BsdDbShelf(Shelf)
`` | `Shelf implementation using the "BSD" db interface. This adds methods first(), next(), previous(), last() and set_location() that have no counterpart in [g]dbm databases. The actual database must be opened using one of the "bsddb" modules "open" routines (i.e. bsddb.hashopen, bsddb.btopen or bsddb.rnopen) and passed to the constructor. See the module's __doc__ string for an overview of the interface.`
| Method resolution order: BsdDbShelf Shelf UserDict.DictMixin Methods defined here: __init__(self, dict, protocol=None, writeback=False) first(self) last(self) next(self) previous(self) set_location(self, key) Methods inherited from Shelf: __contains__(self, key) __del__(self) __delitem__(self, key) __getitem__(self, key) __len__(self) __setitem__(self, key, value) close(self) get(self, key, default=None) has_key(self, key) keys(self) sync(self) Methods inherited from UserDict.DictMixin: __cmp__(self, other) __iter__(self) `# second level definitions support higher levels` __repr__(self) clear(self) items(self) iteritems(self) `# third level takes advantage of second level definitions` iterkeys(self) itervalues(self) `# fourth level uses definitions from lower levels` pop(self, key, *args) popitem(self) setdefault(self, key, default=None) update(self, other=None, **kwargs) values(self)
class DbfilenameShelf(Shelf)
`` | `Shelf implementation using the "anydbm" generic dbm interface. This is initialized with the filename for the dbm database. See the module's __doc__ string for an overview of the interface.`
| Method resolution order: DbfilenameShelf Shelf UserDict.DictMixin Methods defined here: __init__(self, filename, flag='c', protocol=None, writeback=False) Methods inherited from Shelf: __contains__(self, key) __del__(self) __delitem__(self, key) __getitem__(self, key) __len__(self) __setitem__(self, key, value) close(self) get(self, key, default=None) has_key(self, key) keys(self) sync(self) Methods inherited from UserDict.DictMixin: __cmp__(self, other) __iter__(self) `# second level definitions support higher levels` __repr__(self) clear(self) items(self) iteritems(self) `# third level takes advantage of second level definitions` iterkeys(self) itervalues(self) `# fourth level uses definitions from lower levels` pop(self, key, *args) popitem(self) setdefault(self, key, default=None) update(self, other=None, **kwargs) values(self)
class Shelf(UserDict.DictMixin)
`` | `Base class for shelf implementations. This is initialized with a dictionary-like object. See the module's __doc__ string for an overview of the interface.`
| Methods defined here: __contains__(self, key) __del__(self) __delitem__(self, key) __getitem__(self, key) __init__(self, dict, protocol=None, writeback=False) __len__(self) __setitem__(self, key, value) close(self) get(self, key, default=None) has_key(self, key) keys(self) sync(self) Methods inherited from UserDict.DictMixin: __cmp__(self, other) __iter__(self) `# second level definitions support higher levels` __repr__(self) clear(self) items(self) iteritems(self) `# third level takes advantage of second level definitions` iterkeys(self) itervalues(self) `# fourth level uses definitions from lower levels` pop(self, key, *args) popitem(self) setdefault(self, key, default=None) update(self, other=None, **kwargs) values(self)

Functions
`` |  | open(filename, flag='c', protocol=None, writeback=False) `Open a persistent dictionary for reading and writing. The filename parameter is the base filename for the underlying database. As a side-effect, an extension may be added to the filename and more than one file may be created. The optional flag parameter has the same interpretation as the flag parameter of anydbm.open(). The optional protocol parameter specifies the version of the pickle protocol (0, 1, or 2). See the module's __doc__ string for an overview of the interface.`

Data
`` |  | __all__ = ['Shelf', 'BsdDbShelf', 'DbfilenameShelf', 'open']