
Whats new in ZODB 3.8.2
=======================

Bugs Fixed:

- Fixed vulnerabilities in the ZEO network protocol that allow:

  CVE-2009-0668 Arbitrary Python code execution in ZODB ZEO storage servers
  CVE-2009-0669 Authentication bypass in ZODB ZEO storage servers

  The vulnerabilities only apply if you are using ZEO to share a
  database among multiple applications or application instances and if
  untrusted clients are able to connect to your ZEO servers.

Whats new in ZODB 3.8.1
=======================

Bugs Fixed:

- (3.8.1) Reset ``_cache`` on a connection's ``_reader`` object when
  resetting the cache, to prevent reads from the old cache object, e.g.
  during Zope2's auto-refresh of products.
  (https://bugs.launchpad.net/zodb/+bug/142667).

- (3.8.1) An exception would be raised when an error occured attempting to
  lock a file and logging of said error was enabled.

- (3.8.1) Fixed a bug to allow opening of deep-copied blobs.

- (3.8.1) Fixed bug #189542 by prepending the module to an undefined name.

- (3.8.1) Failures in tpc_finish of client-storages weren't handled
  correctly, leaving the client storage in an inconsistent state.

- (3.8.1) If there is a failure while FileStorage is finalizing a transaction,
  the file storage is closed because it's internal meta data may be
  invalid.

- (3.8.1) FileStorages previously saved indexes after a certain
  number of writes.  This was done during the last phase of two-phase
  commit, which made this critical phase more subject to errors than
  it should have been.  Also, for large databases, saves were done so
  infrequently as to be useless.  The feature was removed to reduce
  the chance for errors during the last phase of two-phase commit.

- (3.8.1) File storages previously kept an internal object id to
  transaction id mapping as an optimization. This mapping caused
  excessive memory usage and failures during the last phase of
  two-phase commit. This optimization has been removed.

- (3.8.1) Fixed a bug that caused deep copying of blobs to fail.

- (3.8.1) Refactored handling of invalidations on ZEO clients to fix
  a possible ordering problem for invalidation messages.

- (3.8.1) An ZEO cache internal data structure can get out of sync
  with the data in a way that prevents data from being loaded into the
  cache. We don't yet know why, but added an exception handler to
  prevent this error from being fatal.

- (3.8.1) Fixed setup.py use of setuptools vs distutils, so .c and .h
  files are included in the bdist_egg.

- (3.8.1) On many systems, it was impossible to create more than 32K
  blobs. Added a new blob-directory layout to work around this
  limitation.

- (3.8.1) Fixed a bug, introduced in an earlier beta, that allowed
  clients to connect to out of date servers.

- (3.8.1) Fixed bug that could lead to memory errors due to the use
  of a Python dictionary for a mapping that can grow large.

- (3.8.1) Fixed bug #251037: Made packing of blob storages non-blocking.

- (3.8.1) Fixed a bug that could cause InvalidObjectReference errors
  for objects that were explicitly added to a database if the object
  was modified after a savepoint that added the object.

- (3.8.1) Fixed several bugs that caused ZEO cache corruption when connecting
  to servers. These bugs affected both persistent and non-persistent caches. 

- (3.8.1) Improved the the ZEO client shutdown support to try to
  avoid spurious errors on exit, especially for scripts, such as zeopack.

- (3.8.1) Packing failed for databases containing cross-database references.

- (3.8.1) Cross-database references to databases with empty names
  weren't constructed properly.

- (3.8.1) The cache used an excessive amount of memory, causing applications
  with large caches to exhaust available memory. 

- Fixed a number of bugs in the handling of persistent ZEO caches:

  - Cache records are written in several steps.  If a process exits
    after writing begins and before it is finishes, the cache will be
    corrupt on restart.  The way records are writted was changed to
    make cache record updates atomic.

  - There was no lock file to prevent opening a cache multiple times
    at once, which would lead to corruption.  Persistent caches now
    use lock files, in the same way that file storages do.

  - A bug in the cache-opening logic led to cache failure in the
    unlikely event that a cache has no free blocks.

- When using ZEO Client Storages, Errors occured when trying to store
  objects too big to fit in the ZEO cache file.

- Fixed bug in blob filesystem helper: the `isSecure` check was inversed.

- Fixed bug in transaction buffer: a tuple was unpacked incorrectly in
  `clear`.

- Fixed bug in Connection.TmpStore: load() would not defer to the
  backend storage for loading blobs.

- Fixed bug #190884: Wrong reference to `POSKeyError` caused NameError.

- Completed implementation of ZEO authentication. This fixes issue 220856.

What's new on ZODB 3.8.0
========================

General
-------

- The ZODB Storage APIs have been documented and cleaned up.

- ZODB versions are now officially deprecated and support for them
  will be removed in ZODB 3.9.  (They have been widely recognized as
  deprecated for quite a while.)

- Changed the automatic garbage collection when opening a connection to only
  apply the garbage collections on those connections in the pool that are
  closed. (This fixed issue 113923.)

- (3.8.0b3) Document conflict resolution (see ZODB/ConflictResolution.txt).

- (3.8.0b3) Bugfix the situation in which comparing persistent objects (for
  instance, as members in BTree set or keys of BTree) might cause data
  inconsistency during conflict resolution.

- (3.8.0b3) Support multidatabase references in conflict resolution.

- (3.8.0b3) Make it possible to examine oid and (in some situations) database
  name of persistent object references during conflict resolution.

- (3.8.0b3) Added missing data attribute for conflict errors.

- (3.8.0b5) Fixed bug 153316: persistent and BTrees gave errors on x86_64
  Intel XEON platforms.

ZEO
---

- (3.8.0b6) Bug #98275: Made ZEO cache more tolerant when invalidating current
  versions of objects.

- (3.8.0b4, 3.8.0b5) Fixed a serious bug that could cause client I/O to stop
  (hang). This was accomonied by a critical log message along the
  lines of: "RuntimeError: dictionary changed size during iteration".
  (In b4, the bug was only partially fixed.)

- (3.8a1) ZEO's strategoes for avoiding client cache verification were
  improved in the case that servers are restarted.  Before, if
  transactions were committed after the restart, clients that were up
  to date or nearly up to date at the time of the restart and then
  connected had to verify their caches.  Now, it is far more likely
  that a client that reconnects soon after a server restart won't have
  to verify its cache.

- (3.8a1) Fixed a serious bug that could cause clients that disconnect from and
  reconnect to a server to get bad invalidation data if the server
  serves multiple storages with active writes.

- (3.8a1) It is now theoretically possible to use a ClientStorage in a storage
  server.  This might make it possible to offload read load from a
  storage server at the cost of increasing write latency.  This should
  increase write throughput by offloading reads from the final storage
  server.  This feature is somewhat experimental.  It has tests, but
  hasn't been used in production.

Transactions
------------

- (3.8a1) Add a doom() and isDoomed() interface to the transaction module.

  First step towards the resolution of
  http://www.zope.org/Collectors/Zope3-dev/655

  A doomed transaction behaves exactly the same way as an active transaction
  but raises an error on any attempt to commit it, thus forcing an abort.

  Doom is useful in places where abort is unsafe and an exception cannot be
  raised.  This occurs when the programmer wants the code following the doom to
  run but not commit. It is unsafe to abort in these circumstances as a
  following get() may implicitly open a new transaction.

  Any attempt to commit a doomed transaction will raise a DoomedTransaction
  exception.

- (3.8a1) Clean up the ZODB imports in transaction.

  Clean up weird import dance with ZODB. This is unnecessary since the
  transaction module stopped being imported in ZODB/__init__.py in rev 39622.

- (3.8a1) Support for subtransactions has been removed in favor of
  save points.

Blobs
-----

- (3.8b5) Fixed bug #130459: Packing was broken by uncommitted blob data.

- (3.8b4) Fixed bug #127182: Blobs were subclassable which was not desired.

- (3.8b3) Fixed bug #126007: tpc_abort had untested code path that was
  broken.

- (3.8b3) Fixed bug #129921: getSize() function in BlobStorage could not
  deal with garbage files

- (3.8b1) Updated the Blob implementation in a number of ways.  Some
  of these are backward incompatible with 3.8a1:

  o The Blob class now lives in ZODB.blob

  o The blob openDetached method has been replaced by the committed method.

- (3.8a1) Added new blob feature. See the ZODB/Blobs directory for
  documentation.

  ZODB now handles (reasonably) large binary objects efficiently. Useful to
  use from a few kilobytes to at least multiple hundred megabytes.

BTrees
------

- (3.8a1) Added support for 64-bit integer BTrees as separate types.  

  (For now, we're retaining compile-time support for making the regular
  integer BTrees 64-bit.)

- (3.8a1) Normalize names in modules so that BTrees, Buckets, Sets, and
  TreeSets can all be accessed with those names in the modules (e.g.,
  BTrees.IOBTree.BTree).  This is in addition to the older names (e.g.,
  BTrees.IOBTree.IOBTree).  This allows easier drop-in replacement, which
  can especially be simplify code for packages that want to support both
  32-bit and 64-bit BTrees.

- (3.8a1) Describe the interfaces for each module and actually declare
  the interfaces for each.

- (3.8a1) Fix module references so klass.__module__ points to the Python
  wrapper module, not the C extension.

- (3.8a1) introduce module families, to group all 32-bit and all 64-bit
  modules.
