ZODB, a Python object-oriented database
.. image:: https://img.shields.io/pypi/v/ZODB.svg :target: https://pypi.org/project/ZODB/ :alt: Latest release
.. image:: https://img.shields.io/pypi/pyversions/ZODB.svg :target: https://pypi.org/project/ZODB/ :alt: Supported Python versions
.. image:: https://github.com/zopefoundation/ZODB/actions/workflows/tests.yml/badge.svg :target: https://github.com/zopefoundation/ZODB/actions/workflows/tests.yml :alt: Build status
.. image:: https://coveralls.io/repos/github/zopefoundation/ZODB/badge.svg :target: https://coveralls.io/github/zopefoundation/ZODB :alt: Coverage status
.. image:: https://readthedocs.org/projects/zodb-docs/badge/?version=latest :target: https://zodb-docs.readthedocs.io/en/latest/ :alt: Documentation status
ZODB provides an object-oriented database for Python that provides a high-degree of transparency. ZODB runs on Python 2.7 or Python 3.4 and above. It also runs on PyPy.
no separate language for database operations
very little impact on your code to make objects persistent
no database mapper that partially hides the database.
Using an object-relational mapping is not like using an object-oriented database.
almost no seam between code and database.
ZODB is an ACID Transactional database.
To learn more, visit: https://zodb-docs.readthedocs.io
The github repository is at https://github.com/zopefoundation/zodb
If you're interested in contributing to ZODB itself, see the
developer notes <https://github.com/zopefoundation/ZODB/blob/master/DEVELOPERS.rst>
_.
Fix racetest
problems.
For details see #376 <https://github.com/zopefoundation/ZODB/pull/376>
_.
Fix --with-verify
argument in script repozo --recover
.
For details see #381 <https://github.com/zopefoundation/ZODB/pull/381>
_.
Add support for Python 3.11.
Expand and refactor tests for race conditions.
Fix TypeError: can't concat str to bytes
when running fsoids.py script
with Python 3.
See issue 350 <https://github.com/zopefoundation/ZODB/issues/350>
_.
Readd transaction size information to fsdump
output;
adapt fsstats
to fsdump
's exchanged order for size
and class
information in data records;
(fixes `#354 https://github.com/zopefoundation/ZODB/issues/354_).
Make fsdump
callable via Python's -m
command line option.
Fix UnboundLocalError when running fsoids.py script.
See issue 285 <https://github.com/zopefoundation/ZODB/issues/285>
_.
Rework fsrefs
script to work significantly faster by optimizing how it
does IO. See PR 340 <https://github.com/zopefoundation/ZODB/pull/340>
_.
Require Python 3 to build the documentation.
Fix deprecation warnings occurring on Python 3.10.
Add support for Python 3.9 and 3.10.
Fix race with invalidations when starting a new transaction. The bug
affected Storage implementations that rely on mvccadapter, and could result
in data corruption (oid loaded at wrong serial after a concurrent commit).
See issue 290 <https://github.com/zopefoundation/ZODB/issues/290>
_.
As mentionned in pull request #307, interfaces are clarified about the fact
that storage implementations must update at a precise moment the value that
is returned by lastTransaction(): just after invalidate() or
tpc_finish callback.
Improve volatile attribute _v_
documentation.
Make repozo's recover mode atomic by recovering the backup in a temporary file which is then moved to the expected output file.
Add a new option to repozo in recover mode which allows to verify backups integrity on the fly.
Drop support for Python 3.4.
Add support for Python 3.8.
Fix DB.undo()
and DB.undoMultiple()
to close the storage
they open behind the scenes when the transaction is committed or
rolled back. See issue 268 <https://github.com/zopefoundation/ZODB/issues/268>
_.
Make TransactionMetaData in charge of (de)serializing extension data.
A new extension_bytes
attribute converts automatically from
extension
, or vice-versa. During storage iteration, extension_bytes
holds bytes as they are stored (i.e. no deserialization happens).
See issue 207 <https://github.com/zopefoundation/ZODB/pull/207>
_.
Make a connection's savepoint storage implement its own
(approximate) getSize
method instead of relying on the original
storage. Previously, this produced confusing DEBUG logging. See
issue 282 <https://github.com/zopefoundation/ZODB/issues/282>
_.
Fix tests with transaction 3.0.
Fix inconsistent resolution order with zope.interface v5.
Remove ConnectionPool.map()
. Instead, ConnectionPool
is now
iterable. See PR 280 <https://github.com/zopefoundation/ZODB/pull/280>
_.
transaction
package.
See issue 208 <https://github.com/zopefoundation/ZODB/issues/208>
_.Add support for Python 3.7.
Bump the dependency on zodbpickle to at least 1.0.1. This is
required to avoid a memory leak on Python 2.7. See issue 203 <https://github.com/zopefoundation/ZODB/issues/203>
_.
Bump the dependency on persistent to at least 4.4.0.
Make the internal support functions for dealing with OIDs (p64
and u64
) somewhat faster and raise more informative
exceptions on certain types of bad input. See issue 216 <https://github.com/zopefoundation/ZODB/issues/216>
_.
Remove support for python setup.py test
. It hadn't been working
for some time. See issue #218 <https://github.com/zopefoundation/ZODB/issues/218>
_.
Make the tests run faster by avoiding calls to time.sleep()
.
ZODB now uses pickle protocol 3 for both Python 2 and Python 3.
(Previously, protocol 2 was used for Python 2.)
The zodbpickle package provides a zodbpickle.binary
string type
that should be used in Python 2 to cause binary strings to be saved
in a pickle binary format, so they can be loaded correctly in
Python 3. Pickle protocol 3 is needed for this to work correctly.
Object identifiers in persistent references are saved as
zodbpickle.binary
strings in Python 2, so that they are loaded
correctly in Python 3.
If an object is missing from the index while packing a FileStorage
,
report its full oid
.
Storage imports are a bit faster.
Storages can be important from non-seekable sources, like file-wrapped pipes.
Add support for Python 3.6.
Drop support for Python 3.3.
Ensure that the HistoricalStorageAdapter
forwards the release
method to
its base instance. See issue 78 <https://github.com/zopefoundation/ZODB/issues/788>
_.
Use a higher pickle protocol (2) for serializing objects on Python 2; previously protocol 1 was used. This is much more efficient for new-style classes (all persistent objects are new-style), at the cost of being very slightly less efficient for old-style classes.
.. note:: On Python 2, this will now allow open file
objects
(but not open blobs or sockets) to be pickled (loading
the object will result in a closed file); previously this
would result in a TypeError
. Doing so is not
recommended as they cannot be loaded in Python 3.
See issue 179 <https://github.com/zopefoundation/ZODB/pull/179>
_.
DB.close
now explicitly frees internal resources. This is
helpful to avoid false positives in tests that check for leaks.
Optimize getting the path to a blob file. See
issue 161 <https://github.com/zopefoundation/ZODB/pull/161>
_.
All classes are new-style classes on Python 2 (they were already
new-style on Python 3). This improves performance on PyPy. See
issue 160 <https://github.com/zopefoundation/ZODB/pull/160>
_.
issue 158 <https://github.com/zopefoundation/ZODB/issues/158>
_.umask
).
See issue 155 <https://github.com/zopefoundation/ZODB/issues/155>
_.Call new afterCompletion API on storages to allow them to free
resources after transaction complete.
See issue 147 <https://github.com/zodb/relstorage/issues/147>
__.
Take advantage of the new transaction-manager explicit mode to avoid starting transactions unnecessarily when transactions end.
Connection.new_oid
delegates to its storage, not the DB. This is
helpful for improving concurrency in MVCC storages like RelStorage.
See issue 139 <https://github.com/zopefoundation/ZODB/issues/139>
_.
persistent
is no longer required at setup time.
See issue 119 <https://github.com/zopefoundation/ZODB/issues/119>
_.
Connection.close
and Connection.open
no longer race on
self.transaction_manager
, which could lead to
AttributeError
. This was a bug introduced in 5.0.1. See issue 142 <https://github.com/zopefoundation/ZODB/pull/143>
_.
ZODB.Connection.TransactionMetaData
didn't support custom data
storage that some storages rely on.ZODB now translates transaction meta data, user
and
description
from text to bytes before passing them to storages,
and converts them back to text when retrieving them from storages in
the history
, undoLog
and undoInfo
methods.
The IDatabase
interface was updated to reflect that history
,
undoLog
and undoInfo
are available on database objects.
(They were always available, but not documented in the interface.)
Fix an AttributeError that DemoStorage could raise if it was asked
to store a blob into a temporary changes before reading a blob. See
issue 103 <https://github.com/zopefoundation/ZODB/issues/103>
_.
Call _p_resolveConflict() even if a conflicting change doesn't change the state. This reverts to the behaviour of 3.10.3 and older.
Closing a Connection now reverts its transaction_manager
to
None. This helps prevent errors and release resources when the
transaction_manager
was the (default) thread-local manager. See
issue 114 <https://github.com/zopefoundation/ZODB/issues/114>
_.
Many docstrings have been improved.
Major internal improvements and cleanups plus:
Added a connection prefetch
method that can be used to request
that a storage prefetch data an application will need::
conn.prefetch(obj, ...)
Where arguments can be objects, object ids, or iterables of objects or object ids.
Added optional prefetch
methods to the storage APIs. If a
storage doesn't support prefetch, then the connection prefetch
method is a noop.
fstail: print the txn offset and header size, instead of only the data offset. fstail can now be used to truncate a DB at the right offset.
Drop support for old commit protocol. All of the build-in storages implement the new protocol. This new protocol allows storages to provide better write performance by allowing multiple commits to execute in parallel.
Numerous internal cleanups, including:
Changed the way the root object was created. Now the root object is created using a database connection, rather than by making low-level storage calls.
Drop support for the old commit protocol.
Internal FileStorage-undo fixes that should allow undo in some cases where it didn't work before.
Drop the version
argument to some methods where it was the last
argument and optional.
Added a connection prefetch
method that can be used to request
that a storage prefect data an application will need::
conn.prefetch(obj, ...)
Where arguments can be objects, object ids, or iterables of objects or object ids.
Added optional prefetch
methods to the storage APIs. If a
storage doesn't support prefetch, then the connection prefetch
method is a noop.
Drop support for old commit protocol. All of the build-in storages implement the new protocol. This new protocol allows storages to provide better write performance by allowing multiple commits to execute in parallel.
See 4.4.2.
See 4.4.1.
See 4.4.0.
Major internal implementation changes to the Multi Version Concurrency Control (MVCC) implementation:
For storages that implement IMVCCStorage (RelStorage), no longer implement MVCC in ZODB.
For other storages, MVCC is implemented using an additional storage
layer. This underlying layer works by calling loadBefore
. The
low-level storage load
method isn't used any more.
This change allows server-based storages like ZEO and NEO to be implemented more simply and cleanly.
Internal FileStorage-undo fixes that should allow undo in some cases where it didn't work before.
fstail: print the txn offset and header size, instead of only the data offset. fstail can now be used to truncate a DB at the right offset.
Better support of the new commit protocol. This fixes issues with blobs and undo. See pull requests #77, #80, #83
Added IMultiCommitStorage to directly represent the changes in the 4.4.0 release and to make complient storages introspectable.
This release begins evolution to a more effcient commit protocol that
allows storage implementations, like NEO <http://www.neoppod.org/>
_,
to support multiple transactions committing at the same time, for
greater write parallelism.
This release updates IStorage:
The committed transaction's ID is returned by tpc_finish
, rather
than being returned in response store and tpc_vote results.
tpc_vote
is now expected to return None
or a list of object
ids for objects for which conflicts were resolved.
This release works with storages that implemented the older version of the storage interface, but also supports storages that implement the updated interface.
Drop support for Python 2.6 and 3.2.
Make the zodbpickle
dependency required and not conditional.
This fixes various packaging issues involving pip and its wheel
cache. zodbpickle was only optional under Python 2.6 so this change
only impacts users of that version. See
https://github.com/zopefoundation/ZODB/pull/42.
Add support for Python 3.5.
Avoid failure during cleanup of nested databases that provide MVCC on storage level (Relstorage). https://github.com/zopefoundation/ZODB/issues/45
Remove useless dependency to zdaemon
in setup.py. Remove ZEO documentation.
Both were leftovers from the time where ZEO was part of this repository.
Fix possible data corruption after FileStorage is truncated to roll back a transaction. https://github.com/zopefoundation/ZODB/pull/52
DemoStorage: add support for conflict resolution and fix history() https://github.com/zopefoundation/ZODB/pull/58
Fixed a test that depended on implementation-specific behavior in tpc_finish
Log failed conflict resolution attempts at DEBUG
level. See:
https://github.com/zopefoundation/ZODB/pull/29.
Fix command-line parsing of --verbose
and --verify
arguments.
(The short versions, -v
and -V
, were parsed correctly.)
Add support for PyPy.
Fix the methods in ZODB.serialize
that find object references
under Python 2.7 (used in scripts like referrers
, netspace
,
and fsrecover
among others). This requires the addition of the
zodbpickle
dependency.
FileStorage: fix an edge case when disk space runs out while packing,
do not leave the .pack
file around. That would block any write to the
to-be-packed Data.fs
, because the disk would stay at 0 bytes free.
See https://github.com/zopefoundation/ZODB/pull/21.
Fix registration of custom logging level names ("BLATHER", "TRACE").
We have been registering them in the wrong order since 2004. Before
Python 3.4, the stdlib logging
module masked the error by registering
them in both directions.
Add support for Python 3.4.
Fix POSKeyError
during transaction.commit
when after
savepoint.rollback
. See
https://github.com/zopefoundation/ZODB/issues/16
Ensure that the pickler used in PyPy always has a persistent_id
attribute (inst_persistent_id
is not present on the pure-Python
pickler). (PR #17)
Provide better error reporting when trying to load an object on a closed connection.
Finally released.
Switch to using non-backward-compatible pickles (protocol 3, without storing bytes as strings) under Python 3. Updated the magic number for file-storage files under Python3 to indicate the incompatibility.
Fixed: A UnicodeDecodeError
could happen for non-ASCII OIDs
when using bushy blob layout.
Extended the filename renormalizer used for blob doctests to support the filenames used by ZEO in non-shared mode.
Added url
parameter to setup()
(PyPI says it is required).
Skipped non-unit tests in setup.py test
. Use the buildout to run tests
requiring "layer" support.
Included the filename in the exception message to support debugging in case
loadBlob
does not find the file.
Added support for Python 3.2 / 3.3.
.. note::
ZODB 4.0.x is supported on Python 3.x for new applications only. Due to changes in the standard library's pickle support, the Python3 support does not provide forward- or backward-compatibility at the data level with Python2. A future version of ZODB may add such support.
Applications which need migrate data from Python2 to Python3 should plan to script this migration using separte databases, e.g. via a "dump-and-reload" approach, or by providing explicit fix-ups of the pickled values as transactions are copied between storages.
_p_serial
of persistent objects (fixes
compatibility with recent persistent releases).The persistent
and BTrees
packages are now released as separate
distributions, on which ZODB now depends.
ZODB no longer depends on zope.event. It now uses ZODB.event, which uses zope.event if it is installed. You can override ZODB.event.notify to provide your own event handling, although zope.event is recommended.
BTrees allowed object keys with insane comparison. (Comparison inherited from object, which compares based on in-process address.) Now BTrees raise TypeError if an attempt is made to save a key with comparison inherited from object. (This doesn't apply to old-style class instances.)
Ensured that the export file and index file created by repozo
share
the same timestamp.
https://bugs.launchpad.net/zodb/+bug/993350
Pinned the transaction
and manuel
dependencies to Python 2.5-
compatible versions when installing under Python 2.5.
.. note:: Please see https://github.com/zopefoundation/ZODB/blob/master/HISTORY.rst for older versions of ZODB.