PEP: 3123 Title: Making PyObject\_HEAD conform to standard C Version:
$Revision$ Last-Modified: $Date$ Author: Martin von Löwis
<martin@v.loewis.de> Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 27-Apr-2007 Python-Version: 3.0 Post-History:

Abstract
========

Python currently relies on undefined C behavior, with its usage of
`PyObject_HEAD`. This PEP proposes to change that into standard C.

Rationale
=========

Standard C defines that an object must be accessed only through a
pointer of its type, and that all other accesses are undefined behavior,
with a few exceptions. In particular, the following code has undefined
behavior::

struct FooObject{ PyObject\_HEAD int data; };

PyObject *foo(struct FooObject*f){ return (PyObject\*)f; }

int bar(){ struct FooObject *f = malloc(sizeof(struct FooObject));
struct PyObject *o = foo(f); f-\>ob\_refcnt = 0; o-\>ob\_refcnt = 1;
return f-\>ob\_refcnt; }

The problem here is that the storage is both accessed as if it where
struct `PyObject`, and as struct `FooObject`.

Historically, compilers did not have any problems with this code.
However, modern compilers use that clause as an optimization
opportunity, finding that `f->ob_refcnt` and `o->ob_refcnt` cannot
possibly refer to the same memory, and that therefore the function
should return 0, without having to fetch the value of ob\_refcnt at all
in the return statement. For GCC, Python now uses `-fno-strict-aliasing`
to work around that problem; with other compilers, it may just see
undefined behavior. Even with GCC, using `-fno-strict-aliasing` may
pessimize the generated code unnecessarily.

Specification
=============

Standard C has one specific exception to its aliasing rules precisely
designed to support the case of Python: a value of a struct type may
also be accessed through a pointer to the first field. E.g. if a struct
starts with an `int`, the `struct *` may also be cast to an `int *`,
allowing to write int values into the first field.

For Python, `PyObject_HEAD` and `PyObject_VAR_HEAD` will be changed to
not list all fields anymore, but list a single field of type
`PyObject`/`PyVarObject`::

typedef struct \_object { \_PyObject\_HEAD\_EXTRA Py\_ssize\_t
ob\_refcnt; struct \_typeobject \*ob\_type; } PyObject;

typedef struct { PyObject ob\_base; Py\_ssize\_t ob\_size; }
PyVarObject;

\#define PyObject\_HEAD PyObject ob\_base; \#define PyObject\_VAR\_HEAD
PyVarObject ob\_base;

Types defined as fixed-size structure will then include PyObject as its
first field, PyVarObject for variable-sized objects. E.g.::

typedef struct { PyObject ob\_base; PyObject *start, *stop, \*step; }
PySliceObject;

typedef struct { PyVarObject ob\_base; PyObject \*\*ob\_item;
Py\_ssize\_t allocated; } PyListObject;

The above definitions of `PyObject_HEAD` are normative, so extension
authors MAY either use the macro, or put the `ob_base` field explicitly
into their structs.

As a convention, the base field SHOULD be called ob\_base. However, all
accesses to ob\_refcnt and ob\_type MUST cast the object pointer to
PyObject\* (unless the pointer is already known to have that type), and
SHOULD use the respective accessor macros. To simplify access to
ob\_type, ob\_refcnt, and ob\_size, macros::

\#define Py\_TYPE(o) (((PyObject*)(o))-\>ob\_type) \#define
Py\_REFCNT(o) (((PyObject*)(o))-\>ob\_refcnt) \#define Py\_SIZE(o)
(((PyVarObject\*)(o))-\>ob\_size)

are added. E.g. the code blocks ::

\#define PyList\_CheckExact(op) ((op)-\>ob\_type == &PyList\_Type)

return func-\>ob\_type-\>tp\_name;

needs to be changed to::

\#define PyList\_CheckExact(op) (Py\_TYPE(op) == &PyList\_Type)

return Py\_TYPE(func)-\>tp\_name;

For initialization of type objects, the current sequence ::

PyObject\_HEAD\_INIT(NULL) 0, /\* ob\_size \*/

becomes incorrect, and must be replaced with ::

PyVarObject\_HEAD\_INIT(NULL, 0)

Compatibility with Python 2.6
=============================

To support modules that compile with both Python 2.6 and Python 3.0, the
`Py_*` macros are added to Python 2.6. The macros `Py_INCREF` and
`Py_DECREF` will be changed to cast their argument to `PyObject *`, so
that module authors can also explicitly declare the `ob_base` field in
modules designed for Python 2.6.

Copyright
=========

This document has been placed in the public domain.

 .. Local Variables: mode: indented-text indent-tabs-mode: nil
sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
