PEP: 231 Title: **findattr**() Version: $Revision$ Last-Modified: $Date$
Author: barry\@python.org (Barry Warsaw) Status: Rejected Type:
Standards Track Content-Type: text/x-rst Created: 30-Nov-2000
Python-Version: 2.1 Post-History:

Introduction
============

This PEP describes an extension to instance attribute lookup and
modification machinery, which allows pure-Python implementations of many
interesting programming models. This PEP tracks the status and ownership
of this feature. It contains a description of the feature and outlines
changes necessary to support the feature. This PEP summarizes
discussions held in mailing list forums, and provides URLs for further
information, where appropriate. The CVS revision history of this file
contains the definitive historical record.

Background
==========

The semantics for Python instances allow the programmer to customize
some aspects of attribute lookup and attribute modification, through the
special methods `__getattr__()` and `__setattr__()` \[1\]\_.

However, because of certain restrictions imposed by these methods, there
are useful programming techniques that can not be written in Python
alone, e.g. strict Java Bean-like \[2\]\_ interfaces and Zope style
acquisitions \[3\]*. In the latter case, Zope solves this by including a
C extension called ExtensionClass \[5\]* which modifies the standard
class semantics, and uses a metaclass hook in Python's class model
called alternatively the "Don Beaudry Hook" or "Don Beaudry Hack"
\[6\]\_.

While Zope's approach works, it has several disadvantages. First, it
requires a C extension. Second it employs a very arcane, but truck-sized
loophole in the Python machinery. Third, it can be difficult for other
programmers to use and understand (the metaclass has well-known brain
exploding properties). And fourth, because ExtensionClass instances
aren't "real" Python instances, some aspects of the Python runtime
system don't work with ExtensionClass instances.

Proposals for fixing this problem have often been lumped under the
rubric of fixing the "class/type dichotomy"; that is, eliminating the
difference between built-in types and classes \[7\]\_. While a laudable
goal itself, repairing this rift is not necessary in order to achieve
the types of programming constructs described above. This proposal
provides an 80% solution with a minimum of modification to Python's
class and instance objects. It does nothing to address the type/class
dichotomy.

Proposal
========

This proposal adds a new special method called `__findattr__()` with the
following semantics:

-   If defined in a class, it will be called on all instance attribute
    resolutions instead of `__getattr__()` and `__setattr__()`.

-   `__findattr__()` is never called recursively. That is, when a
    specific instance's `__findattr__()` is on the call stack, further
    attribute accesses for that instance will use the standard
    `__getattr__()` and `__setattr__()` methods.

-   `__findattr__()` is called for both attribute access ('getting') and
    attribute modification ('setting'). It is not called for attribute
    deletion.

-   When called for getting, it is passed a single argument (not
    counting 'self'): the name of the attribute being accessed.

-   When called for setting, it is called with third argument, which is
    the value to set the attribute to.

-   `__findattr__()` methods have the same caching semantics as
    `__getattr__()` and `__setattr__()`; i.e. if they are present in the
    class at class definition time, they are used, but if they are
    subsequently added to a class later they are not.

Key Differences with the Existing Protocol
==========================================

`__findattr__()`'s semantics are different from the existing protocol in
key ways:

First, `__getattr__()` is never called if the attribute is found in the
instance's `__dict__`. This is done for efficiency reasons, and because
otherwise, `__setattr__()` would have no way to get to the instance's
attributes.

Second, `__setattr__()` cannot use "normal" syntax for setting instance
attributes, e.g. "self.name = foo" because that would cause recursive
calls to `__setattr__()`.

`__findattr__()` is always called regardless of whether the attribute is
in `__dict__` or not, and a flag in the instance object prevents
recursive calls to `__findattr__()`. This gives the class a chance to
perform some action for every attribute access. And because it is called
for both gets and sets, it is easy to write similar policy for all
attribute access. Further, efficiency is not a problem because it is
only paid when the extended mechanism is used.

Related Work
============

PEP 213 \[9\]\_ describes a different approach to hooking into attribute
access and modification. The semantics proposed in PEP 213 can be
implemented using the `__findattr__()` hook described here, with one
caveat. The current reference implementation of `__findattr__()` does
not support hooking on attribute deletion. This could be added if it's
found desirable. See example below.

Examples
========

One programming style that this proposal allows is a Java Bean-like
interface to objects, where unadorned attribute access and modification
is transparently mapped to a functional interface. E.g.

::

    class Bean:
        def __init__(self, x):
            self.__myfoo = x

        def __findattr__(self, name, *args):
            if name.startswith('_'):
                # Private names
                if args: setattr(self, name, args[0])
                else:    return getattr(self, name)
            else:
                # Public names
                if args: name = '_set_' + name
                else:    name = '_get_' + name
                return getattr(self, name)(*args)

        def _set_foo(self, x):
            self.__myfoo = x

        def _get_foo(self):
            return self.__myfoo


    b = Bean(3)
    print b.foo
    b.foo = 9
    print b.foo

A second, more elaborate example is the implementation of both implicit
and explicit acquisition in pure Python::

    import types

    class MethodWrapper:
        def __init__(self, container, method):
            self.__container = container
            self.__method = method

        def __call__(self, *args, **kws):
            return self.__method.im_func(self.__container, *args, **kws)


    class WrapperImplicit:
        def __init__(self, contained, container):
            self.__contained = contained
            self.__container = container

        def __repr__(self):
            return '<Wrapper: [%s | %s]>' % (self.__container,
                                             self.__contained)

        def __findattr__(self, name, *args):
            # Some things are our own
            if name.startswith('_WrapperImplicit__'):
                if args: return setattr(self, name, *args)
                else:    return getattr(self, name)
            # setattr stores the name on the contained object directly
            if args:
                return setattr(self.__contained, name, args[0])
            # Other special names
            if name == 'aq_parent':
                return self.__container
            elif name == 'aq_self':
                return self.__contained
            elif name == 'aq_base':
                base = self.__contained
                try:
                    while 1:
                        base = base.aq_self
                except AttributeError:
                    return base
            # no acquisition for _ names
            if name.startswith('_'):
                return getattr(self.__contained, name)
            # Everything else gets wrapped
            missing = []
            which = self.__contained
            obj = getattr(which, name, missing)
            if obj is missing:
                which = self.__container
                obj = getattr(which, name, missing)
                if obj is missing:
                    raise AttributeError, name
            of = getattr(obj, '__of__', missing)
            if of is not missing:
                return of(self)
            elif type(obj) == types.MethodType:
                return MethodWrapper(self, obj)
            return obj


    class WrapperExplicit:
        def __init__(self, contained, container):
            self.__contained = contained
            self.__container = container

        def __repr__(self):
            return '<Wrapper: [%s | %s]>' % (self.__container,
                                             self.__contained)

        def __findattr__(self, name, *args):
            # Some things are our own
            if name.startswith('_WrapperExplicit__'):
                if args: return setattr(self, name, *args)
                else:    return getattr(self, name)
            # setattr stores the name on the contained object directly
            if args:
                return setattr(self.__contained, name, args[0])
            # Other special names
            if name == 'aq_parent':
                return self.__container
            elif name == 'aq_self':
                return self.__contained
            elif name == 'aq_base':
                base = self.__contained
                try:
                    while 1:
                        base = base.aq_self
                except AttributeError:
                    return base
            elif name == 'aq_acquire':
                return self.aq_acquire
            # explicit acquisition only
            obj = getattr(self.__contained, name)
            if type(obj) == types.MethodType:
                return MethodWrapper(self, obj)
            return obj

        def aq_acquire(self, name):
            # Everything else gets wrapped
            missing = []
            which = self.__contained
            obj = getattr(which, name, missing)
            if obj is missing:
                which = self.__container
                obj = getattr(which, name, missing)
                if obj is missing:
                    raise AttributeError, name
            of = getattr(obj, '__of__', missing)
            if of is not missing:
                return of(self)
            elif type(obj) == types.MethodType:
                return MethodWrapper(self, obj)
            return obj


    class Implicit:
        def __of__(self, container):
            return WrapperImplicit(self, container)

        def __findattr__(self, name, *args):
            # ignore setattrs
            if args:
                return setattr(self, name, args[0])
            obj = getattr(self, name)
            missing = []
            of = getattr(obj, '__of__', missing)
            if of is not missing:
                return of(self)
            return obj


    class Explicit(Implicit):
        def __of__(self, container):
            return WrapperExplicit(self, container)


    # tests
    class C(Implicit):
        color = 'red'

    class A(Implicit):
        def report(self):
            return self.color

    # simple implicit acquisition
    c = C()
    a = A()
    c.a = a
    assert c.a.report() == 'red'

    d = C()
    d.color = 'green'
    d.a = a
    assert d.a.report() == 'green'

    try:
        a.report()
    except AttributeError:
        pass
    else:
        assert 0, 'AttributeError expected'


    # special names
    assert c.a.aq_parent is c
    assert c.a.aq_self is a

    c.a.d = d
    assert c.a.d.aq_base is d
    assert c.a is not a


    # no acquisition on _ names
    class E(Implicit):
        _color = 'purple'

    class F(Implicit):
        def report(self):
            return self._color

    e = E()
    f = F()
    e.f = f
    try:
        e.f.report()
    except AttributeError:
        pass
    else:
        assert 0, 'AttributeError expected'


    # explicit
    class G(Explicit):
        color = 'pink'

    class H(Explicit):
        def report(self):
            return self.aq_acquire('color')

        def barf(self):
            return self.color

    g = G()
    h = H()
    g.h = h
    assert g.h.report() == 'pink'

    i = G()
    i.color = 'cyan'
    i.h = h
    assert i.h.report() == 'cyan'

    try:
        g.i.barf()
    except AttributeError:
        pass
    else:
        assert 0, 'AttributeError expected'

C++-like access control can also be accomplished, although less cleanly
because of the difficulty of figuring out what method is being called
from the runtime call stack::

    import sys
    import types

    PUBLIC = 0
    PROTECTED = 1
    PRIVATE = 2

    try:
        getframe = sys._getframe
    except ImportError:
        def getframe(n):
            try: raise Exception
            except Exception:
                frame = sys.exc_info()[2].tb_frame
            while n > 0:
                frame = frame.f_back
                if frame is None:
                    raise ValueError, 'call stack is not deep enough'
            return frame


    class AccessViolation(Exception):
        pass


    class Access:
        def __findattr__(self, name, *args):
            methcache = self.__dict__.setdefault('__cache__', {})
            missing = []
            obj = getattr(self, name, missing)
            # if obj is missing we better be doing a setattr for
            # the first time
            if obj is not missing and type(obj) == types.MethodType:
                # Digusting hack because there's no way to
                # dynamically figure out what the method being
                # called is from the stack frame.
                methcache[obj.im_func.func_code] = obj.im_class
            #
            # What's the access permissions for this name?
            access, klass = getattr(self, '__access__', {}).get(
                name, (PUBLIC, 0))
            if access is not PUBLIC:
                # Now try to see which method is calling us
                frame = getframe(0).f_back
                if frame is None:
                    raise AccessViolation
                # Get the class of the method that's accessing
                # this attribute, by using the code object cache
                if frame.f_code.co_name == '__init__':
                    # There aren't entries in the cache for ctors,
                    # because the calling mechanism doesn't go
                    # through __findattr__().  Are there other
                    # methods that might have the same behavior?
                    # Since we can't know who's __init__ we're in,
                    # for now we'll assume that only protected and
                    # public attrs can be accessed.
                    if access is PRIVATE:
                        raise AccessViolation
                else:
                    methclass = self.__cache__.get(frame.f_code)
                    if not methclass:
                        raise AccessViolation
                    if access is PRIVATE and methclass is not klass:
                        raise AccessViolation
                    if access is PROTECTED and not issubclass(methclass,
                                                              klass):
                        raise AccessViolation
            # If we got here, it must be okay to access the attribute
            if args:
                return setattr(self, name, *args)
            return obj

    # tests
    class A(Access):
        def __init__(self, foo=0, name='A'):
            self._foo = foo
            # can't set private names in __init__
            self.__initprivate(name)

        def __initprivate(self, name):
            self._name = name

        def getfoo(self):
            return self._foo

        def setfoo(self, newfoo):
            self._foo = newfoo

        def getname(self):
            return self._name

    A.__access__ = {'_foo'      : (PROTECTED, A),
                    '_name'     : (PRIVATE, A),
                    '__dict__'  : (PRIVATE, A),
                    '__access__': (PRIVATE, A),
                    }

    class B(A):
        def setfoo(self, newfoo):
            self._foo = newfoo + 3

        def setname(self, name):
            self._name = name

    b = B(1)
    b.getfoo()

    a = A(1)
    assert a.getfoo() == 1
    a.setfoo(2)
    assert a.getfoo() == 2

    try:
        a._foo
    except AccessViolation:
        pass
    else:
        assert 0, 'AccessViolation expected'

    try:
        a._foo = 3
    except AccessViolation:
        pass
    else:
        assert 0, 'AccessViolation expected'

    try:
        a.__dict__['_foo']
    except AccessViolation:
        pass
    else:
        assert 0, 'AccessViolation expected'


    b = B()
    assert b.getfoo() == 0
    b.setfoo(2)
    assert b.getfoo() == 5
    try:
        b.setname('B')
    except AccessViolation:
        pass
    else:
        assert 0, 'AccessViolation expected'

    assert b.getname() == 'A'

Here's an implementation of the attribute hook described in PEP 213
(except that hooking on attribute deletion isn't supported by the
current reference implementation).

::

    class Pep213:
        def __findattr__(self, name, *args):
            hookname = '__attr_%s__' % name
            if args:
                op = 'set'
            else:
                op = 'get'
            # XXX: op = 'del' currently not supported
            missing = []
            meth = getattr(self, hookname, missing)
            if meth is missing:
                if op == 'set':
                    return setattr(self, name, *args)
                else:
                    return getattr(self, name)
            else:
                return meth(op, *args)


    def computation(i):
        print 'doing computation:', i
        return i + 3


    def rev_computation(i):
        print 'doing rev_computation:', i
        return i - 3


    class X(Pep213):
        def __init__(self, foo=0):
            self.__foo = foo

        def __attr_foo__(self, op, val=None):
            if op == 'get':
                return computation(self.__foo)
            elif op == 'set':
                self.__foo = rev_computation(val)
            # XXX: 'del' not yet supported

    x = X()
    fooval = x.foo
    print fooval
    x.foo = fooval + 5
    print x.foo
    # del x.foo

Reference Implementation
========================

The reference implementation, as a patch to the Python core, can be
found at this URL:

http://sourceforge.net/patch/?func=detailpatch&patch\_id=102613&group\_id=5470

References
==========

.. \[1\]
http://docs.python.org/reference/datamodel.html\#customizing-attribute-access
.. \[2\] http://www.javasoft.com/products/javabeans/ .. \[3\]
http://www.digicool.com/releases/ExtensionClass/Acquisition.html ..
\[5\] http://www.digicool.com/releases/ExtensionClass .. \[6\]
http://www.python.org/doc/essays/metaclasses/ .. \[7\]
http://www.foretec.com/python/workshops/1998-11/dd-ascher-sum.html ..
\[8\] http://docs.python.org/howto/regex.html .. \[9\] PEP 213,
Attribute Access Handlers, Prescod
http://www.python.org/dev/peps/pep-0213/

Rejection
=========

There are serious problems with the recursion-protection feature. As
described here it's not thread-safe, and a thread-safe solution has
other problems. In general, it's not clear how helpful the
recursion-protection feature is; it makes it hard to write code that
needs to be callable inside `__findattr__` as well as outside it. But
without the recursion-protection, it's hard to implement `__findattr__`
at all (since `__findattr__` would invoke itself recursively for every
attribute it tries to access). There seems to be no good solution here.

It's also dubious how useful it is to support `__findattr__` both for
getting and for setting attributes -- `__setattr__` gets called in all
cases already.

The examples can all be implemented using `__getattr__` if care is taken
not to store instance variables under their own names.

Copyright
=========

This document has been placed in the Public Domain.

.. Local Variables: mode: indented-text indent-tabs-mode: nil End:
