PEP: 362 Title: Function Signature Object Version: $Revision$
Last-Modified: $Date$ Author: Brett Cannon <brett@python.org>, Jiwon Seo
<seojiwon@gmail.com>, Yury Selivanov <yury@edgedb.com>, Larry Hastings
<larry@hastings.org> Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 21-Aug-2006 Python-Version: 3.3 Post-History:
04-Jun-2012 Resolution:
https://mail.python.org/pipermail/python-dev/2012-June/120682.html

Abstract
========

Python has always supported powerful introspection capabilities,
including introspecting functions and methods (for the rest of this PEP,
"function" refers to both functions and methods). By examining a
function object you can fully reconstruct the function's signature.
Unfortunately this information is stored in an inconvenient manner, and
is spread across a half-dozen deeply nested attributes.

This PEP proposes a new representation for function signatures. The new
representation contains all necessary information about a function and
its parameters, and makes introspection easy and straightforward.

However, this object does not replace the existing function metadata,
which is used by Python itself to execute those functions. The new
metadata object is intended solely to make function introspection easier
for Python programmers.

Signature Object
================

A Signature object represents the call signature of a function and its
return annotation. For each parameter accepted by the function it stores
a `Parameter object`\_ in its `parameters` collection.

A Signature object has the following public attributes and methods:

-   return\_annotation : object The "return" annotation for the
    function. If the function has no "return" annotation, this attribute
    is set to `Signature.empty`.

-   parameters : OrderedDict An ordered mapping of parameters' names to
    the corresponding Parameter objects.

-   bind(\*args, \*\*kwargs) -\> BoundArguments Creates a mapping from
    positional and keyword arguments to parameters. Raises a `TypeError`
    if the passed arguments do not match the signature.

-   bind\_partial(\*args, \*\*kwargs) -\> BoundArguments Works the same
    way as `bind()`, but allows the omission of some required arguments
    (mimics `functools.partial` behavior.) Raises a `TypeError` if the
    passed arguments do not match the signature.

-   replace(parameters=<optional>, \*, return\_annotation=<optional>)
    -\> Signature Creates a new Signature instance based on the instance
    `replace` was invoked on. It is possible to pass different
    `parameters` and/or `return_annotation` to override the
    corresponding properties of the base signature. To remove
    `return_annotation` from the copied `Signature`, pass in
    `Signature.empty`.

    Note that the '=<optional>' notation, means that the argument is
    optional. This notation applies to the rest of this PEP.

Signature objects are immutable. Use `Signature.replace()` to make a
modified copy: ::

    >>> def foo() -> None:
    ...     pass
    >>> sig = signature(foo)

    >>> new_sig = sig.replace(return_annotation="new return annotation")
    >>> new_sig is not sig
    True
    >>> new_sig.return_annotation != sig.return_annotation
    True
    >>> new_sig.parameters == sig.parameters
    True

    >>> new_sig = new_sig.replace(return_annotation=new_sig.empty)
    >>> new_sig.return_annotation is Signature.empty
    True

There are two ways to instantiate a Signature class:

-   Signature(parameters=<optional>, \*,
    return\_annotation=Signature.empty) Default Signature constructor.
    Accepts an optional sequence of `Parameter` objects, and an optional
    `return_annotation`. Parameters sequence is validated to check that
    there are no parameters with duplicate names, and that the
    parameters are in the right order, i.e. positional-only first, then
    positional-or-keyword, etc.
-   Signature.from\_function(function) Returns a Signature object
    reflecting the signature of the function passed in.

It's possible to test Signatures for equality. Two signatures are equal
when their parameters are equal, their positional and positional-only
parameters appear in the same order, and they have equal return
annotations.

Changes to the Signature object, or to any of its data members, do not
affect the function itself.

Signature also implements `__str__`: ::

    >>> str(Signature.from_function((lambda *args: None)))
    '(*args)'

    >>> str(Signature())
    '()'

Parameter Object
================

Python's expressive syntax means functions can accept many different
kinds of parameters with many subtle semantic differences. We propose a
rich Parameter object designed to represent any possible function
parameter.

A Parameter object has the following public attributes and methods:

-   name : str The name of the parameter as a string. Must be a valid
    python identifier name (with the exception of `POSITIONAL_ONLY`
    parameters, which can have it set to `None`.)

-   default : object The default value for the parameter. If the
    parameter has no default value, this attribute is set to
    `Parameter.empty`.

-   annotation : object The annotation for the parameter. If the
    parameter has no annotation, this attribute is set to
    `Parameter.empty`.

-   kind Describes how argument values are bound to the parameter.
    Possible values:

    -   `Parameter.POSITIONAL_ONLY` - value must be supplied as a
        positional argument.

        Python has no explicit syntax for defining positional-only
        parameters, but many built-in and extension module functions
        (especially those that accept only one or two parameters) accept
        them.

    -   `Parameter.POSITIONAL_OR_KEYWORD` - value may be supplied as
        either a keyword or positional argument (this is the standard
        binding behaviour for functions implemented in Python.)

    -   `Parameter.KEYWORD_ONLY` - value must be supplied as a keyword
        argument. Keyword only parameters are those which appear after a
        \"\*\" or "\*args" entry in a Python function definition.

    -   `Parameter.VAR_POSITIONAL` - a tuple of positional arguments
        that aren't bound to any other parameter. This corresponds to a
        "\*args" parameter in a Python function definition.

    -   `Parameter.VAR_KEYWORD` - a dict of keyword arguments that
        aren't bound to any other parameter. This corresponds to a
        "\*\*kwargs" parameter in a Python function definition.

    Always use `Parameter.*` constants for setting and checking value of
    the `kind` attribute.

-   replace(\*, name=<optional>, kind=<optional>, default=<optional>,
    annotation=<optional>) -\> Parameter Creates a new Parameter
    instance based on the instance `replaced` was invoked on. To
    override a Parameter attribute, pass the corresponding argument. To
    remove an attribute from a `Parameter`, pass `Parameter.empty`.

Parameter constructor:

-   Parameter(name, kind, \*, annotation=Parameter.empty,
    default=Parameter.empty) Instantiates a Parameter object. `name` and
    `kind` are required, while `annotation` and `default` are optional.

Two parameters are equal when they have equal names, kinds, defaults,
and annotations.

Parameter objects are immutable. Instead of modifying a Parameter
object, you can use `Parameter.replace()` to create a modified copy like
so: ::

    >>> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42)
    >>> str(param)
    'foo=42'

    >>> str(param.replace())
    'foo=42'

    >>> str(param.replace(default=Parameter.empty, annotation='spam'))
    "foo:'spam'"

BoundArguments Object
=====================

Result of a `Signature.bind` call. Holds the mapping of arguments to the
function's parameters.

Has the following public attributes:

-   arguments : OrderedDict An ordered, mutable mapping of parameters'
    names to arguments' values. Contains only explicitly bound
    arguments. Arguments for which `bind()` relied on a default value
    are skipped.
-   args : tuple Tuple of positional arguments values. Dynamically
    computed from the 'arguments' attribute.
-   kwargs : dict Dict of keyword arguments values. Dynamically computed
    from the 'arguments' attribute.

The `arguments` attribute should be used in conjunction with
`Signature.parameters` for any arguments processing purposes.

`args` and `kwargs` properties can be used to invoke functions: ::

    def test(a, *, b):
        ...

    sig = signature(test)
    ba = sig.bind(10, b=20)
    test(*ba.args, **ba.kwargs)

Arguments which could be passed as part of either `*args` or `**kwargs`
will be included only in the `BoundArguments.args` attribute. Consider
the following example: ::

    def test(a=1, b=2, c=3):
        pass

    sig = signature(test)
    ba = sig.bind(a=10, c=13)

    >>> ba.args
    (10,)

    >>> ba.kwargs:
    {'c': 13}

Implementation
==============

The implementation adds a new function `signature()` to the `inspect`
module. The function is the preferred way of getting a `Signature` for a
callable object.

The function implements the following algorithm:

-   If the object is not callable - raise a TypeError

-   If the object has a `__signature__` attribute and if it is not
    `None` - return it

-   If it has a `__wrapped__` attribute, return
    `signature(object.__wrapped__)`

-   If the object is an instance of `FunctionType`, construct and return
    a new `Signature` for it

-   If the object is a bound method, construct and return a new
    `Signature` object, with its first parameter (usually `self` or
    `cls`) removed. (`classmethod` and `staticmethod` are supported too.
    Since both are descriptors, the former returns a bound method, and
    the latter returns its wrapped function.)

-   If the object is an instance of `functools.partial`, construct a new
    `Signature` from its `partial.func` attribute, and account for
    already bound `partial.args` and `partial.kwargs`

-   If the object is a class or metaclass:

    -   If the object's type has a `__call__` method defined in its MRO,
        return a Signature for it

    -   If the object has a `__new__` method defined in its MRO, return
        a Signature object for it

    -   If the object has a `__init__` method defined in its MRO, return
        a Signature object for it

-   Return `signature(object.__call__)`

Note that the `Signature` object is created in a lazy manner, and is not
automatically cached. However, the user can manually cache a Signature
by storing it in the `__signature__` attribute.

An implementation for Python 3.3 can be found at \[\#impl\]*. The python
issue tracking the patch is \[\#issue\]*.

Design Considerations
=====================

No implicit caching of Signature objects
----------------------------------------

The first PEP design had a provision for implicit caching of `Signature`
objects in the `inspect.signature()` function. However, this has the
following downsides:

-   If the `Signature` object is cached then any changes to the function
    it describes will not be reflected in it. However, If the caching is
    needed, it can be always done manually and explicitly

-   It is better to reserve the `__signature__` attribute for the cases
    when there is a need to explicitly set to a `Signature` object that
    is different from the actual one

Some functions may not be introspectable
----------------------------------------

Some functions may not be introspectable in certain implementations of
Python. For example, in CPython, built-in functions defined in C provide
no metadata about their arguments. Adding support for them is out of
scope for this PEP.

Signature and Parameter equivalence
-----------------------------------

We assume that parameter names have semantic significance--two
signatures are equal only when their corresponding parameters are equal
and have the exact same names. Users who want looser equivalence tests,
perhaps ignoring names of VAR\_KEYWORD or VAR\_POSITIONAL parameters,
will need to implement those themselves.

Examples
========

Visualizing Callable Objects' Signature
---------------------------------------

Let's define some classes and functions:

::

    from inspect import signature
    from functools import partial, wraps


    class FooMeta(type):
        def __new__(mcls, name, bases, dct, *, bar:bool=False):
            return super().__new__(mcls, name, bases, dct)

        def __init__(cls, name, bases, dct, **kwargs):
            return super().__init__(name, bases, dct)


    class Foo(metaclass=FooMeta):
        def __init__(self, spam:int=42):
            self.spam = spam

        def __call__(self, a, b, *, c) -> tuple:
            return a, b, c

        @classmethod
        def spam(cls, a):
            return a


    def shared_vars(*shared_args):
        """Decorator factory that defines shared variables that are
           passed to every invocation of the function"""

        def decorator(f):
            @wraps(f)
            def wrapper(*args, **kwargs):
                full_args = shared_args + args
                return f(*full_args, **kwargs)

            # Override signature
            sig = signature(f)
            sig = sig.replace(tuple(sig.parameters.values())[1:])
            wrapper.__signature__ = sig

            return wrapper
        return decorator


    @shared_vars({})
    def example(_state, a, b, c):
        return _state, a, b, c


    def format_signature(obj):
        return str(signature(obj))

Now, in the python REPL:

::

    >>> format_signature(FooMeta)
    '(name, bases, dct, *, bar:bool=False)'

    >>> format_signature(Foo)
    '(spam:int=42)'

    >>> format_signature(Foo.__call__)
    '(self, a, b, *, c) -> tuple'

    >>> format_signature(Foo().__call__)
    '(a, b, *, c) -> tuple'

    >>> format_signature(Foo.spam)
    '(a)'

    >>> format_signature(partial(Foo().__call__, 1, c=3))
    '(b, *, c=3) -> tuple'

    >>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
    '(*, c=20) -> tuple'

    >>> format_signature(example)
    '(a, b, c)'

    >>> format_signature(partial(example, 1, 2))
    '(c)'

    >>> format_signature(partial(partial(example, 1, b=2), c=3))
    '(b=2, c=3)'

Annotation Checker
------------------

::

    import inspect
    import functools

    def checktypes(func):
        '''Decorator to verify arguments and return types

        Example:

            >>> @checktypes
            ... def test(a:int, b:str) -> int:
            ...     return int(a * b)

            >>> test(10, '1')
            1111111111

            >>> test(10, 1)
            Traceback (most recent call last):
              ...
            ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
        '''

        sig = inspect.signature(func)

        types = {}
        for param in sig.parameters.values():
            # Iterate through function's parameters and build the list of
            # arguments types
            type_ = param.annotation
            if type_ is param.empty or not inspect.isclass(type_):
                # Missing annotation or not a type, skip it
                continue

            types[param.name] = type_

            # If the argument has a type specified, let's check that its
            # default value (if present) conforms with the type.
            if param.default is not param.empty and not isinstance(param.default, type_):
                raise ValueError("{func}: wrong type of a default value for {arg!r}". \
                                 format(func=func.__qualname__, arg=param.name))

        def check_type(sig, arg_name, arg_type, arg_value):
            # Internal function that encapsulates arguments type checking
            if not isinstance(arg_value, arg_type):
                raise ValueError("{func}: wrong type of {arg!r} argument, " \
                                 "{exp!r} expected, got {got!r}". \
                                 format(func=func.__qualname__, arg=arg_name,
                                        exp=arg_type.__name__, got=type(arg_value).__name__))

        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            # Let's bind the arguments
            ba = sig.bind(*args, **kwargs)
            for arg_name, arg in ba.arguments.items():
                # And iterate through the bound arguments
                try:
                    type_ = types[arg_name]
                except KeyError:
                    continue
                else:
                    # OK, we have a type for the argument, lets get the corresponding
                    # parameter description from the signature object
                    param = sig.parameters[arg_name]
                    if param.kind == param.VAR_POSITIONAL:
                        # If this parameter is a variable-argument parameter,
                        # then we need to check each of its values
                        for value in arg:
                            check_type(sig, arg_name, type_, value)
                    elif param.kind == param.VAR_KEYWORD:
                        # If this parameter is a variable-keyword-argument parameter:
                        for subname, value in arg.items():
                            check_type(sig, arg_name + ':' + subname, type_, value)
                    else:
                        # And, finally, if this parameter a regular one:
                        check_type(sig, arg_name, type_, arg)

            result = func(*ba.args, **ba.kwargs)

            # The last bit - let's check that the result is correct
            return_type = sig.return_annotation
            if (return_type is not sig._empty and
                    isinstance(return_type, type) and
                    not isinstance(result, return_type)):

                raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
                                 format(func=func.__qualname__, exp=return_type.__name__,
                                        got=type(result).__name__))
            return result

        return wrapper

Acceptance
==========

PEP 362 was accepted by Guido, Friday, June 22, 2012 \[\#accepted\]\_ .
The reference implementation was committed to trunk later that day.

References
==========

.. \[\#impl\] pep362 branch
(https://bitbucket.org/1st1/cpython/overview) .. \[\#issue\] issue 15008
(http://bugs.python.org/issue15008) .. \[\#accepted\] "A Desperate Plea
For Introspection (aka: BDFAP Needed)"
(https://mail.python.org/pipermail/python-dev/2012-June/120682.html)

Copyright
=========

This document has been placed in the public domain.

 .. Local Variables: mode: indented-text indent-tabs-mode: nil
sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
