.. Copyright 2013 Reahl Software Services (Pty) Ltd. All rights reserved.
 
Stubble -- A collection of tools for writing stubs in unit tests (reahl.stubble)
================================================================================

Introduction
^^^^^^^^^^^^

Stubble is a set of tools which can be used to simplify Python unit
testing code by means of the responsible use of so-called "stub"
classes. Stubble includes several special classes that can be used
to stub out (and restore) sys.stdout or to mock methods on a class 
temporarily.

Stubble allows you to write arbitrary classes for use as stubs instead
of real classes while testing.  This can be dangerous in a test, since
your ad-hoc stub classes may implement interfaces that are not
representative of the real classes they stand in for.

However, Stubble lets you link a stub class loosely to the real class
which it is a stub for.  This information is then used to ensure that
tests will break if there is a discrepancy between the interface
supported by your stub class and that of the real class it stands in
for.

Stubble also includes some support for stubbing a Python egg and its
associated meta information when you are using setuptools.

(More information is provided in the Overview below.)




Stub classes
^^^^^^^^^^^^

What stub classes are
---------------------

A stub class is a class you use in a test as a substitute for another
class.  Here are a few reasons for wanting to use such stub classes:

- You may need to provide an instance of, say, MyBulkyRealClass in a
  test fixture.  But constructing such an instance may be difficult,
  cumbersome, or outright impossible.  For example: your test code may
  only need to call a single method on the provided MyBulkyRealClass
  instance - but merely in order to construct the needed instance, you
  have to construct a forest of other instances.  This is
  unnecessary.  Another example is where you need to construct an
  instance of a GUI library object, such as a Window...but, the GUI
  toolkit you are using may not allow instantiation of such objects
  outside of its control.

  In both these examples, it would have been much easier to just
  create a simple class containing the single method which would be
  called by your testing code - possibly even with an implementation
  hard-coded for this particular scenario.

- You may want to decouple the unit tests of different modules: you
  don't really want to have the unit tests of one module fail because
  of errors in another module.  By using stub classes for real classes
  in other modules, the unit tests of the module being tested are not
  dependent on the actual other module, just on its expected interface.

- You may want to monitor certain things in a test, such as the values
  of parameters that were passed to a particular method call.  Or, you
  may want to hard-code a special implementation for a particular
  method specifically for your test scenario.

The careful use of stub classes can greatly simplify and speed up the
writing of unit test code.


Prerequisites for examples
--------------------------

This section shows by example how the stub classes in Stubble can be
used.  The examples all assume the following real class::

  class RealClass(object):
      b = 123
    
      def foo(self, a):
          print 'i am the real foo'

      def bar(self):
          print 'i am the real bar'
 

Basic functionality
-------------------

Any class can be a Stubble-enabled stub.  To create a stub class, you
just prepend the class with a descriptor like this::

  from reahl.stubble import stubclass
  
  @stubclass(RealClass)
  class Stub(object):
      def foo(self, a):
          'i am a fake foo'

The class ``Stub`` is now declared as being a stub **of RealClass**.
When Python encounters the declaration, it will check *each* method
declared on this class and make sure that:

  - There is a similarly named method on the real class; and
  - the corresponding method on the real class has the exact same
    signature as declared by the stub.

If any of these conditions are not met, an exception is raised.

The simplest solution is usually to derive stub classes from
``object``.  But, sometimes it is useful to derive from the real class
which is being stubbed.

The main reason for *not* deriving from the real class is usually that
parameters (or other side-effects) needed for the __init__ of the real
class are cumbersome to provide (or unwanted).

But, sometimes, you want to inherit from the real class, to inherit
some its real behaviour, but also override some of it::

  from reahl.stubble import stubclass
  
  @stubclass(RealClass)
  class Stub(RealClass):
      def foo(self, a):
          self.foo_called = True


Exempt methods
--------------

Not all methods on a stub class need to be strictly checked against
those in the real class - utility methods you use in the test, for
instance.  To mark a method as being *exempt* from the checking, you
can use a decorator on that method::

  from reahl.stubble import stubclass, exempt
  
  @stubclass(RealClass)
  class Stub(object):
      @exempt
      def my_own_method(self):
          print 'i am my own method'


Attributes
----------

At first it was thought important to let Stubble do strict checking on
any class attribute similarly to what it does for methods.  In
practice we found that it is a bit bothersome to do that.  But if you
really want to, you can do the following::

  from reahl.stubble import stubclass
  
  @stubclass(RealClass, check_attributes_also=True)
  class Stub(object):
      b = 'a value'

This would additionally check that RealClass has an attribute "b" and
fail if it does not have one.



EasterEggs
^^^^^^^^^^

Setuptools provide (amongst others) support for Python eggs.  Python
eggs are somewhat similar to OSGI bundles: they are components which
consist (mainly) of Python code, but also have metadata associated
with them.  Amongst other things, this allows components to publish
their interfaces, and it allows components to specify how they can be
extended by 3rd parties.  Setuptools also includes a method (the
ResourceManager API) by which packages can request the contents of
"files" regardless of how these files have been packaged or where they
are physically located.

To help here, Stubble provides the EasterEgg. EasterEggs are not real
Python eggs. A correctly initialised global EasterEgg instance is
constructed as ``reahl.stubble.easter_egg``.  It should suffice for most
purposes, but you *can* construct additional instances if needed.

Just be sure to always add your EasterEggs to
``pkg_resources.working_set``  (or similar), else they won't have any
effect.  Also, each added EasterEgg should be named uniquely (or it
won't be added).



Stub entry points
-----------------

If you work with setuptools you may be testing code to which you want
to supply stub objects via the setuptools entry point mechanism.

EasterEgg has two methods for adding stub classes as entry points,
exemplified here::

  reahl.stubble.easter_egg.add_entry_point_from_line(group_name, 
                        'test1 = examples.setuptools:StubClass1')
  reahl.stubble.easter_egg.add_entry_point(group_name, 'test2', StubClass2)

Actual code under test would now probably do something like this (and be
oblivious to the fact that the provided classes are stubs)::

  #  (we just print out each class it finds...)
  for i in pkg_resources.iter_entry_points(group_name):
      print i.load()


Where you tear down test fixtures after a test run, you should clear
the fake entry points registered with the easter_egg::

  reahl.stubble.easter_egg.clear()


Stubbed resources
-----------------

The EasterEgg can also be used for testing code that makes use of the
ResourceManager API in setuptools.  You can put the real files that
the ResourceManager API should provide in a directory somewhere - and
then specify that location as the EasterEgg's module_path::

  reahl.stubble.easter_egg.set_module_path('/some/where')

(By default, this path is os.getcwd())


Intercepting calls
^^^^^^^^^^^^^^^^^^

Stubble includes a number of classes that can be used as context
managers to temporarily intercept calls to code that may not even be
under your control. This is done by swapping certain methods out
temporarily for special ones that are restored after a particular
block of code finished executing.

SystemOutStub
-------------

.. autoclass:: reahl.stubble.intercept.SystemOutStub
   :members:

CallMonitor
-----------

.. autoclass:: reahl.stubble.intercept.CallMonitor
   :members:

MonitoredCall
-------------

.. autoclass:: reahl.stubble.intercept.MonitoredCall
   :members:

InitMonitor
-----------

.. autoclass:: reahl.stubble.intercept.InitMonitor
   :members:

replaced
--------

.. autofunction:: reahl.stubble.intercept.replaced



Experimental features
^^^^^^^^^^^^^^^^^^^^^

The simple functionality of Stubble explained so far is really what it
is all about.  But, having started off as an experiment, Stubble
provides several interesting experimental features.  However
interesting some of these may sound, they are not really used at all in
practice... sometimes because they're just theoretically nice ideas
with little use in practice; sometimes because they're nice ideas that
proved too difficult to implement transparently.

They're provided as part of Stubble for interest's sake.  Maybe a
skilled Python programmer out there feels like the challenge...


Impostors
---------

Passing a stub class instead of the real class to code that is being
tested works well in most cases.  The notable exception is when the
code actually checks the type of the class itself, such as with
``isinstance`` or ``issubclass``, etc.

Ideally speaking, you'd want a stub that pretends in all respects to
be the real thing to the code being tested.  Impostors are an attempt
at such stub classes.

To make your stub class an Impostor, you have to let it derive from
``reahl.stubble.Impostor``::

  from reahl.stubble import Impostor, stubclass
  
  @stubclass(RealClass)
  class Stub(Impostor):
      pass


With such a declaration, you gain the dubious benefit expressed in the
code::

  assert not issubclass(Stub, RealClass)  #issubclass catches Impostors out
  s = Stub()
  assert isinstance(s, RealClass)         #but the foolery works well here


The value derived from this is debatable... most often you get more
benefit by deriving your stub class from the real class it is a stub
for.


Delegation
----------

Delegation is actually a more useful idea than Impostors.  It arose
from the problem sometimes encounters where the programmer does not
have control over the creation of the instance that has to be stubbed.

For example, a GUI framework may create a bunch of instances for you,
and you just want to *replace* one node in this object forest with a
special kind of stub *instance*.  Also, you actually would only want
to override a single method, but have the rest of the behaviour
delegated to the real instance "as usual".

Delegation (in this context) is the strategy of creating a stub class
whose instance is 'superimposed' upon an instance of a real class.

A delegate is declared like this::

  from reahl.stubble import stubclass, Delegate
  
  @stubclass(RealClass)
  class Stub(Delegate):

      shadowed = ['foo', 'aa']

      def foo(self, a):
          print 'i am a fake foo'
 

And instance of it is then created like this::

  real_instance = RealClass()  # first we need an instance to delegate to
  s = Stub(real_instance)      # the stub instance


Our stub instance how has the following interesting behaviour::

  assert isinstance(s, RealClass)  # Delegates act like Impostors do
  s.foo(1)                         # calls the fake
  s.bar()                          # calls the real

  try:
      s.aa                         # breaks regardless of
				   #  whether or not aa is on real_instance
  except:
      pass

  s.aa = 123                              # `aa` is set on the fake
  assert s.aa == 123                      # `aa` is read from the fake
  assert not hasattr(real_instance, 'aa')  # see, it was not set there


The unsolved problem with Delegation
------------------------------------

At present, although such Delegates would have been nice to have, their
implementation (which is a bit tricky) has a serious flaw which will
make Delegates behave contrary to what you'd expect in certain
circumstances.

The problem is when one shadows attributes or methods that are
accessed by the real class itself from within code delegated to.  An
example illustrates best::

  from reahl.stubble import stubclass, Delegate
  
  class AnotherRealClass(object):
      def foo(self):
          self.aa = 123

  real_instance = AnotherRealClass()

  @stubclass(AnotherRealClass)
  class Stub(Delegate):
    shadowed = ['aa']

  s = Stub(real_instance)

The behaviour we would expect here is that, upon calling s.foo, the
real foo is called, which sets 'aa' on the stub, even though the
setting of 'aa' happens in real code::


  s.foo()

  try:
      #this expected behaviour would have been tested
      # like this:
      assert s.aa == 123
      assert not hasattr(real_instance, 'aa')
  except:
      pass

But alas, we cannot do that... The variable is in fact set on the real
class.  

The same problem manifests itself if you call a method which is
delegated to the real class, and the real class in turn tries to call
a method which is being shadowed on the stub.

This problem severely limits the use of Delegates and can cause bugs
in tests that are very difficult to find -- hence Delegates are not
used in practice.

Up for a challenge?
~~~~~~~~~~~~~~~~~~~

Anyone interested in giving it a bash: the one solution is to change
reahl.stubble.Delegate.__init__, so it changes the __class__ of 'real' to a
substitute with a __getattribute__ which can do the necessary voodoo.

That's the solution in theory only.  In practice, object layout
differences prevent this particular solution...  But, you may know
better.



Dealing with Instance variables
-------------------------------

In Python, it is very difficult to check anything when it comes to
instance variables... simply because the class does not have any
information about which instance variables it will have.

Some people use __slots__ as a way to specify instance variables for
this reason.  *However*, that is **not** what __slots__ is intended
for.  __slots__ is an optimisation feature which, if used for other
reasons will behave contrary to your expectations.

Such as in this example originally posted on comp.lang.python
by Blair Hall on Apr 10 2003 (I modified it a bit, though...)::

  class A(object):
      def __init__(self):
          pass

  class B(A):
      __slots__ = ('z',)
      def __init__(self):
          super(B, self).__init__()

  # now, if you used __slots__ thinking that, since 'c' is not 
  # in B.__slots__ and that the code above would complain, 
  # you're in for a surprise...
  b = B()
  b.c = 3     # passes


Using __slots__ also interferes with a number of other pythonic
flexibilities, so its use is not really recommended unless really
necessary.

If you *are*, however, interested in checking instance variables too,
other conventions are possible.  For example:

 - Always initialise all your instance variables in __init__
   (even if they are == None), so you can expect your class invariant
   to include 'instances of this class has all those attributes')
 - Always put class variables in the class for each variable
   instances of it would have (the class variable values also serve as
   handy default values for unset instance variables).

Using the latter approach, stubble could be used to check attributes
by your either specifying a default value for the attribute in the
stub class, or by using reahl.stubble.checkedinstance::

  from reahl.stubble import stubclass, checkedinstance

  class RealClassFollowingConventions(object):
      a = None
      b = None
    
  @stubclass(RealClassFollowingConventions, check_attributes_also=True)
  class Stub(object):
      a = 'asd'
      b = checkedinstance()


- (remove either a or b in the real class to see it complain)
- (PS: the difference between a and b below is that for b we do not
   give a default value, we just state that it should be in both)

For the stubborn, who insist on using __slots__, and who even insist
on using it as a checked list of allowed attributes, stubble deals in
the following drug (remove 'aa' from __slots__ to see it complain)::

  from reahl.stubble import stubclass, slotconstrained
  
  class StubbornRealClass(object):
      __slots__ = ('aa')
    
  @stubclass(StubbornRealClass)
  class Stub(object):
      aa = slotconstrained()

  s = Stub()
  try:
      s.aa                  #even though declared as class attributes, 
                            # these behave like instance attributes, 
                            # so its not there if not set
  except AttributeError, e:
      pass
                               
  s.aa = 123                #as usual
  assert s.aa == 123        #as usual



