Running trial tests within Twisted
==================================

Trial tests are used to test in some circumstances the integration between
Zope and Twisted. The standard Zope test suites don't work since a lot of the
methods we are testing require use to return deferred objects. These deferred
objects contain callbacks and error callbacks which we need to test.

In order to set up the test suite correctly in order to test the deferreds
we need our test suite to extend the twisted.trial.unittest.TestCase object.
Then our setup methods must call the zope.testing.trial.setUp method and our
teardown method must call the zope.testing.trial.tearDown methods.

Our first test we will rerun a test suite which has 1 failing test, 1 test
which contains an error and a successful test.

    >>> import os.path, sys
    >>> from zope.testing import testrunner
    >>> defaults = [
    ...     '--path', this_directory,
    ...     '--tests-pattern', '^trialtest$',
    ...     ]

    >>> sys.argv = 'test --tests-pattern ^trialtest$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running zope.testing.testrunner.layer.UnitTests tests:
      Set up zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtest.TestTrialTests)
    Exception: this test is a broken trial test :-)
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Failure in test test_failure (trialtest.TestTrialTests)
    FailTest: I am a failed trial test
    <BLANKLINE>
      Ran 3 tests with 1 failures and 1 errors in N.NNN seconds.
    Tearing down left over layers:
      Tear down zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    True

Here is the output from the same tests above but the setup method contains
errors.

    >>> sys.argv = 'test --tests-pattern ^trialtestfs$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running zope.testing.testrunner.layer.UnitTests tests:
      Set up zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_assert_ok (trialtestfs.TestTrialBrokenSetup)
    Exception: broken setup
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtestfs.TestTrialBrokenSetup)
    Exception: broken setup
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_failure (trialtestfs.TestTrialBrokenSetup)
    Exception: broken setup
    <BLANKLINE>
      Ran 3 tests with 0 failures and 3 errors in N.NNN seconds.
    Tearing down left over layers:
      Tear down zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    True

Here is the output from the same tests but the teardown method contains
errors.

    >>> sys.argv = 'test --tests-pattern ^trialtestft$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running zope.testing.testrunner.layer.UnitTests tests:
      Set up zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_assert_ok (trialtestft.TestTrialBrokenTeardown)
    Exception: broken tearDown
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtestft.TestTrialBrokenTeardown)
    Exception: this test is a broken trial test :-)
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtestft.TestTrialBrokenTeardown)
    Exception: broken tearDown
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Failure in test test_failure (trialtestft.TestTrialBrokenTeardown)
    FailTest: I am a failed trial test
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_failure (trialtestft.TestTrialBrokenTeardown)
    Exception: broken tearDown
    <BLANKLINE>
      Ran 3 tests with 1 failures and 4 errors in N.NNN seconds.
    Tearing down left over layers:
      Tear down zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    True

Make sure that errors are handled correctly if the postCaseCleanup method
fails for what ever reason.

    >>> import twisted.trial.util
    >>> old_doCleanReactor = twisted.trial.util._Janitor.do_cleanReactor
    >>> def new_doCleanReactor(cls):
    ...     raise Exception("Can't clean the reactor")
    ...
    >>> twisted.trial.util._Janitor.do_cleanReactor = new_doCleanReactor

    >>> sys.argv = 'test --tests-pattern ^trialtest$ '.split()
    >>> testrunner.run(defaults)
    ... # doctest: +NORMALIZE_WHITESPACE
    Running zope.testing.testrunner.layer.UnitTests tests:
      Set up zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    <BLANKLINE>
    <BLANKLINE>
    Error in test test_error (trialtest.TestTrialTests)
    Exception: this test is a broken trial test :-)
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    Failure in test test_failure (trialtest.TestTrialTests)
    FailTest: I am a failed trial test
    <BLANKLINE>
      Ran 3 tests with 1 failures and 1 errors in N.NNN seconds.
    Tearing down left over layers:
      Tear down zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
    True

Now un-patch the twisted trial janitor, so that it does actually clean the
reactor.

    >>> twisted.trial.util._Janitor.do_cleanReactor = old_doCleanReactor
