Metadata-Version: 2.0
Name: pytest-spark
Version: 0.2.0
Summary: pytest plugin to run the tests with support of pyspark.
Home-page: https://github.com/malexer/pytest-spark
Author: Alex (Oleksii) Markov
Author-email: alex@markovs.me
License: MIT
Keywords: pytest spark pyspark unittest test
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.2
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Topic :: Software Development :: Testing
Requires-Dist: findspark
Requires-Dist: pytest

pytest-spark
############

pytest_ plugin to run the tests with support of pyspark (`Apache Spark`_).

This plugin will allow to specify SPARK_HOME directory in ``pytest.ini``
and thus to make "pyspark" importable in your tests which are executed
by pytest.

Also it defines session scope fixture ``spark_context`` which can be
used in your tests.


Install
=======

.. code-block:: shell

    $ pip install pytest-spark


Usage
=====

Set Spark location
------------------

To run tests with required spark_home location just add "spark_home"
value to ``pytest.ini`` in your project directory::

    [pytest]
    spark_home = /opt/spark

pytest-spark will try to import ``pyspark`` from specified location.


Using fixture
-------------

Use fixture ``spark_context`` in your tests as a regular pyspark fixture.
SparkContext instance will be created once and reused for the whole test
session.

Example::

    def test_my_case(spark_context):
        test_rdd = spark_context.parallelize([1, 2, 3, 4])
        # ...



.. _pytest: http://pytest.org/
.. _Apache Spark: https://spark.apache.org/


