Haul
====

.. image:: https://travis-ci.org/vinta/Haul.png?branch=master
    :alt: Build Badge
    :target: https://travis-ci.org/vinta/Haul

.. image:: https://coveralls.io/repos/vinta/Haul/badge.png?branch=master
    :alt: Coverage Badge
    :target: https://coveralls.io/r/vinta/Haul

.. image:: https://badge.fury.io/py/haul.png
    :alt: Version Badge
    :target: http://badge.fury.io/py/haul

.. image:: https://d2weczhvl823v0.cloudfront.net/vinta/haul/trend.png
    :alt: Bitdeli Badge
    :target: https://bitdeli.com/free


Find thumbnails and original images from URL or HTML file.

Demo
====

`Hauler on Heroku <http://hauler.herokuapp.com/>`_

Installation
============

on Ubuntu

.. code-block:: bash

    $ sudo apt-get install build-essential python-dev libxml2-dev libxslt1-dev
    $ pip install haul

on Mac OS X

.. code-block:: bash

    $ pip install haul

Fail to install haul? `It is probably caused by lxml <http://lxml.de/installation.html>`_.

Usage
=====

Find images from ``img src``, ``a href`` and even ``background-image``:

.. code-block:: python

    import haul

    url = 'http://gibuloto.tumblr.com/post/62525699435/fuck-yeah'
    result = haul.find_images(url)

    print(result.image_urls)
    """
    output:
    [
        'http://25.media.tumblr.com/3f5f10d7216f1dd5eacb5eb3e302286a/tumblr_mtpcwdzKBT1qh9n5lo1_500.png',
        ...
        'http://24.media.tumblr.com/avatar_a3a119b674e2_16.png',
        'http://25.media.tumblr.com/avatar_9b04f54875e1_16.png',
        'http://31.media.tumblr.com/avatar_0acf8f9b4380_16.png',
    ]
    """

Find original (or bigger size) images with ``extend=True``:

.. code-block:: python

    import haul

    url = 'http://gibuloto.tumblr.com/post/62525699435/fuck-yeah'
    result = haul.find_images(url, extend=True)

    print(result.image_urls)
    """
    output:
    [
        'http://25.media.tumblr.com/3f5f10d7216f1dd5eacb5eb3e302286a/tumblr_mtpcwdzKBT1qh9n5lo1_500.png',
        ...
        'http://24.media.tumblr.com/avatar_a3a119b674e2_16.png',
        'http://25.media.tumblr.com/avatar_9b04f54875e1_16.png',
        'http://31.media.tumblr.com/avatar_0acf8f9b4380_16.png',
        # bigger size, extended from above urls
        'http://25.media.tumblr.com/3f5f10d7216f1dd5eacb5eb3e302286a/tumblr_mtpcwdzKBT1qh9n5lo1_1280.png',
        ...
        'http://24.media.tumblr.com/avatar_a3a119b674e2_128.png',
        'http://25.media.tumblr.com/avatar_9b04f54875e1_128.png',
        'http://31.media.tumblr.com/avatar_0acf8f9b4380_128.png',
    ]
    """

Advanced Usage
==============

Custom finder / extender pipeline:

.. code-block:: python

    from haul import Haul
    from haul.compat import str


    def img_data_src_finder(pipeline_index,
                            soup,
                            finder_image_urls=[],
                            *args, **kwargs):
        """
        Find image URL in <img>'s data-src attribute
        """

        now_finder_image_urls = []

        for img in soup.find_all('img'):
            src = img.get('data-src', None)
            if src:
                src = str(src)
                now_finder_image_urls.append(src)

        output = {}
        output['finder_image_urls'] = finder_image_urls + now_finder_image_urls

        return output

    MY_FINDER_PIPELINE = (
        'haul.finders.pipeline.html.img_src_finder',
        'haul.finders.pipeline.css.background_image_finder',
        img_data_src_finder,
    )

    GOOGLE_SITES_EXTENDER_PIEPLINE = (
        'haul.extenders.pipeline.google.blogspot_s1600_extender',
        'haul.extenders.pipeline.google.ggpht_s1600_extender',
        'haul.extenders.pipeline.google.googleusercontent_s1600_extender',
    )

    url = 'http://fashion-fever.nl/dressing-up/'
    h = Haul(parser='lxml',
             finder_pipeline=MY_FINDER_PIPELINE,
             extender_pipeline=GOOGLE_SITES_EXTENDER_PIEPLINE)
    result = h.find_images(url, extend=True)

Run Tests
=========

.. code-block:: bash

    $ cd tests
    $ python test.py
