.. raw:: html

    <p align="center">
    <img alt="Flametree Logo" title="Flametree Logo" src="https://raw.githubusercontent.com/Edinburgh-Genome-Foundry/Flametree/master/images/logo.png" width="500">
    </p>
    <h2 align="center"> Python file operations made easy </h2>



.. image:: https://github.com/Edinburgh-Genome-Foundry/Flametree/actions/workflows/build.yml/badge.svg
   :target: https://github.com/Edinburgh-Genome-Foundry/Flametree/actions/workflows/build.yml
   :alt: GitHub CI build status

.. image:: https://coveralls.io/repos/github/Edinburgh-Genome-Foundry/Flametree/badge.svg
   :target: https://coveralls.io/github/Edinburgh-Genome-Foundry/Flametree


Flametree is a Python library which provides a simple syntax for handling files and folders
(no ``os.path.join``, ``os.listdir`` etc.), and works the same way for different file systems.

Write a Flametree program to read/write files in disk folders, and your code will also be
able to read/write in zip archives and virtual (in-memory) archives - which is particularly
useful on web servers.

As an illustration, here is how to use Flametree to read a file ``texts/poems/the_raven.txt``, replace all
occurences of the word "raven" by "seagull" in the text, and write the result to a new
file ``the_seagull.txt`` in the same folder:

.. code:: python

     from flametree import file_tree

     with file_tree("texts") as root:
         poem_text = root.poems.the_raven_txt.read()
         new_text = poem_text.replace("raven", "seagull")
         root.poems._file("the_seagull.txt").write(new_text)

Even in this very simple use case, the syntax is clearer than the ``os`` way,
which would write as follows:

.. code:: python

    import os

    with open(os.path.join("poems", "the_raven.txt"), "r") as f:
        poem_text = f.read()
    new_text = poem_text.replace("raven", "seagull")
    with open(os.path.join("poems", "the_seagull.txt"), "w") as f:
        content = f.write(new_text)

Moreover, the same Flametree code also works for files inside a zip archive:

.. code:: python

     with file_tree("my_archive.zip") as root:
         poem_text = root.poems.the_raven_txt.read()
         new_text = poem_text.replace("raven", "seagull")
         root.poems._file("the_seagull.txt").write(new_text)

Now in hard mode: suppose that your server receives binary zip data of an
archive containing ``poems/the_raven.txt``, and must return back a new zip
containing a file ``poems/the_seagull.txt``. Here again, the syntax of the core
operations is the same:

.. code:: python

     destination_zip = file_tree("@memory") # Create a new virtual zip
     with file_tree(the_raven_zip_data) as root:
         poem_text = root.poems.the_raven_txt.read()
         new_text = poem_text.replace("raven", "seagull")
         destination_zip._dir("poems")._file("the_seagull.txt").write(new_text)
     destination_zip_data = destination_zip._close()
     # Now send the data to the client

See section *Usage* below for more examples and features.

Installation
------------

Flametree should work on Windows/Max/Linux, with Python 2 and 3, and has no external dependency.

It can be installed by unzipping the source code in one directory and using this command: ::

    python setup.py install

You can also install it directly from the Python Package Index with this command: ::

    pip install flametree


Contribute
----------

Flametree is an open-source software originally written by Zulko_ and released on Github_
under the MIT licence (Copyright 2017 Edinburgh Genome Foundry, University of Edinburgh).
Everyone is welcome to contribute!
In particular if you have ideas of new kinds of file systems to add to Flametree.


Usage
-----

Opening a file tree
~~~~~~~~~~~~~~~~~~~

Here is how you open different kinds of file systems:

.. code:: python

     from flametree import file_tree

     # Open a directory from the disk's file system:
     root = file_tree("my_folder/")

     # Open a zip archive on the disk:
     root = file_tree("my_archive.zip")

     # Connect to a file-like object (file handle, StringIO...) of a zip:
     root = file_tree(file_like_object)

     # Create a virtual 'in-memory' zip file:
     root = file_tree("@memory")

     # Open some data string representing a zip to read
     root = file_tree(some_big_zip_data_string)



In the two first examples, if ``my_folder`` or ``my_archive.zip`` do not exist, they
will be automatically created. If they do exist, it is possible to completely overwrite
them with the option ``replace=True``.

Exploring a file tree:
~~~~~~~~~~~~~~~~~~~~~~

Once you have created the ``root`` element with one of the methods above, you can display the whole
file tree with ``root._tree_view()`` :

.. code::

    >>> print (root._tree_view())
    texts/
      poems/
        dover_beach.txt
        the_raven.txt
        the_tyger.txt
      todo_list.txt
    figures/
      figure1.png
      figure2.png
    Readme.md


The attributes of a directory like ``root`` are its files and subdirectories.
For instance to print the content of ``dover_beach.txt`` you would write:

.. code:: python

  print( root.texts.poems.dover_beach_txt.read() )

or even simpler:

.. code:: python

    root.texts.poems.dover_beach_txt.print_content()

Notice that the ``.`` before ``txt`` was replaced by ``_`` so as to form a valid
 attribute name.

This syntactic sugar is particularly useful to explore a file tree in
IPython Notebooks or other editors offering auto-completion:


.. image:: https://raw.githubusercontent.com/Edinburgh-Genome-Foundry/Flametree/master/images/autocomplete.png
   :alt: [illustration]
   :align: center

Alternatively, you can access files and directories using dictionary calls:

.. code:: python

    root["texts"]["poems"]["dover_beach.txt"]

To iterate through the subdirectories of a directory, use the ``_dirs`` attribute:

.. code:: python

    for subdirectory in root._dirs:
        print (subdirectory._name) # Will print 'texts' and 'figures'

To iterate through the files of a directory, use the ``_files`` attribute:

.. code:: python

    for f in root.figures._files:
        print (f._name) # Will print 'figure1.png' and 'figure2.png'

Finally, use ``_all_files`` to iterate through all files nested in a directory.
The snippet below prints the content of all ``.txt`` files in the file tree:

.. code:: python

    for f in root._all_files:
        if f._name.endswith(".txt"):
            f.print_content()

Creating files and folders
~~~~~~~~~~~~~~~~~~~~~~~~~~

To create a new subdirectory use ``_dir``:

.. code:: python

    root._dir("data") # create a 'data' folder at the root
    root.data._dir("reports") # create a 'reports' folder under `root/data`

To create a new file use ``_file``:

.. code:: python

    root._file("joke.txt") # create a 'joke.txt' file at the root.
    root.texts._file("hello.txt") # create 'hello.txt' in `root/texts`.

To write content in a file, use ``.write``:

.. code:: python

    root.joke_txt.write("A plateau is the highest form of flattery.")

Writing to a file will use mode ``a`` (append) by default. To overwrite
the file set the write mode to ``"w"``. Let's erase and rewrite that ``joke.txt``:

.. code:: python

    root.joke_txt.write("'DNA' stands for National Dyslexic Association.", "w")

File and directory creation commands can be chained.
Let us create some new folders ``data/`` and ``data/test_1/``, and
write to file ``data/test_1/values.csv``, all in a single line:

.. code:: python

    root._dir("data")._dir("test_1")._file("values.csv").write("1,15,25")

Beware that ``._dir`` and ``._file`` **overwrite their target by default**, which means that if you write:

.. code:: python

    root._dir("data")._file("values_1.csv").write("1,4,7")
    root._dir("data")._file("values_2.csv").write("2,9,7")

The directory ``data`` will only contain ``values_2.csv``, because the second
line's ``_dir("data")`` erases the ``data`` directory and starts a new one. To avoid this,
either use ``root.data`` in the second line:

.. code:: python

    root._dir("data")._file("values_1.csv").write("1,4,7")
    root.data._file("values_2.csv").write("2,9,7")

Or use ``replace=False`` in ``_dir``:

.. code:: python

    root._dir("data")._file("values_1.csv").write("1,4,7")
    root._dir("data", replace=False)._file("values_2.csv").write("2,9,7")


Other operations
~~~~~~~~~~~~~~~~

You can move, copy, and delete a file with ``.move(folder)``, ``.copy(folder)``,
``.delete()``, and a directory with ``._move(folder)``, ``._copy(folder)``,
``._delete()``.

.. code:: python

    root.data.values1_csv.delete() # delete file 'values1.csv'
    root.data._delete() # delete directory 'data'
    # Move folder `plots` from `root/figures` to `other_root/figures`
    root.figures.plots._move(other_root.figures)
    # Move file `fig.png` from `root/figures` to `other_root/figures`
    root.figures.fig_png.move(other_root.figures)

Special rules for ZIP archives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is not currently possible to modify/delete a file that is already zipped
into an archive (because zips are not really made for that, it would
be doable but would certainly be a hack).

When creating files and folders in a zip with Flametree, the changes in the actual zip
will only be performed by closing the ``root`` with ``root._close()``
(after which the ``root`` can't be used any more). If it is an in-memory zip, ``root._close()``
returns the value of the zip content as a string (Python 2) or bytes (Python 3).

Here are a few examples:

.. code:: python

    root = file_tree("archive.zip")
    root._file("hello.txt").write("Hi there !")
    root._close()

    # Equivalent to the previous, using `with`:
    with file_tree("archive.zip") as root:
        root._file("hello.txt").write("Hi there !")

    # Getting binary data of an in-memory zip file:
    root = file_tree("@memory")
    root._file("hello.txt").write("Hi there !")
    binary_data = root._close()


Using file writers from other libraries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Some libraries have file-generating methods which expect a file name or a file
object to write too.
You can also feed Flametree files to these functions. for instance here is
how to use Weasyprint to create a PDF ``pdfs/report.pdf``

.. code:: python

    import weasyprint
    from flametree import file_tree
    root = file_tree(".") # or 'archive.zip' to write in an archive.
    html = weasyprint.HTML(string="<b>Hello</b> world!", base_url='.')
    html.write_pdf(root._dir("pdfs")._file("test.pdf"))

And here is how you would save a Matplotlib figure in a zip archive:

.. code:: python

    import matplotlib.pyplot as plt
    from flametree import file_tree
    fig, ax = plt.subplots(1)
    ax.plot([1, 2, 3], [3, 1, 2])
    with file_tree("archive.zip") as root:
        fig.savefig(root._dir("plots")._file("figure.png"), format="png")

That's all folks !


.. _Zulko: https://github.com/Zulko/
.. _Github: https://github.com/Edinburgh-Genome-Foundry/flametree
