T2K Data Manager - t2kdm
========================

Install
-------

You will need a working setup of the DIRAC UI to access the grid data.
Follow the instructions here:
https://gridpp.ac.uk/wiki/Quick_Guide_to_Dirac#Dirac_client_installation

You will also need a valid Grid certificate. Instructions on how to obtain one are available at: https://t2k.org/nd280/datacomp/gridandirac/gridcert

DIRAC comes with its own Python interpreter. When you source DIRAC's bashrc,
you will havea somewhat isolated Python environment. Within this environment
you can install t2kdm releases directly with pip:

    $ pip install t2kdm

Or you can clone and install the HEAD version from the repository:

    $ git clone git@github.com:t2k-software/t2kdm.git
    $ pip install -e ./t2kdm

Configuration
-------------

Simply run:

    $ t2kdm-config

Test
----

Make sure you have a valid grid proxy.
Then simply run:

    $ t2kdm-tests

Scripts
-------

The T2K Data Manager provides a bunch of scripts for handling the data.
They all start with `t2kdm-*` and come with (basic) instructions when called with `-h`.

Command Line Interface
----------------------

The T2K Data Manager also provides a command line interface (CLI),
which behaves similar to the classic `ftp` client:

    $ t2kdm-cli
    Welcome to the T2K Data Manager CLI.
      ____  ___   _  _  ____  __  __       ___  __    ____
     (_  _)(__ \ ( )/ )(  _ \(  \/  )___  / __)(  )  (_  _)
       )(   / _/ |   (  )(_) ))    ((___)( (__  )(__  _)(_
      (__) (____)(_)\_)(____/(_/\/\_)     \___)(____)(____)

    Type 'help' or '?' to list commands.

    (t2kdm)

The CLI can be quit by typing `quit`, `exit`, or simply pushing `CTRL-C`.

Examples
--------

List files on the grid:

    $ t2kdm-ls /test/t2kdm

Download a file from the grid:

    $ t2kdm-get /test/t2kdm/test1.txt

Download all files in a folder that match a regular expression:

    $ t2kdm-get /test/t2kdm -r 'test[1-3]\.txt'

List replicas of a file:

    $ t2kdm-replicas /test/t2kdm/test1.txt

List all available storage elements:

    $ t2kdm-SEs

Recursively replicate a folder to a specific storage element:

    $ t2kdm-replicate /test/t2kdm UKI-SOUTHGRID-OX-HEP-disk -r

Check which files are replicated to a given storage element:

    $ t2kdm-check /test/t2kdm -s UKI-SOUTHGRID-OX-HEP-disk -r

Remove replicas of files from a specififc storage element:

    $ t2kdm-remove /test/t2kdm/test1.txt UKI-SOUTHGRID-OX-HEP-disk

Contributing
------------

Please use `pre-commit <https://pre-commit.com/>`_ to check your code before
checking in anything. To install pre-commit and the repository hooks just run
these commands inside the repository:

    $ pip3 install pre-commit
    $ pre-commit install

That's it. Now pre-commit should check your code for compliance whenever you
try to check it in.

Among other things, it will check that the code adheres to the `Black
<https://black.readthedocs.io/en/stable/index.html>`_ code style.
