Metadata-Version: 2.1
Name: ska-sdc2-scoring-utils
Version: 1.0.0
Summary: Utility scripts for interacting with SKA SDC2 scoring service.
Home-page: https://gitlab.com/ska-telescope/sdc/sdc2-scoring-utils
Author: SKA Organisation
License: License :: OSI Approved :: BSD License
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: docopt
Requires-Dist: python-keycloak
Requires-Dist: requests

# Science Data Challenge 2 scoring service CLI

This package contains two simple command line interface (CLI) applications
for interaction with the SDC2 online scoring service:

- `sdc2-score` : A tool for upload and check the status of submissions.
- `sdc2-score-admin`: A tool managing SDC2 users and groups.

## Installation

This package can be installed with:

```shell
pip install ska-sdc2-scoring-utils
```

## Using the `sdc2-score` command

Create a submission and generate its SUBMISSION_ID using

```shell
sdc2-score create CATALOGUE_FILE
```

Retrieve information about the submission using

```shell
sdc2-score info SUBMISSION_ID
```

All submissions belonging to a team can be printed using

```shell
sdc2-score ls
```

By default, all commands are run against the 'full' dataset. The -c flag can be used to run commands against the 'dev' dataset.

### Credentials:

Credentials  can either be specified using the command line options as follows:

```shell
sdc2-score create -u USER -p PASSWORD CATALOGUE_FILE
```

or are loaded from the following environment variables:

- `SDC2_SCORER_USER`
- `SDC2_SCORER_PASSWORD`

If both are supplied, the command line option flags are used.

Summary usage:

```console
Usage:
    sdc2-score leaderboard [--sub-info] [--score-info] [--raw] [-c VERSION]
    sdc2-score create CATALOGUE_FILE [-u USERNAME] [-p PASSWORD] [-c VERSION] [--hidden]
    sdc2-score info SUBMISSION_ID [-c VERSION] [--raw]
    sdc2-score ls [-u USERNAME] [-p PASSWORD] [-n LIMIT] [--score-info] [--raw] [-c VERSION]
    sdc2-score -h | --help | --version

Options:
    --sub-info           Display additional submission information.
    --score-info         Display additional score information.
    --raw                Display the raw response from the SDC scoring service.
    --hidden             If passed, submission will be hidden from leaderboards
    -c VERSION           Set the catalogue version (1.dev, 1.ldev, 1.full) [default: 1.full].
    -n LIMIT             Maximum number of submissions to list [default: 20].
    -u USER              Scoring service username.
    -p PASSWORD          Scoring service user password.
    -h, --help           Show this screen.
    --version            Show the version.

Arguments
    CATALOGUE_FILE      Path to catalogue file to submit.
    SUBMISSION_ID       Submission ID, minimally matched.
```


## Using the `sdc2-score-admin` command

The admin client is intended for SDC2 challenge administrators and requires
that the following environment variables are set:

- `SDC2_SCORER_ADMIN_USER`
- `SDC2_SCORER_ADMIN_PASSWORD`

Once these are set, the basic usage settings are as follows:

```console
Usage:
    sdc2-score-admin group ls [-u] [--raw]
    sdc2-score-admin group add GROUP_NAME...
    sdc2-score-admin group rm GROUP_ID...
    sdc2-score-admin user ls [--raw]
    sdc2-score-admin user add FIRST_NAME LAST_NAME USERNAME EMAIL GROUP [PASSWORD]
    sdc2-score-admin user rm USER_ID...
    sdc2-score-admin submission ls [-n LIMIT] [-c VERSION] [--raw]
    sdc2-score-admin submission rm SUBMISSION_ID [-c VERSION]
    sdc2-score-admin (-h | --help | --version)

Options:
    -u                   Show users in the group.
    --raw                Display unformatted API response JSON object.
    -n LIMIT             Maximum number of submissions to show [default: 20].
    -c VERSION           Catalogue version (1.full, 1.dev) [default: 1.full].
    -h, --help           Show this screen.
    --version            Show the version.

Arguments:
    EMAIL                Email address of the user.
    FIRST_NAME           User's first name.
    GROUP_ID             Group name or ID, minimally matched.
    GROUP_NAME           Name of the group.
    LAST_NAME            User's last name.
    PASSWORD             Password for the user. Optional, autogenerated if not set.
    SUBMISSION_ID        UUID for a submission, minimally matched
    USER_ID              User name or ID, minimally matched.
    USERNAME             User's user name
```

## Developer notes

For local development, commands can also be run and tested using:

```shell
python -m ska_sdc2_scoring_utils.sdc2_score
python -m ska_sdc2_scoring_utils.sdc2_score_admin
```


