NGP IRIS 5 Tutorial - NGPIris 5.0.0 documentation

Hide navigation sidebar
Hide table of contents sidebar
Skip to content
Back to top
View this page
Toggle table of contents sidebar

# NGP IRIS 5 Tutorial

- Introduction

- CLI

  - The `iris` command

    - Example use cases

      - Listing buckets/namespaces

      - Downloading a file

      - Uploading a file

      - Searching for a file

      - Delete a file

      - Delete a folder

  - The `iris_generate_credentials_file` command

- Package

  - The `HCPHandler` class

    - Example use cases

      - Listing buckets/namespaces

      - Downloading a file

      - Uploading a file

      - Uploading a folder

      - Searching for a file

      - Delete a file

      - Delete a folder

  - The `HCIHandler` class

    - Example use cases

      - List index names

      - Look up information of an index

      - Make queries

## Introduction

IRIS 5 is a complete overhaul of the previous versions of IRIS, mainly in terms of its codebase. The general functionality like download from and upload to the HCP are still here, but might differ from previous versions from what you are used to. This document will hopefully shed some light on what you (the user) can expect and how your workflow with IRIS might change in comparison to previous versions of IRIS.

IRIS 5, like previous versions of IRIS, consists of two main parts: a Python package and an associated Command Line Interface (CLI), which are described below.

## CLI

IRIS 5 features a CLI like recent versions of IRIS. However, the new CLI is a bit different compared to before; the general structure of subcommands for the `iris` command are totally different, but it still has the subcommands you would come to expect. A new command, `iris_generate_credentials_file`, has also been added. It will generate an empty credentials file that can be filled in with your own NGPr credentials.

### The `iris` command

Typing `iris --help` yields the following:

    Usage: iris [OPTIONS] CREDENTIALS COMMAND [ARGS]...

      NGP Intelligence and Repository Interface Software, IRIS.

      CREDENTIALS refers to the path to the JSON credentials file.

    Options:
      --version  Show the version and exit.
      --help     Show this message and exit.

    Commands:
      delete-folder    Delete a folder from an HCP bucket/namespace.
      delete-object    Delete an object from an HCP bucket/namespace.
      download         Download files from an HCP bucket/namespace.
      list-buckets     List the available buckets/namespaces on the HCP.
      list-objects     List the objects in a certain bucket/namespace on the...
      simple-search    Make simple search using substrings in a...
      test-connection  Test the connection to a bucket/namespace.
      upload           Upload files to an HCP bucket/namespace.

- `delete-folder`: Deletes a folder on the HCP

- `delete-object`: Deletes an object on the HCP

- `download`:

  - Downloads a file from a bucket/namespace on the HCP

  - `iris path/to/credentials.json download --help`:

    -     Usage: iris CREDENTIALS download [OPTIONS] OBJECT BUCKET LOCAL_PATH

            Download files from an HCP bucket/namespace.

            OBJECT is the name of the object to be downloaded.

            BUCKET is the name of the upload destination bucket.

            LOCAL_PATH is the path to where the downloaded objects are to be stored
            locally.

          Options:
            --help  Show this message and exit.

- `list-buckets`: Lists all buckets that the user is allowed to see

- `list-objects`: Lists all objects that the user is allowed to see

- `simple-search`: Performs a simple search using a substring in order to find matching objects in a bucket/namespace

- `upload`:

  - Uploads either a file or a folder to a bucket/namespace on the HCP

  - `iris path/to/credentials.json upload --help`:

    -     Usage: iris CREDENTIALS upload [OPTIONS] FILE_OR_FOLDER BUCKET

            Upload files to an HCP bucket/namespace.

            FILE-OR-FOLDER is the path to the file or folder of files to be uploaded.

            BUCKET is the name of the upload destination bucket.

          Options:
            --help  Show this message and exit.

#### Example use cases

The following subsections contain examples of simple use cases for IRIS 5. Of course, correct paths and bucket names should be replaced for your circumstances.

##### Listing buckets/namespaces

    iris path/to/your/credentials.json list-buckets

##### Downloading a file

    iris path/to/your/credentials.json download path/to/your/file/on/the/bucket the_name_of_the_bucket path/on/your/local/machine

##### Uploading a file

    iris path/to/your/credentials.json upload destination/path/on/the/bucket the_name_of_the_bucket path/to/your/file/on/your/local/machine

##### Searching for a file

By default, the `simple-search` command is case insensitive:

    iris path/to/your/credentials.json simple-search the_name_of_the_bucket your_search_string

This can be changed with the `--case_sensitive` option:

    iris path/to/your/credentials.json simple-search --case_sensitive True the_name_of_the_bucket your_case_sensitive_search_string

##### Delete a file

    iris path/to/your/credentials.json delete-object path/to/your/file/on/the/bucket the_name_of_the_bucket

##### Delete a folder

    iris path/to/your/credentials.json delete-folder path/to/your/folder/on/the/bucket/ the_name_of_the_bucket

### The `iris_generate_credentials_file` command

IRIS 5 comes with a new separate command for generating your NGPr credentials: `iris_generate_credentials_file`. The idea with this command is to make it easier for anyone to ensure the correct structure of their credentials file. Typing `iris_generate_credentials_file --help` yields the following:

    Usage: iris_generate_credentials_file [OPTIONS]

      Generate blank credentials file for the HCI and HCP.

      WARNING: This file will store sensitive information (such as passwords) in
      plaintext.

    Options:
      --path TEXT  Path for where to put the new credentials file
      --name TEXT  Custom name for the credentials file
      --help       Show this message and exit.

Simply running `iris_generate_credentials_file` will generate a blank credentials file (which is just a JSON file) like the following:

    {
      "hcp": {
          "endpoint": "",
          "aws_access_key_id": "",
          "aws_secret_access_key": ""
      },
      "hci": {
          "username": "",
          "password": "",
          "address": "",
          "auth_port": "",
          "api_port": ""
      }
    }

## Package

The updated codebase for IRIS 5 contains some major changes to use of the package, but should still be familiar. The use cases of IRIS 5 is sill intended to be the same as in previous versions. The difference between IRIS 5 and previous versions is the new syntax and names of classes, methods and functions. Everything in IRIS 5 was inspired by the previous implementations of the `boto3` library, which means that most functionality should still exist, but in a different form; methods and functions may have new names, and they might be combined or separated.

### The `HCPHandler` class

IRIS 5 comes with a class for handling connections with the HCP:

    HCPHandler(
      self,
      credentials_path : str,
      use_ssl : bool = False,
      proxy_path : str = "",
      custom_config_path : str = ""
    )

Generally, a bucket must be mounted before any methods can be executed. If not, IRIS will throw a `NoBucketMounted` error. A bucket can be mounted with the following code:

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

#### Example use cases

##### Listing buckets/namespaces

There is no need for mounting a bucket when listing all available buckets. However, credentials are still needed. As such, we can list all buckets with the following:

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    print(hcp_h.list_buckets())

##### Downloading a file

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

    hcp_h.download_file("path/to/object/in/bucket", "path/on/local/machine")

##### Uploading a file

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

    hcp_h.upload_file("path/on/local/machine", "path/to/object/in/bucket")

##### Uploading a folder

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

    hcp_h.upload_folder("path/on/local/machine/", "path/to/folder/in/bucket/")

##### Searching for a file

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

    print(hcp_h.search_objects_in_bucket("a search string"))

##### Delete a file

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

    hcp_h.delete_object("path/to/object/in/bucket/")

##### Delete a folder

    from NGPIris.hcp import HCPHandler

    hcp_h = HCPHandler("credentials.json")

    hcp_h.mount_bucket("myBucket")

    hcp_h.delete_folder("path/to/folder/in/bucket/")

### The `HCIHandler` class

IRIS 5 comes with a class for handling connections with the HCI:

    HCIHandler(
      self,
      credentials_path : str,
      use_ssl : bool = False
    )

A token can be requested to the handler, which in turn allows us to make requests to the HCI:

    from NGPIris.hci import HCIHandler

    hci_h = HCIHandler("credentials.json")

    hci_h.request_token()

#### Example use cases

##### List index names

    from NGPIris.hci import HCIHandler

    hci_h = HCIHandler("credentials.json")
    hci_h.request_token()

    print(hci_h.list_index_names())

##### Look up information of an index

    from NGPIris.hci import HCIHandler
    from pprint import pprint
    from json import dumps

    hci_h = HCIHandler("credentials.json")
    hci_h.request_token()

    pprint(
        dumps(
            hci_h.look_up_index("myIndex"), # A dictionary is returned, so we use
            indent = 4                      # dumps and pprint in order to make the
        )                                   # output more readable
    )

##### Make queries

    from NGPIris.hci import HCIHandler
    from pprint import pprint
    from json import dumps

    hci_h = HCIHandler("credentials.json")
    hci_h.request_token()

    query = {
        "indexName" : "myIndex",
        "facetRequests" : [
            {
                "fieldName" : "ref"
            },
            {
                "fieldName" : "alt"
            }
        ]
    }
    pprint(
        dumps(
            hci_h.raw_query(query), # A dictionary is returned, so we use
            indent = 4              # dumps and pprint in order to make the
        )                           # output more readable
    )