Using Metapack Packages — Metapack 0.7.11 documentation

# Using Metapack Packages

This section will use and example package, which is available online in three forms at:

- http://library.metatab.org/example.com-full-2017-us-1.zip

- http://library.metatab.org./example.com-full-2017-us-1.xlsx

- http://library.metatab.org/example.com-full-2017-us-1.csv

Additionally, the section will refer to the filesystem package, which is just the ZIP package unpacked:

    $ wget http://library.metatab.org/example.com-full-2017-us-1.zip
    $ unzip example.com-full-2017-us-1.zip

## Command Line Operations

Note

See the commands page for detailed documentation of the commands in the base metapack package.

Command line programs that operate on packages take a URL to a local or remote package, with a URL fragment to refer to a resource.

Metapack has a single CLI entrypoint, mp, to run commands. The entrypoint is exensible, so other python modules can install additinoal commands. You can see the commands that are avilable, and their version numebrs, with mp config

    $ mp config

    Package        Version
    -------------  ---------
    metapack       0.9.1
    metatab        0.7.16
    metatabdecl    1.6
    rowgenerators  0.8.31
    publicdata     0.3.8
    tableintuit    0.1.5

    Subcommand    Package
    ------------  -------------------------------------
    config        metapack 0.9.1
    doc           metapack 0.9.1
    index         metapack 0.9.1
    info          metapack 0.9.1
    open          metapack 0.9.1
    run           metapack 0.9.1
    search        metapack 0.9.1
    build         metapack-build 0.0.2
    edit          metapack-build 0.0.2
    new           metapack-build 0.0.2
    s3            metapack-build 0.0.2
    stats         metapack-build 0.0.2
    update        metapack-build 0.0.2
    url           metapack-build 0.0.2
    wp            metapack-wp 0.0.6.post0.dev1+g4c7a6bc

Commands which operate on packages take a URL as the last argument. There are a few common forms for these URLs:

Full url | http://library.metatab.org/example.com-full-2017-us-1.zip
Local file | example.com-full-2017-us-1.zip
Metatab.csv in current directory | . or ‘’

Commands that operate on resources use URLs that have a fragment and the name of the resource:

Full url | http://library.metatab.org/example.com-full-2017-us-1.zip#renter_cost
Local file | example.com-full-2017-us-1.zip#renter_cost
Metatab.csv in current directory | .#renter_cost

For instance, to list the resources in a package:

    # Remote Package
    $ mp info http://library.metatab.org/example.com-full-2017-us-1.zip

    # Local filesystem package
    $ cd example.com-full-2017-us-1
    $ mp info -r

To display the resources:

    # Remote Package
    $ mp info http://library.metatab.org/example.com-full-2017-us-1.zip#renter_cost

    # Local filesystem package
    $ cd example.com-full-2017-us-1
    $ mp info -r .#renter_cost

Note that in a filesystem package, the reference to the package (‘.’) can be omitted when referencing the package, but is included when referencing a resource. This is because the shell considers strings that start with ‘#’ to be comments. To avoid this, the ‘#’ can also be escaped, mp info -r \#renter_cost, but this form is not commonly used.

### Opening Packages

Opening packages with the CLI primarily means opening the CSV files included in the package, which you can do from direct reference. Just open the metadata file and read the resource URLs. However, you can also use the mp run program to dump the data in a variety of formats. The URL requires a resource fragment, but if you don’t include it, mp run will show the resources.

    $ mp run example.com-full-2017-us-1.zip
    Select a resource to run:
    Type       Ref                       Url
    ---------  ------------------------  --------------------------------------
    Resource   #renter_cost              data/renter_cost.csv
    Resource   #simple-example-altnames  data/simple-example-altnames.csv
    Resource   #simple-example           data/simple-example.csv
    ...

    $ mp run example.com-full-2017-us-1.zip#renter_cost
    id,col2,renter_cost_gt_30,renter_cost_gt_30_cv, ...
    1,,1447,13.6176070905,42.2481751825,8.272140707
    2,,5581,6.235932071,49.2803532009,4.9333693054
    3,,525,17.6481586483,45.2196382429,13.2887199931

If you just want to list all of the resources and references, use mp info -r.

## Using Packages In Python

Using packages in Python involves opening the package, and then acessing a resource.

    import metapack as mp

    # Open the package.
    pkg = mp.open_package('http://library.metatab.org/example.com-full-2017-us-1.csv')
    print(pkg) # display documentation

    # Get a handle on a resource
    r = pkg.resource('renter_cost')

    # Get a dataframe for the resource
    df = r.dataframe()