PAPIS(1)

NAME
papis - Papis Documentation

Papis is a command-line based document and bibliography manager. Its command-line interface (CLI) is heavily tailored after Git.

QUICK START

This is a tutorial that should be enough to get you started using papis. Papis tries to be as simple and lightweight as possible, therefore its document model should be too as simple as possible.

But before taking a look at its database structure let us show the daily usage of papis for a regular user. This tutorial is command-line based, so you should be familiar with opening a terminal window on your system and do some general operations with it, like creating folders and files.

  Creating a new library

We will illustrate the process by creating a first library with a couple of pdf documents in it. Papis can be highly configured using configuration files. Many programs use configuration files maybe without you being aware of it. Papis' configuration files are stored together inside the folder

~/.config/papis

Bear in mind that ~ means "Home Directory". Inside this directory a configuration file is found,

~/.config/papis/config

Right now we will open this file for editing and we will create a library. In papis everything should be human-readable and human-editable. So adding a library is as easy as adding two lines to this configuration file.

Say that you want to create a "papers" library, where you can finally order all those pdf's hanging around on your computer. We create this library by putting these two lines inside the config file:

[papers]
dir = ~/Documents/mypapers

In the above lines we have created a library with the name papers which is located in the directory ~/Documents/mypapers. So all the documents that we will be adding to the library will be located inside ~/Documents/mypapers, and nowhere else. Everything that papis needs to take care of your papers library is inside the ~/Documents/mypapers directory, self-contained.

If you have not already, add the two lines to the ~/.config/papis/config file and save it, and we will proceed to add some documents. Of course, you have to make sure that the folder ~/Documents/mypapers exists, so go ahead and create it

mkdir -p ~/Documents/mypapers

  Adding the first document

If you don't have any special pdf lying around let me choose one for you: link. You can download this document and we are going to add it into the papers library.

Assuming that you have the document in the current directory and you have renamed the document to document.pdf, do the following to add the pdf into your library:

papis add document.pdf --set author "Newton" --set title "Principia Mathematica"

And it's done! We have added our first book to the library.

Let us see how this works exactly. Papis consists of many commands, and one of these commands is add. add itself has many flags, which are options for the given command. In the example above we have used the flags author and title to tell papis to use Newton as the author's name and Principia Mathematica as the document's title. You can see all the possible flags for the command add if you use the help flag, i.e., if you issue the following command

papis add --help

Now you are asking yourself, what happened to the pdf-file? Where is it stored? Is it stored in an obscure database somewhere in my computer? No, papis just copied the document.pdf file into a folder inside the library folder ~/Documents/papers/. If you now go there, you will see that a folder with a weird name has been created. Inside of the folder there is the document.pdf file and another file, info.yaml.

If you open the info.yaml file you will see the following contents:

author: Newton
title: Principia Mathematica
files:
- document.pdf

This file is all that papis uses to store the information of your newly added document. It is stored in a nicely readable YAML format.

Now you already have your first document, and.. you can open it! Just do

papis open

and the document should open in your default pdf-viewer. You can change the default pdf-viewer in your configuration file (see section on Configuration file).

Now you can try to repeat the same process with another pdf-file lying around. When you hit papis open again, it will ask you which one you want. If you input parts of the title or the author's name it will try to match what you typed with the paper you are looking for, so that you can get the desired paper very easily.

Of course papis shines really in other areas, for instance imagine you are browsing this paper prl paper and you want to add it to your library, as of version v0.9 you can issue one of these commands

papis add https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.124.171801/
papis add --from doi 10.1103/PhysRevLett.124.171801/

Here you can see it in action using the smart matching first alternative

INSTALLATION
.SS Using pip

The easiest way of installing papis is using the PyPi repositories and the package manager pip3, just open a terminal and type in

pip3 install papis

If you are on GNU/Linux like systems you might need to type sudo

sudo pip3 install papis

or if you prefer installing it locally then simply type

pip3 install --user papis

You can also update papis with pip

pip3 install --upgrade papis

  Archlinux

•   The package papis is also found in the archlinux repositories here.

•   If you want to use the git version of papis instead refer to papis-git package. Thanks Julian!.

  NixOS

If you are running NixOS or you have the nix package manager installed, you can install papis by running:

nix-env -i papis

If you like papis, just clone the repository

git clone git@github.com:papis/papis.git
cd papis

and start hacking it with:

nix-shell --expr 'with import <nixpkgs> {}; papis.overrideDerivation (drv: { src = ./.; })'

This command will provide you a shell with all the dependencies required.

  From source

First of all you have to get the code, open a terminal and hit

git clone https://github.com/papis/papis.git

or download the zip file.

Go inside of the papis source folder and you can install it with python3.

The general command that you have to hit is by using the setup.py script:

python3 setup.py install

Again, if you want to install it locally because you don't have administrative rights on your computer you simply type

python3 setup.py install --user

If you want to work on the code, you can alternatively hit

python3 setup.py develop --user

WARNING: If you install the package locally, the program papis will be installed by default into your ~/.local/bin directory, so that you will have to set your PATH accordingly.

One way of doing this in bash shells (Linux and the like, also Ubuntu on Windows or cygwin) is by adding the following line to your ~/.bashrc file

export PATH=$PATH:$HOME/.local/bin

  Requirements

Papis needs the following packages that are sometimes not installed with the system python3 distribution

python3-setuptools

However if you have a general enough python distribution they should be installed.

  Running tests

In order to run the necessary tests to submit a pull request, make sure that the following commands pass

python -m pytest papis/ tests/ --cov=papis
python -m mypy papis
python -m flake8 papis

for it, make sure that you have pytest, flake8 and mypy installed.

You can make sure that you have everything you need to run the tests by doing in the root directory

pip install .[develop]

this command installs the necessary dependencies for developing and running the tests. Look inside of the file setup.py for further information.

You can also look at the folder tools for scripts used in the CI testing phase for further context.

CONFIGURATION FILE

Papis uses a configuration file in *INI* format.

The basic configuration unit is a library. Imagine you want to have a library called papers and another called books. You can have these libraries work independently from each other.

You would declare these libraries telling papis where the folders are in your system, like so:

# my library for papers and stuff
[papers]
dir = ~/Documents/papers
# my library for books and related documents
[books]
dir = ~/Documents/books

One important aspect of the configuration system is that you can override settings on a per library basis, this means that you can set settings that should have a value for the library papers and another value if you're currently using the library books. The settings have to be set in the section under the library definition. For example, let's suppose you want to open your documents in papers using the pdf reader okular however in books you want to open the documents in firefox, for some reason, the you would write

# my library for papers and stuff
[papers]
dir = ~/Documents/papers
opentool = okular
# my library for books and related documents
[books]
dir = ~/Documents/books
opentool = firefox
[settings]
opentool = evince
default-library = papers

Here we wrote also the special section [settings], which sets global settings that are valid in all libraries. Of course, every setting set within [settings] can be overridden by any library through the mechanism previously discussed.

A more complete example of a configuration file is the following

#
# This is a general section, the settings set here will be set for
# all libraries
#
[settings]
#
# General file opener program, rifle is a nice python program
# If you're on macOS, you can write "open", if you're on linux
# you can also write "xdg-open", on windows-cygwin, you can set it to
# "cygstart"
#
opentool = rifle
# Use ranger as a file browser, a nice python program
file-browser = ranger
# Ask for confirmation when doing papis add
add-confirm = True
# Edit the info.yaml file before adding a doc into the library
# papis add --edit
add-edit = True
# Open the files before adding a document into the library
# papis add --open
add-open = True
#
# Define custom default match and header formats
#
match-format = {doc[tags]}{doc.subfolder}{doc[title]}{doc[author]}{doc[year]}
#
# Define header format with colors and multiline support
#
header-format = <red>{doc.html_escape[title]}</red>
  <span color='#ff00ff'>  {doc.html_escape[author]}</span>
  <yellow>   ({doc.html_escape[year]})</yellow>
[tui]
editmode = vi
options_list.selected_margin_style = bg:ansigreen fg:ansired
options_list.unselected_margin_style =
# Define a lib
[papers]
dir = ~/Documents/papers
# override settings from the section tui only for the papers library
# you have to prepend "tui-" to the settings
tui-editmode = emacs
tui-options_list.unselected_margin_style = bg:blue
# use whoosh as a database for papers
database-backend = whoosh
# rename files added by author and title
add-file-name = {doc[author]}{doc[title]}
# Define a lib for books
[books]
dir = ~/Documents/books
database-backend = whoosh
# Define a lib for Videos
[videos]
dir = ~/Videos/courses
# Define a lib for contacts, why not?
# To make it work you just have to define some default settings
[contacts]
dir = ~/contacts/general
database-backend = papis
mode = contact
header-format = {doc[first_name]} {doc[last_name]}
match-format = {doc[org]} {doc[first_name]} {doc[last_name]}
browse-query-format = {doc[first_name]} {doc[last_name]}
add-open = False

  Local configuration files

Papis also offers the possibility of creating local configuration files. The name of the local configuration file can be configured with the local-config-file setting.

The local configuration file can be found in the current directory of where you are issuing the papis command or in the directory of the library that you are considering in the papis command.

For instance let us suppose that you are in some project folder ~/Documents/myproject and you have a local config file there with a definition of a new library. Then whenever you are in the ~/Documents/myproject directory papis will also read the local configuration file found there.

On the other hand, also if you have a configuration file in the library folder for your papers, for instance in

~/Documents/papers/.papis.config

then every time that you use this library papis will also source this configuration file.

An example of a project using a local configuration file can be seen here , where the repository includes documents for component datasheets and every time papis is using that library the .papis.config file is also read and some settings will be getting overridden.

  Python configuration file

For some users it would be useful to have a python file that gets loaded together with the usual configuration file, this file lives in your papis configuration directory with the name config.py, for instance for most users it will be in

~/.config/papis/config.py

  General settings

local-config-file

•   Default: '.papis.config'

    Name AND relative path of the local configuration file that papis will additionally read if the file is present in the current directory or in the base directory of a given library.

    This is useful, for instance, if you have a library somewhere for which you want special configuration settings but do not want these settings to cluster in your configuration file. It is also useful if you're sharing a library with someone else and you want them to have the same settings in that library as you. Imagine you're sharing a library of datasheets with your friend Fulano. You have your library at

    ~/Documents/lib-with-fulano

    and you've set a local configuration file there

    ~/Documents/lib-with-fulano/.papis.config

    then whenever Fulano uses that library and the file is also present, his papis program will also read the configuration settings at the path above.

dir-umask

•   Default: 493

    This is the default umask that will be used to create the new documents' directories.

use-git

•   Default: False

    Some commands will issue git commands if this option is set to True. For example in mv or rename.

browse-query-format

•   Default: '{doc[title]} {doc[author]}'

    The query string that is to be searched for in the browse command whenever a search engine is used.

search-engine

•   Default: 'https://duckduckgo.com'

    Search engine to be used by some commands like browse.

user-agent

•   Default: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3)'

    User agent used by papis whenever it obtains information from external servers.

scripts-short-help-regex

•   Default: '.*papis-short-help: *(.*)'

    This is the format of the short help indicator in external papis commands.

info-name

•   Default: 'info.yaml'

    The default name of the information files.

doc-url-key-name

•   Default: 'doc_url'

    Some documents might have, apart from an url, also a file url associated with them. The key name appearing in the information file is defined by this setting.

default-library

•   Default: 'papers'

    The name of the library that is to be searched when papis is run without library arguments.

format-doc-name

•   Default: 'doc'

    This setting controls the name of the document in the papis format strings like in format strings such as match-format or header-format. For instance, if you are managing videos, you might want to set this option to vid in order to set the header-format to {vid[title]} - {vid[director]} - {vid[duration]}.

match-format

•   Default: '{doc[tags]}{doc.subfolder}{doc[title]}{doc[author]}{doc[year]}'

    Default format that is used to match a document against in order to select it. For example if the match-format is equal to {doc[year]} {doc[author]} then the title of a document will not work to match a document, only the year and author.

header-format

•

Default:

    <ansired>{doc.html_escape[title]}</ansired>
     <ansigreen>{doc.html_escape[author]}</ansigreen>
      <ansiblue>({doc.html_escape[year]})</ansiblue> [<ansiyellow>{doc.html_escape[tags]}</ansiyellow>]

    Default format that is used to show a document in order to select it.

header-format-file

•   Default: None

    This option should have the path of a file with the header-format template. Sometimes templates can get big so this is a way of not cluttering the config file with text.

    As an example you would set

    [papers]
    header-format-file = ~/.papis/config/styles/header.txt

info-allow-unicode

•   Default: True

    This flag is to be set if you want to allow unicode characters in your info file or not. If it is set to false then a representation for the unicode characters will be written in its place. Since we should be living in an unicode world, it is set to True by default.

  Tools options

opentool

•   Default: 'xdg-open'

    This is the general program that will be used to open documents. As for now papis is not intended to detect the type of document to be opened and decide upon how to open the document. You should set this to the right program for the tool. If you are on linux you might want to take a look at ranger or let the default handle it in your system. For mac users you might set this to open.

browser

•   Default: '$BROWSER'

    Program to be used for opening websites, the default is the environment variable $BROWSER.

picktool

•   Default: 'papis'

    This is the program used whenever papis asks you to pick a document or options in general.

Only option is:

•   papis

editor

•   Default: '$EDITOR'

    Editor used to edit files in papis, e.g., for the papis edit command. It defaults to the $EDITOR environment variable, if this is not set then it will default to the $VISUAL environment variable. Otherwise the default editor in your system will be used.

file-browser

•   Default: 'xdg-open'

    File browser to be used when opening a directory. It defaults to the default file browser in your system, however, you can set it to different file browsers such as dolphin, thunar or ranger just to name a few.

  Bibtex options

bibtex-journal-key

•   Default: 'journal'

    Journal publishers may request abbreviated journal titles. This option allows the user to set the key for the journal entry when using papis export --bibtex.

    Set as full_journal_title or abbrev_journal_title for whichever style required. Default is journal.

extra-bibtex-keys

•   Default: '[]'

    When exporting documents in bibtex format, you might want to add non-standard bibtex keys such as doc_url or tags. You can add these as a valid python list of strings, for instance:

    [mylib]
    extra-bibtex-keys = ["tags", "doc_url"]

bibtex-ignore-keys

•   Default: '[]'

    When exporting a document to the BibTeX format, do not export the keys appearing in this list. This might be useful if you have some keys that have a lot of content, such as abstract, or maybe you have used a valid BibTeX key for some other purposes, like the note key.

extra-bibtex-types

•   Default: '[]'

    Allow non-standard bibtex types to be recognized, e.g,

    [mylib]
    extra-bibtex-types = ["wikipedia", "video", "song"]

    See bibtex reference.

multiple-authors-format

•   Default: '{au[family]}, {au[given]}'

    When retrieving automatic author information from services like crossref.org, papis usually builds the author field for the given document. The format how every single author name is built is given by this setting, for instance you could customize it by the following:

    multiple-authors-format = {au[surname]} -- {au[given_name]}

    which would given in the case of Albert Einstein the string Einstein -- Albert.

multiple-authors-separator

•   Default: ' and '

    Similarly to multiple-authors-format, this is the string that separates single authors in the author field. If it is set to `` and `` then you would have <author 1> and <author 2> and .... in the author field.

bibtex-unicode

•   Default: False

    Whether or not to allow direct unicode characters in the document fields to be exported into the bibtex text.

  papis add options

ref-format

•   Default: '{doc[title]:.15} {doc[author]:.6} {doc[year]}'

    This flag is set to change the ref flag in the info.yaml file when a document is imported. For example: I prefer the format FirstAuthorYear e.g. Plews2019. This would be achieved by the following:

    ref-format = {doc[author_list][0][surname]}{doc[year]}

    In general however I recomment the default behaviour of just using the author key of the document, i.e.,

    ref-format = {doc[title]:.15} {doc[author]:.6} {doc[year]}

    The spaces in the value of the format will be important in order to capitalize the string, i.e., if you have a title like STUDIES ABOUT EARTH AND HIMMEL and and an author list like mesh-ki-ang-nuna then the built reference will be StudiesAboutEMeshKi.

    NOTE: Special characters will be replaced when generating the ref entry (e.g. Ö → O, . and other symbols will be striped from the string).

    If you want to add some punctuation, dots (.) and underscores (_) can be escaped by a backslash. For example,

    ref-format = {doc[author_list][0][surname]}\.{doc[year]}

    would result in 'Plews.2019'. To ensure correct capitalization you might consider inserting whitespaces after an escaped character.

add-confirm

•   Default: False

    If set to True, every time you run papis add the flag --confirm will be added automatically. If is set to True and you add it, i.e., you run papis add --confirm, then it will have the contrary effect, i.e., it will not ask for confirmation.

add-folder-name

•   Default: 'empty string'

    Default name for the folder of newly added documents. For example, if you want the folder of your documents to be named after the format author-title then you should set it to the papis format: {doc[author]}-{doc[title]}. You can create formatted subfolders by using path separators (i.e., /) in this format string, e.g., {doc[year]} / {doc[title]}. Per default a hash followed by the author name is created.

add-file-name

•   Default: None

    Same as add-folder-name, but for files, not folders. If it is not set, the names of the files will be cleaned and taken as-is.

add-interactive

•   Default: False

    If set to True, every time you run papis add the flag --interactive will be added automatically. If is set to True and you add it, i.e., you run papis add --interactive, then it will have the contrary effect, i.e., it will not run in interactive mode.

add-edit

•   Default: False

    If set to True, every time you run papis add the flag --edit will be added automatically. If it is set to True and you add something, i.e., you run papis add --edit, then it will have the contrary effect, i.e., it will not prompt to edit the info file.

add-open

•   Default: False

    If set to True, every time you run papis add the flag --open will be added automatically. If it is set to True and you add something, i.e., you run papis add --open, then it will have the contrary effect, i.e., it will not open the attached files before adding the document to the library.

add-subfolder

•   Default: 'empty string'

    Configure a default for the --subfolder command line option. Note that, this setting is not allowed to contain formatting options. However, one can also specify nested sub-folders.

  papis browse options

browse-key

•   Default: 'url'

    This command provides the key that is used to generate the url. For users that run papis add --from-doi, setting browse-key to doi constructs the url from dx.doi.org/<DOI>, providing a much more accurate url.

    Default value is set to url. If you need functionality with the search-engine option, set the option to an empty string e.g.

    browse-key = ''

  papis edit options

notes-name

•   Default: 'notes.tex'

    In papis edit you can edit notes about the document. notes-name is the default name of the notes file, which by default is supposed to be a TeX file. The notes-name is formated by the formater, so that the filename of notes can be dynamically defined, e.g.:

    notes-name = notes_{doc[title]:.15}.tex

notes-template

•   Default: ''

    When editing notes for the first time, a preliminary note will be generated based on a template. The path to this template is specified by notes-template. The template will then be formated by formater. This can be useful to enforce the same style in the notes for all documents.

    Default value is set to "", which will return an empty notes file. If no file is found at the path to the template, then also an empty notes file will be generated.

  Citations options

You can change the name of the citation files, however we discourage this.

citations-file-name

•   Default: 'citations.yaml'

    The name of the file to store the citations of the documents.

cited-by-file-name

•   Default: 'cited-by.yaml'

    The name of the file to store the citations to the document.

  Marks

open-mark

•   Default: False

    If this option is set to True, every time papis opens a document it will ask to open a mark first. If it is set to False, then doing

    papis open --mark

    will avoid opening a mark.

mark-key-name

•   Default: 'marks'

    This is the default key name for the marks in the info file. For example, if you set mark-key-name = bookmarks then you would have in your info.yaml file

    author: J. Krishnamurti
    bookmarks:
    - name: Chapter 1
      value: 120

mark-format-name

•   Default: 'mark'

    This is the name of the mark to be passed to the options mark-header-format etc... E.g. if you set mark-format-name = m then you could set mark-header-format = {m[value]} - {m[name]}.

mark-header-format

•   Default: '{mark[name]} - {mark[value]}'

    This is the format in which the mark will appear whenever the user has to pick one. You can change this in order to make marks work in the way you like. Per default it is assumed that every mark has a name and a value key.

mark-match-format

•   Default: '{mark[name]} - {mark[value]}'

    Format in which the mark name has to match the user input.

mark-opener-format

•   Default: 'xdg-open'

    Due to the difficulty to generalize opening a general document at a given bookmark, the user should set this in whichever way it suits their needs. For example

•   If you are using the pdf viewer evince and you want to open a mark, you would use

    mark-opener-format = evince -p {mark[value]}

•   If you are using okular you would use

    mark-opener-format = okular -p {mark[value]}

•   If you are using zathura, do

    mark-opener-format = zathura -P {mark[value]}

  Downloaders

downloader-proxy

•   Default: None

    There is the possibility of download papers using a proxy. To know more you can checkout this link.

isbn-service

•   Default: 'openl'

    Sets the ISBN service used by the ISBN importer. Available plugins are documented here.

  Databases

default-query-string

•   Default: '.'

    This is the default query that a command will take if no query string is typed in the command line. For example this is the query that is passed to the command open whenever no search string is typed:

    papis open

    Imagine you want to open all papers authored by John Smith whenever you do not specify an input query string, i.e., papis open. Then setting

    default-query-string = author:"John Smith"

    would do the trick. Notice that the current example has been done assuming the database-backend = papis.

database-backend

•   Default: 'papis'

    The backend to use in the database. As for now papis supports the own database system papis and whoosh.

use-cache

•   Default: True

    Set to False if you do not want to use the cache for the given library. This is only effective if you're using the papis database-backend.

cache-dir

•   Default: '$XDG_CACHE_HOME'

whoosh-schema-fields

•   Default: "['doi']"

    Python list with the TEXT fields that should be included in the whoosh database schema. For instance, say that you want to be able to search for the doi and ref of the documents, then you could include

    whoosh-schema-fields = ['doi', 'ref']

whoosh-schema-prototype

•

Default:

    {
    "author": TEXT(stored=True),
    "title": TEXT(stored=True),
    "year": TEXT(stored=True),
    "tags": TEXT(stored=True),
    }

    This is the model for the whoosh schema, check the documentation for more information.

  Terminal user interface (picker)

These options are for the terminal user interface (tui). They are defined in the section tui which means that you can set them in your configuration file globally like

[tui]
status_line_format = "F1: Help"
...

or inside the library sections prepending a tui-,

[papers]
tui-status_line_format = "Library papers**
...

status_line_format

•   Default: '{selected_index}/{number_of_documents} F1:help c-l:redraw '

    This is the format of the string that appears at the bottom in the status line. Right now there are only two variables defined:

•   selected_index

•   number_of_documents

    Which are self-explanatory.

status_line_style

•   Default: 'bg:ansiwhite fg:ansiblack'

    The style the status line should have. Examples are fg:#ff00aa bg:black etc... More information can be found here .

message_toolbar_style

•   Default: 'bg:ansiyellow fg:ansiblack'

    The style of the message toolbar, this toolbar is the one where messages of the echo command are rendered for instance.

options_list.selected_margin_style

•   Default: 'bg:ansiblack fg:ansigreen'

    Style of the margin of the selected document in the picker.

options_list.unselected_margin_style

•   Default: 'bg:ansiwhite'

    Style of the margin of the unselected documents in the picker. If you don't want any coloring for them you can just set this setting to the empty string as such

    tui-options_list.unselected_margin_style =

error_toolbar_style

•   Default: 'bg:ansired fg:ansiblack'

    The style for the error messages.

editmode

•   Default: 'emacs'

    Whenever the user is typing text, one can use either emacs like keybindings or vi. If this does not tell you anything, you can just leave it as is.

move_down_key

•   Default: 'down'

move_up_key

•   Default: 'up'

move_down_while_info_window_active_key

•   Default: 'c-n'

move_up_while_info_window_active_key

•   Default: 'c-p'

focus_command_line_key

•   Default: 'tab'

edit_document_key

•   Default: 'c-e'

open_document_key

•   Default: 'c-o'

show_help_key

•   Default: 'f1'

show_info_key

•   Default: 's-tab'

go_top_key

•   Default: 'home'

go_bottom_key

•   Default: 'end'

mark_key

•   Default: 'c-t'

  FZF integration

From version 0.12 papis ships with an out-of-the-box fzf integration for the picker. A minimal terminal user interface is provided and together with options for its customization. You can set the picktool to fzf by setting

picktool = fzf

in the configuration section of your library.

In comparison to the built-in papis tui the advantage of the fzf picker is that it is much faster, however a disadvantage is that it is restricted to one-line entries. Also it is important to notice that fzf will only match against what is shown on the terminal screen, as opposed to the papis matcher, that can match against the whole title and whole author text since this is controlled by the match-format setting. However, for many uses it might not bother the user to have this limitation of fzf.

fzf-binary

•   Default: 'fzf'

    Path to or name of the fzf binary.

fzf-extra-flags

•   Default: ['--ansi', '--multi', '-i']

    Extra flags to be passed to fzf every time it gets called.

fzf-extra-bindings

•   Default: ['ctrl-s:jump']

    Extra bindings to fzf as a python list. Refer to the fzf documentation for more details.

fzf-header-format

•   Default: '{c.Fore.MAGENTA}{doc[title]:<70.70}{c.Style.RESET_ALL} :: {c.Fore.CYAN}{doc[author]:<20.20}{c.Style.RESET_ALL}{c.Fore.YELLOW}«{doc[year]:4}»{c.Style.RESET_ALL}:{doc[tags]}'

    Format for the entries for fzf. Notice that if you want colors you should have in fzf-extra-flags the --ansi flag and include the colors in the header-format as ansi escape sequences.

    The papis format string is given the additional variable c which contains the package colorama in it. Refer to the colorama documentation to see which colors are available here. For instance, if you want the title in red you would put in your fzf-header-format

    "{c.Fore.RED}{doc[title]}{c.Style.RESET_ALL}"

  fzf with a preview window

fzf has the disadvantage that it does not support multiline output and it matches only against what it shows on the screen.

You can go around this issue by composing an fzf customization. The following configuration

fzf-extra-flags = ["--ansi", "--multi", "-i",
                   "--preview", "echo {} | sed -r 's/~~/\\n/g; /^ *$/d' ",
                   "--preview-window", "bottom:wrap:20%%",
                   "--color", "preview-fg:#F6E6E4,preview-bg:#5B6D5B"]
fzf-extra-bindings = ["ctrl-s:jump",
                      "ctrl-t:toggle-preview"]
fzf-header-format = {c.Fore.MAGENTA}{doc[title]}{c.Style.RESET_ALL}~~ {c.Fore.CYAN}{doc[author]}{c.Style.RESET_ALL}~~ {c.Fore.YELLOW}«{doc[year]}»{c.Style.RESET_ALL}~~ {c.Fore.YELLOW}{doc[journal]}{c.Style.RESET_ALL}~~ :{doc[tags]}

will have unrestricted titles, author, journal etc fields against which the query will match and it will show in the fzf preview window a tidy description of the currently selected field by replacing the token ~~ by a newline. You can try this out and play with fzf customizations. Please note that bottom:wrap:20%% has two % since the config file interpolator uses % as a reserved symbol, so it must be escaped by writing two of them.

  Other

unique-document-keys

•   Default: "['doi','ref','isbn','isbn10','url','doc_url']"

    Whenever you add a new document, papis tries to figure out if you have already added this document before. This is partially done checking for some special keys, and checking if they match. Which keys are checked against is decided by this option, which should be formatted as a python list, just as in the default value.

    For instance, if you add a paper with a given doi, and then you add another document with the same doi, then papis will notify you that there is already another document with this doi because the doi key is part of the unique-document-keys option.

document-description-format

•   Default: '{doc[title]} - {doc[author]}'

    papis sometimes will have to tell you which document it is processing through text, for instance, imagine you are updating a document

    author: Albert Einstein
    title: General Relativity

    and papis is doing something with it. Then if your document-description-format is set to {doc[title]} - {doc[author]}, you will see that papis tells you

    .....
    Updating 'General Relativity - Albert Einstein'
    ...

    so you will know exactly what is going on.

sort-field

•   Default: None

    As of version 0.10, some command line commands have the --sort option to sort the documents according to a given field. If you set sort-field in your configuration file, this will sort by default the documents according to this sort field. For instance, if you want your documents by default to be sorted by year, you would set sort-field = year.

sort-reverse

•   Default: False

    Augments sort-field by allowing the documents to be sorted in reverse order. For example, when sorting by year, this allows sorting ascendingly or descendingly. This is a boolean option that can be set to True or False.

time-stamp

•   Default: True

    Whether or not to add a timestamp to a document when is being added to papis. If documents have a timestamp, then they will be sortable using --sort time-added option.

formater

•   Default: 'python'

    The formatting language in python can be configured through plugins.

class papis.format.PythonFormater
    Construct a string using a PEP 3101 (str.format based) format string.

    This formater is named "python" and can be set using the formater setting in the configuration file (see General settings).

class papis.format.Jinja2Formater
    Construct a string using Jinja2 templates.

    This formater is named "jinja2" and can be set using the formater setting in the configuration file (see General settings).

THE INFO.YAML FILE

At the heart of papis there is the information file. The info file contains all information about the documents.

It uses the YAML syntax to store information, which is a very human-readable language. It is quite format-free: papis does not assume that any special information should be there. However it will interpret the field files as the files linked to the document for the papis open command. The files field should be formatted as a YAML list.

For instance, if are storing papers with papis, then you most probably would like to store author and title in there like this:

author: Isaac Newton
title: Opticks, or a treatise of the reflections refractions, inflections and
  colours of light
files:
  - document.pdf

Here we have used the files field to tell papis that the paper has a pdf document attached to it. You can of course attach many other documents so that you can open them when you are opening it with the papis open command. For instance if you have a paper with supporting information, you could store it like such

author: Isaac Newton
title: Opticks, or a treatise of the reflections refractions, inflections and
  colours of light
files:
  - document.pdf
  - supporting-information.pdf

Therefore, in the folder where this document lives we have the following structure

.
└── paper-folder
    ├── info.yaml
    ├── document.pdf
    └── supporting-information.pdf

THE LIBRARY STRUCTURE

One of the things that makes papis interesting is the fact that its library structure is nearly nonexistent.

A papis library is linked to a directory, where all the documents are (and possibly sublibraries). What papis does is simply to go to the library folder and look for all subfolders that contain a information file, which by default is a info.yaml file.

Every subfolder that has an info.yaml file in it is a valid papis document. As an example let us consider the following library

/home/fulano/Documents/papers/
├── folder1
│   └── paper.pdf
├── folder2
│   ├── folder3
│   │   ├── info.yaml
│   │   └── blahblahblah.pdf
│   └── folder4
│       ├── info.yaml
│       └── output.pdf
├── classics
│   └── folder5
│       ├── info.yaml
│       └── output.pdf
├── physics
│   └── newton
│       └── principia
│           ├── document.pdf
│           ├── supplements.pdf
│           └── info.yaml
└─── rpa
    └── bohm
        ├── info.yaml
        ├── notes.tex
        └── output.pdf

The first thing that you might notice is that there are many folders. Just to check that you understand exactly what is a document, please think about which of these pdfs is not a valid papis document... That's right!, folder1/paper.pdf is not a valid document since the folder1 does not contain any info.yaml file. You see also that it does not matter how deep the folder structure is in your library: you can have a physics folder in which you have a newton folder in which you have a folder containing the actual book document.pdf plus some supplementary information supplements.pdf. In this case, inside the info.yaml you would have the following file section

files:
- document.pdf
- supplements.pdf

which tells papis that this folder contains two relevant files.

THE DATABASE

One of the things that makes papis interesting is the fact that there can be many backends for the database system, including no database.

Right now there are three types of databases that the user can use:

•

No database

    database-backend = papis
    use-cache = False

•

Simple cache based database.

    database-backend = papis

•

Whoosh based database.

    database-backend = whoosh

    If you just plan to have up to 3000 documents in your library, you will have ample performance with the two first options. However if you're reaching higher numbers, you'll probably want to use the Whoosh backend for very good performance.

    You can select a database by using the flag database-backend.

  Papis database

The fact that there is no database means that papis should crawl through the library folder and see which folders have an info.yaml file, which is for slow computers (and harddrives) quite bad.

Papis implements a very rudimentary caching system. A cache is created for every library. Inside the cache the whole information already converted into python is stored.

These cache files are stored per default in

~/.cache/papis/

Notice that most papis commands will update the cache if it has to be the case. For instance the edit command will let you edit your document's information and after you are done editing it will update the information for the given document in the cache. If you go directly to the document and edit the info file without passing through the papis edit command, the cache will not be updated and therefore papis will not know of these changes, although they will be there. In such cases you will have to clear the cache.

  Clearing the cache

To clear the cache for a given library you can use the flag --clear-cache, e.g.

papis --clear-cache

  Query language

Since version 0.3 there is a query language in place for the searching of documents. The queries can contain any field of the info file, e.g., author:einstein publisher : review will match documents that have a matching author with einstein AND have a publisher matching review. The AND part here is important, since only the AND filter is implemented in this simple query language. At the moment it is not possible to do an OR. If you need this, you should consider using the Whoosh database.

For illustration, here are some examples:

•   Open documents where the author key matches 'albert' (ignoring case) and year matches '19' (i.e., 1990, 2019, 1920):

    papis open 'author : albert year : 05'

•   Add the restriction to the previous search that the usual matching matches the substring 'licht' in addition to the previously selected

    papis open 'author : albert year : 05 licht'

    This is not to be mixed with the restriction that the key year matches '05 licht', which will not match any year, i.e.

    papis open 'author : albert year : "05 licht"'

  Disabling the cache

You can disable the cache using the configuration setting use-cache and set it to False, e.g.

[settings]
use-cache = False
[books]
# Use cache for books but don't use for the rest of libraries
use-cache = True

  Whoosh database

Papis has also the possibility to use the blazing fast and pure python Whoosh library. Its performance is orders of magnitude better than the crude cache based database.

Of course, the performance comes at a cost. To achieve more performance a database backend should create an index with information about the documents. Parsing a user query means going to the index and matching the query to what is found in the index. This means that the index can not in general have all the information that the info file of the documents includes.

In other words, the whoosh index will store only certain fields from the document's info files. The good news is that we can tell papis exactly which fields we want to index. These flags are

•   whoosh-schema-fields

•   whoosh-schema-prototype

    The prototype is for advanced users. If you just want to, say, include the publisher to the fields that you can search in, then you can put

    whoosh-schema-fields = ['publisher']

    and you will be able to find documents by their publisher. For example, without this line set for publisher, the query

    papis open publisher:*

    will not return anything, since the publisher field is not being stored.

  Query language

The whoosh database uses the whoosh query language which is much more advanced than the query language in the Papis database.

The whoosh query language supports both AND and OR, for instance

papis open '(author:einstein AND year:1905) OR title:einstein'

will give papers of einstein in the year 1905 together with all papers where einstein appears in the title.

You can read more about the whoosh query language here.

THE PAPIS_ID KEY

Every papis document should have a papis_id key created at random by the papis database.

If you manually add a document into your library, you will have to clear the library cache

papis --cc

in order to trigger a database building the next time that you issue a papis command. When the library scans the document added manually, it will create a papis_id key automatically and it will edit your info.yaml file accordingly. We stress again that the database will edit the info.yaml file, without committing the changes (in the case that you are using a git repository), so that you can inspect the changes manually.

Please note that if you add a document manually with an existing papis_id to your library, papis will not check if there is an id clash. A clash of ids has a very low probability. Please refer to the papis-doctor help for checking for clashes.

  Use of papis_id in scripts

Since the papis_id key is a unique identifier, it is quite useful for scripts that do not depend on the actual path to the document in your system.

For instance you can get the papis_id of a document in bash like such

id=$(papis list --id)

and subsequently use the id variable to trigger other commands, for instance you can open the file attached to the document like such

papis open papis_id:${id}

CITATIONS OF DOCUMENTS: CITATIONS.YAML AND CITED-BY.YAML FILES

Papis has support for downloading and exploring citations that documents reference, and also cited-by type references.

If your document has a doi associated and you use the updater from this doi, or you added information from the doi when you added the document, then chances are that the info.yaml file has a citations key within it.

In this case, papis can actually get metadata from these dois and store it in a citations.yaml file, for references that the document has within it.

You can generate this file either from the web application or from the papis citations command. Refer to their respective documentations in order to know more about it.

As of version v0.13, it is also possible to generate a cited-by.yaml file with the information of other papers that cite your document. This is done by scanning your papis library for documents that cite said document. You can also generate this file from the web application or from the papis citations command.

The citation files try to include always first information already existing in the library. This is, before doing any online query, papis tries to find the relevant information in your library.

Notice that papis copies most of the metadata to the citations.yaml and cited-by.yaml files. Even though this might seem quite heavy on disk space, as a rule of thumb all the citation.yaml files of a library with 2k papers containing physics papers will amount to only around 30MB.

COMMANDS

  Add

The add command is one of the central commands of the papis command line interface. It is a very versatile command with a fair amount of options.

There are also customization settings available for this command, check out the configuration page for add.

  Examples

•   Add a document located in ~/Documents/interesting.pdf and name the folder where it will be stored in the database interesting-paper-2021

    papis add ~/Documents/interesting.pdf \
        --folder-name interesting-paper-2021

    if you want to add directly some key values, like author, title and tags, you can also run the following:

    papis add ~/Documents/interesting.pdf \
        --folder-name interesting-paper-2021 \
        --set author 'John Smith' \
        --set title 'The interesting life of bees' \
        --set year 1985 \
        --set tags 'biology interesting bees'

•   Add a paper that you have locally in a file and get the paper information through its doi identifier (in this case 10.10763/1.3237134 as an example):

    papis add ~/Documents/interesting.pdf --from doi 10.10763/1.3237134

•   Add paper to a library named machine-learning from arxiv.org

    papis -l machine-learning add \
        --from arxiv https://arxiv.org/abs/1712.03134

•   If you do not want copy the original pdfs into the library, you can also tell papis to just create a link to them, for example

    papis add --link ~/Documents/interesting.pdf \
        --from doi 10.10763/1.3237134

    will add an entry into the papis library, but the pdf document will remain at ~/Documents/interesting.pdf, and in the document's folder there will be a link to ~/Documents/interesting.pdf instead of the file itself. Of course you always have to be sure that the document at ~/Documents/interesting.pdf does not disappear, otherwise you will end up without a document to open.

•   Papis also tries to make sense of the inputs that you have passed to the command, for instance you could provide only a doi and papis will try to know if this is indeed a doi

    papis add 10.1103/PhysRevLett.123.156401

    or from a url

    papis add journals.aps.org/prl/abstract/10.1103/PhysRevLett.123.156401
    papis add https://arxiv.org/abs/1712.03134

•   You can also download citations alongside the information of the paper if the papers is able to obtain a doi identifier. You can pass the --fetch-citations flag in order to create a citations.yaml file.

  Examples in python

There is a python function in the add module that can be used to interact in a more effective way in python scripts,

papis.commands.add.run(paths: List[str], data: Dict[str, Any] | None = None, folder_name: str | None = None, file_name: str | None = None, subfolder: str | None = None, base_path: str | None = None, confirm: bool = False, open_file: bool = False, edit: bool = False, git: bool = False, link: bool = False, citations: Sequence[Dict[str, Any]] = ()) -> None

Parameters

•   paths -- Paths to the documents to be added

•   data -- Data for the document to be added. If more data is to be retrieved from other sources, the data dictionary will be updated from these sources.

•   folder_name -- Name of the folder where the document will be stored

•   file_name -- File name of the document's files to be stored.

•   subfolder -- Folder within the library where the document's folder should be stored.

•   confirm -- Whether or not to ask user for confirmation before adding.

•   open_file -- Whether or not to ask the user for opening the file before adding.

•   edit -- Whether or not to ask user for editing the info file before adding.

•   git -- Whether or not to ask user for committing before adding, in the case of course that the library is a git repository.

  Command-line Interface

Add a document into a given library

papis add [OPTIONS] [FILES]...

Options

-h, --help
    Show this message and exit.

-s, --set <set_list>
    Set some information before

-d, --subfolder <subfolder>
    Subfolder in the library

-p, --pick-subfolder
    Pick from existing subfolders

--folder-name <folder_name>
    Name for the document's folder (papis format)

--file-name <file_name>
    File name for the document (papis format)

--from <from_importer>
    Add document from a specific importer (arxiv, bibtex, crossref, dblp, doi, folder, isbn, lib, pdf2arxivid, pdf2doi, pmid, yaml)

-b, --batch
    Batch mode, do not prompt or otherwise

--confirm, --no-confirm
    Ask to confirm before adding to the collection

--open, --no-open
    Open file before adding document

--edit, --no-edit
    Edit info file before adding document

--link, --no-link
    Instead of copying the file to the library, create a link to its original location

--git, --no-git
    Git add and commit the new document

--list-importers, --li
    List all available papis importers

--force-download, --fd
    Download file with importer even if local file is passed

--fetch-citations
    Fetch citations from doi

    Arguments

FILES
    Optional argument(s)

  Addto

This command adds files to existing papis documents in some library.

For instance imagine you have two pdf files, a.pdf and b.pdf that you want to add to a document that matches with the query string einstein photon definition, then you would use

papis addto 'einstein photon definition' -f a.pdf -f b.pdf

notice that we repeat two times the flag -f, this is important.

  Command-line Interface

Add files to an existing document

papis addto [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--git, --no-git
    Add and commit files

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

-f, --files <files>
    File fullpaths to documents

-u, --urls <urls>
    URLs to documents

--file-name <file_name>
    File name for the document (papis format)

--doc-folder <doc_folder>
    Document folder on which to apply action

    Arguments

QUERY
    Optional argument

  Browse

This command will try its best to find a source in the internet for the document at hand.

Of course if the document has an url key in its info file, it will use this url to open it in a browser. Also if it has a doc_url key, or a doi, it will try to compose urls out of these to open it.

If none of the above work, then it will try to use a search engine with the document's information (using the browse-query-format). You can select which search engine you want to use using the search-engine setting.

It uses the configuration option browse-key to form an url according to which key is given in the document. You can bypass this option using the -k flag issuing the command.

papis browse -k doi einstein

This will form an url through the DOI of the document.

papis browse -k isbn

This will form an url through the ISBN of the document using isbnsearch.org.

papis browse -k ads

This will form an url using the gread ADS service and there you can check for similar papers, citations, references and much more. Please note that for this to work the document should have a DOI attached to it.

papis browse -k whatever

This will consider the key whatever of the document to be a valid url, I guess at this point you'll know what you're doing.

papis browse -k search-engine

This is the default, it will do a search-engine search with the data of your paper and hopefully you'll find it.

  Command-line Interface

Open document's url in a browser

papis browse [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

-k, --key <key>
    Use the value of the document's key to open in the browser, e.g. doi, url, doc_url ...

-n, --print
    Just print out the url, do not open it with browser

-a, --all
    Apply action to all matching documents

--doc-folder <doc_folder>
    Document folder on which to apply action

    Arguments

QUERY
    Optional argument

  Bibtex

This command helps to interact with bib files in your LaTeX projects.

  Examples

I use it for opening some papers for instance

papis bibtex read new_papers.bib open

or to add papers to the bib

papis bibtex           \
  read new_papers.bib  \ # Read bib file
  add -q einstein      \ # Pick a doc with query 'einstein' from library
  add -q heisenberg    \ # Pick a doc with query 'heisenberg' from library
  save new_papers.bib    # Save in new_papers.bib

or if I update some information in my papis yaml files then I can do

papis bibtex          \
  read new_papers.bib \ # Read bib file
  update -f           \ # Update what has been read from papis library
  save new_papers.bib   # save everything to new_papers.bib, overwriting

  Local configuration file

If you are working in a local folder where you have a bib file called main.bib, you'll grow sick and tired of writing always read main.bib and save main.bib, so you can write a local configuration file .papis.config for papis bibtex to read and write automatically

[bibtex]
default-read-bibfile = main.bib
default-save-bibfile = main.bib
auto-read = True

with this setup, you can just do

papis bibtex add -q einstein save

  Check references quality

When you're collaborating with someone, you might come across malformed or incomplete references. Most journals want to have all the
``
doi``s and urls available. You can automate this diagnostic with

For this you kan use the command doctor

papis bibtex read mybib.bib doctor

Mostly I want to have only the references in my project's bib file that are actually cited in the latex file, you can check which references are not cited in the tex files by doing

papis bibtex iscited -f main.tex -f chapter-2.tex

and you can then filter them out using the command filter-cited.

To monitor the health of the bib project's file, I mostly have a target in the project's Makefile like

.PHONY: check-bib
check-bib:
    papis bibtex iscited -f main.tex doctor

it does not solve all problems under the sun, but it is really better than no check!

  Vim integration

Right now, you can easily use it from vim with these simple lines

function! PapisBibtexRef()
  let l:temp = tempname()
  echom l:temp
  silent exec "!papis bibtex ref -o ".l:temp
  let l:olda = @a
  let @a = join(readfile(l:temp), ',')
  normal! "ap
  redraw!
  let @a = l:olda
endfunction
command! -nargs=0 BibRef call PapisBibtexRef()
command! -nargs=0 BibOpen exec "!papis bibtex open"

And use like such: asciicast

  Command-line Interface

A papis script to interact with bibtex files

papis bibtex [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

Options

-h, --help
    Show this message and exit.

--noar, --no-auto-read
    Do not auto read even if the configuration file says it

    Commands

add Add a reference to the bibtex file

bibtex
    Import documents from a bibtex file

browse
    browse a document in the documents list

doctor
    Check bibfile for correctness, missing...

edit
    Tries to find the document in the list...

filter-cited
    Filter cited documents from the read bib...

import
    Import documents to papis e.g.

iscited
    Check which documents are not cited e.g.

open
    Open a document in the documents list

ref Print the reference for a document

rm  Remove a document from the documents list

save
    Save the documents imported in bibtex format

sort
    Sort documents

unique
    Remove duplicate BibTeX entries.

update
    Update documents from and to the library

  Config

The config command allows you to query the settings used by Papis on your system.

The config command returns the value used by Papis. Therefore, if you have not customized some setting, it will return the default value. In contrast, if you have customized it, it will return the value set in the configuration file. For example, to find out to what your "default-library" is set to, call:

papis config default-library

The config command can also be used to query a settings' default value. This is done by adding the --default flag. This ignores all settings set in your Papis configuration file (note, however, that settings set in a config.py script can count as default values). Check the default "default-library" with:

papis config --default default-library

Settings from a specific section in the configuration file can also be accessed. To take an example, the Bibtex command's settings can be accessed with:

papis config --section bibtex
papis config --default --section bibtex

or with papis config (without the section argument) to show the settings available for all the known sections.

You can also query a specific setting within a section. For example like this:

papis config --section bibtex default-read-bibfile
papis config --default --section bibtex default-read-bibfile

Alternatively, you can also use the <section.setting> format to query the value of a setting in a specific section:

papis config bibtex.default-read-bibfile
papis config --default bibtex.default-read-bibfile

For some more advanced usage, we can also query multiple settings at once. Here, sections specified with <section>.<setting> override the section specified by --setting <section>. This can be achieved by:

papis config --section sec1 key1 key2 key3 sec2.key4 sec3.key5

You can find a list of all available settings in the configuration section at General settings. Commands and other plugins can define their own settings, which are documented separately.

  Command-line Interface

Print configuration values

papis config [OPTIONS] [OPTIONS]...

Options

-h, --help
    Show this message and exit.

-s, --section <section>
    select a default section for the options

-d, --default
    List default configuration setting values, instead of those in the configuration file

--json
    Print settings in a JSON format

    Arguments

OPTIONS
    Optional argument(s)

  Main

  Examples

•   To override some configuration options, you can use the flag --set, for instance, if you want to override the editor used and the opentool to open documents, you can just type

    papis --set editor gedit --set opentool firefox edit
    papis --set editor gedit --set opentool firefox open

•   If you want to list the libraries and pick one before sending a database query to papis, use --pick-lib as such

    papis --pick-lib open 'einstein relativity'

  Command-line Interface

papis [OPTIONS] COMMAND [ARGS]...

Options

-h, --help
    Show this message and exit.

--version
    Show the version and exit.

-v, --verbose
    Make the output verbose (equivalent to --log DEBUG)

--profile <profile>
    Print profiling information into file

-l, --lib <lib>
    Choose a library name or library path (unnamed library)

-c, --config <config>
    Configuration file to use

--pick-lib
    Pick library to use

--cc, --clear-cache
    Clear cache of the library used

-s, --set <set_list>
    Set key value, e.g., --set info-name information.yaml --set opentool evince

--color <color>
    Prevent the output from having color

Options
    always | auto | no

--log <log>
    Logging level

Options
    INFO | DEBUG | WARNING | ERROR | CRITICAL

--logfile <logfile>
    File to dump the log

--np <np>
    Use number of processors for multicore functionalities in papis

    Commands

add Add a document into a given library

add-contact
    Add new contact

add-datasheet
    Add new datasheet

add-from-mail
    Add contact from mail file

add-mark
    Add mark to document

addto
    Add files to an existing document

bibtex
    A papis script to interact with bibtex files

browse
    Open document's url in a browser

call
    Print contact information in mutt format

citations
    Handle document citations

commit-citations
    No help message available

config
    Print configuration values

crawl
    Crawl dois using a seed doi

dmenu
    Rough dmenu-based gui

doctor
    Check for common problems in documents

doi Parse doi from url and download via papis-scihub

edit
    Edit document information from a given...

exec
    Execute a python file in the environment...

explore
    Explore new documents using a variety of...

export
    Export a document from a given library

foreach
    No help message available

ga  Git add <document>

gacitations
    No help message available

git Run git command in a library or document...

list
    List documents' properties

mail
    Email a paper to my friend

merge
    Merge two documents from a given library

mutt
    Print contact information in mutt format

mv  Move a document into some other path

octopart
    No help message available

open
    Open document from a given library

parse-dois
    No help message available

rename
    Rename entry

rl  No help message available

rm  Delete a document, a file, or a notes-file

run Run an arbitrary shell command in the...

serve
    Start a papis server

tags
    No help message available

update
    Update a document from a given library.

update-scripts
    Update all scripts from papis repository

zathura-citations
    Run papis-hubation from zathura

zathura-open
    Reopen document taking into account marks

  Edit

This command edits the information of the documents. The editor used is defined by the editor configuration setting.

  Command-line Interface

Edit document information from a given library

papis edit [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--doc-folder <doc_folder>
    Document folder on which to apply action

--git, --no-git
    Add changes made to the info file

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

-n, --notes
    Edit notes associated to the document

-a, --all
    Apply action to all matching documents

-e, --editor <editor>
    Editor to be used

    Arguments

QUERY
    Optional argument

  Explore

This command is in an experimental stage but it might be useful for many people.

Imagine you want to search for some papers online, but you don't want to go into a browser and look for it. Explore gives you way to do this, using several services available online, more should be coming on the way.

An excellent such resource is crossref, which you can use by using the subcommand crossref:

papis explore crossref --author 'Freeman Dyson'

If you issue this command, you will see some text but basically nothing will happen. This is because explore is conceived in such a way as to concatenate commands, doing a simple

papis explore crossref -h

will tell you which commands are available. Let us suppose that you want to look for some documents on crossref, say some papers of Schroedinger, and you want to store them into a bibtex file called lib.bib, then you could concatenate the commands crossref and export --format bibtex as such

papis explore crossref -a 'Schrodinger' export --format bibtex lib.bib

This will store everything that you got from crossref in the file lib.bib and store in bibtex format. explore is much more flexible than that, you can also pick just one document to store, for instance let's assume that you don't want to store all retrieved documents but only one that you pick, the pick command will take care of it

papis explore crossref -a 'Schrodinger' pick export --format bibtex lib.bib

notice how the pick command is situated before the export. More generally you could write something like

papis explore \
    crossref -a Schroedinger \
    crossref -a Einstein \
    arxiv -a 'Felix Hummel' \
    export --format yaml docs.yaml \
    pick  \
    export --format bibtex specially-picked-document.bib

The upper command will look in crossref for documents authored by Schrodinger, then also by Einstein, and will look on the arxiv for papers authored by Felix Hummel. At the end, all these documents will be stored in the docs.yaml. After that we pick one document from them and store the information in the file specially-picked-document.bib, and we could go on and on.

If you want to follow-up on these documents and get them again to pick one, you could use the yaml command to read in document information from a yaml file, i.e., the previously created docs.yaml

papis explore \
    yaml docs.yaml \
    pick \
    cmd 'papis scihub {doc[doi]}' \
    cmd 'firefox {doc[url]}'

In this last example, we read the documents' information from docs.yaml and pick a document, which then feed into the explore cmd command, that accepts a papis formatting string to issue a general shell command. In this case, the picked document gets fed into the papis scihub command which tries to download the document using scihub, and also this very document is tried to be opened by firefox (in case the document does have a url).

  Command-line Interface

Explore new documents using a variety of resources

papis explore [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

Options

-h, --help
    Show this message and exit.

  add

papis explore add [OPTIONS]

  arxiv

Look for documents on ArXiV.org.

Examples of its usage are papis explore arxiv -a 'Hummel' -m 100 arxiv -a 'Garnet Chan' pick

If you want to search for the exact author name 'John Smith', you should enclose it in extra quotes, as in the example below papis explore arxiv -a '"John Smith"' pick

papis explore arxiv [OPTIONS]

Options

-h, --help
    Show this message and exit.

-q, --query <query>

-a, --author <author>

-t, --title <title>

--abstract <abstract>

--comment <comment>

--journal <journal>

--report-number <report_number>

--category <category>

--id-list <id_list>

--page <page>

-m, --max <max>

  bibtex

Import documents from a bibtex file

Examples of its usage are

papis explore bibtex lib.bib pick

papis explore bibtex [OPTIONS] BIBFILE

Options

-h, --help
    Show this message and exit.

    Arguments

BIBFILE
    Required argument

  citations

Query the citations of a paper

Example:

Go through the citations of a paper and export it in a yaml file papis explore citations 'einstein' export --format yaml einstein.yaml

papis explore citations [OPTIONS] [QUERY]

Options

--doc-folder <doc_folder>
    Document folder on which to apply action

-h, --help
    Show this message and exit.

-b, --cited-by
    Use the cited-by citations

-a, --all
    Apply action to all matching documents

    Arguments

QUERY
    Optional argument

  cmd

Run a general command on the document list

Examples of its usage are:

Look for 200 Schroedinger papers, pick one, and add it via papis-scihub

papis explore crossref -m 200 -a 'Schrodinger'
    pick cmd 'papis scihub {doc[doi]}'

    papis explore cmd [OPTIONS] COMMAND

    Options

-h, --help
    Show this message and exit.

    Arguments

COMMAND
    Required argument

  crossref

Look for documents on crossref.org.

Examples of its usage are

papis explore crossref -a 'Albert einstein' pick export --bibtex lib.bib

papis explore crossref [OPTIONS]

Options

-h, --help
    Show this message and exit.

-q, --query <query>
    General query

-a, --author <author>
    Author of the query

-t, --title <title>
    Title of the query

-m, --max <_ma>
    Maximum number of results

-f, --filter <_filters>
    Filters to apply

-o, --order <order>
    Order of appearance according to sorting

Default
    desc

Options
    desc | asc

-s, --sort <sort>
    Sorting parameter

Default
    score

Options
    published-print | score | indexed | relevance | references-count | issued | published-online | published | is-referenced-by-count | updated | deposited

  dblp

Look for documents on dblp.org.

Examples of its usage are

papis explore dblp -a 'Albert einstein' pick export --bibtex lib.bib

papis explore dblp [OPTIONS]

Options

-h, --help
    Show this message and exit.

-q, --query <query>
    General query

-m, --max <max_results>
    Maximum number of results

  dissemin

Look for documents on dissem.in

Examples of its usage are

papis explore dissemin -q 'Albert einstein' pick cmd 'firefox {doc[url]}'

papis explore dissemin [OPTIONS]

Options

-h, --help
    Show this message and exit.

-q, --query <query>

  export

Export retrieved documents into various formats for later use

Examples of its usage are

papis explore crossref -m 200 -a 'Schrodinger' export --yaml lib.yaml

papis explore export [OPTIONS]

Options

-h, --help
    Show this message and exit.

-f, --format <fmt>
    Format for the document

Options
    bibtex | json | yaml

-o, --out <out>
    Outfile to write information to

  isbn

Look for documents using isbnlib

Examples of its usage are

papis explore isbn -q 'Albert einstein' pick cmd 'firefox {doc[url]}'

papis explore isbn [OPTIONS]

Options

-h, --help
    Show this message and exit.

-q, --query <query>

-s, --service <service>

Options
    default | goob | openl | wiki

  json

Import documents from a json file

Examples of its usage are

papis explore json lib.json pick

papis explore json [OPTIONS] JSONFILE

Options

-h, --help
    Show this message and exit.

    Arguments

JSONFILE
    Required argument

  lib

Query for documents in your library

Examples of its usage are papis lib -l books einstein pick

papis explore lib [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--doc-folder <doc_folder>
    Document folder on which to apply action

-l, --library <library>
    Papis library to look

    Arguments

QUERY
    Optional argument

  pick

Pick a document from the retrieved documents

Examples of its usage are

papis explore bibtex lib.bib pick

papis explore pick [OPTIONS]

Options

-h, --help
    Show this message and exit.

-n, --number <number>
    Pick automatically the n-th document

  yaml

Import documents from a YAML file.

Examples of its usage are:

papis explore yaml lib.yaml pick

papis explore yaml [OPTIONS] YAMLFILE

Options

-h, --help
    Show this message and exit.

    Arguments

YAMLFILE
    Required argument

  Export

The export command is useful to work with other programs such as bibtex.

Some examples of its usage are:

•   Export one of the documents matching the author with einstein to bibtex:

    papis export --format bibtex 'author : einstein'

    or export all of them

    papis export --format bibtex --all 'author : einstein'

•   Export all documents to bibtex and save them into a lib.bib file

    papis export --all --format bibtex --out lib.bib

•   Export a folder of one of the documents matching the word krebs into a folder named, interesting-document

      papis export --folder --out interesting-document krebs
    this will create the folder ``interesting-document`` containing the
    ``info.yaml`` file, the linked documents and a ``bibtex`` file for
    sharing with other people.

    NOTE: Every document exported also comes with the key _papis_local_folder associated that points to the full local folder path where the document is stored in the file system. This is done for the convenience of third party apps.

  Command-line Interface

Export a document from a given library

papis export [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--doc-folder <doc_folder>
    Document folder on which to apply action

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

-a, --all
    Apply action to all matching documents

--folder
    Export document folder to share

-o, --out <out>
    Outfile or outdir

-f, --format <fmt>
    Format for the document

Options
    bibtex | json | yaml

    Arguments

QUERY
    Optional argument

  Git

This command is useful if your library is itself a git repository. You can use this command to issue git commands in your library repository without having to change your current directory.

  CLI Examples

•   Check the status of the library repository:

    papis git status

•   Commit all changes:

    papis git commit -a

  List

This command is to list contents of a library.

  Examples

•   List all document files associated will all entries:

    papis list --all --file

•   List all document year and title with custom formatting:

    papis list --all --format '{doc[year]} {doc[title]}'

•   List all documents according to the bibitem formatting (stored in a template file bibitem.template):

    papis list --all --template bibitem.template

•   For scripting, printing the id of a series of documents is valuable in order to further use the id in other scripts.

    papis_id=$(papis list --id)
    papis open papis_id:${papis_id}
    papis edit papis_id:${papis_id}
    # etc.

  Command-line Interface

List documents' properties

papis list [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

-i, --info
    Show the info file name associated with the document

--id
    Show the papis_id

-f, --file
    Show the file name associated with the document

-d, --dir
    Show the folder name associated with the document

-n, --notes
    List notes files, if any

--format <_format>
    List entries using a custom papis format, e.g. '{doc[year] {doc[title]}

--template <template>
    Template file containing a papis format to list entries

--downloaders
    List available downloaders

--libraries
    List defined libraries

-a, --all
    Apply action to all matching documents

--doc-folder <doc_folder>
    Document folder on which to apply action

    Arguments

QUERY
    Optional argument

  Mv

  Command-line Interface

Move a document into some other path

papis mv [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--git, --no-git
    Commit changes to git

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

--doc-folder <doc_folder>
    Document folder on which to apply action

    Arguments

QUERY
    Optional argument

  Open

The open command is a very important command in the papis workflow. With it you can open documents, folders or marks.

  Marks

One of special things about this command is the possibility of creating marks for documents. As you would imagine, it is in general difficult to create marks for any kind of data. For instance, if our library consists of pdf files and epub files for instance, we would like to define bookmarks in order to go back to them at some later point.

How you define marks can be customized through the marks configuration settings here. The default way of doing it is just by defining a marks list in a document. Let us look at a concrete example:

author: Isaiah Shavitt, Rodney J. Bartlett
edition: '1'
files: [book.pdf]
isbn: 052181832X,9780521818322
marks:
- {name: Intermediates definition, value: 344}
- {name: EOM equations, value: 455}
publisher: Cambridge University Press
ref: book:293288
series: Cambridge Molecular Science
title: 'Many-Body Methods in Chemistry and Physics'
type: book
year: '2009'

This book has defined two marks. Each mark has a name and a value. If you tell the open command to open marks, then it will look for the marks and open the value (page number). This is the default behaviour, however if you go to the configuration you'll see that you can change the convention to what it suits you.

  Examples

•   Open a pdf file linked to a document matching the string bohm

    papis open bohm

•   Open the folder where this last document is stored

    papis open -d bohm

    Please notice that the file browser used will be also related to the file-browser setting.

•   Open a mark defined in the info file

    papis open --mark bohm

  Command-line Interface

Open document from a given library

papis open [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

--doc-folder <doc_folder>
    Document folder on which to apply action

-a, --all
    Apply action to all matching documents

--tool <tool>
    Tool for opening the file (opentool)

-d, --dir
    Open directory

-m, --mark, --no-mark
    Open mark

    Arguments

QUERY
    Optional argument

  Rename

  Command-line Interface

Rename entry

papis rename [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--git, --no-git
    Commit changes to git

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

--doc-folder <doc_folder>
    Document folder on which to apply action

    Arguments

QUERY
    Optional argument

  Rm

  Command-line Interface

Delete a document, a file, or a notes-file

papis rm [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--git, --no-git
    Remove in git

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

--doc-folder <doc_folder>
    Document folder on which to apply action

--file
    Remove files from a document instead of the whole folder

-n, --notes
    Remove the notes file from a document instead of the whole folder

-f, --force
    Do not confirm removal

-a, --all
    Apply action to all matching documents

    Arguments

QUERY
    Optional argument

  Run

This command is useful to issue commands in the directory of your library.

  Examples

•   List files in your directory

    papis run ls

•   Find a file in your directory using the find command

    papis run find -name 'document.pdf'

•   Find all pdfs in the document folders matching einstein

      papis run -p einstein --all -- find . -name '*.pdf'
    notice that in general, the symbol ``--`` is advisable
    so that the arguments after it are considered as positional arguments
    for the shell commands.
    In this example you could also use pipes, for instance to print the
    absolute path to the files, in linux you can use the command
    ``readlink -f`` and a pipe ``|`` to do this, i.e.:

    papis run -p einstein \
            --all -- "find . -name '*.pdf' | xargs readlink -f"

•   Replace some text in all info.yaml files by something. For instance imagine you want to replace all note field names in the info.yaml files by _note so that the note field does not get exported to bibtex. You can do

    papis run -a -- sed -i "s/^note:/_note:/" info.yaml

  Command-line Interface

Run an arbitrary shell command in the library or command folder

papis run [OPTIONS] <COMMANDS>

Options

-h, --help
    Show this message and exit.

-p, --pick <QUERY>
    Give a query to pick a document to run the command in its folder

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

--doc-folder <doc_folder>
    Document folder on which to apply action

-a, --all
    Apply action to all matching documents

--prefix <PREFIX>
    Prefix shell commands by a prefix command

    Arguments

<COMMANDS>
    Optional argument(s)

  Update

This command is to update the information of the documents.

Some examples of the usage are given below

  Examples

•   Update a document automatically and interactively (searching by doi number in crossref, or in other sources...)

    papis update --auto -i "author : dyson"

•   Update your library from a bib(la)tex file where many entries are listed. papis will try to look for documents in your library that match these entries and will ask you entry per entry to update it (of course this is done if you use the -i flag for interactively doing it). In the example libraryfile.bib is a file containing many entries.

    papis update --from bibtex libraryfile.bib -i

•   Tag all einstein papers with the tag classics

    papis update --all --set tags classics einstein

    and add the tag of physics to all papers tagged as classics

    papis update --all --set tags '{doc[tags]} physics' einstein

  Command-line Interface

Update a document from a given library.

papis update [OPTIONS] [QUERY]

Options

-h, --help
    Show this message and exit.

--git, --no-git
    Commit changes to git

--doc-folder <doc_folder>
    Document folder on which to apply action

-a, --all
    Apply action to all matching documents

--sort <FIELD>
    Sort documents with respect to the FIELD

--reverse
    Reverse sort order

--auto
    Try to parse information from different sources

--from <from_importer>
    Add document from a specific importer (arxiv, bibtex, crossref, dblp, doi, folder, isbn, lib, pdf2arxivid, pdf2doi, pmid, yaml)

-s, --set <set_tuples>
    Update document's information with key value. The value can be a papis format.

    Arguments

QUERY
    Optional argument

  Citations

The citations command updates and creates the citations.yaml and cited.yaml files for every document.

  Examples

•   Create the citations.yaml file for a document that you pick

    papis citations -c

    or longer

    papis citations --fetch-citations

•   Create the citations.yaml file for all documents matching an author

    papis citations --all -c author:einstein

•   Overwrite the citations.yaml file with the --force flag for all Einstein papers

    papis citations --force -c author:einstein
    # or
    papis citations -fc author:einstein

•   Update the citations.yaml file with citations of documents existing in your library

    papis citations --all --update-from-database author:einstein

•   Create the cited-by.yaml for all documents in your library (this might take a while)

    papis citations --fetch-cited-by --all

WEB APPLICATION

Since v0.12, papis ships with an experimental, simple yet handy web application.

You can start the web application by issuing the following command

papis serve --port 8181 --host localhost

Then input the following url in your browser

http://localhost:8181

This web application can be useful if you intend to read access to your documents from a tablet or from another computer.

Further documentation will be available soon, but bear in mind that this web application is experimental, bug reports and suggestions are highly appreciated.

GUI

Papis is a program mainly intended for the command line, however, some experimental frontends have been conceived for it and are already downloadable from pip.

See for instance

•   papis-rofi

•   papis-dmenu

    And more to come!

EDITOR SUPPORT

There are some plugins for the major open source editors vim and emacs.

  1 VIM

There is a new well-maintained and high-quality package for neovim papis.nvim.

If neovim is a deal breaker for you, you can take a look at the original vim plugin papis-vim.

  2 Emacs

papis.el is a compatibility layer for papis that tries to use as much of the existing packages as possible. In particular it tries to make use also of org-ref to try not to reinvent the wheel. If you are used to org-ref or org-cite it strives to work seamless with them and still have your papis library intact.

HOOKS

From version 0.12 papis has a minimal hook infrastructure. Some parts of papis define and run hooks so that users and plugin writers can also tap into this functionality.

A hook is declared in the same way as a plugin, in fact they are implemented in the same way within the stevedore plugin.

  1 Writing hooks as a user

Right now the only way to add a hook as a user is using your config.py configuration file, which gets loaded when your papis configuration gets loaded.

As an example you can add a function to the on_edit_done hook like

import papis.hooks
papis.hooks.add("on_edit_done", lambda: print(42))

  2 Writing hooks as a developer

To add a hook as a plugin writer or a developer you can just add the entry_point to the setup.py file, for instance for the on_edit_done hook you would write

"papis.hook.on_edit_done" : [
    "my_hook_name=path.to.my:function,
]

CUSTOM SCRIPTS

As in git, you can write custom scripts to include them in the command spectrum of papis.

  Example: Mail script

Imagine you want to write a script to send papers to someone via the email client mutt (you can try to do it with another mail client), you could write the following script called papis-mail:

#! /usr/bin/env bash
# papis-short-help: Email a paper to my friend
folder_name=$1
zip_name="${folder_name}.zip"
papis -l ${PAPIS_LIB} export --folder --out ${folder_name}
zip -r ${zip_name} ${folder_name}
mutt -a ${zip_name}

Papis defines environment variables such as PAPIS_LIB so that external scripts can make use of the user input.

To use the script you can put it somewhere in your PATH or alternatively inside the ~/.papis/scripts folder. If this is the case then you can run

papis -h

and you will see that there is another command besides the default called mail. In fact, you will see

positional arguments:
  command               For further information for every command, type in 'papis <command> -h'
    add                 Add a document into a given library
    .............       ..........................
    mail                Email a paper to my friend
optional arguments:
  -h, --help            show this help message and exit
  ... .........         .... ... ...... ....... ........... .. ..... ......

where the description Email a paper to my friend is there because we have defined the comment # papis-short-help: Email a paper to my friend in the header of the script.

Then, if you type

papis -l mylib mail this_paper

this will create a folder called this_paper with a selection of a document, zip it and send it to whoever you choose to.

  Example: Accessing papis from within mutt

You may want to pick documents to attach to your email in mutt from the papis interface.

Add this code to your muttrc

# # macro to attach paper from papis
macro attach,compose \cp \
"\
<enter-command>unset wait_key<enter>\                                 # Don't require 'press any key'
<shell-escape>rm -rf /tmp/paper /tmp/paper.zip<enter>\                # remove the folder /tmp/paper if it already exists
<shell-escape>papis export --folder -o /tmp/paper<enter>\             # start papis with the --folder flag
<shell-escape>zip -r /tmp/paper.zip /tmp/paper<enter>\                # zip the directory
<attach-file>/tmp/paper.zip<enter>\                                   # attach zip file to the email
"

Try it out with Ctrl-p on your Compose screen. This makes use of the papis export --folder flag that moves the paper folder you choose to a temporary location (/tmp/paper). Mutt will then attach the paper to the email, which you can rename to be more descriptive with R.

  Example: Define papis mode in i3wm

This is an example of using papis with the window manager i3.

# Enter papis mode
bindsym $mod+Ctrl+p mode "papis"
# Define papis mode
mode "papis" {
  # open documents
  bindsym $mod+o exec python3 -m papis.main \
    --pick-lib --set picktool dmenu open
  # edit documents
  bindsym $mod+e exec python3 -m papis.main \
    --pick-lib --set picktool dmenu --set editor gvim edit
  # open document's url
  bindsym $mod+b exec python3 -m papis.main \
    --pick-lib --set picktool dmenu browse
  # return to default mode
  bindsym Ctrl+c mode "default"
  bindsym Return mode "default"
  bindsym Escape mode "default"
}

  Useful links

•   Get paper references with papis

    citget() {
        query=$1
        shift
        papis explore               \\
            citations -s "$query" \\
            pick                  \\
            cmd "papis add --from doi {doc[doi]} $@"
    }

API

This module describes which functions are intended to be used by users to create papis scripts.

class papis.api.T

papis.api.get_lib_name() -> str
    Get current library.

    It either retrieves the library from the environment PAPIS_LIB variable, the command-line arguments passed in by the user or the configuration files.

Returns
    the name of the library.

    >>> get_lib_name() is not None
    True

papis.api.set_lib_from_name(library: str) -> None
    Set current library.

    It either sets the library from the environment PAPIS_LIB variable, the command-line args passed by the user or the configuration files.

Parameters
    library -- name of a library (as defined in the configuration files) or a path to an existing library.

papis.api.get_libraries() -> List[str]
    Get all the libraries declared in the configuration files.

    A library in the configuration files is a section that has the dir or dirs keys defined.

Returns
    a list of library names.

    >>> len(get_libraries()) >= 1
    True

papis.api.pick_doc(documents: Sequence[Document]) -> Sequence[Document]
    Pick a subset of documents from the given documents.

Parameters
    documents -- a sequence of documents.

Returns
    a subset of documents corresponding to the user selected ones.

papis.api.pick(items: Sequence[T], default_index: int = 0, header_filter: Callable[[T], str] | None = None, match_filter: Callable[[T], str] | None = None) -> Sequence[T]
    Pick a subset of items from the given items.

Parameters

•   items -- a sequence of items.

•   default_index -- index used when no explicit item is picked.

•   header_filter -- a callable to stringify the given item for display.

•   match_filter -- a callable to stringify the given item for display.

papis.api.open_file(file_path: str, wait: bool = True) -> None
    Open the given file using the configured opentool.

Parameters

•   file_path -- a path to a file.

•   wait -- if True, wait for the completion of the opener program before continuing execution (blocking behavior).

papis.api.open_dir(dir_path: str, wait: bool = True) -> None
    Open the given directory using the configured file-browser.

Parameters

•   dir_path -- a path to a folder.

•   wait -- if True, wait for the completion of the opener program before continuing execution (blocking behavior).

papis.api.edit_file(file_path: str, wait: bool = True) -> None
    Edit the given file using the configured editor.

Parameters

•   file_path -- a path to a file.

•   wait -- if True, wait for the completion of the editor before continuing execution (blocking behavior).

papis.api.get_all_documents_in_lib(library: str | None = None) -> List[Document]
    Get all documents in the given library.

Parameters
    library -- a library name.

Returns
    a list of all known documents in the library.

    >>> import tempfile
    >>> folder = tempfile.mkdtemp()
    >>> set_lib_from_name(folder)
    >>> docs = get_all_documents_in_lib(folder)
    >>> len(docs)
    0

papis.api.get_documents_in_dir(directory: str, search: str = '') -> List[Document]
    Get documents contained in the given folder.

Parameters

•   directory -- a path to a folder containing documents.

•   search -- a search string used to filter the documents.

Returns
    a list of filtered documents from directory.

    >>> import tempfile
    >>> docs = get_documents_in_dir(tempfile.mkdtemp())
    >>> len(docs)
    0

papis.api.get_documents_in_lib(library: str | None = None, search: Dict[str, Any] | str = '') -> List[Document]
    Get documents contained in the given library.

Parameters

•   library -- a library name.

•   search -- a search parameter used to filter the documents.

Returns
    a list of filtered documents from library.

papis.api.clear_lib_cache(lib: str | None = None) -> None
    Clear the cache associated with a library.

    If no library is given, then the current library is used.

Parameters
    lib -- a library name.

    >>> clear_lib_cache()

papis.api.doi_to_data(doi: str) -> Dict[str, Any]
    Get metadata for the given doi by querying Crossref.

Parameters
    doi -- a valid DOI (Document Object Identifier).

Returns
    metadata for the given identifier.

papis.api.save_doc(doc: Document) -> None
    Save the document to disk.

    This commits the new document to the database and saves it to disk by updating its info.yaml file.

Parameters
    doc -- an existing document.

PLUGIN ARCHITECTURE

  General architecture

Papis uses the package stevedore for general plugin management.

The only papis module invoking stevedore should be papis/plugin.py.

The different plugins in papis like papis.command, papis.exporter etc. define a so-called ExtensionManager which loads various objects that have been declared in a python package somewhere.

For example, the yaml exporter is defined as

def exporter(documents: List[papis.document.Document]) -> str:
    string = yaml.dump_all(
        [papis.document.to_dict(document) for document in documents],
        allow_unicode=True)
    return str(string)

and declared in setup.py as

setup(
    ...
    entry_points={
        'papis.exporter': [
            'bibtex=papis.bibtex:exporter',
            'json=papis.json:exporter',
            'yaml=papis.yaml:exporter',
        ],
    ...
)

and the exporter can be used as in the code below

import papis.plugin
extension_manager = papis.plugin.get_extension_manager("papis.exporter")
# yaml_exporter is the function defined above
yaml_exporter = extension_manager["yaml"].plugin
yaml_string = yaml_exporter(mydocs)

Now a developer is able to write another exporter in some package and install the package in the system. The extension_manager will be able to access the provided functions in the package if they have been declared in the entry points of the setup.py script of the named package.

  Exporter

TO DOCUMENT

  Command

TO DOCUMENT

  Importer

TO DOCUMENT

  Explore

TO DOCUMENT

GIT

Papis is conceived to work well with the tool git, this would also work with mercurial or subversion.

Here you will find a description of a possible workflow using git with papis. This is not the only workflow, but it is the most obvious.

Let's say you have a library named books in the directory ~/Documents/MyNiceBooks. You could turn the books library into a git repository, just doing for example

papis -l books run git init

or just going to the library directory and running the command there:

cd ~/Documents/MyNiceBooks
git init

Now you can add everything you have in the library with git add . if you are in the library's directory or

papis -l books git add .

if you want to do it using the papis' git command.

  Interplay with other commands

Some papis commands give you the opportunity of using git to manage changes. For instance, if you are adding a new document, you could use the --commit flag to also add a commit into your library, so if you do

papis add --set author "Pedrito" --set title "Super book" book.pdf --commit

then also papis will do an automatic commit for the book, so that you can push your library afterwards to a remote repository.

You can imagine that papis commands like rename and mv should also offer such functionality, and they indeed do through the --git flag. Go to their documentation for more information.

  Updating the library

You can use papis' simple git wrapper,

papis git pull

  Usual workflow

Usually the workflow is like this:

When adding a document that you know for sure you want in your library:

•   Add the document and commit it, either by git add --commit or committing the document after adding it to the library.

•   Pull changes from the remote library, maybe you pushed something at work (reference changes etc..) and you do not have it yet there, you would do something like

    papis git pull

•   Push what you just added

    papis git push

•   Review the status of the library

    papis git status

    When editing a document's info file:

•   Edit the file and then take a look at the diff

    papis git diff

•   Add the changes

    papis git add --all

•   Commit

    papis git commit

•   Pull/push:

    papis git pull
    papis git push

    Of course these workflows are just very basic examples. Your optimal workflow could look completely different.

SCIHUB SUPPORT

Papis has a script that uses the scihub platform to download scientific papers. Due to legal caution the script is not included directly as a papis command, and it has its own PyPi repository.

To install it, just type

pip3 install papis-scihub

Now you can type

papis scihub -h

and see the help message of the script.

Some usage examples are:

•   Download via the doi number:

    papis scihub 10.1002/andp.19053220607 \\
      add -d einstein_papers --folder-name photon_definition

•   Download via a url that contains the doi number in the format .*/doi/<doinumber>

    papis scihub https://physicstoday.scitation.org/doi/10.1063/1.881498 \\
      add --folder-name important_paper

•   Download via the doi.org url:

    papis scihub https://doi.org/10.1016/j.physrep.2016.12.002 add

IMPORTING FROM BIBTEX OR ZOTERO

Many users will want to import a library from a bibtex file that another program such as zotero, mendeley or jabref will have exported. These programs usually have a field called FILE or file that points to a path where the document file can be found.

To import such a library you can use a provided script originally intended for zotero which is, however, general enough to work for other programs too.

To install this script you can install the project papis-zotero and follow the instructions therein to import a bibliography file into papis.

You can also take a look at the zotero2papis project in order to see a different workflow and different implementation of a converter script. This blogpost explains the workflow using papis with zotero and SyncThing.

SHELL COMPLETION

Papis has shell completion for bash, fish and zsh through the click framework that comes with it when installed through pip. To control the directory in which the completions gets installed use the environment variables

PAPIS_<SHELL>_COMPLETION_DIR=my/custom/directory

where <SHELL> is the uppercase name of the shell (e.g. BASH) and the paths are considered subdirectories of the chosen prefix. The default paths for each shell are given below.

•   bash: the completion script is installed in $PREFIX/share/bash-completion/completions and works directly with the bash-completion package. It can also be sourced manually using (or added to your .bashrc)

    source $PREFIX/share/bash-completion/completions/papis.bash

•   fish: the completion script is installed in $PREFIX/share/fish/vendor_completions.d, which should be sourced automatically (see the fish docs for more details). It can also be sourced manually using (or added to your config.fish)

    source $PREFIX/share/fish/vendor_completions.d/papis.fish

•   zsh: the completion script is installed in $PREFIX/share/zsh/site-functions, which is sourced automatically starting with version 5.0.7 (see the zsh docs for more details). It can also be sourced manually using (or added to your .zshrc)

    source $PREFIX/share/zsh/site-functions/_papis

    Alternatively, the completion can be generated on-the-fly by running (see more in the click docs)

    eval "$(_PAPIS_COMPLETE=<shell>_source papis)"

    where <shell> is one of the shells supported by click. Note that older versions of click used source_<shell> instead for the values of _PAPIS_COMPLETE.

DEVELOPER API REFERENCE

WARNING: The APIs documented here are not stable and may change from one version to another. This is meant to be used by developers, both of papis itself and any external plugins.

  papis.cli

papis.cli.query_argument(**attrs: Any) -> Callable[[...], Any]
    Adds a query argument as a click decorator.

papis.cli.query_option(**attrs: Any) -> Callable[[...], Any]
    Adds a -q, --query option as a click decorator.

papis.cli.sort_option(**attrs: Any) -> Callable[[...], Any]
    Adds a --sort and a --reverse option as a click decorator.

papis.cli.doc_folder_option(**attrs: Any) -> Callable[[...], Any]
    Adds a --doc-folder argument as a click decorator.

papis.cli.all_option(**attrs: Any) -> Callable[[...], Any]
    Adds a --all option as a click decorator.

papis.cli.git_option(**attrs: Any) -> Callable[[...], Any]
    Adds a --git option as a click decorator.

papis.cli.handle_doc_folder_or_query(query: str, doc_folder: str | None) -> List[Document]
    Query database for documents.

    This handles the query_option() and doc_folder_option() command-line arguments. If a doc_folder is given, then the document at that location is loaded, otherwise the database is queried using query.

Parameters

•   query -- a database query string.

•   doc_folder -- existing document folder (see papis.document.from_folder()).

papis.cli.handle_doc_folder_query_sort(query: str, doc_folder: str | None, sort_field: str | None, sort_reverse: bool) -> List[Document]
    Query database for documents.

    Similar to handle_doc_folder_or_query(), but also handles the sort_option() arguments. It sorts the resulting documents according to sort_field and reverse_field.

Parameters

•   sort_field -- field by which to sort the resulting documents (see papis.document.sort()).

•   sort_reverse -- if True, the fields are sorted in reverse order.

papis.cli.handle_doc_folder_query_all_sort(query: str, doc_folder: str, sort_field: str | None, sort_reverse: bool, _all: bool) -> List[Document]
    Query database for documents.

    Similar to handle_doc_folder_query_sort(), but also handles the all_option() argument.

Parameters
    _all -- if False, the user is prompted to pick a subset of documents (see papis.api.pick_doc()).

papis.cli.bypass(group: Group, command: Command, command_name: str) -> Callable[[...], Any]
    Overwrite existing papis commands.

    This function is specially important for developing scripts in papis.

    For example, consider augmenting the add command, as seen when using papis add. In this case, we may want to add some additional options or behaviour before calling papis.commands.add, but would like to avoid writing it from scratch. This function can then be used as follows to allow this

    import click
    import papis.cli
    import papis.commands.add
    @click.group()
    def main():
        """Your main app"""
        pass
    @papis.cli.bypass(main, papis.commands.add.cli, "add")
    def add(**kwargs):
        # do some logic here...
        # and call the original add command line function by
        papis.commands.add.cli.bypassed(**kwargs)

  papis.document

Module defining the main document type.

class papis.document.KeyConversion(_typename, _fields=None, /, **kwargs)
    A dict that contains a key and an action. The key contains the name of a key in another dictionary and the action contains a callable that can pre-processes the value.

papis.document.EmptyKeyConversion = {'action': None, 'key': None}
    A default KeyConversion.

class papis.document.KeyConversionPair(foreign_key, list)

foreign_key: str
    A string denoting the foreign key (in the input data).

list: List[KeyConversion]
    A list of KeyConversion mappings used to rename and post-process the foreign_key and its value.

papis.document.keyconversion_to_data(conversions: Sequence[KeyConversionPair], data: Dict[str, Any], keep_unknown_keys: bool = False) -> Dict[str, Any]
    Function to convert between dictionaries.

    This can be used to define a fixed set of translation rules between, e.g., JSON data obtained from a website API and standard papis key names and formatting. The implementation is completely generic.

    For example, we have the simple dictionary

    data = {"id": "10.1103/physrevb.89.140501"}

    which contains the DOI of a document with the wrong key. We can then write the following rules

    conversions = [
        KeyConversionPair("id", [
            {"key": "doi", "action": None},
            {"key": "url": "action": lambda x: "https://doi.org/{}".format(x)}
        ])
    ]
    new_data = keyconversion_to_data(conversions, data)

    to rename the "id" key to the standard "doi" key used by papis and a URL. Any number of such rules can be written, depending on the complexity of the incoming data. Note that any errors raised on the application of the action will be silently ignored and the corresponding key will be skipped.

Parameters

•   conversions -- a sequence of KeyConversionPairs used to convert the data.

•   data -- a dict to be convert according to conversions.

•   keep_unknown_keys -- if True unknown keys from data are kept in the resulting dictionary. Otherwise, only keys from conversions are present.

Returns
    a new dict containing the entries from data converted according to conversions.

papis.document.author_list_to_author(data: Dict[str, Any]) -> str
    Convert a list of authors into a single author string.

    This uses the multiple-authors-separator and the multiple-authors-format settings to construct the concatenated authors.

Parameters
    data -- a dict that contains an "author_list" key to be converted into a single author string.

    >>> author1 = {"given": "Some", "family": "Author"}
    >>> author2 = {"given": "Other", "family": "Author"}
    >>> author_list_to_author({"author_list": [author1, author2]})
    'Author, Some and Author, Other'

papis.document.split_authors_name(authors: List[str], separator: str = 'and') -> List[Dict[str, Any]]
    Convert list of authors to a fixed format.

    This uses bibtexparser.customization.splitname() to correctly split and determine the first and last names of an author in the list. Note that this is just a heuristic and can give incorrect results for certain author names.

Parameters

•   authors -- a list of author names, where each entry can consists of multiple authors separated by separator.

•   separator -- a separator for entries in authors that contain multiple authors.

class papis.document.DocHtmlEscaped(doc: Document)
    Small helper class to escape HTML elements in a document.

    >>> DocHtmlEscaped(from_data({"title": '> >< int & "" "'}))['title']
    '&gt; &gt;&lt; int &amp; &quot;&quot; &quot;'

class papis.document.Document(folder: str | None = None, data: Dict[str, Any] | None = None)
    An abstract document in a papis library.

    This class inherits from a standard dict and implements some additional functionality.

html_escape
    A DocHtmlEscaped instance that can be used to escape keys in the document for use in HTML documents.

has(key: str) -> bool
    Check if key is in the document.

set_folder(folder: str) -> None
    Set the document's main folder.

    This also updates the location of the info file and other attributes. Note, however, that it will not load any data from the given folder even if it contains another info file (see from_folder() for this functionality).

Parameters
    folder -- an absolute path to a new main folder for the document.

get_main_folder() -> str | None

Returns
    the root path in the filesystem where the document is stored, if any.

get_main_folder_name() -> str | None

Returns
    the folder name of the document, i.e. the basename of the path returned by get_main_folder().

get_info_file() -> str

Returns
    path to the info file, which can also be an empty string if no such file has been created.

get_files() -> List[str]
    Get the files linked to the document.

    The files in a document are stored relative to its main folder. If no main folder is set on the document (see set_folder()), then this function will not return any files. To retrieve the relative file paths only, access doc["files"] directly.

Returns
    a list of absolute file paths in the document's main folder, if any.

save() -> None
    Saves the current document fields into the info file.

load() -> None
    Load information from the info file.

papis.document.from_data(data: Dict[str, Any]) -> Document
    Construct a Document from a dictionary.

Parameters
    data -- a dictionary to be made into a new document.

papis.document.from_folder(folder_path: str) -> Document
    Construct a Document from a folder.

Parameters
    folder_path -- absolute path to a valid papis folder.

papis.document.to_json(document: Document) -> str
    Export the document to JSON.

Returns
    a JSON string corresponding to all the entries in the document.

papis.document.to_dict(document: Document) -> Dict[str, Any]
    Convert a document back into a standard dict.

Returns
    a dict corresponding to all the entries in the document.

papis.document.dump(document: Document) -> str
    Dump the document into a formatted string.

    The format of the string is not fixed and is meant to be used to display the document entries in a consistent way across papis.

Returns
    a string containing all the entries in the document.

    >>> doc = from_data({'title': 'Hello World'})
    >>> dump(doc)
    'title:     Hello World'

papis.document.delete(document: Document) -> None
    Delete a document from the filesystem.

    This function delete the main folder of the document (recursively), but it does not delete the in-memory version of the document.

papis.document.describe(document: Document | Dict[str, Any]) -> str

Returns
    a string description of the current document using document-description-format.

papis.document.move(document: Document, path: str) -> None
    Move the document to a new main folder at path.

    This supposes that the document exists in the location document.get_main_folder() and will change the folder in the input document as a result.

Parameters
    path -- absolute path where the document should be moved to. This path is expected to not exist yet and will be created by this function.

    >>> doc = from_data({'title': 'Hello World'})
    >>> doc.set_folder('path/to/folder')
    >>> import tempfile; newfolder = tempfile.mkdtemp()
    >>> move(doc, newfolder)
    Traceback (most recent call last):
    ...
    FileExistsError: There is already...

papis.document.sort(docs: Sequence[Document], key: str, reverse: bool = False) -> List[Document]
    Sort a list of documents by the given key.

    The sort is performed on the key with a priority given to the type of the value. If the key does not exist in the document, this is given the lowest priority and left at the end of the list.

Parameters

•   docs -- a sequence of documents.

•   key -- a key in the documents by which to sort.

•   reverse -- if True, the sorting is done in reverse order (descending instead of ascending).

Returns
    a list of documents sorted by key.

papis.document.new(folder_path: str, data: Dict[str, Any], files: Sequence[str] | None = None) -> Document
    Creates a complete document with data and existing files.

    The document is saved to the filesystem at folder_path and all the given files are copied over to the main folder.

Parameters

•   folder_path -- a main folder for the document.

•   data -- a dict with key and values to be used as metadata in the document.

•   files -- a sequence of files to add to the document.

Raises
    FileExistsError -- if folder_path already exists.

  papis.downloaders

class papis.downloaders.Importer(uri: str = '')
    Importer that tries to get data and files from implemented downloaders.

    This importer simply calls get_info_from_url() on the given URI.

classmethod match(uri: str) -> Importer | None
    Check if the importer can process the given URI.

    For example, an importer that supports links from the arXiv can check that the given URI matches using:

    re.match(r".*arxiv.org.*", uri)

    This can then be used to instantiate and return a corresponding Importer object.

Parameters
    uri -- An URI where the document information should be retrieved from.

Returns
    An importer instance if the match to the URI is successful or None otherwise.

fetch() -> None
    Fetch metadata and files for the given uri.

    This method calls Importer.fetch_data() and Importer.fetch_files() to get all the information available for the document. It is recommended to implement the two methods separately, if possible, for maximum flexibility.

    The imported data is stored in ctx and it is not queried again on subsequent calls to this function.

fetch_data() -> None
    Fetch metadata from the given uri.

    The imported metadata is stored in ctx.

fetch_files() -> None
    Fetch files from the given uri.

    The imported files are stored in ctx.

class papis.downloaders.Downloader(uri: str = '', name: str = '', ctx: Context | None = None, expected_document_extension: Sequence[str] | str | None = None, cookies: Dict[str, str] | None = None, priority: int = 1)
    A base class for downloader instances implementing common functionality.

    In general, downloaders are expected to implement a subset of the methods below, depending on the generality. A simple downloader could only implement get_bibtex_url() and get_document_url().

expected_document_extension
    A single extension or a list of extensions supported by the downloader. The extensions do not contain the leading dot, e.g. ["pdf", "djvu"].

priority
    A priority given to the downloader. This is used when trying to automatically determine a prefered downloader for a given URL.

session
    A requests.Session that is used for all the requests made by the downloader.

classmethod match(url: str) -> Downloader | None
    Check if the downloader can process the given URL.

    For example, an importer that supports links from the arXiv can check that the given URL matches using:

    re.match(r".*arxiv.org.*", uri)

    This can then be used to instantiate and return a corresponding Downloader object.

Parameters
    url -- An URL where the document information should be retrieved from.

Returns
    A downloader instance if the match to the URL is successful or None otherwise.

fetch() -> None
    Fetch metadata and files for the given uri.

    This method calls Downloader.fetch_data() and Downloader.fetch_files() to get all the information available for the document. It is recommended to implement the two methods separately, if possible, for maximum flexibility.

    The imported data is stored in ctx and it is not queried again on subsequent calls to this function.

fetch_data() -> None
    Fetch metadata for the given URL.

    The imported metadata is stored in ctx. To fetch the metadata, the following steps are followed

•   Call get_data() to import any scraped metadata.

•   Call get_bibtex_data() to import any metadata from BibTeX files available remotely.

    Note that previous steps overwrite any information, i.e. the BibTeX data will take priority.

fetch_files() -> None
    Fetch files from the given uri.

    The imported files are stored in ctx. The file is downloaded with download_document() and stored as a temporary file.

get_bibtex_url() -> str | None

Returns
    an URL to a valid BibTeX file that can be used to extract metadata about the document.

get_bibtex_data() -> str | None
    Get BibTeX data available at get_bibtex_url(), if any.

Returns
    a string containing the BibTeX data, which can be parsed.

download_bibtex() -> None
    Download and store that BibTeX data from get_bibtex_url().

    Use get_bibtex_data() to access the metadata from the BibTeX URL.

get_data() -> Dict[str, Any]
    Retrieve general metadata from the given URL.

    This function is meant to be as general as possible and should not contain data imported from BibTeX (use get_bibtex_data() instead). For example, this can be used for web scrapping or calling other website APIs to gather metadata about the document.

get_doi() -> str | None

Returns
    a DOI for the document, if any.

get_document_url() -> str | None

Returns
    a URL to a file that should be downloaded.

get_document_data() -> bytes | None
    Get data for the downloaded file that is given by get_document_url().

Returns
    the bytes (stored in memory) for the downloaded file.

download_document() -> None
    Download and store the file that is given by get_document_url().

    Use get_document_data() to access the file binary contents.

check_document_format() -> bool
    Check if the document downloaded by download_document() has a file type supported by the downloader.

    If the downloader has no prefered type, then all files are accepted.

Returns
    True if the document has a supported file type and False otherwise.

papis.downloaders.get_available_downloaders() -> List[Type[Downloader]]
    Get all declared downloader classes.

papis.downloaders.get_matching_downloaders(url: str) -> List[Downloader]
    Get downloaders matching the given url.

Parameters
    url -- a URL to match.

Returns
    a list of downloaders (sorted by priority).

papis.downloaders.get_downloader_by_name(name: str) -> Type[Downloader]
    Get a specific downloader by its name.

Parameters
    name -- the name of the downloader. Note that this is the name of the entry point used to define the downloader. In general, this should be the same as its name, but this is not enforced.

Returns
    a downloader class.

papis.downloaders.get_info_from_url(url: str, expected_doc_format: str | None = None) -> Context
    Get information directly from the given url.

Parameters

•   url -- the URL of a resource.

•   expected_doc_format -- an expected document file type, that is used to override the file type defined by the chosen downloader.

  papis.importer

class papis.importer.ImporterT
    Invariant TypeVar bound to the Importer class.

    alias of TypeVar('ImporterT', bound=Importer)

papis.importer.cache(meth: Callable[[ImporterT], None]) -> Callable[[ImporterT], None]
    Decorator used to cache Importer methods.

    The data is cached in the Importer.ctx of each importer instance. The method meth is only called if the context is empty.

Parameters
    meth -- a method of an Importer.

class papis.importer.Context

data
    A dict of fields retrieved by the Importer. These are generally not processed.

files
    A list of files retrieved by the Importer.

class papis.importer.Importer(uri: str = '', name: str = '', ctx: Context | None = None)

name
    A name given to the importer (that is not necessarily unique).

uri The URI (Uniform Resource Identifier) that the importer is to extract data from. This can be an URL, a local or remote file name, an object identifier (e.g. DOI), etc.

ctx A Context that stores the data retrieved by the importer.

classmethod match(uri: str) -> Importer | None
    Check if the importer can process the given URI.

    For example, an importer that supports links from the arXiv can check that the given URI matches using:

    re.match(r".*arxiv.org.*", uri)

    This can then be used to instantiate and return a corresponding Importer object.

Parameters
    uri -- An URI where the document information should be retrieved from.

Returns
    An importer instance if the match to the URI is successful or None otherwise.

classmethod match_data(data: Dict[str, Any]) -> Importer | None
    Check if the importer can process the given metadata.

    This method can be used to search for valid URIs inside the data that can then be processed by the importer. For example, if the metadata contains a DOI field, this can be used to import additional information.

Parameters
    data -- An dict with metadata to inspect and match against.

Returns
    An importer instance if matching metadata is found or None otherwise.

fetch() -> None
    Fetch metadata and files for the given uri.

    This method calls Importer.fetch_data() and Importer.fetch_files() to get all the information available for the document. It is recommended to implement the two methods separately, if possible, for maximum flexibility.

    The imported data is stored in ctx and it is not queried again on subsequent calls to this function.

fetch_data() -> None
    Fetch metadata from the given uri.

    The imported metadata is stored in ctx.

fetch_files() -> None
    Fetch files from the given uri.

    The imported files are stored in ctx.

papis.importer.get_import_mgr() -> stevedore.extension.ExtensionManager
    Retrieve the stevedore.extension.ExtensionManager for importer plugins.

papis.importer.available_importers() -> List[str]
    Get a list of available importer names.

papis.importer.get_importers() -> List[Type[Importer]]
    Get a list of available importer classes.

papis.importer.get_importer_by_name(name: str) -> Type[Importer]
    Get an importer class by name.

  papis.utils

class papis.utils.A
    Invariant typing.TypeVar

    alias of TypeVar('A')

class papis.utils.B
    Invariant typing.TypeVar

    alias of TypeVar('B')

papis.utils.get_session() -> requests.Session
    Create a requests.Session for papis.

    This session has the expected User-Agent (see user-agent), proxy (see downloader-proxy) and other settings used for papis. It is recommended to use it instead of creating a requests.Session at every call site.

papis.utils.parmap(f: Callable[[A], B], xs: Iterable[A], np: int | None = None) -> List[B]
    Apply the function f to all elements of xs.

    When available, this function uses the multiprocessing module to apply the function in parallel. This can have a noticeable performance impact when the number of elements of xs is large, but can also be slower than a sequential map().

    The number of processes can also be controlled using the PAPIS_NP environment variable. Setting this variable to 0 will disable the use of multiprocessing on all platforms.

Parameters

•   f -- a callable to apply to a list of elements.

•   xs -- an iterable of elements to apply the function f to.

•   np -- number of processes to use when applying the function f in parallel. This value defaults to PAPIS_NP or os.cpu_count().

papis.utils.general_open(file_name: str, key: str, default_opener: str | None = None, wait: bool = True) -> None
    Open a file with a configured open tool (executable).

Parameters

•   file_name -- a file path to open.

•   key -- a key in the configuration file to determine the opener used, e.g. opentool.

•   default_opener -- an existing executable that can be used to open the file given by file_name. By default, the opener given by key, if any, or the default papis opener are used.

•   wait -- if True wait for the process to finish, otherwise detach the process and return immediately.

papis.utils.open_file(file_path: str, wait: bool = True) -> None
    Open file using the configured opentool.

Parameters

•   file_path -- a file path to open.

•   wait -- if True wait for the process to finish, otherwise detach the process and return immediately.

papis.utils.get_folders(folder: str) -> List[str]
    Get all folders with papis documents inside of folder.

    This is the main indexing routine. It looks inside folder and crawls the whole directory structure in search of subfolders containing an info file. The name of the file must match the configured info-name.

Parameters
    folder -- root folder to look into.

Returns
    List of folders containing an info file.

papis.utils.create_identifier(input_list: str | None = None, skip: int = 0) -> Iterator[str]
    Creates an infinite list of identifiers based on input_list.

    This creates a generator object capable of iterating over lists to create unique products of increasing cardinality (see here). This is mainly intended to create suffixes for existing strings, e.g. file names, to ensure uniqueness.

Parameters

•   input_list -- list to iterate over

•   skip -- number of identifiers to skip.

    >>> import string
    >>> m = create_identifier(string.ascii_lowercase)
    >>> next(m)
    'a'

papis.utils.clean_document_name(doc_path: str, is_path: bool = True) -> str
    Clean a string to only contain visible ASCII characters.

    This function uses slugify to create ASCII strings that can be used safely as file names or printed to consoles that do not necessarily support full unicode.

    By default, it assumes that the input is a path and will only look at its basename. This can have unintended results for other strings and can be disabled by setting is_path to False.

Parameters

•   doc_path -- a string to be cleaned.

•   is_path -- if True, only the basename of doc_path is cleaned, as obtained from os.path.basename().

Returns
    a cleaned ASCII string.

papis.utils.locate_document_in_lib(document: Document, library: str | None = None) -> Document
    Locate a document in a library.

    This function uses the unique-document-keys to determine if the current document matches any document in the library. The first document for which a key matches exactly will be returned.

Parameters

•   document -- the document to search for.

•   library -- the name of a valid papis library.

Returns
    a full document as found in the library.

Raises
    IndexError -- No document found in the library.

papis.utils.locate_document(document: Document, documents: Iterable[Document]) -> Document | None
    Locate a document in a list of documents.

    This function uses the unique-document-keys to determine if the current document matches any document in the list. The first document for which a key matches exactly will be returned.

Parameters

•   document -- the document to search for.

•   documents -- an iterable of existing documents to match against.

Returns
    a document from documents which matches the given document or None if no document is found.

papis.utils.folders_to_documents(folders: Iterable[str]) -> List[Document]
    Load a list of documents from their respective folders.

Parameters
    folders -- a list of folder paths to load from.

Returns
    a list of document objects.

papis.utils.update_doc_from_data_interactively(document: Document | Dict[str, Any], data: Dict[str, Any], data_name: str) -> None
    Shows a TUI to update the document interactively with fields from data.

Parameters

•   document -- a document (or a mapping convertible to a document) which is going to be updated.

•   data -- additional data to select and merge into document.

•   data_name -- an identifier for the data to show in the TUI.

papis.utils.get_cache_home() -> str
    Get default cache directory.

    This will retrieve the cache-dir configuration setting. It is XDG standard compatible.

Returns
    the absolute path for the cache main folder.

papis.utils.get_matching_importer_or_downloader(uri: str, only_data: bool = True) -> List[Importer]
    Gets all the importers and downloaders that match uri.

    This function tries to match the URI using match() and extract the data using fetch(). Only importers that fetch the data without issues are returned.

Parameters

•   uri -- an URI to match the importers against.

•   only_data -- if True, attempt to only import document data, not files.

papis.utils.get_matching_importer_by_name(name_and_uris: Iterable[Tuple[str, str]], only_data: bool = True) -> List[Importer]
    Get importers that match the given URIs.

    This function tries to match the URI using match() and extract the data using fetch(). Only importers that fetch the data without issues are returned.

Parameters

•   name_and_uris -- an list of (name, uri) of importer names and URIs to match them against.

•   only_data -- if True, attempt to only import document data, not files.

papis.utils.collect_importer_data(importers: Iterable[Importer], batch: bool = True, only_data: bool = True) -> Context
    Collect all data from the given importers.

    It is assumed that the importers have called the needed fetch methods, so all data has been downloaded and converted. This function is meant to only do the aggregation.

Parameters

•   batch -- if True, overwrite data from previous importers, otherwise ask the user to manually merge.

•   only_data -- if True, only import document data, not files.

papis.utils.is_relative_to(path: str, other: str) -> bool
    Check if paths are relative to each other.

    This is equivalent to pathlib.PurePath.is_relative_to().

Returns
    True if path is relative to the other path.

papis.utils.dump_object_doc(objects: Iterable[Tuple[str, Any]], sep: str = '\n\t', bright: bool = True) -> List[str]
    Dumps the documentation for each of the object in objects to a string.

    This function is meant to provide a short description for each object based on the first line of its documentation.

    >>> from papis.arxiv import Importer
    >>> dump_object_doc([("arxiv", Importer)], sep=": ", bright=False)
    ['arxiv: Importer accepting an arXiv ID and downloading files and data']

Parameters

•   objects -- an iterable of (name, object) to be displayed.

•   sep -- the separator between the name and the description.

•   bright -- if True, the object name is styled bold for command-line printing.

Returns
    a list of strings describing each object.

  papis.yaml

papis.yaml.data_to_yaml(yaml_path: str, data: Dict[str, Any]) -> None
    Save data to yaml_path in the YAML format.

Parameters

•   yaml_path -- path to a file.

•   data -- data to write to the file as a YAML document.

papis.yaml.list_to_path(data: Sequence[Dict[str, Any]], filepath: str) -> None
    Save a list of dicts to a YAML file.

Parameters

•   data -- a sequence of dictionaries to save as YAML documents.

•   filepath -- path to a file.

papis.yaml.yaml_to_data(yaml_path: str, raise_exception: bool = False) -> Dict[str, Any]
    Read a YAML document from yaml_path.

Parameters

•   yaml_path -- path to a file.

•   raise_exception -- if True an exception is raised when loading the data has failed. Otherwise just a log message is emitted.

Returns
    a dict containing the data from the YAML document.

Raises
    ValueError -- if the document cannot be loaded due to YAML parsing errors.

papis.yaml.yaml_to_list(yaml_path: str, raise_exception: bool = False) -> List[Dict[str, Any]]
    Read a list of YAML documents.

    This is analogous to yaml_to_data(), but uses yaml.load_all to read multiple documents (see PyYAML docs).

Parameters

•   yaml_path -- path to a file containing YAML documents.

•   raise_exception -- if True an exception is raised when loading the data has failed. Otherwise just a log message is emitted.

Returns
    a list of dict objects, one for each YAML document in the file.

Raises
    ValueError -- if the documents cannot be loaded due to YAML parsing errors.

papis.yaml.exporter(documents: List[Document]) -> str
    Returns a YAML string containing all documents in the input list.

class papis.yaml.Importer(uri: str)
    Importer that parses a YAML file

classmethod match(uri: str) -> Importer | None
    Check if the uri points to an existing YAML file.

fetch_data() -> Any
    Fetch metadata from the YAML file.

FAQ

Here are some problems that users have come across often:

•   When I remove a folder manually in a library or I synchronize the library manually, I do not see the new papers in the library. Answer: You probably need to update the cache because papis did not know anything about your changes in the library since you did it by yourself.

    papis --clear-cache

    will do.

    For more information you can also check the github faq link.

•   Index

•   Module Index

•   Search Page

AUTHOR
Alejandro Gallo

COPYRIGHT
2017, Alejandro Gallo