Pearl
=====

Because only in the best Shells you will find a Pearl...

<h1 align="center">
    <a href="https://github.com/pearl-core/pearl"><img
        alt="Pearl"
        width=300px
        src="https://rawgit.com/pearl-core/logo/master/pearl.png"></a>
</h1>

|Project Status|Communication|
|:-----------:|:-----------:|
|[![Build status](https://api.travis-ci.org/pearl-core/pearl.png?branch=master)](https://travis-ci.org/pearl-core/pearl) | [![Join the gitter chat at https://gitter.im/pearl-core/pearl](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/pearl-core/pearl?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |

**Table of Contents**
- [Description](#description)
- [Quickstart](#quickstart)
- [Installation](#installation)
  - [Dependencies](#dependencies)
  - [Linux](#linux)
  - [OSX](#osx)
- [Create your own Pearl package](#create-your-own-pearl-package)
- [Create your own Pearl repository](#create-your-own-pearl-repository)
- [Troubleshooting](#troubleshooting)
- [Contributing](#contributing)

Description
===========
**Pearl** is a lightweight package manager for automating reproducible environments
between different systems (Linux and OSX).
It can be used for dotfiles, plugins, programs and any form of code
accessible via git.

As soon as a package gets installed, its content can be activated out of the box
according to certain events, like, for instance, a shell startup (Bash, Zsh or Fish) or
an editor startup (Vim or Emacs). This is possible via a smart and simple
[hook mechanism](#create-your-own-pearl-package)
that integrates the package content within the Pearl ecosystem.

The main advantages on using Pearl are:

- Create your own Pearl package in a very simple way.
- Full control and sync of your dotfiles across different systems.
- Automatic bootstrap of the package content whenever shells or editors get started.
- Access to a wide range of existing packages via the [OPH (Official Pearl Hub)](https://github.com/pearl-hub).
- Allows to create your own shareable package repository.
- Stable codebase with 100+ unit tests and exhaustive integration tests via [Travis](https://travis-ci.org/pearl-core/pearl) for Linux and OSX.
- Small number of [dependencies](#dependencies) needed in order to ensure compatibility with most of the systems.

Comparison with similar solution: Ansible
-----------------------------------------
You could achieve something similar from what Pearl provide by using
[Ansible](https://www.ansible.com/). Ansible is a powerful software for IT
automation which can be widely used for many use cases.
Despite of this, Ansible has few drawbacks when using it for lightweight forms of automation:

- Pearl uses bash for writing simple scripts for automation:
  - it makes easier the integration with other programs in the system (without existing Playbooks may be hard and tedious to achieve this in Ansible);
  - bash is a powerful, accessible and well-known language;
- Ansible requires way more dependencies than Pearl;
- Ansible requires knowledge about how Ansible Playbooks works;
- Pearl uses built-in [functions](https://github.com/fsquillace/buava/blob/master/README.md#table-of-buava-functions) and [variables](#structure-of-a-pearl-package) which heavily simplify construction of scripts for automation;
- Pearl makes easier to remove packages and restore the system to an initial state;

Quickstart
==========
The Pearl command allows to: `create`, `list`, `search`, `install`, `update`, `emerge`,
`remove` the Pearl packages defined according to the configuration located in
`$XDG_CONFIG_HOME/pearl/pearl.conf` (defaults to `~/.config/pearl/pearl.conf`)

![quickstart](https://raw.githubusercontent.com/pearl-core/resources/master/pearl-opt3.gif)

Create
------
- Command `create` helps you create a new local Pearl package:

```sh
$ pearl create mydotfiles ~/dotfiles
```
This will create a directory `pearl-config` in `~/dotfiles` containing all the templates to help you
start writing a Pearl package. `~/dotfiles` does not need to be an empty directory.
Additionally, the local repository in `$XDG_CONFIG_HOME/pearl/pearl.conf` will be updated with
the new package entry called `mydotfiles`.

For more information about the `pearl-config` content look at the [section](#create-your-own-pearl-package) below.

List
----
- List all the available packages:

```sh
$ pearl list
...
pearl/dot-git
    Awesome git dotfiles (https://github.com/pearl-hub/git)
pearl/sesaila [installed]
    Awesome aliases for Bash, Zsh and Fish shells (https://github.com/pearl-hub/sesaila)
pearl/airline [installed]
    Status/tabline for vim (https://github.com/vim-airline/vim-airline)
pearl/trash-cli [installed]
    Command line interface to the freedesktop.org trashcan (https://github.com/pearl-hub/trash-cli)
...
```

Search
------
- Search for `vim` Pearl packages:

```sh
$ pearl search vim
* Updating https://github.com/pearl-hub/repo.git repository
pearl/dot-vim
    Awesome vim dotfiles (https://github.com/pearl-hub/vim)
```

Install
-------
- Install `pearl/dot-vim` package (as soon as the package is installed the package is ready out of the box in vim editor!):

```sh
$ pearl install dot-vim
* Updating https://github.com/pearl-hub/repo.git repository
* Installing pearl/dot-vim package
```

- Install `pearl/trash-cli` package:

```sh
$ pearl install trash-cli
* Updating https://github.com/pearl-hub/repo.git repository
* Installing pearl/trash-cli package
$ trash -h
Usage: trash [OPTION]... FILE...

Put files in trash
...
...
```

Update
-------
- Update `pearl/dot-vim` package:

```sh
$ pearl update dot-vim
* Updating https://github.com/pearl-hub/repo.git repository
* Updating pearl/dot-vim package
```

- Update Pearl and all its packages installed:

```sh
$ pearl update
...
* Updating https://github.com/pearl-hub/repo.git repository
* Updating Pearl script
* Updating pearl/dot-vim package
* Updating pearl/airline package
* Updating pearl/trash-cli package
* Updating pearl/caprica package
...
```

Emerge
------
Emerge is an idempotent command for either installing or updating a package
depending whether the package is already installed or not. This command turns
to be particularly useful for establishing dependencies between packages.
See the section [below](#define-dependencies-between-pearl-packages)
for more details.

Remove
-------
- Remove `pearl/dot-vim` package:

```sh
$ pearl remove dot-vim
* Updating https://github.com/pearl-hub/repo.git repository
* Removing pearl/dot-vim package
```

- Remove Pearl and all its packages installed:

```sh
$ pearl remove
...
Are you sure to REMOVE all the Pearl packages in $PEARL_HOME folder? (N/y)
* Updating https://github.com/pearl-hub/repo.git repository
* Removing pearl/dot-vim package
* Removing pearl/airline package
* Removing pearl/trash-cli package
* Removing pearl/caprica package
...
```

Recommended Pearl Hub packages to install:
-------------------------------

- [cmd](https://github.com/pearl-hub/cmd)
- [kyrat](https://github.com/pearl-hub/kyrat)
- [ranger](https://github.com/pearl-hub/ranger)
- [sesaila](https://github.com/pearl-hub/sesaila)
- [trash-cli](https://github.com/pearl-hub/trash-cli)
- [txum](https://github.com/pearl-hub/txum)

For dotfiles packages take a look [here](https://github.com/pearl-hub?q=dot).

Check out the [OPH (Official Pearl Hub)](https://github.com/pearl-hub)
for more packages you might be interested.

Installation
============

Dependencies
------------
Before installing Pearl be sure that all dependencies are properly installed in your system.
The Pearl dependencies are the following:

### Mandatory
- [python (>=3.5)](https://www.python.org/)
- [bash (>=4.1)](https://www.gnu.org/software/bash/)
- [git (>=1.8)](https://git-scm.com/)

### Optional
The following are not mandatory dependencies but can be handy to have for the hook functions in Pearl package.
All the Linux distributions have these dependencies already installed.

- [GNU coreutils](https://www.gnu.org/software/coreutils/)
- [grep](https://www.gnu.org/software/grep/) 
- [sed](https://www.gnu.org/software/sed/) 

### Additional shells supported
Pearl supports also the following shells:

- [fish (>=2.2.0)](https://fishshell.com/)
- [zsh (>=5.2)](http://www.zsh.org/)


Linux
-----

### Arch Linux

Pearl can be installed in Arch Linux through AUR.
The package is [pearl-git](https://aur.archlinux.org/packages/pearl-git/).

For example, to install Pearl via [yay](https://github.com/Jguer/yay) AUR helper:
```
yay -S pearl-git
```

Any other AUR helpers can be found [here](https://wiki.archlinux.org/index.php/AUR_helpers).

### Other Linux distributions

Assuming all Pearl [dependencies](#dependencies) are properly installed
in the system, to install Pearl you can use the `pip` command.
Unless there is a specific use case, it is not a good option to use virtual environments such as
`virtualenv` or `conda` because otherwise Pearl will be only visible within that environment.
It is recommended to use the system-wide `pip` which is generally locate in `/usr/bin/pip`.
The following will install the package in your `$HOME` directory (`~/.local/`):
```
/usr/bin/pip install --user pearl
export PATH="~/.local/bin:$PATH"
```

Pearl command will be located in `~/.local/bin/pearl`

To create the `$PEARL_HOME` directory and the new pearl configuration file from template, run: 
```
pearl init
```

OSX
---
In order to install all Pearl dependencies, you first need to install [Homebrew](http://brew.sh/).

To install all the needed dependencies via Homebrew:
```sh
brew update
brew install bash git coreutils grep gnu-sed python
```

The following will install the package under `/usr/local`:
```
/usr/local/bin/pip3 install pearl
# If the bin path is not already in $PATH:
export PATH="/usr/local/bin:$PATH"
```

Pearl command will be located in `/usr/local/bin/pearl`

To create the `$PEARL_HOME` directory and the new pearl configuration file from template, run: 
```
pearl init
```

**IMPORTANT NOTE**: Pearl gets loaded through `~/.bashrc`. The problem is that in OSX,
the terminal opens a login shell and only `~/.bash_profile` will get executed.
Run the following only if `~/.bashrc` is not loaded within `~/.bash_profile` file:

```sh
echo "[[ -f ~/.bashrc ]] && source ~/.bashrc" >> ~/.bash_profile
```

This will make sure that `~/.bashrc` will run at shell startup.

Create your own Pearl package
===============
**Any git repository is already a Pearl package**. For instance, in order
to manage a dotfiles repository in Pearl, you just need to change
the Pearl configuration file located in `$XDG_CONFIG_HOME/pearl/pearl.conf`.

Add the following line to `pearl.conf` file:

    PEARL_PACKAGES = {
        "mydotfiles": {
            "url": "https://github.com/user/mydotfiles.git",
            "description": "My dotfiles"
        },
    }

In other words, update the `PEARL_PACKAGES` dictionary with a new entry containing the
name of the package (i.e. `mydotfiles`),
the git url (i.e. `https://github.com/user/mydotfiles.git`) and an optional description.

***That's it!*** The package will be ready to be [installed](#install),
[updated](#update), [emerged](#emerge) and [removed](#remove) via the Pearl system.

## Structure of a Pearl package ##
Your own git repository can contain an **optional** directory
named `pearl-config` used by Pearl to integrate the package with the Pearl environment.

    / (package root)
    │
    ├── pearl-config (optional directory)
    │   │
    │   ├── hooks.sh
    │   ├── config.sh
    │   ├── config.bash
    │   ├── config.zsh
    │   ├── config.fish
    │   ├── config.vim
    │   ├── config.el
    │   └── package.conf
    │
    └── (additional package content)

The files inside `pearl-config` are also **optional** script/configuration files:

- `hooks.sh` - contains the [hooks functions](#hook-functions) executed during the `install`, `update` and `remove` events.
- `config.sh` - will be sourced whenever a new Bash/Zsh shell is starting up.
- `config.bash` - will be sourced whenever a new Bash shell is starting up.
- `config.zsh` - will be sourced whenever a new Zsh shell is starting up.
- `config.fish` - will be sourced whenever a new Fish shell is starting up.
- `config.vim` - will be executed whenever Vim editor is starting up.
- `config.el` - will be sourced whenever Emacs editor is starting up.
- `package.conf` - contains optional metadata information (name, author, description, keywords, etc) about the package that are useful when indexing the package in a repository list.

The following variables can be used in any of the previous scripts:

- `PEARL_HOME`          - Pearl location (`$XDG_DATA_HOME/pearl` which by default is `$HOME/.local/share/pearl`)
- `PEARL_PKGDIR`        - Pearl package location
- `PEARL_PKGVARDIR`     - Pearl package location containing data needed for package
- `PEARL_PKGNAME`       - Pearl package name
- `PEARL_PKGREPONAME`   - Pearl package repo name (useful to detect and interact with packages within the same repo)

Additionally, the script `hooks.sh` can use the utility functions available in
[Buava](https://github.com/fsquillace/buava) and Pearl [*utils*](lib/utils) directory that
make easier the integration with Pearl ecosystem.

Useful examples of Pearl packages can be checked in the
[Official Pearl Hub](https://github.com/pearl-hub).

### The hooks.sh script ###
#### Hook functions ####
- `post_install`  - Called *after* an installation of the package occurs.
- `pre_update`    - Called *before* an update of the package occurs.
- `post_update`   - Called *after* an update of the package occurs.
- `pre_remove`    - Called *before* a removal of the package occurs.

#### An hooks.sh script example ####

    post_install() {
        warn "Remember to setup your config located in: ~/.dotfile"
        # Do a smart backup before modifying the file
        backup ${HOME}/.dotfile
        "# New dotfile" > ${HOME}/.dotfile
        link tmux "$PEARL_PKGDIR/mytmux.conf"

        info "Awesome - new package installed!"
        return 0
    }
    post_update() {
        post_install
        return 0
    }
    pre_remove() {
        info "dotfiles package removed"
        unlink tmux "$PEARL_PKGDIR/mytmux.conf"

        # Do an idempotent delete
        delete ${HOME}/.dotfile
        return 0
    }

The `info` and `warn` are functions that print a message
using different colors (namely cyan and yellow).

The `link` `unlink` are idempotent functions (the result will not change
if the function will be called multiple times) that are able
to link/unlink a config file in order to be loaded at startup by a certain program.

The `backup` keeps the last three backups of the file and do not perform backup
if the file has not been modified since the latest backup. The `delete` is a
function for idempotent remove (it will not raise an error if the file
no longer exist).

All these functions belong to the [Buava](https://github.com/fsquillace/buava) package
in [`utils.sh`](https://github.com/fsquillace/buava/blob/master/lib/utils.sh)
and to the Pearl [`utils.sh`](lib/utils/utils.sh) script. You can use them
inside the `hooks.sh` to any hook function.

**Very important note**: All the hook functions **must** be
[**idempotent**](https://en.wikipedia.org/wiki/Idempotence)
(the commands of each hook function must produce the same result even if
the command gets executed multiple times).
All buava commands are idempotent and this will help to write hook functions
very quickly.

**Note**: For OSX system, the GNU version `sed` and `grep` are automatically
imported in `hooks.sh` and can be directly used if needed.

### The package.conf file
`package.conf` is meant to contain metadata about the package. To simplify the creation of packages, this file
is completely optional. It contains name of package, description, author, os compatibility, license and more.
In case you include such information, they may be used to populate the repository file `repo.conf` so
that searching for packages will be easier.
It can be also used to establish dependencies between packages.
If you do not need such features for your package, just ignore this file.

## Create a Pearl package from a local directory ##
Pearl package system will work even for local directories. This is particularly useful
whenever a Pearl package needs to be tested before pushing to a git repository.

For instance, the following lines in `pearl.conf` file will add a package located in
`/home/joe/dotfiles`:

    PEARL_PACKAGES = {
        "mydotfiles": {
            "url": "/home/user/mydotfiles",
            "description": "My dotfiles"
        },
    }

The directory path must be an absolute path.

The package will be ready to be [installed](#install), [updated](#update),
[emerged](#emerge) and [removed](#remove) via the Pearl system.

The directory content can be structured in the exact way as described
in the [section](#structure-of-a-pearl-package) above.

## Define dependencies between Pearl packages ##
Suppose you have a package `mypack` which depends on another package `mydep`,
you can update the `mypack` `hooks.sh` file in this way:

    post_install() {
        # Install/update the dependency here:
        pearl emerge ${PEARL_PKGREPONAME}/mydep
    }
    post_update() {
        post_install
    }
    pre_remove() {
        # Uncomment below to strictly remove the dependency
        # during the removal of the current package:
        #pearl remove ${PEARL_PKGREPONAME}/mydep
    }

The `PEARL_PKGREPONAME` variable will make sure to define dependencies only
between packages of the same repository.
To see a real example in Pearl Hub, take a look at the [Kyrat hooks.sh](https://github.com/pearl-hub/kyrat/blob/master/pearl-config/hooks.sh).

## Use third-party git repository not available in Pearl Hub ##
If you want to use a third-party git repository
that is not available in the [Official Pearl Hub](https://github.com/pearl-hub),
you can:

* Create your own git repository and use the `PEARL_PKGVARDIR` directory (recommended)
* Create your own git repository and use [git submodule](https://git-scm.com/docs/git-submodule)
* Point directly to the third-party git repository

To see examples of Pearl packages from third-party git repos take a look at the
[Official Pearl Hub](https://github.com/pearl-hub).

### Create your own git repository and use the `PEARL_PKGVARDIR` directory (recommended) ###
You can use the `PEARL_PKGVARDIR` directory during the installation phase to install the third-party git repository.
This is the best way to incorporate third-party project into Pearl ecosystem.

Here it is an example of `hooks.sh` file which install the ranger file manager into the directory `${PEARL_PKGVARDIR}/ranger`:

    function post_install(){
        install_or_update_git_repo https://github.com/ranger/ranger.git "${PEARL_PKGVARDIR}/ranger" master
    }

    function post_update(){
        post_install
    }

    function pre_remove(){
        rm -rf ${PEARL_PKGVARDIR}/ranger
    }

The function `install_or_update_git_repo` comes from the [Buava](https://github.com/fsquillace/buava)
library in [`utils.sh`](https://github.com/fsquillace/buava/blob/master/lib/utils.sh)
which is natively available in Pearl during the installation.
You can even use the functions `install_git_repo` or `update_git_repo` which respectively install or update the git repository.

For a full example take a look at the [ranger](https://github.com/pearl-hub/ranger) Pearl Hub package.

### Create your own git repository and use git submodule ###
Inside your git repository, you just need to add the third-party git repo as a
[git submodule](https://git-scm.com/docs/git-submodule).
For instance, to add the [powerline](https://github.com/powerline/powerline) in your Pearl package,
you can introduce a submodule in the `module` directory:

    git submodule add https://github.com/powerline/powerline.git module

The filesystem structure of the package will become something like this:

    / (package root)
    │
    ├── pearl-config   (optional directory)
    ├── module/        (contains third-party code)
    └── (additional package content)

Then, you just need to modify the config scripts in order to integrate the third-party
project inside Pearl environment.

### Point directly to the third-party git repository ###
Let's suppose you want to install the [vim-rails](https://github.com/tpope/vim-rails) plugin.
In your Pearl configuration (`$XDG_CONFIG_HOME/pearl/pearl.conf`), add your new Pearl package:

    PEARL_PACKAGES = {
        "vim-rails": {
            "url": "https://github.com/tpope/vim-rails.git",
            "description": "Ruby on Rails power tools"
        },
    }

Install the package:

    pearl install vim-rails

Voila', your new vim plugin is ready to be used!

This approach is particularly useful whenever you do not need to specify
any pearl config to *"enrich"* the third-party project inside
the Pearl environment.


Create your own Pearl repository
===============
A Pearl repository is just a git repository containing a file located in `pearl-config/pearl.conf`
with a list of packages. For instance, the *OPH* repository is available
[here](https://github.com/pearl-hub/repo-v2).

In order to use the new repository (i.e. "https://github.com/myrepo/pearl-repo.git"),
update the `pearl.conf` file by adding the following line:

    PEARL_REPOS += ("https://github.com/myrepo/pearl-repo.git")

Troubleshooting
===============

## Corrupted Pearl Home directory ##

> **Q**: What should I do if I accidentally removed files/packages in `$PEARL_HOME`?

> **A**: You can recover the structure of the `$PEARL_HOME` by running:

    $> pearl init

> The command will create all the essential directories and symlinks in `$PEARL_HOME`.
> It is harmless to run the `init` command multiple times since it is idempotent.

## Corrupted package ##

> **Q**: Why I can no longer update/remove a package?

> **A**: This is probably because either one of the hook functions
> is failing or the package content is corrupted. You can forcely remove the package:

    $> pearl remove <packagename>

> which bypass hook functions that are failing. If that does not even work,
> you can delete a package by simply removing its directory:

    $> rm -rf $PEARL_HOME/packages/pearl/<packagename>

> After that, you can reinstall the package again.
> The Pearl packages contain a dedicated directory `var` for storing
> data needed for the package itself.
> The `var` data are always managed by the package and they never gets deleted by Pearl
> during the package removal.
> If you want to delete the content in `var` package:

    $> rm -rf $PEARL_HOME/var/pearl/<packagename>

## Package shell variables/functions not visible in current shell after installation ##

> **Q**: Why are not package's environment variables/functions visible in
> my current shell after installing/updating the package?

> **A**: After package install/update, the variables or
> functions related to the current shell and defined in `pearl-config/config.*`
> may not be available because a reload of Pearl configuration file is required.
> You can fix this by simply run the function:

    pearl-source
    
> which reloads the configuration.
> The use of such function is not always required but depends
> whether the variables/functions involve the current shell where the
> package `install`/`update` occurred (i.e. a new variable defined in `config.sh`
> and the current shell is a bash or zsh). Alternatively, user can always
> create a new shell and the package resources will be available as
> expected.

## Error during package install

> Q: Why Do I get the following error:

    Error on executing 'post_install' hook. Rolling back...

> A: This occurs when the `post_install` hook function fails.
> Pearl will attempt to roll back and force a removal of the package. In this way
> you can attempt to install the package again once the hook function gets
> fixed.

Contributing
============

You could help improving Pearl and the [OPH](https://github.com/pearl-hub) in the following ways:

- [Reporting Bugs](CONTRIBUTING.md#reporting-bugs)
- [Suggesting Enhancements](CONTRIBUTING.md#suggesting-enhancements)
- [Writing Code](CONTRIBUTING.md#your-first-code-contribution)
