<!--- Copyright 2022 eprbell --->

<!--- Licensed under the Apache License, Version 2.0 (the "License"); --->
<!--- you may not use this file except in compliance with the License. --->
<!--- You may obtain a copy of the License at --->

<!---     http://www.apache.org/licenses/LICENSE-2.0 --->

<!--- Unless required by applicable law or agreed to in writing, software --->
<!--- distributed under the License is distributed on an "AS IS" BASIS, --->
<!--- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --->
<!--- See the License for the specific language governing permissions and --->
<!--- limitations under the License. --->

# The DaLI Configuration File

## Table of Contents
* **[Introduction](#introduction)**
* **[Data Loader Plugin Sections](#data-loader-plugin-sections)**
  * [Coinbase Section (REST)](#coinbase-section-rest)
  * [Coinbase Section Pro (REST)](#coinbase-pro-section-rest)
  * [Trezor Section (CSV)](#trezor-section-csv)
  * [Trezor Old Section (CSV)](#trezor-old-section-csv)
  * [Manual Section (CSV)](#trezor-section-csv)
* **[Builtin Sections](#builtin-sections)**
  * [Transaction Hints Section](#transaction-hints-section)
  * [Header Sections](#header-sections)

## Introduction

The configuration file is in [INI format](https://en.wikipedia.org/wiki/INI_file) and it is used to initialize data loaders (plugins) and configure DaLI's behavior. It contains a sequence of configuration sections, which are of two types:
* plugin sections select data loader plugins (e.g. Coinbase REST, Trezor CSV, etc.) to run and initialize them.
* builtin sections configure general DaLI behavior (format of the output ODS, hints on how to generate certain transactions, etc.);

An example of a configuration file can be found in [test_config.ini](../config/test_config.ini)

## Data Loader Plugin Sections

A data loader plugin has the purpose of reading crypto data from a source (CSV file or REST-based service). Data loader plugins are initialized in a section of the INI file that is structured as described below. Note that:
* *`<type>`* is one of: `csv` or `rest`, depending on the type of plugin;
* *`<plugin>`* is the name of the plugin;
* *`<qualifiers>`* is an optional sequence of words with the purpose of distinguishing multiple sections initializing the same plugin (e.g. two different Trezor wallets belonging to the same user, or two Coinbase accounts belonging to two people filing together);
* *`<parameter>`* and *`<value>`* are plugin-specific name-value pairs used to initialize a specific instance of the plugin.

<pre>
[dali.plugin.input.<em>&lt;type&gt;</em>.<em>&lt;plugin&gt;</em> <em>&lt;qualifiers&gt;</em>]
<em>&lt;parameter_1&gt;</em> = <em>&lt;value_1&gt;</em>
...
<em>&lt;parameter_n&gt;</em> = <em>&lt;value_n&gt;</em>
</pre>

DaLI comes with a few builtin plugins, but more are needed: help us make DaLI a robust open-source, community-driven crypto data loader by contributing plugins for exchanges and wallets! Read the [Contributing to the Repository](../CONTRIBUTING.md#contributing-to-the-repository) document to learn more.

### Coinbase Section (REST)
This plugin is REST-based and requires setting up API Keys in your Coinbase account Settings (click on the API link).

**IMPORTANT NOTE**:
* When setting up API key/secret, only use read permissions (DaLI does NOT need write permissions).
* store your API key and secret safely and NEVER share it with anyone!

Initialize this plugin section as follows:
<pre>
[dali.plugin.input.rest.coinbase <em>&lt;qualifiers&gt;</em>]
account_holder = <em>&lt;account_holder&gt;</em>
api_key = <em>&lt;api_key&gt;</em>
api_secret = <em>&lt;api_secret&gt;</em>
</pre>

### Coinbase Pro Section (REST)
This plugin is REST-based and requires setting up API Keys in your Coinbase Pro account Settings (click on the API link).

**IMPORTANT NOTE**:
* When setting up API key/secret/passphrase, only use read permissions (DaLI does NOT need write permissions).
* store your API key, secret and passphrase safely and NEVER share it with anyone!

Initialize this plugin section as follows:
<pre>
[dali.plugin.input.rest.coinbase_pro <em>&lt;qualifiers&gt;</em>]
account_holder = <em>&lt;account_holder&gt;</em>
api_key = <em>&lt;api_key&gt;</em>
api_secret = <em>&lt;api_secret&gt;</em>
api_passphrase = <em>&lt;api_passphrase&gt;</em>
</pre>

### Trezor Section (CSV)
This plugin is CSV-based and parses CSV files generated by Trezor Suite. Initialize it as follows:
<pre>
[dali.plugin.input.csv.trezor <em>&lt;qualifiers&gt;</em>]
account_holder = <em>&lt;account_holder&gt;</em>
account_nickname = <em>&lt;account_nickname&gt;</em>
currency = <em>&lt;currency&gt;</em>
timezone = <em>&lt;timezone&gt;</em>
csv_file = <em>&lt;csv_file&gt;</em>
</pre>

Notes:
* account_nickname is a user-selected identifier for the account;
* currency is the crypto the account is denominated in;
* timezone identifies the timezone where the transactions occurred (Trezor doesn't provide this information in the CSV file)

### Trezor Old Section (CSV)
This plugin is CSV-based and parses older CSV files generated by the Trezor web interface (only use this if the Trezor plugin doesn't work). Initialize it as follows:
<pre>
[dali.plugin.input.csv.trezor_old <em>&lt;qualifiers&gt;</em>]
account_holder = <em>&lt;account_holder&gt;</em>
account_nickname = <em>&lt;account_nickname&gt;</em>
currency = <em>&lt;currency&gt;</em>
timezone = <em>&lt;timezone&gt;</em>
csv_file = <em>&lt;csv_file&gt;</em>
</pre>

Notes:
* account_nickname is a user-selected identifier for the account;
* currency is the crypto the account is denominated in;
* timezone identifies the timezone where the transactions occurred (Trezor doesn't provide this information in the CSV file)

### Manual Section (CSV)
The manual CSV plugin is used to describe transactions for exchanges and wallets that are not yet supported. Initialize it as follows:
<pre>
[dali.plugin.input.csv.trezor_old <em>&lt;qualifiers&gt;</em>]
in_csv_file = <em>&lt;in_csv_file&gt;</em>
out_csv_file = <em>&lt;out_csv_file&gt;</em>
intra_csv_file = <em>&lt;intra_csv_file&gt;</em>
</pre>

The in_csv_file contains transactions describing crypto being acquired. Line 1 is considered a header line and it's ignored. Subsequent lines have the following format:
* unique_id (optional): unique id identifying the transaction. It's only useful to match partial intra transactions (see below) and it can be omitted in other cases;
* timestamp: time at which the transaction occurred. DaLI can parse most timestamp formats, but timestamps must always include: year, month, day, hour, minute, second and timezone (milliseconds are optional). E.g.: "2020-01-21 11:15:00+00:00";
* asset: which cryptocurrency was transacted (e.g. BTC, ETH, etc.);
* exchange: exchange or wallet on which the transaction occurred;
* holder: exchange account or wallet owner;
* transaction_type: AIRDROP, BUY, DONATE, GIFT, HARDFORK, INCOME, INTEREST, MINING, STAKING or WAGES;
* spot_price: value of 1 unit of the given cryptocurrency at the time the transaction occurred;
* crypto_in: how much of the given cryptocurrency was acquired with the transaction;
* fiat_fee: fiat value of the transaction fees;
* fiat_in_no_fee (optional): fiat value of the transaction without fees;
* fiat_in_with_fee (optional): fiat value of the transaction with fees;
* notes (optional): user-provided description of the transaction.

The out_csv_file contains transactions describing crypto being disposed of. Line 1 is considered a header line and it's ignored. Subsequent lines have the following format:
* unique_id (optional): unique id identifying the transaction. It's only useful to match partial intra transactions (see below) and it can be omitted in other cases;
* timestamp: time at which the transaction occurred. DaLI can parse most timestamp formats, but timestamps must always include: year, month, day, hour, minute, second and timezone (milliseconds are optional). E.g.: "2020-01-21 11:15:00+00:00";
* asset: which cryptocurrency was transacted (e.g. BTC, ETH, etc.);
* exchange: exchange or wallet on which the transaction occurred;
* holder: exchange account or wallet owner;
* transaction_type: DONATE, GIFT or SELL;
* spot_price: value of 1 unit of the given cryptocurrency at the time the transaction occurred;
* crypto_out_no_fee: how much of the given cryptocurrency was sold or sent with the transaction (excluding fees);
* crypto_fee: crypto value of the transaction fees;
* crypto_out_with_fee (optional): how much of the given cryptocurrency was sold or sent with the transaction (including fees);
* fiat_out_no_fee (optional): fiat value of the transaction without fees;
* fiat_fee (optional): fiat value of the transaction fees;
* notes (optional): user-provided description of the transaction.

The intra_csv_file contains transactions describing crypto being moved across accounts controlled by the same user (or people filing together). Line 1 is considered a header line and it's ignored. Subsequent lines have the following format:
* unique_id: unique id identifying the transaction. It's only useful to match partial intra transactions (see below) and it can be omitted in other cases;
* timestamp: time at which the transaction occurred. DaLI can parse most timestamp formats, but timestamps must always include: year, month, day, hour, minute, second and timezone (milliseconds are optional). E.g.: "2020-01-21 11:15:00+00:00";
* asset: which cryptocurrency was transacted (e.g. BTC, ETH, etc.);
* from_exchange (optional): exchange or wallet from which the transfer of cryptocurrency occurred;
* from_holder (optional): owner of the exchange account or wallet from which the transfer of cryptocurrency occurred;
* to_exchange (optional): exchange or wallet to which the transfer of cryptocurrency occurred;
* to_holder (optional): owner of the exchange account or wallet to which the transfer of cryptocurrency occurred;
* spot_price (optional): value of 1 unit of the given cryptocurrency at the time the transaction occurred;
* crypto_sent (optional): how much of the given cryptocurrency was sent with the transaction;
* crypto_received (optional): how much of the given cryptocurrency was received with the transaction;
* notes (optional): user-provided description of the transaction.

Note that empty (separator) lines are allowed in all three CSV files to increase readability.

The manual CSV plugin is typically used for two purposes:
* add self-contained in/out/intra transactions from exchanges or wallets that are yet unsupported by DaLI;
* add NON self-contained in/out/intra transactions from exchanges or wallets that are yet unsupported by DaLI.

The difference between self-contained and non-self-contained transactions is that the latter describe a part of an intra transaction, whereas the former are full transactions. Non self-contained transactions are used when an intra transaction is sent from a wallet or exchange supported by DaLI and is received at a wallet or exchange that is not yet supported by DaLI or viceversa. This causes DaLI to generate an incomplete transaction.

For the case of supported origin and unsupported destination, DaLI generates:
1. an intra transaction with defined from_exchange/from_holder/crypto_sent and empty to_exchange/to_holder/crypto_received empty or
2. an out transaction (modeling the source side of the intra transaction)

For the case of unsupported origin and supported destination, DaLI generates:

3. an intra transaction with empty from_exchange/from_holder/crypto_sent and defined to_exchange/to_holder/crypto_received or
4. an in transaction (modeling the destination side of the intra transaction)

The manual CSV allows users to complete the partially generated transactions:
* in case 1 the user adds a partial intra transaction with empty from_exchange/from_holder/crypto_sent and defined to_exchange/to_holder/crypto_received to the *`intra_csv_file`*;
* in case 2 the user adds an in_transaction to *`in_csv_file`* (modeling the destination side of the intra transaction). Transaction type can be omitted.
* in case 3 the user adds a partial intra transaction with defined from_exchange/from_holder,crypto_sent and empty to_exchange/to_holder/crypto_received to the *`intra_csv_file`*;
* in case 4 the user adds an out_transaction to *`out_csv_file`* (modeling the source side of the intra transaction). Transaction type can be omitted.

The `unique_id` field is critical and it is used to pair incomplete transactions: make sure it matches the unique id of the generated incomplete transaction you want to complete.
Self-contained transactions don't need unique_id.

**Example 1)**

If DaLI generates the following incomplete intra transaction:
unique_id   |timestamp                 |asset|from_exchange|from_holder|to_exchange|to_holder|spot_price|crypto_sent|crypto_received
------------|--------------------------|-----|-------------|-----------|-----------|---------|----------|-----------|---------------
389ded74b35f|2020-03-01 10:45:23 +00:00|BTC  |Coinbase     |Alice      |-          |-        |15100     |0.5        |-

The user can complete it by adding the following to the *`intra_csv_file`*:
unique_id   |timestamp                 |asset|from_exchange|from_holder|to_exchange|to_holder|spot_price|crypto_sent|crypto_received
------------|--------------------------|-----|-------------|-----------|-----------|---------|----------|-----------|---------------
389ded74b35f|2020-03-01 11:25:18 +00:00|BTC  |-            |-          |FTX        |Alice    |-         |-          |0.49

The next time DaLI is run with the new information in the *`intra_csv_file`*, it will generate a full intra transaction:
unique_id   |timestamp                 |asset|from_exchange|from_holder|to_exchange|to_holder|spot_price|crypto_sent|crypto_received
------------|--------------------------|-----|-------------|-----------|-----------|---------|----------|-----------|---------------
389ded74b35f|2020-03-01 11:25:18 +00:00|BTC  |Coinbase     |Alice      |FTX        |Alice    |15100     |0.5        |0.49

**Example 2)**

If DaLI generates the following incomplete out transaction:
unique_id   |timestamp                 |asset|exchange|holder|transaction_type|spot_price|crypto_out_no_fee|crypto_fee
------------|--------------------------|-----|--------|------|----------------|----------|-----------------|----------
389ded74b35f|2020-03-01 10:45:23 +00:00|BTC  |Coinbase|Alice |-               |15100     |0.5              |0.01

The user can complete it by adding the following to the *`in_csv_file`*:
unique_id   |timestamp                 |asset|exchange|holder|transaction_type|spot_price|crypto_out_no_fee|crypto_fee
------------|--------------------------|-----|--------|------|----------------|----------|-----------------|----------
389ded74b35f|2020-03-01 11:25:18 +00:00|BTC  |FTX     |Alice |-               |15100     |0.49             |0.01

The next time DaLI is run with the new information in the *`in_csv_file`*, it will generate a full intra transaction:
unique_id   |timestamp                 |asset|from_exchange|from_holder|to_exchange|to_holder|spot_price|crypto_sent|crypto_received
------------|--------------------------|-----|-------------|-----------|-----------|---------|----------|-----------|---------------
389ded74b35f|2020-03-01 11:25:18 +00:00|BTC  |Coinbase     |Alice      |FTX        |Alice    |15100     |0.5        |0.49

## Builtin Sections
Builtin sections are used as global configuration of DaLI behavior.

### Transaction Hints Section
The transaction_hints section is optional and it is used to force a transaction to be generated as in, out or intra. This is useful because DaLI doesn't know the user's intentions and so in certain cases it needs hints. For example an out transaction could be represented as a partial intra transaction (as described above) or as a normal out transaction describing a gift to another person: only the user knows the correct representation. In such cases a transaction_hint line can be used to clarify which option is preferred.

The format of this section is as follows:
<pre>
[transaction_hints]
<em>&lt;unique_id_1&gt;</em> = <em>&lt;direction&gt;</em>:<em>&lt;transaction_type&gt;</em>:<em>&lt;notes&gt;</em>
...
<em>&lt;unique_id_n&gt;</em> = <em>&lt;direction&gt;</em>:<em>&lt;transaction_type&gt;</em>:<em>&lt;notes&gt;</em>
</pre>

Where:
* *`unique_id`* is the unique_id of the transaction that needs to be recast;
* *`direction`* is denotes how to represent the transaction in generated file and it's one of: `in`, `out` or `intra`;
* *`transaction_type`* is the transaction type (see [Manual Section (CSV)](#trezor-section-csv) for more details on this field);
* *`notes`* is an optional English comment.

### Header Sections
There are three header sections (all of which are optional):
* in_header
* out_header
* intra_header

They are used to describe the format of the generated output file. They are populated with `<field> = <integer>`* assignments (the fields in each section are the same ones described in [Manual Section (CSV)](#trezor-section-csv)): this assigns a column to each field in the output file.

For example:
<pre>
[in_header]
timestamp = 0
asset = 1
exchange = 2
holder = 3
transaction_type = 4
spot_price = 6
crypto_in = 7
fiat_fee = 8
fiat_in_no_fee = 9
fiat_in_with_fee = 10
unique_id = 12
notes = 13

[out_header]
timestamp = 0
asset = 1
exchange = 2
holder = 3
transaction_type = 4
spot_price = 6
crypto_out_no_fee = 7
crypto_fee = 8
crypto_out_with_fee = 9
fiat_out_no_fee = 10
fiat_fee = 11
unique_id = 12
notes = 13

[in_header]
timestamp = 0
asset = 1
from_exchange = 2
from_holder = 3
to_exchange = 4
to_holder = 5
spot_price = 6
crypto_sent = 7
crypto_received = 8
unique_id = 12
notes = 13
</pre>
