# quollio-data-profiler

# Description
This system collects advanced metadata like table to table lineage or anomaly record and ingests them to QDC.


# Prerequisite
Before you begin to use this, you need to do the following.
- Add your assets to QDC with metadata agent.
- Issue External API client id and client secret on QDC.

# Install
Install with the following command.  
`pip install quollio-data-profiler`

# Usage
Here is an example of creating a view for snowflake lineage. Please enter any values for <>.  
```
from quollio_data_profiler.repository.qdc import QDCExternalAPIClient
from quollio_data_profiler.repository.snowflake import SnowflakeConnectionConfig
from quollio_data_profiler.snowflake_lineage_profiler import execute


def view_build_only():
    company_id = "<company id issued by quollio.>"
    build_view_connection = SnowflakeConnectionConfig(
            account_id="<Snowflake account id. Please use the same id of metadata agent.>",
            account_role="<Role necessary for creating view in your account>",
            account_user="<user name>",
            account_password="<password>",
            account_warehouse="<compute warehouse>", 
    )
    qdc_client = QDCExternalAPIClient(
        client_id="<client id issued on QDC.>",
        client_secret="<client secret>",
        base_url="<base endpoint>",
    )
    execute(
        company_id=company_id,
        sf_build_view_connections=build_view_connection,
        qdc_client=qdc_client,
        is_view_build_only=True,
    )

if __name__ == "__main__":
    view_build_only()
``` 
Please refer to the scripts in `./examples` for other usages.


# Development
## How to test
### Unittest
1. Run `make test`
