Metadata-Version: 2.2
Name: tk_core
Version: 1.1.9
Summary: Centralized core functionality for Terakeet SD, DS, DE, etc.
Description-Content-Type: text/markdown
Requires-Dist: boto3
Requires-Dist: botocore
Requires-Dist: pandas==2.1.4
Requires-Dist: pydantic
Requires-Dist: python-dotenv
Requires-Dist: scrapingbee
Requires-Dist: serpapi
Requires-Dist: snowflake-connector-python[pandas]
Requires-Dist: snowflake-snowpark-python[pandas]
Requires-Dist: ujson
Requires-Dist: redis
Requires-Dist: retry
Requires-Dist: rich
Requires-Dist: trafilatura==1.7.0
Requires-Dist: extruct
Requires-Dist: beautifulsoup4
Requires-Dist: newspaper3k
Requires-Dist: opengraph
Requires-Dist: jsonschema
Requires-Dist: psutil
Requires-Dist: tqdm
Requires-Dist: lxml==5.1.0
Provides-Extra: dev
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"

# tk-core

[![PyPI Latest Release](https://img.shields.io/pypi/v/tk-core.svg)](https://pypi.org/project/tk-core/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

Centralized core functionality for Terakeet SD, DS, DE, etc.

## Installation Steps

Installation of the tk-core package requires a github personal access token (PAT) and the selectin of which branch you would like to install.

### Github Personal Access Token (PAT)

This is the authentication method used to install the tk-core library. To create a github PAT please follow the steps below. The steps are listed in the <a href=https://docs.github.com/en/enterprise-server@3.9/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens>github doucmentation</a>.

1. Log into GitHub
2. Click on your profile picture, then click “Settings”
3. Scroll to the bottom of the settings and select “<> Developer Settings”
4. Select “Personal Access Tokens” to drop down the menu
5. Select “Tokens” (classic)
6. Select “Generate new token” then “Generate new token (classic)”
7. Fill out the Note, Expiration, and Scope. 
    a. The only permissions needed are “Repo”
8. Scroll to the bottom, select “Generate Token”
9. Copy the token once generated
10. Paste into your code editor environment variable as “GITHUB_PAT”
    a. GITHUB_PAT=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

### pip install

Once you have the access token in your .env file of your environment you can pip install the library. Because of the access token, you should not need to put in a password.

```
pip install git+https://github.com/terakeet/tk-core.git@<branch>

# Example
pip install git+https://github.com/terakeet/tk-core.git@staging
```

# APIs

## Current

- [common](src/tk_core/common/README.md)
  - anything you find yourself using frequently could be added here
- [serp_api](src/tk_core/serp_api/README.md)
  - anything related to <a href=https://serpapi.com>SERPAPI</a>
  - there is a base class for all Google endpoints: `serp_api.base.py:SERPAPI`
- [snowflake](src/tk_core/snowkeet/README.md)
  - anything related to snowflake
- [core](src/tk_core/core/README.md)
  - functionality that is core to the other sub-modules in the package
- [async module](src/tk_core/core/async_module/README.md)
  - helper functions and async functionality that is core to the other sub-modules in the package

## Future

- google
- open_ai
- semrush

## Examples

Each sub-packages should have their own directory inside the `examples` directory. These will be built out over time to help (along with the documentation) understand functionality and common use cases for the tk-core package.

# Project Structure

```
tk-core
|
├── src
│   └── tk_core
│       ├── common
│       │   ├── de_service.py
│       │   ├── dictionary.py
│       │   ├── files.py
│       │   ├── hasher.py
│       │   └── s3.py
|       |   |__ timeout.py
|       |
│       ├── core
│       │   └── models.py
|       |
│       ├── google (future)
│       │   ├── gsc.py
│       │   ├── ga.py
│       │   ├── sheets.py
│       │   ├── calendar.py
│       │   └── gmail.py
|       |
│       ├── open_ai (future)
|       |
│       ├── semrush (future)
|       |
│       ├── serp_api
│       │   ├── README.md
│       │   ├── base.py
│       │   ├── batch_serp.py
│       │   ├── format_for_snowflake.py
│       │   ├── models.py
│       │   ├── serp.py
│       │   ├── trend.py
│       │   └── util.py
|       |
│       └── snowflake
│       │   ├── error_wrapper.py
│       │   ├── snowkeet_new.py
│           └── snowkeet.py
└── test
    ├── common
    ├── core
    └── serpapi
|
├── README.md
├── .coveragerc
├── .pre-commit-config.yaml
├── pyproject.toml
├── requirements.txt
├── requirements-dev.txt
```

# Test PYPI

In case you need to development version, it can be found here.
[![TEST PyPI Latest Release](https://img.shields.io/pypi/v/tk-core.svg)](https://test.pypi.org/project/tk-core/)

# Questions, Concerns, Bugs

Clone the repo, create a PR and give it a shot your self. Make sure to write some tests--or update the existing ones--with any changing functionality. Feel free to reach out to the engineering team for help.
