Metadata-Version: 2.1
Name: pg2elastic
Version: 0.1.0
Summary: Enhanced PostgreSQL to Elasticsearch Data Synchronization
Home-page: https://github.com/robert-walters/pg2elastic
Author: Dorin Musteata RW
Author-email: dorin.musteata-contractor@robertwalters.com
Maintainer: Dorin Musteata RW
Maintainer-email: dorin.musteata-contractor@robertwalters.com
License: LGPLv3
Project-URL: Bug Reports, https://github.com/robert-walters/pg2elastic/issues
Project-URL: Source, https://github.com/robert-walters/pg2elastic
Keywords: pg2elastic,elasticsearch,postgres,change data capture
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Requires-Python: >=3.7.0
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: async-timeout (==4.0.2)
Requires-Dist: black (==23.7.0)
Requires-Dist: boto3 (==1.28.8)
Requires-Dist: botocore (==1.31.8)
Requires-Dist: certifi (==2023.7.22)
Requires-Dist: charset-normalizer (==3.2.0)
Requires-Dist: click (==8.1.6)
Requires-Dist: elasticsearch (==7.13.4)
Requires-Dist: elasticsearch-dsl (==7.4.1)
Requires-Dist: environs (==9.5.0)
Requires-Dist: faker (==19.2.0)
Requires-Dist: greenlet (==2.0.2)
Requires-Dist: idna (==3.4)
Requires-Dist: isort (==5.11.5)
Requires-Dist: jmespath (==1.0.1)
Requires-Dist: marshmallow (==3.20.1)
Requires-Dist: mypy-extensions (==1.0.0)
Requires-Dist: opensearch-dsl (==2.1.0)
Requires-Dist: opensearch-py (==2.2.0)
Requires-Dist: packaging (==23.1)
Requires-Dist: pathspec (==0.11.1)
Requires-Dist: platformdirs (==3.9.1)
Requires-Dist: psycopg2-binary (==2.9.6)
Requires-Dist: python-dateutil (==2.8.2)
Requires-Dist: python-dotenv (==0.21.1)
Requires-Dist: redis (==4.6.0)
Requires-Dist: requests (==2.31.0)
Requires-Dist: requests-aws4auth (==1.2.3)
Requires-Dist: s3transfer (==0.6.1)
Requires-Dist: six (==1.16.0)
Requires-Dist: sqlalchemy (==1.4.49)
Requires-Dist: sqlparse (==0.4.4)
Requires-Dist: tomli (==2.0.1)
Requires-Dist: typing-extensions (==4.7.1)
Requires-Dist: urllib3 (==1.26.16)
Requires-Dist: loguru (==0.7.0)

# ⚡ pg2elastic

Enhanced PostgreSQL to Elasticsearch Data Synchronization

### 📚 Description

pg2elastic is fork of official [pgsync](https://pgsync.com/). Everything that pgsync got, pg2elastic got too.

Welcome to pg2elastic, a fork of the official [pgsync](https://pgsync.com/) package, designed to provide seamless and
efficient data synchronization between PostgreSQL databases and Elasticsearch clusters.
Building upon the solid foundation of pgsync, pg2elastic inherits all of its powerful capabilities and takes them a step
further.

#### Key Features:

* High-Performance Sync: pg2elastic inherits the robust data synchronization engine from pgsync, ensuring lightning-fast
  and reliable transfers.
* Real-time Indexing: Seamlessly mirror your PostgreSQL data into Elasticsearch indices, keeping them in sync in
  real-time.
* Schema Mapping: Easily define and customize the mapping of PostgreSQL schemas to Elasticsearch indexes, giving you
  full control over the data structure.
* Efficient Data Types Handling: pg2elastic effortlessly handles data type conversions, ensuring accurate representation
  across platforms.
* Continuous Enhancements: We are committed to actively maintaining and enhancing pg2elastic, incorporating the latest
  advancements in both PostgreSQL and Elasticsearch technologies.
* Whether you're working on a data-driven application or performing complex data analysis, pg2elastic empowers you with
  a streamlined and feature-rich solution for harmonizing your PostgreSQL and Elasticsearch ecosystems.

---

### 🛠️ Prerequisites

[PGSync Requirements](https://github.com/toluaina/pgsync#requirements)

---

### ✨ Key Enhancements

* [Loguru, Better Logging Module](https://github.com/Delgan/loguru)
* `PG_SCHEMA` environment variable to enhance performance by eliminating the need to scan all schemas
* `REDIS_USERNAME` environment variable to specify redis username
* `REDIS_PASSWORD` environment variable to specify redis password
* `REDIS_ENDPOINT` environment variable to specify redis connection endpoint

---

### 🚀 Deployment

#### Manual Deployment

You need to run `pg2elastic` command in order to initialize it.

- Create a .env file using the `cp .env.sample .env` command and replace the existing environment variables with
  personal configuration settings.

- Download dependencies using `python setup.py develop`

- Start the app in pre-production mode by using `pg2elastic` command for development

If you do not run the full setup, you will get errors when running this package.

---

### ✅ Testing

```bash
$ export PG_SCHEMA=
$ flake8 pg2elastic tests
$ python setup.py test
```

---

### 🔊 Logs

This project comes with a [loguru](https://github.com/Delgan/loguru) module for logging, the configurations
for loguru can be found in `pg2elastic` bin file.

---

### 🚚 Deployment

```bash
$ python setup.py sdist bdist_wheel
$ twine upload dist/*
```

---
