# Pegasus: A State-of-the-Art Multimodal Embedding Model

![Pegasus Banner](./assets/banner.png)

Welcome to Pegasus, the latest and most advanced model for creating high-quality embeddings from multimodal data. We're pushing the boundaries of what's possible with machine learning, enabling tasks and applications that were once mere visions of the future.

In essence, Pegasus is designed to transform the way we look at data. Our aim is to make it easier for anyone, regardless of their domain or discipline, to generate task-specific, high-quality embeddings from any type of data, be it text, image, video, audio, or even more complex data types.

## The Vision

When we began crafting Pegasus, we asked ourselves a fundamental question: What if an algorithm could see and understand all the modalities of data the same way we do, and even beyond?

The answer lies in Pegasus. Powered by a unique blend of artificial intelligence and machine learning, Pegasus is capable of parsing and comprehending multi-modal data types, generating valuable insights in the process.

Much like the mythical creature it's named after, Pegasus is all about bridging worlds - in our case, the world of raw, multi-modal data and the world of actionable, contextualized information.

## The Features

Pegasus is not just another run-of-the-mill machine learning model. We've painstakingly crafted this framework, ensuring it includes features that set it apart:

1. **Multimodal Data Understanding:** From text to images, audio, and more, Pegasus is designed to handle and understand a wide array of data types.

2. **Personalized for Any Task:** Pegasus adapts to your specific task, generating high-quality, task-specific embeddings for a wide variety of applications.

3. **Scalability & Performance:** Pegasus has been optimized for efficiency and can scale according to the demands of your tasks, ensuring seamless operation even with large amounts of data.

4. **Open Source:** We believe in the power of community and collaboration. Pegasus is an open-source project, welcoming contributions and improvements from the global developer community.

## Get Involved

We're just at the beginning of our journey. As we continue to develop and refine Pegasus, we invite you to join us. Whether you're a developer, researcher, or simply an enthusiast, your insights and contributions can help shape the future of Pegasus.

# Contributing to Pegasus

We are thrilled to invite you to be a part of the Pegasus project. This is not just an open source project but a community initiative, and we value your expertise and creativity. To show our appreciation, we have instituted a unique rewards system that directly compensates contributors from the revenue generated by the Pegasus API.

## Why Contribute

Contributing to Pegasus not only enhances your skills and profile but also comes with financial rewards. When you contribute code, documentation, or any form of improvement to the Pegasus project, you are adding value. As such, we believe it's only fair that you share in the rewards.

## Rewards Program

Here's how the Pegasus Rewards Program works:

1. **Submit a Pull Request:** This can be a code enhancement, bug fix, documentation update, new feature, or any improvement to the project.

2. **Review and Approval:** Our team will review your contribution. If it gets approved and merged, you become eligible for the rewards program.

3. **Revenue Share:** Once your pull request is merged, you will receive a percentage of the revenue generated by the Pegasus API. The percentage will be determined based on the significance and impact of your contribution. 

This means you're not just contributing to an open source project; you're becoming a part of the Pegasus ecosystem. Your efforts can yield ongoing benefits as the Pegasus API grows and evolves.

## Becoming a Paid API

As part of our growth strategy, we will be deploying Pegasus as a Paid API. The revenue generated from this API will not only sustain and further the project, but also fund the rewards program.

## How to Start Contributing

If you're ready to become a part of Pegasus and contribute to the future of multimodal embeddings, here's what you need to do:

1. Fork the repository.

2. Make your improvements or additions in your forked repository.

3. Submit a pull request detailing the changes you've made.

4. Our team will review your submission. If it's approved, it will be merged into the main repository, and you will become part of the Pegasus Rewards Program.

Thank you for considering contributing to Pegasus. Your expertise and commitment to this project are what make it thrive. Let's build the future of multimodal embeddings together.

# Pegasus: The Future of Multimodal Embeddings

Welcome to Pegasus, a state-of-the-art embedding model designed for a new era of multi-modality data understanding. We are at the forefront of AI research, and our mission is to provide high-quality, task-specific embeddings for any data type you're working with.

## Roadmap

Pegasus is a constant work in progress, and we're always striving for better. Our roadmap provides a snapshot of where we're heading. This iterative, continuous improvement process is fundamental to achieving our goal of creating the most effective SOTA omni-modality embeddings model. 

### Phase 1: Reconfiguring the ImageBind Model

To improve our handling of diverse data, we are reconfiguring the ImageBind model to utilize Flash Attention. This shift will allow us to manage longer context lengths and handle more complex inputs effectively.

### Phase 2: Pretraining with Diverse Datasets

Quality embeddings require quality training. To ensure our model is versatile and robust, we're pretraining Pegasus using the same datasets that ImageBind has been trained on. This step ensures our model inherits the benefits of proven training methodologies while also incorporating our enhancements.

### Phase 3: Benchmarking

It's important to know where we stand. After we've reconfigured and pretrained our model, we will conduct comprehensive benchmarking tests. This process will highlight any areas of strength or potential improvement, allowing us to further refine our model.

### Phase 4: Finetuning on Long Samples

We believe that Pegasus can handle more than short snippets of data. To prove this, we'll finetune our model using long data samples, pushing the boundaries of what's possible with embedding models.

### Phase 5: Continued Innovation

Our roadmap doesn't stop with finetuning. As we move forward, we're excited to explore new methodologies and techniques to enhance Pegasus. Potential areas of exploration include:

1. **Advanced Training Techniques:** We'll look into more sophisticated methods to make the training process faster and more efficient. 

2. **Expanding Modality Types:** We aim to support more types of modalities, ensuring that Pegasus is truly a universal tool for multi-modal data.


3. **Integration with More Frameworks:** We want Pegasus to be accessible and easy to use with popular machine learning and data processing frameworks.

4. **Optimizing for Real-Time Processing:** We're focused on making Pegasus capable of generating embeddings in real-time, a critical feature for many applications.

5. **Community Driven Enhancements:** We're excited to see what the community suggests and contributes - the possibilities are endless!

6. **Production-Level API Deployment:** Pegasus will enter Agora's paid API line up so you can effortlessly make API requests and recieve your embeddings no complicated setup necessary

As we continue our journey, we invite you to join us. Let's create the best SOTA omni-modality embeddings model together. Welcome to the future with Pegasus.



# Roadmap

* Make it extremely fast through quantization

* Parrelization, ASYNCHORNNY + other optimizations