Metadata-Version: 2.1
Name: agent_search
Version: 0.0.1
Summary: LocalSearch: An open source framework and dataset for webscale local search.
License: Apache-2.0
Author: Owen Colegrove
Author-email: owen@sciphi.ai
Requires-Python: >=3.9,<3.12
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: configparser (>=5.0.0,<6.0.0)
Requires-Dist: fire (>=0.5.0,<0.6.0)
Requires-Dist: numpy (>=1.25.2,<2.0.0)
Requires-Dist: pandas (>=2.1.2,<3.0.0)
Requires-Dist: pydantic (>=1.10.13,<2.0.0)
Requires-Dist: pysqlite3 (==0.5.2)
Requires-Dist: qdrant_client (>=1.7.0,<2.0.0)
Requires-Dist: requests (>=2.31.0,<3.0.0)
Requires-Dist: transformers (>=4.33.1,<5.0.0)
Description-Content-Type: text/markdown

# AgentSearch [ΨΦ]: A Comprehensive Agent-First Framework and Dataset for Webscale Search

AgentSearch is a powerful new tool that allows you to operate a webscale search engine locally, catering to both Large Language Models (LLMs) and human users. This open-source initiative provides access to over one billion high-quality embeddings sourced from a wide array of content, including selectively filtered Creative Commons data and the entirety of Arxiv, Wikipedia, and Project Gutenberg.

## Features of AgentSearch

- **Gated Access**: Controlled and secure access to the search engine, ensuring data integrity and privacy.
- **Offline Support**: Ability to operate in a fully offline environment.
- **Customizable**: Upload your own local data or tailor the provided datasets according to your needs.
- **API Endpoint**: AgentSearch offers a fully managed access through a dedicated API, facilitating easy and efficient integration into various workflows.

## Quickstart Guide for AgentSearch

Follow this guide for a streamlined setup and demonstration of the AgentSearch project.

### Prerequisites

Make sure Docker is installed on your system. If not, download and install it from [Docker's official website](https://www.docker.com/).

### Quick Setup

1. Install the AgentSearch client by executing:

   ```shell
   git clone https://github.com/AgentSearch-AI/agent-search.git && cd agent-search
   pip install -e .
   ```

### Running a Query

- To perform a query and witness AgentSearch in action, use:

  ```shell
  python agent_search/script/run_query.py query --query="What is Fermat's last theorem?"
  ```

  Note that this command assumes you have followed the steps below to launch your local agent-first search engine. For remote access to our search engine, please register for a free API key at [AgentSearch](https://www.sciphi.ai/).

### Local Setup and Initialization

1. **Database Population**:
   - Populate the SQLite database with this command:

     ```shell
     python agent_search/script/populate_dbs.py populate_sqlite
     ```

     This creates a SQLite database `open_web_search.db` in the `data` directory. This script can be readily adopted to your own bespoke datasets. For a direct installation of the 1TB data into the database, please use [insert link].

2. **Start Qdrant (vector database) Service with Docker**:
   - Run Qdrant service in a Docker container with this command, which sets up the necessary ports and storage:

     ```shell
     docker run -p 6333:6333 -p 6334:6334 \
         -v $(pwd)/qdrant_storage:/qdrant/storage:z \
         qdrant/qdrant
     ```

     For installation guidance on Qdrant, refer to [their documentation](https://qdrant.tech/documentation/quick-start/).

3. **Run the Server**:
   - Launch the AgentSearch server:

     ```shell
     python agent_search/app/server.py
     ```

### Additional Notes

- Run all commands from the root directory of the AgentSearch project.
- Replace the `query` in the run command with your desired search query.
