Metadata-Version: 2.1
Name: fastspeed
Version: 0.1.1
Summary: Fastspeed is a toolkit for serving LLM.
Home-page: https://github.com/StudyExchange/fastspeed.git
Author: StudyExchange
Author-email: StudyExchange@163.com
License: MIT
Project-URL: Changelog, https://github.com/StudyExchange/fastspeed.git/blob/master/CHANGELOG.md
Project-URL: Source, https://github.com/StudyExchange/fastspeed.git
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi[all]
Requires-Dist: uvicorn[standard]
Requires-Dist: python-multipart
Requires-Dist: httpx
Requires-Dist: requests
Requires-Dist: torch
Requires-Dist: torchaudio
Requires-Dist: torchvision
Requires-Dist: huggingface
Requires-Dist: uvicorn
Requires-Dist: modelscope
Requires-Dist: transformers
Requires-Dist: sse-starlette

# Fastspeed
Fastspeed is a toolkit for serving LLM.

## Requirements
- Python >= 3.6

## Installation
```shell
pip install fastspeed
```

## Example
- server
```sh
fastspeed --model model_name_or_path  # default model_hub=modelscope
fastspeed --model model_name_or_path --model_hub hf
```

- client
```sh
curl -X 'POST' \
  'http://127.0.0.1:8000/chat/completions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ],
  "model": ""
}'
```


# Change Log

## 0.1.1 (2024-09-15)
- Suport half, cuda and trust_remote_code=True

## 0.1.0 (2024-08-29)
- Fastapi with single GPU, model from huggingface or modelscope
