Metadata-Version: 2.1
Name: fastxtend
Version: 0.1.5
Summary: Train fastai models faster (and other useful tools)
Home-page: https://github.com/warner-benjamin/fastxtend
Author: Benjamin Warner
Author-email: me@benjaminwarner.dev
License: MIT License
Project-URL: Documentation, https://fastxtend.benjaminwarner.dev/
Keywords: fastai pytorch extensions
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas >=2.0.3
Requires-Dist: fastcore >=1.5.27
Requires-Dist: fastai >=2.7.9
Requires-Dist: torch >=1.13.1
Provides-Extra: all
Requires-Dist: timm >=0.9.5 ; extra == 'all'
Requires-Dist: kornia >=0.7.0 ; extra == 'all'
Requires-Dist: ffcv >=1.0.2 ; extra == 'all'
Requires-Dist: librosa >=0.10.1 ; extra == 'all'
Requires-Dist: torchaudio >=0.13.1 ; extra == 'all'
Requires-Dist: primePy >=1.3 ; extra == 'all'
Requires-Dist: colorednoise >=2.0 ; extra == 'all'
Requires-Dist: soundfile >=0.10.3 ; extra == 'all'
Requires-Dist: matplotlib <3.8 ; extra == 'all'
Requires-Dist: transformers >=4.32.0 ; extra == 'all'
Requires-Dist: datasets >=2.14.3 ; extra == 'all'
Requires-Dist: wandb >=0.15.10 ; extra == 'all'
Requires-Dist: rich >=13.6.0 ; extra == 'all'
Requires-Dist: typer >=0.9.0 ; extra == 'all'
Requires-Dist: bitsandbytes >=0.41.0 ; extra == 'all'
Provides-Extra: audio
Requires-Dist: librosa >=0.10.1 ; extra == 'audio'
Requires-Dist: torchaudio >=0.13.1 ; extra == 'audio'
Requires-Dist: primePy >=1.3 ; extra == 'audio'
Requires-Dist: colorednoise >=2.0 ; extra == 'audio'
Requires-Dist: soundfile >=0.10.3 ; extra == 'audio'
Requires-Dist: matplotlib <3.8 ; extra == 'audio'
Provides-Extra: dev
Requires-Dist: timm >=0.9.5 ; extra == 'dev'
Requires-Dist: kornia >=0.7.0 ; extra == 'dev'
Requires-Dist: ffcv >=1.0.2 ; extra == 'dev'
Requires-Dist: librosa >=0.10.1 ; extra == 'dev'
Requires-Dist: torchaudio >=0.13.1 ; extra == 'dev'
Requires-Dist: primePy >=1.3 ; extra == 'dev'
Requires-Dist: colorednoise >=2.0 ; extra == 'dev'
Requires-Dist: soundfile >=0.10.3 ; extra == 'dev'
Requires-Dist: matplotlib <3.8 ; extra == 'dev'
Requires-Dist: transformers >=4.32.0 ; extra == 'dev'
Requires-Dist: datasets >=2.14.3 ; extra == 'dev'
Requires-Dist: wandb >=0.15.10 ; extra == 'dev'
Requires-Dist: rich >=13.6.0 ; extra == 'dev'
Requires-Dist: typer >=0.9.0 ; extra == 'dev'
Requires-Dist: bitsandbytes >=0.41.0 ; extra == 'dev'
Requires-Dist: nbdev >=2.3.12 ; extra == 'dev'
Provides-Extra: ffcv
Requires-Dist: timm >=0.9.5 ; extra == 'ffcv'
Requires-Dist: kornia >=0.7.0 ; extra == 'ffcv'
Requires-Dist: ffcv >=1.0.2 ; extra == 'ffcv'
Provides-Extra: text
Requires-Dist: transformers >=4.32.0 ; extra == 'text'
Requires-Dist: datasets >=2.14.3 ; extra == 'text'
Provides-Extra: vision
Requires-Dist: timm >=0.9.5 ; extra == 'vision'
Requires-Dist: kornia >=0.7.0 ; extra == 'vision'

# fastxtend

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

### Train fastai models faster (and other useful tools)

![fastxtend accelerates fastai](nbs/images/imagenette_benchmark.png)

Train fastai models faster with fastxtend’s [fused
optimizers](optimizer.fused.html), [Progressive
Resizing](callback.progresize.html) callback, and integrated [FFCV
DataLoader](ffcv.tutorial.html).

## Feature overview

**Train Models Faster**

- Drop in [fused optimizers](optimizer.fused.html), which are 21 to 293
  percent faster then fastai native optimizers.
- Up to 75% optimizer memory savings with integrated
  [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) [8-bit
  optimizers](optimizer.eightbit.html).
- Increase GPU throughput and decrease training time with the
  [Progressive Resizing](callback.progresize.html) callback.
- Use the highly optimized [FFCV DataLoader](ffcv.tutorial.html), fully
  integrated with fastai.
- Integrated support for `torch.compile` via the
  [Compile](callback.compiler.html) callbacks.

**General Features**

- Fused implementations of modern optimizers, such as
  [Adan](optimizer.adan.html), [Lion](optimizer.lion.html), &
  [StableAdam](optimizer.stableadam.html).
- Hugging Face [Transformers compatibility](text.huggingface.html) with
  fastai
- Flexible [metrics](metrics.html) which can log on train, valid, or
  both. Backwards compatible with fastai metrics.
- Easily use [multiple losses](multiloss.html) and log each individual
  loss on train and valid.
- [Multiple profilers](callback.profiler.html) for profiling training
  and identifying bottlenecks.
- A fast [Exponential Moving Average](callback.ema.html) callback for
  smoother training.

**Vision**

- Apply
  [`MixUp`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#mixup),
  [`CutMix`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#cutmix),
  or Augmentations at once with
  [`CutMixUp`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#cutmixup)
  or
  [`CutMixUpAugment`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#cutmixupaugment).
- Additional [image augmentations](vision.augment.batch.html).
- Support for running fastai [batch transforms on
  CPU](vision.data.html).
- More [attention](vision.models.attention_modules.html) and
  [pooling](vision.models.pooling.html) modules
- A flexible implementation of fastai’s
  [`XResNet`](https://fastxtend.benjaminwarner.dev/vision.models.xresnet.html#xresnet).

Check out the documentation for additional splitters, callbacks,
schedulers, utilities, and more.

## Documentation

<https://fastxtend.benjaminwarner.dev>

## Install

fastxtend is avalible on pypi:

``` bash
pip install fastxtend
```

fastxtend can be installed with task-specific dependencies for `vision`,
`ffcv`, `text`, `audio`, or `all`:

``` bash
pip install "fastxtend[all]"
```

To easily install most prerequisites for all fastxtend features, use
[Conda](https://docs.conda.io/en/latest) or
[Miniconda](https://docs.conda.io/en/latest/miniconda.html):

``` bash
conda create -n fastxtend python=3.11 "pytorch>=2.1" torchvision torchaudio \
pytorch-cuda=12.1 fastai nbdev pkg-config libjpeg-turbo opencv tqdm psutil \
terminaltables numpy "numba>=0.57" librosa timm kornia rich typer wandb \
"transformers>=4.34" "tokenizers>=0.14" "datasets>=2.14" ipykernel ipywidgets \
"matplotlib<3.8" -c pytorch -c nvidia -c fastai -c huggingface -c conda-forge

conda activate fastxtend

pip install "fastxtend[all]"
```

replacing `pytorch-cuda=12.1` with your preferred [supported version of
Cuda](https://pytorch.org/get-started/locally).

To create an editable development install:

``` bash
git clone https://github.com/warner-benjamin/fastxtend.git
cd fastxtend
pip install -e ".[dev]"
```

## Usage

Like fastai, fastxtend provides safe wildcard imports using python’s
`__all__`.

``` python
from fastai.vision.all import *
from fastxtend.vision.all import *
from fastxtend.ffcv.all import *
```

In general, import fastxtend after all fastai imports, as fastxtend
modifies fastai. Any method modified by fastxtend is backwards
compatible with the original fastai code.

## Examples

Use a fused ForEach optimizer:

``` python
Learner(..., opt_func=adam(foreach=True))
```

Or a bitsandbytes 8-bit optimizer:

``` python
Learner(..., opt_func=adam(eightbit=True))
```

Speed up image training using Progressive Resizing:

``` python
Learner(... cbs=ProgressiveResize())
```

Log an accuracy metric on the training set as a smoothed metric and
validation set like normal:

``` python
Learner(..., metrics=[Accuracy(log_metric=LogMetric.Train, metric_type=MetricType.Smooth),
                      Accuracy()])
```

Log multiple losses as individual metrics on train and valid:

``` python
mloss = MultiLoss(loss_funcs=[nn.MSELoss, nn.L1Loss],
                  weights=[1, 3.5], loss_names=['mse_loss', 'l1_loss'])

Learner(..., loss_func=mloss, metrics=RMSE(), cbs=MultiLossCallback)
```

Compile a model with `torch.compile`:

``` python
from fastxtend.callback import compiler

learn = Learner(...).compile()
```

Profile a fastai training loop:

``` python
from fastxtend.callback import simpleprofiler

learn = Learner(...).profile()
learn.fit_one_cycle(2, 3e-3)
```

## Benchmark

To run the benchmark on your own machine, see the [example
scripts](https://github.com/warner-benjamin/fastxtend/tree/main/examples)
for details on how to replicate.
