# Design Decisions

!!! note

    This section is not necessary for users of the library to understand.

    If you are looking to contribute, or curious about why things work the way they do, read on.

## Design Goals

In order of importance:

1. Work as a drop-in replacement for `httpx.Client`, so existing code can gain the benefits of the library without being changed.
2. Preserve the 100% typed interface of `httpx`.
3. Make writing & testing new augmentations easier.
4. Organize code in a way that'll make augmenting future libraries (or `httpx.AsyncClient`) easier. (Based on limitations in porting `scrapelib` that made a partial rewrite easier than porting it from `requests`.)

## PluginClient

Each "suite" of behavior: throttling, retries, caching, robots.txt checking-- consists of a plugin that implements a simple protocol:

```python
class RequestCallable(Protocol):
    def __call__(self, method: str, url: str, **kwargs) -> Self:
        ...

class Plugin(Protocol):
    # a callable that takes (~a version of itself~) and returns another
    wrap: Callable[[RequestCallable, PluginClient], RequestCallable]
    _client: PluginClient | None
```

These plugins define a `wrap` method, which returns a function with the `RequestCallable` signature.

The function takes two arguments, the next callable in the chain, and the underlying `client` itself.

If you'd like to follow along, `throttle.py` is the simplest of them, at around 20 lines long.

You'll see the plugin contains a constructor and a `wrap` function.

```python
def wrap(self, next_handler, client):
    # this inner function will wrap/replace the current _request
    def new_handler(method, url, **kwargs):
        # Plugin logic is placed within this function.
        # unless the request is being blocked for some reason, the handler
        # should call `next_handler`
        # this gives the opportunity to modify parameters and/or responses
        return next_handler(method, url, **kwargs)
    return new_handler
```

Each feature is implemented in a similar way.

The recommended `areful.httpx.make_careful_client` entrypoint is
just a convinient combination of these `make_ZZZ_ciient` functions.

The rest of this document describes *why* this choice was made.

## Why not inheritance?

For 15 years `scrapelib` used a class hierarchy to a similar end, it'd certainly work here too.

The equivalent to a careful client in `scrapelib` is a `scrapelib.Scraper`.
It has a long inheritance hierarchy:

`scrapelib.Scraper` -> `CachingSession` -> `ThrottledSession`  -> `RetrySession` -> `requests.Session`

This hierarchy means that there is no such thing as a `CachingSession` that doesn't use throttling,
and adding new behavior means considering exactly where it works best in the chain and then setting that in stone.

There's arguably no benefit derived from having things set up this way.
It is just too annoying to mix & match behaviors, or add new ones, a single class would have been easier to maintain over the years.

## Why not mixins?

So, we don't want to just give up and go with a single monolithic class... why not [mixins](https://en.wikipedia.org/wiki/Mixin)?

It seems like we could have `ThrottledMixin`, `RetryMixin`, `DevCacheMixin`, etc.

Then to use retry & cache together someone would:

```python
import httpx
from careful.hypothetical import ThrottledMixin, RetryMixin, DevCacheMixin

class CustomClient(RetryMixin, DevCacheMixin, Client):
  pass

client = CustomClient()
```

Honestly, not a great start: having to declare an empty class, and to carefully think about method resolution order rules in ordering them.

But there's a bigger problem lurking here... configuration.

Assume each mixin is configured through its constructor:

```python

class RetryMixin:
  def __init__(self, num_retries=2, retry_delay_seconds=10, **kwargs):
    ...


class DevCacheMixin:
  def __init__(self, cache_backend=..., should_cache=..., **kwargs):
    ...
```

To make this work properly, our custom class needs a constructor too.
It'd wind up looking like:

```python
  def __init__(self, num_retries, retry_delay_seconds, cache_backend, should_cache, ...)
      # Initialize mixins explicitly
      RetryMixin.__init__(self,  num_retries=num_retries,
                          retry_delay_seconds=retry_delay_seconds)

      DevCacheMixin.__init__(self, cache_backend=cache_backend,
                             should_cache=should_cache)

      Client.__init__(self, **kwargs)
```

This makes working with the mixins frustrating, since any new combination requires modifying a repetitive constructor.

This led to a composition-over-inheritance choice, using plugin classes with a helpful `make_careful_client` and `make_careful_client_from_env` for the  most common cases.
