This file is a merged representation of a subset of the codebase, containing files not matching ignore patterns, combined into a single document by Repomix.

================================================================
File Summary
================================================================

Purpose:
--------
This file contains a packed representation of the entire repository's contents.
It is designed to be easily consumable by AI systems for analysis, code review,
or other automated processes.

File Format:
------------
The content is organized as follows:
1. This summary section
2. Repository information
3. Directory structure
4. Multiple file entries, each consisting of:
  a. A separator line (================)
  b. The file path (File: path/to/file)
  c. Another separator line
  d. The full contents of the file
  e. A blank line

Usage Guidelines:
-----------------
- This file should be treated as read-only. Any changes should be made to the
  original repository files, not this packed version.
- When processing this file, use the file path to distinguish
  between different files in the repository.
- Be aware that this file may contain sensitive information. Handle it with
  the same level of security as you would the original repository.

Notes:
------
- Some files may have been excluded based on .gitignore rules and Repomix's configuration
- Binary files are not included in this packed representation. Please refer to the Repository Structure section for a complete list of file paths, including binary files
- Files matching these patterns are excluded: **/.specstory, **/history, **/_private, **/.venv, **/.git, **/REPO_CONTENT.txt, **/CLEANUP.txt, **/uv.lock, **/dev
- Files matching default ignore patterns are excluded

Additional Info:
----------------

================================================================
Directory Structure
================================================================
.cursor/
  rules/
    0project.mdc
    cleanup.mdc
    filetree.mdc
    quality.mdc
.github/
  workflows/
    push.yml
    release.yml
.ruff_cache/
  .gitignore
  CACHEDIR.TAG
plugins/
  repos/
    twat_audio/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_audio/
          __version__.py
          twat_audio.py
      tests/
        test_twat_audio.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      package.toml
      pyproject.toml
      README.md
      VERSION.txt
    twat_cache/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .idea/
        inspectionProfiles/
          profiles_settings.xml
        .gitignore
        misc.xml
        modules.xml
        ruff.xml
        twat_cache.iml
        vcs.xml
        workspace.xml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      docs/
        context_management.md
      examples/
        backend_selection.py
      src/
        twat_cache/
          engines/
            __init__.py
            aiocache.py
            base.py
            cachebox.py
            cachetools.py
            common.py
            diskcache.py
            functools_engine.py
            functools.py
            joblib.py
            klepto.py
            manager.py
            py.typed
            redis.py
          types/
            cachebox.pyi
          __init__.py
          __main__.py
          __version__.py
          backend_selector.py
          cache.py
          config.py
          context.py
          decorators.py
          exceptions.py
          hybrid_cache.py
          logging.py
          paths.py
          py.typed
          type_defs.py
          utils.py
        twat_cache.egg-info/
          dependency_links.txt
          entry_points.txt
          PKG-INFO
          requires.txt
          SOURCES.txt
          top_level.txt
      tests/
        __init__.py
        test_backend_selector.py
        test_cache.py
        test_config.py
        test_constants.py
        test_context_simple.py
        test_context.py
        test_decorators.py
        test_engines.py
        test_exceptions_simple.py
        test_exceptions.py
        test_fallback.py
        test_redis_cache.py
        test_twat_cache.py
      .gitignore
      .pre-commit-config.yaml
      CHANGELOG.md
      cleanup.py
      LICENSE
      NEXT.md
      PROMPT.txt
      pyproject.toml
      README.md
      TODO.md
      VERSION.txt
    twat_coding/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      cursor/
        rules/
          0project.mdc
      dist/
        .gitignore
        .gitkeep
      src/
        twat_coding/
          pystubnik/
            backends/
              __init__.py
              ast_backend.py
              base.py
              mypy_backend.py
            core/
              config.py
              conversion.py
              shared_types.py
              types.py
              utils.py
            processors/
              __init__.py
              docstring.py
              file_importance.py
              importance.py
              imports.py
              stub_generation.py
              type_inference.py
            types/
              docstring.py
              type_system.py
            utils/
              ast_utils.py
              display.py
              memory.py
            __init__.py
            cli.py
            config.py
            errors.py
            read_imports.py
            README.md
          __init__.py
          __version__.py
          twat_coding.py
      tests/
        test_package.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      MANIFEST.in
      mypy.ini
      package.toml
      pyproject.toml
      README.md
      TODO.md
      VERSION.txt
    twat_ez/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_ez/
          __init__.py
          __version__.py
          py_needs.py
      tests/
        test_twat_ez.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      pyproject.toml
      README.md
      VERSION.txt
    twat_fs/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      examples/
        upload_example.py
      src/
        twat_fs/
          upload_providers/
            __init__.py
            async_utils.py
            bashupload.py
            catbox.py
            core.py
            dropbox.py
            factory.py
            fal.py
            filebin.py
            litterbox.py
            pixeldrain.py
            protocols.py
            s3.py
            simple.py
            types.py
            uguu.py
            utils.py
            www0x0.py
          __init__.py
          __main__.py
          __version__.py
          cli.py
          py.typed
          upload.py
      templates/
        authenticated_provider_template.py
        simple_provider_template.py
      tests/
        data/
          test.txt
        __init__.py
        test_async_utils.py
        test_filebin_pixeldrain.py
        test_integration.py
        test_s3_advanced.py
        test_twat_fs.py
        test_upload.py
        test_utils.py
      .gitignore
      .pre-commit-config.yaml
      CHANGELOG.md
      cleanup.py
      IDEAS.md
      LICENSE
      MANIFEST.in
      mypy.ini
      pyproject.toml
      README.md
      TODO.md
      update_providers.py
      VERSION.txt
    twat_genai/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_genai/
          core/
            config.py
            image.py
            models.py
            prompt.py
          engines/
            fal/
              __init__.py
              client.py
              config.py
              lora.py
              models.py
            base.py
          __init__.py
          __main___loras.json
          __main__.py
          __version__.py
          cli.py
      tests/
        test_twat_genai.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      pyproject.toml
      README.md
      VERSION.txt
    twat_hatch/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_hatch/
          themes/
            _shared/
              snippets/
                author.toml.j2
                dependencies.toml.j2
                development.toml.j2
                features.toml.j2
                package.toml.j2
                tools.toml.j2
              base.toml.j2
            default/
              .github/
                workflows/
                  push.yml.j2
                  release.yml.j2
              dist/
                .gitkeep.j2
              tests/
                test_package.py.j2
              .pre-commit-config.yaml.j2
              hidden.gitignore.j2
              LICENSE.j2
              pyproject.toml.j2
              README.md.j2
            package/
              src/
                __package_name__/
                  __package_name__.py.j2
              package.toml.j2
            plugin/
              src/
                __package_name__/
                  __init__.py.j2
              pyproject.toml.j2
              README.md.j2
            plugin_host/
              src/
                __init__.py.j2
              pyproject.toml.j2
              README.md.j2
            __init__.py
            package.toml.j2
            plugin_host.toml.j2
            plugin.toml.j2
          __init__.py
          __main__.py
          __version__.py
          config.py
          hatch.py
          utils.py
      tests/
        test_twat_hatch.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      pyproject.toml
      README.md
      TODO.md
      VERSION.txt
    twat_image/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_image/
          __init__.py
          __version__.py
          gray2alpha.py
      tests/
        test_twat_image.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      pyproject.toml
      README.md
      VERSION.txt
    twat_labs/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_labs/
          __init__.py
          __version__.py
      tests/
        test_twat_labs.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      pyproject.toml
      README.md
      VERSION.txt
    twat_llm/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      docs/
        people-api-tldr.md
        people-api.md
        people.py
        review-copilot.md
        review-cursor.md
        review-o3.md
        review-trae.md
        serp.py
        web-search-api.md
      src/
        twat_llm/
          __version__.py
          twat_llm.py
        funchain.py
        llm_plugins.py
        mallmo.py
      tests/
        test_twat_llm.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      package.toml
      pyproject.toml
      README.md
      VERSION.txt
    twat_mp/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      docs/
        architecture.md
      src/
        twat_mp/
          __init__.py
          __version__.py
          async_mp.py
          mp.py
      tests/
        test_async_mp.py
        test_benchmark.py
        test_twat_mp.py
      .gitignore
      .pre-commit-config.yaml
      API_REFERENCE.md
      CHANGELOG.md
      cleanup.py
      LICENSE
      pyproject.toml
      README.md
      TODO.md
      VERSION.txt
    twat_os/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
        .gitkeep
      src/
        twat_os/
          __init__.py
          __version__.py
          paths.py
          paths.toml
      tests/
        test_package.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      pyproject.toml
      README.md
      VERSION.txt
    twat_search/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .idea/
        inspectionProfiles/
          profiles_settings.xml
        .gitignore
        misc.xml
        modules.xml
        ruff.xml
        twat_search.iml
        vcs.xml
        workspace.xml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      debug_output/
        qwant_analysis.txt
        qwant_content.html
        yahoo_analysis.txt
        yahoo_content.html
      dist/
        .gitignore
      resources/
        brave/
          brave_image.md
          brave_news.md
          brave_video.md
          brave.md
        pplx/
          pplx_urls.txt
          pplx.md
        you/
          you_news.md
          you_news.txt
          you.md
          you.txt
        pricing.md
      src/
        twat_search/
          web/
            engines/
              lib_falla/
                core/
                  __init__.py
                  aol.py
                  ask.py
                  bing.py
                  dogpile.py
                  duckduckgo.py
                  falla.py
                  fetch_page.py
                  gibiru.py
                  google.py
                  mojeek.py
                  qwant.py
                  searchencrypt.py
                  startpage.py
                  yahoo.py
                  yandex.py
                __init__.py
                main.py
                requirements.txt
                settings.py
                utils.py
              __init__.py
              base.py
              bing_scraper.py
              brave.py
              critique.py
              duckduckgo.py
              falla.py
              google_scraper.py
              hasdata.py
              pplx.py
              serpapi.py
              tavily.py
              you.py
            __init__.py
            api.py
            cli.py
            config.py
            engine_constants.py
            exceptions.py
            models.py
            utils.py
          __init__.py
          __main__.py
          __version__.py
      tests/
        unit/
          web/
            engines/
              __init__.py
              test_base.py
            __init__.py
            test_api.py
            test_config.py
            test_exceptions.py
            test_models.py
            test_utils.py
          __init__.py
          mock_engine.py
        web/
          test_bing_scraper.py
        conftest.py
        test_twat_search.py
      .gitignore
      .pre-commit-config.yaml
      CHANGELOG.md
      cleanup.py
      debug_fetch.py
      falla_search.py
      google_debug_Python_programming_language.html
      google_debug_test_query.html
      LICENSE
      pyproject.toml
      README.md
      requirements.txt
      test_async_falla.py
      test_falla.py
      test_google_falla_debug.py
      test_simple.py
      test_sync_falla.py
      TODO.md
      VERSION.txt
    twat_speech/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_speech/
          __version__.py
          twat_speech.py
      tests/
        test_twat_speech.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      package.toml
      pyproject.toml
      README.md
      VERSION.txt
    twat_task/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_task/
          __init__.py
          __main__.py
          __version__.py
          task.py
        testprefect.py
      tests/
        test_twat_task.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      pyproject.toml
      README.md
      VERSION.txt
    twat_text/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_text/
          __version__.py
          twat_text.py
      tests/
        test_twat_text.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      package.toml
      pyproject.toml
      README.md
      VERSION.txt
    twat_video/
      .cursor/
        rules/
          0project.mdc
          cleanup.mdc
          filetree.mdc
          quality.mdc
      .github/
        workflows/
          push.yml
          release.yml
      .ruff_cache/
        .gitignore
        CACHEDIR.TAG
      dist/
        .gitignore
      src/
        twat_video/
          __init__.py
          __version__.py
          twat_video.py
      tests/
        test_twat_video.py
      .gitignore
      .pre-commit-config.yaml
      cleanup.py
      LICENSE
      LOG.md
      package.toml
      pyproject.toml
      README.md
      VERSION.txt
src/
  twat/
    __init__.py
    __version__.py
tests/
  test_twat.py
  VERSION.txt
.gitignore
.pre-commit-config.yaml
cleanup.py
LICENSE
LOG.md
pyproject.toml
README.md
TODO-ALL.md
VERSION.txt

================================================================
Files
================================================================

================
File: .cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: .cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: .cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 864]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [1000]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.6K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [2.7K]  LOG.md
├── [4.2K]  README.md
├── [3.9K]  TODO-ALL.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [ 192]  plugins
│   ├── [  64]  .benchmarks
│   └── [ 128]  repos_dev
│       └── [  96]  .specstory
│           └── [ 128]  history
│               ├── [2.0K]  .what-is-this.md
│               └── [ 17K]  porting-configuration-files-from-@twat-fs-to-@twat-coding.md
├── [6.3K]  pyproject.toml
├── [ 128]  src
│   └── [ 192]  twat
│       └── [2.1K]  __init__.py
└── [ 192]  tests
    ├── [   8]  VERSION.txt
    └── [ 136]  test_twat.py

15 directories, 21 files

================
File: .cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: .github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: .github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: .ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: .ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_audio/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_audio/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_audio/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 800]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 955]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 577]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1.3K]  LOG.md
├── [ 710]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [ 426]  package.toml
├── [7.0K]  pyproject.toml
├── [ 128]  src
│   └── [ 128]  twat_audio
│       └── [1.6K]  twat_audio.py
└── [ 128]  tests
    └── [ 154]  test_twat_audio.py

10 directories, 18 files

================
File: plugins/repos/twat_audio/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_audio/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_audio --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_audio/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-audio
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_audio/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_audio/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_audio/dist/.gitignore
================
*

================
File: plugins/repos/twat_audio/src/twat_audio/__version__.py
================


================
File: plugins/repos/twat_audio/src/twat_audio/twat_audio.py
================
logging.basicConfig(
logger = logging.getLogger(__name__)
class Config:
def process_data(
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        raise ValueError(msg)
def main() -> None:
        config = Config(name="default", value="test", options={"key": "value"})
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        logger.exception("An error occurred: %s", str(e))
    main()

================
File: plugins/repos/twat_audio/tests/test_twat_audio.py
================
def test_version():

================
File: plugins/repos/twat_audio/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_audio/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf] 
  - repo: https://github.com/venomlab/pyproject-pyupgrade
    rev: v0.1.1
    hooks:
      - id: pyupgrade

================
File: plugins/repos/twat_audio/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_audio/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_audio/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v0.0.1] - 2025-02-15

Initial release of the twat-audio package.

### Added

- Basic project structure with modern Python packaging (PEP 621 compliance)
- Initial implementation of `twat_audio.py` with:
  - Configuration management using dataclasses
  - Basic logging setup
  - Type hints and runtime type checking
  - Process data function skeleton
- Comprehensive development setup:
  - Hatch for development workflow management
  - Pre-commit hooks for code quality
  - Ruff for linting and formatting
  - MyPy for type checking
  - Pytest for testing with coverage and benchmarking support
- GitHub Actions workflows for:
  - Build and test on multiple Python versions (3.10, 3.11, 3.12)
  - Code quality checks
  - Automated releases to PyPI
- Project documentation:
  - Basic README.md with installation and usage instructions
  - MIT License

## [Unreleased]

No unreleased changes yet.

[unreleased]: https://github.com/twardoch/twat-audio/compare/v0.0.1...HEAD
[v0.0.1]: https://github.com/twardoch/twat-audio/releases/tag/v0.0.1

================
File: plugins/repos/twat_audio/package.toml
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_audio/pyproject.toml
================
# this_file: twat_audio/pyproject.toml

# this_file: twat_audio/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = ["hatchling>=1.21.0", "hatch-vcs>=0.3.0"]  # Core build backend and VCS plugin
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_audio"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-audio"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = ["twat>=1.0.0"]  # Main twat package dependency

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-audio#readme"
Issues = "https://github.com/twardoch/twat-audio/issues"
Source = "https://github.com/twardoch/twat-audio"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_audio/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = ["pytest", "pytest-cov", "mypy>=1.0.0", "ruff>=0.1.0"]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_audio --cov=tests {args:tests}"
type-check = "mypy src/twat_audio tests"
lint = ["ruff check src/twat_audio tests", "ruff format src/twat_audio tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",
    "ruff>=0.9.6",
    "pyupgrade>=3.19.1",
    "pyupgrade-directories>=0.3.0"
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_audio tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
fix = ["find . -name '*.py' -exec pyupgrade --py310-plus {} +", "ruff check --fix --unsafe-fixes {args:.}", "ruff format {args:.}"]
all = ["style", "typing", "fix", "fmt"]


# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_audio", "tests"]
branch = true
parallel = true
omit = ["src/twat_audio/__about__.py"]

# Coverage path mappings
[tool.coverage.paths]
twat_audio = ["src/twat_audio", "*/twat-audio/src/twat_audio"]
tests = ["tests", "*/twat-audio/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",
    "ruff>=0.9.6",
    "mypy>=1.15.0",
    "pyupgrade>=3.19.1",
    "pyproject-pyupgrade>=0.1.1",
    "pyupgrade-directories>=0.3.0",
]

all = [
    "twat>=1.8.1",
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_audio --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_audio/README.md
================
# 



## Features

- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-audio
```

## Usage

```python
import twat_audio
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_audio/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_cache/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_cache/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_cache/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[1.1K]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [3.4K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.6K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [  96]  .specstory
│   └── [ 800]  history
│       ├── [2.7K]  .what-is-this.md
│       ├── [302K]  2025-03-04_03-25-codebase-analysis-and-todo-md-update.md
│       ├── [ 80K]  2025-03-04_05-35-implementation-of-todo-phases-1,-2,-3.md
│       ├── [115K]  2025-03-04_08-00-project-maintenance-and-documentation-update.md
│       ├── [ 35K]  caching-strategies-for-twat-cache-library.md
│       ├── [ 54K]  codebase-improvement-and-dependency-management-plan.md
│       ├── [124K]  documentation-update-and-task-management.md
│       ├── [217K]  implementing-todo-item.md
│       ├── [ 44K]  merging-files-from-wrong-location.md
│       ├── [238K]  project-documentation-and-command-execution.md
│       ├── [135K]  project-overview-and-todo-execution.md
│       ├── [ 13K]  project-review-and-implementation.md
│       ├── [142K]  project-setup-and-implementation-steps.md
│       ├── [162K]  project-setup-and-task-implementation.md
│       ├── [ 56K]  project-setup-and-task-management.md
│       ├── [ 53K]  refactoring-plan-for-twat-cache-system.md
│       ├── [173K]  refining-decorators-and-integration-goals-1.md
│       ├── [282K]  refining-decorators-and-integration-goals.md
│       ├── [ 95K]  running-lint-fix-command.md
│       ├── [ 11K]  todo-list-update-and-progress-logging.md
│       ├── [170K]  update-todo-and-log-for-project.md
│       ├── [182K]  updating-todo-and-running-development-commands.md
│       └── [260K]  updating-todos-and-project-documentation.md
├── [5.9K]  CHANGELOG.md
├── [ 939]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [ 19K]  LOG.md
├── [7.7K]  NEXT.md
├── [1.2K]  PROMPT.txt
├── [ 38K]  README.md
├── [377K]  REPO_CONTENT.txt
├── [5.3K]  TODO.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [  96]  docs
│   └── [7.7K]  context_management.md
├── [  96]  examples
│   └── [6.9K]  backend_selection.py
├── [1.5K]  pyproject.toml
├── [ 160]  src
├── [ 576]  tests
│   ├── [  67]  __init__.py
│   ├── [ 13K]  test_backend_selector.py
│   ├── [ 13K]  test_cache.py
│   ├── [3.3K]  test_config.py
│   ├── [2.0K]  test_constants.py
│   ├── [6.0K]  test_context.py
│   ├── [2.5K]  test_context_simple.py
│   ├── [ 12K]  test_decorators.py
│   ├── [4.3K]  test_engines.py
│   ├── [6.9K]  test_exceptions.py
│   ├── [2.8K]  test_exceptions_simple.py
│   ├── [6.3K]  test_fallback.py
│   ├── [9.5K]  test_redis_cache.py
│   └── [7.6K]  test_twat_cache.py
├── [188K]  twat_search.txt
└── [ 90K]  uv.lock

13 directories, 61 files

================
File: plugins/repos/twat_cache/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_cache/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_cache --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_cache/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-cache
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_cache/.idea/inspectionProfiles/profiles_settings.xml
================
<component name="InspectionProjectProfileManager">
  <settings>
    <option name="USE_PROJECT_PROFILE" value="false" />
    <version value="1.0" />
  </settings>
</component>

================
File: plugins/repos/twat_cache/.idea/.gitignore
================
# Default ignored files
/shelf/
/workspace.xml

================
File: plugins/repos/twat_cache/.idea/misc.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="ProjectRootManager" version="2" project-jdk-name="Python 3.12 (twat_cache)" project-jdk-type="Python SDK" />
</project>

================
File: plugins/repos/twat_cache/.idea/modules.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="ProjectModuleManager">
    <modules>
      <module fileurl="file://$PROJECT_DIR$/.idea/twat_cache.iml" filepath="$PROJECT_DIR$/.idea/twat_cache.iml" />
    </modules>
  </component>
</project>

================
File: plugins/repos/twat_cache/.idea/ruff.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="RuffConfigService">
    <option name="globalRuffExecutablePath" value="/Library/Frameworks/Python.framework/Versions/3.12/bin/ruff" />
  </component>
</project>

================
File: plugins/repos/twat_cache/.idea/twat_cache.iml
================
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
  <component name="NewModuleRootManager">
    <content url="file://$MODULE_DIR$">
      <excludeFolder url="file://$MODULE_DIR$/.venv" />
    </content>
    <orderEntry type="jdk" jdkName="Python 3.12 (twat_cache)" jdkType="Python SDK" />
    <orderEntry type="sourceFolder" forTests="false" />
  </component>
  <component name="PyDocumentationSettings">
    <option name="format" value="PLAIN" />
    <option name="myDocStringFormat" value="Plain" />
  </component>
  <component name="TestRunnerService">
    <option name="PROJECT_TEST_RUNNER" value="py.test" />
  </component>
</module>

================
File: plugins/repos/twat_cache/.idea/vcs.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="VcsDirectoryMappings">
    <mapping directory="" vcs="Git" />
  </component>
</project>

================
File: plugins/repos/twat_cache/.idea/workspace.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="ChangeListManager">
    <list default="true" id="c7e45542-6d33-435d-ad9e-4cdc20ca9961" name="Changes" comment="">
      <change beforePath="$PROJECT_DIR$/.specstory/history/2025-03-04_08-41-managing-todo-md-tasks.md" beforeDir="false" afterPath="$PROJECT_DIR$/.specstory/history/2025-03-04_08-41-managing-todo-md-tasks.md" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/CHANGELOG.md" beforeDir="false" afterPath="$PROJECT_DIR$/CHANGELOG.md" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/LOG.md" beforeDir="false" />
      <change beforePath="$PROJECT_DIR$/TODO.md" beforeDir="false" afterPath="$PROJECT_DIR$/TODO.md" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_cache/decorators.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_cache/decorators.py" afterDir="false" />
    </list>
    <option name="SHOW_DIALOG" value="false" />
    <option name="HIGHLIGHT_CONFLICTS" value="true" />
    <option name="HIGHLIGHT_NON_ACTIVE_CHANGELIST" value="false" />
    <option name="LAST_RESOLUTION" value="IGNORE" />
  </component>
  <component name="Git.Settings">
    <option name="RECENT_GIT_ROOT_PATH" value="$PROJECT_DIR$" />
  </component>
  <component name="GitHubPullRequestSearchHistory">{
  &quot;lastFilter&quot;: {
    &quot;state&quot;: &quot;OPEN&quot;,
    &quot;assignee&quot;: &quot;twardoch&quot;
  }
}</component>
  <component name="GithubPullRequestsUISettings">{
  &quot;selectedUrlAndAccountId&quot;: {
    &quot;url&quot;: &quot;https://github.com/twardoch/twat-cache.git&quot;,
    &quot;accountId&quot;: &quot;e13b4c44-74d4-444d-8d41-a0014a4f5e46&quot;
  }
}</component>
  <component name="ProjectColorInfo">{
  &quot;associatedIndex&quot;: 8
}</component>
  <component name="ProjectId" id="2tqSVti5U24zs1IzhjeXPuMBHsU" />
  <component name="ProjectViewState">
    <option name="hideEmptyMiddlePackages" value="true" />
    <option name="showLibraryContents" value="true" />
  </component>
  <component name="PropertiesComponent">{
  &quot;keyToString&quot;: {
    &quot;RunOnceActivity.ShowReadmeOnStart&quot;: &quot;true&quot;,
    &quot;RunOnceActivity.git.unshallow&quot;: &quot;true&quot;,
    &quot;git-widget-placeholder&quot;: &quot;main&quot;,
    &quot;settings.editor.selected.configurable&quot;: &quot;preferences.pluginManager&quot;
  }
}</component>
  <component name="SharedIndexes">
    <attachedChunks>
      <set>
        <option value="bundled-python-sdk-14705d77f0bb-aa17d162503b-com.jetbrains.pycharm.community.sharedIndexes.bundled-PC-243.25659.43" />
      </set>
    </attachedChunks>
  </component>
  <component name="SpellCheckerSettings" RuntimeDictionaries="0" Folders="0" CustomDictionaries="0" DefaultDictionary="application-level" UseSingleDictionary="true" transferred="true" />
  <component name="TaskManager">
    <task active="true" id="Default" summary="Default task">
      <changelist id="c7e45542-6d33-435d-ad9e-4cdc20ca9961" name="Changes" comment="" />
      <created>1741078138937</created>
      <option name="number" value="Default" />
      <option name="presentableId" value="Default" />
      <updated>1741078138937</updated>
    </task>
    <servers />
  </component>
</project>

================
File: plugins/repos/twat_cache/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_cache/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_cache/dist/.gitignore
================
*

================
File: plugins/repos/twat_cache/docs/context_management.md
================
# Cache Context Management

The `twat_cache` library provides powerful context management utilities that allow you to control the lifecycle of cache engines and ensure proper resource cleanup. This document explains how to use these context management features effectively.

## Overview

Cache context management in `twat_cache` serves several important purposes:

1. **Resource Management**: Ensures that cache resources (files, connections, memory) are properly cleaned up when no longer needed
2. **Temporary Configuration**: Allows you to temporarily override cache settings for specific code blocks
3. **Engine Selection**: Provides a way to explicitly select a specific cache engine for a block of code
4. **Error Handling**: Guarantees cleanup even when exceptions occur

## Available Context Managers

### `engine_context`

The `engine_context` function is a context manager that creates and manages a cache engine instance:

```python
from twat_cache.context import engine_context

# Basic usage
with engine_context() as cache:
    # Use the cache engine
    cache.set("key", "value")
    value = cache.get("key")

# With specific configuration
with engine_context(maxsize=100, folder_name="my_cache") as cache:
    # Use the cache engine with custom configuration
    cache.set("key", "value")

# With specific engine
with engine_context(engine_name="redis") as cache:
    # Use the Redis cache engine
    cache.set("key", "value")
```

### `CacheContext` Class

The `CacheContext` class provides a reusable context manager for working with cache engines:

```python
from twat_cache.context import CacheContext
from twat_cache.config import create_cache_config

# Create a context with default settings
context = CacheContext()
with context as cache:
    # Use the cache engine
    cache.set("key", "value")

# Create a context with specific configuration
config = create_cache_config(maxsize=100, folder_name="my_cache")
context = CacheContext(config=config)
with context as cache:
    # Use the cache engine with custom configuration
    cache.set("key", "value")

# Create a context with a specific engine
redis_context = CacheContext(engine_name="redis")
with redis_context as cache:
    # Use the Redis cache engine
    cache.set("key", "value")
```

## Manual Engine Creation

If you need more control over the engine lifecycle, you can use the `get_or_create_engine` function:

```python
from twat_cache.context import get_or_create_engine

# Create an engine
engine = get_or_create_engine(maxsize=100, folder_name="my_cache")

try:
    # Use the engine
    engine.set("key", "value")
    value = engine.get("key")
finally:
    # Important: You must manually clean up the engine
    engine.cleanup()
```

> **Warning**: When using `get_or_create_engine`, you are responsible for calling `cleanup()` on the engine when it's no longer needed. For automatic cleanup, use `engine_context` or `CacheContext` instead.

## Practical Examples

### Temporary Cache Configuration

```python
from twat_cache.context import engine_context
from twat_cache.decorators import ucache

# Default cache configuration
@ucache()
def slow_function(x):
    # ... expensive computation ...
    return result

# Override cache configuration for a specific section
with engine_context(maxsize=1000, ttl=3600) as cache:
    # Use the cache directly
    key = f"special_key_{x}"
    result = cache.get(key)
    if result is None:
        result = compute_expensive_result(x)
        cache.set(key, result)
```

### Using Different Cache Backends

```python
from twat_cache.context import engine_context

# Process small data with in-memory cache
with engine_context(engine_name="cachetools") as memory_cache:
    for item in small_data:
        process_with_cache(item, memory_cache)

# Process large data with disk cache
with engine_context(engine_name="diskcache") as disk_cache:
    for item in large_data:
        process_with_cache(item, disk_cache)
```

### Redis Cache for Distributed Applications

```python
from twat_cache.context import engine_context

# Use Redis for distributed caching
with engine_context(
    engine_name="redis",
    redis_host="redis.example.com",
    redis_port=6379,
    redis_password="secret",
    ttl=3600,
) as redis_cache:
    # Cache is now shared across all application instances
    result = redis_cache.get("shared_key")
    if result is None:
        result = compute_expensive_result()
        redis_cache.set("shared_key", result)
```

## Best Practices

1. **Use Context Managers**: Always use context managers (`with` statements) when possible to ensure proper resource cleanup.

2. **Specify Engine Requirements**: When you need specific features, explicitly specify the engine name or configuration parameters that provide those features.

3. **Handle Exceptions**: Context managers will clean up resources even when exceptions occur, but you should still handle exceptions appropriately for your application logic.

4. **Reuse Contexts**: For repeated operations with the same configuration, create a `CacheContext` instance once and reuse it.

5. **Avoid Nesting**: While it's technically possible to nest context managers, it can lead to confusion. Try to keep your cache context structure simple.

## Advanced Usage

### Custom Engine Selection Logic

```python
from twat_cache.context import engine_context
from twat_cache.engines.manager import get_engine_manager

# Get available engines
manager = get_engine_manager()
available_engines = manager.get_available_engines()

# Choose an engine based on custom logic
if "redis" in available_engines and is_distributed_environment():
    engine_name = "redis"
elif "diskcache" in available_engines and data_size > 1_000_000:
    engine_name = "diskcache"
else:
    engine_name = "cachetools"

# Use the selected engine
with engine_context(engine_name=engine_name) as cache:
    # Use the cache
    cache.set("key", "value")
```

### Combining with Decorators

```python
from twat_cache.context import engine_context
from twat_cache.decorators import ucache
from functools import wraps

def with_custom_cache(func):
    """Decorator that uses a custom cache configuration."""
    @wraps(func)
    def wrapper(*args, **kwargs):
        with engine_context(maxsize=1000, ttl=3600) as cache:
            # Create a cache key
            key = f"{func.__name__}:{args}:{kwargs}"
            
            # Check cache
            result = cache.get(key)
            if result is not None:
                return result
                
            # Call function and cache result
            result = func(*args, **kwargs)
            cache.set(key, result)
            return result
    return wrapper

# Use the custom cache decorator
@with_custom_cache
def expensive_function(x, y):
    # ... expensive computation ...
    return result
```

## Error Handling

The context managers in `twat_cache` are designed to handle errors gracefully:

```python
from twat_cache.context import engine_context
from twat_cache.exceptions import CacheOperationError

try:
    with engine_context() as cache:
        # This might raise an exception
        cache.set("key", complex_object)
        result = cache.get("key")
except CacheOperationError as e:
    # Handle cache-specific errors
    print(f"Cache operation failed: {e}")
except Exception as e:
    # Handle other errors
    print(f"An error occurred: {e}")
finally:
    # The cache engine will be cleaned up automatically
    # No need to call cache.cleanup() here
    print("Cache resources have been cleaned up")
```

## Conclusion

Cache context management in `twat_cache` provides a flexible and robust way to work with cache engines while ensuring proper resource management. By using these context management utilities, you can write cleaner, more maintainable code that efficiently handles cache resources.

================
File: plugins/repos/twat_cache/examples/backend_selection.py
================
def example_type_specific_config():
    print("\n=== Example 1: Type-specific Configuration Helpers ===")
        @ucache(config=configure_for_numpy())
        def process_array(data: np.ndarray) -> np.ndarray:
            print("  Processing NumPy array...")
            time.sleep(1)  # Simulate processing time
        arr = np.array([1, 2, 3, 4, 5])
        print("  First call (cache miss):")
        start = time.time()
        result1 = process_array(arr)
        print(f"  Result: {result1}")
        print(f"  Time: {time.time() - start:.4f} seconds")
        print("\n  Second call (cache hit):")
        result2 = process_array(arr)
        print(f"  Result: {result2}")
        print("  NumPy not installed, skipping NumPy example")
    @ucache(config=configure_for_json())
    def fetch_json_data(url: str) -> dict[str, Any]:
        print(f"  Fetching JSON data from {url}...")
        time.sleep(1)  # Simulate network request
    print("\n  JSON example:")
    json_result1 = fetch_json_data("https://example.com/api/data")
    print(f"  Result: {json_result1}")
    json_result2 = fetch_json_data("https://example.com/api/data")
    print(f"  Result: {json_result2}")
def example_hybrid_caching():
    print("\n=== Example 2: Hybrid Caching Based on Result Size ===")
    @hybrid_cache()
    def get_data(size: str) -> dict[str, Any] | list[int]:
        print(f"  Generating {size} data...")
            return list(range(100000))  # Large list
    print("\n  Small data example:")
    small_result1 = get_data("small")
    print(f"  Result size: {len(str(small_result1))} bytes")
    small_result2 = get_data("small")
    print(f"  Result size: {len(str(small_result2))} bytes")
    print("\n  Large data example:")
    large_result1 = get_data("large")
    print(f"  Result size: {len(str(large_result1))} bytes")
    large_result2 = get_data("large")
    print(f"  Result size: {len(str(large_result2))} bytes")
def example_smart_caching():
    print("\n=== Example 3: Smart Caching with Automatic Backend Selection ===")
    @smart_cache()
    def process_data(data_type: str, size: int) -> Any:
        print(f"  Processing {data_type} data of size {size}...")
            return {f"key_{i}": f"value_{i}" for i in range(size)}
            return list(range(size))
            print(f"\n  {data_type.capitalize()} data of size {size}:")
            result1 = process_data(data_type, size)
            result_size = len(str(result1)) if data_type != "int" else 8
            print(f"  Result type: {type(result1).__name__}, size: {result_size} bytes")
            result2 = process_data(data_type, size)
            print(f"  Result type: {type(result2).__name__}, size: {result_size} bytes")
    print("Backend Selection Strategy Examples")
    print("==================================")
    example_type_specific_config()
    example_hybrid_caching()
    example_smart_caching()
    print("\nAll examples completed successfully!")

================
File: plugins/repos/twat_cache/src/twat_cache/engines/__init__.py
================
    engines["functools"] = cast(type[BaseCacheEngine[Any, Any]], FunctoolsCacheEngine)
    engines["cachetools"] = cast(type[BaseCacheEngine[Any, Any]], CacheToolsEngine)
    engines["diskcache"] = cast(type[BaseCacheEngine[Any, Any]], DiskCacheEngine)
    engines["aiocache"] = cast(type[BaseCacheEngine[Any, Any]], AioCacheEngine)
    engines["klepto"] = cast(type[BaseCacheEngine[Any, Any]], KleptoEngine)
    engines["joblib"] = cast(type[BaseCacheEngine[Any, Any]], JoblibEngine)
    engines["cachebox"] = cast(type[BaseCacheEngine[Any, Any]], CacheBoxEngine)
    engines["redis"] = cast(type[BaseCacheEngine[Any, Any]], RedisCacheEngine)
def get_available_engines() -> list[str]:
        for name, engine_cls in engines.items()
        if hasattr(engine_cls, "is_available") and engine_cls.is_available()

================
File: plugins/repos/twat_cache/src/twat_cache/engines/aiocache.py
================
class AioCacheEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
        return is_package_available("aiocache")
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
            raise ImportError(msg)
        if is_package_available("redis"):
            self._cache: Cache = RedisCache(
        elif is_package_available("pymemcache"):
            self._cache = MemcachedCache(
            self._cache = SimpleMemoryCache(
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
            raise RuntimeError(msg)
        return cast(
            cached(
    def get(self, key: CacheKey) -> R | None:
        return cast(R | None, self._cache.get(str(key)))
    def set(self, key: CacheKey, value: R) -> None:
        self._cache.set(str(key), value, ttl=self._config.ttl)
    def clear(self) -> None:
        self._cache.clear()
    def stats(self) -> dict[str, Any]:
            "size": len(self._cache),
    def is_available(self) -> bool:

================
File: plugins/repos/twat_cache/src/twat_cache/engines/base.py
================
def is_package_available(package_name: str) -> bool:
    return util.find_spec(package_name) is not None
class BaseCacheEngine(ABC, Generic[P, R]):
    def __init__(self, config: CacheConfig) -> None:
        self._created_at = time.time()
        self.validate_config()
                self._cache_path = get_cache_path(self._config.folder_name)
                    ensure_dir_exists(self._cache_path, mode=0o700 if self._config.secure else 0o755)
                logger.error(f"Failed to initialize cache path: {e}")
                raise PathError(msg) from e
        logger.debug(f"Initialized {self.__class__.__name__} with config: {config}")
    def stats(self) -> dict[str, Any]:
            "path": str(self._cache_path) if self._cache_path else None,
            "uptime": time.time() - self._created_at,
    def validate_config(self) -> None:
                validate_cache_path(self._config.folder_name)
                logger.error(f"Invalid cache folder configuration: {e}")
                raise ConfigurationError(msg) from e
            logger.error(msg)
            raise ConfigurationError(msg)
    def is_available(cls) -> bool:
    def cleanup(self) -> None:
        logger.debug(f"Cleaning up {self.__class__.__name__} resources")
        self._cache.clear()
    def _check_ttl(self, timestamp: float | None) -> bool:
        return (time.time() - timestamp) < self._config.ttl
    def _make_key(self, func: Callable[P, R], args: Any, kwargs: Any) -> CacheKey:
        return create_cache_key(func, args, kwargs)
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
    def _get_cached_value(self, key: CacheKey) -> R | None:
    def _set_cached_value(self, key: CacheKey, value: R) -> None:
    def clear(self) -> None:
    def get(self, key: str) -> R | None:
        return self._get_cached_value(key)
    def set(self, key: str, value: R) -> None:
        self._set_cached_value(key, value)
    def name(self) -> str:
class CacheEngine(BaseCacheEngine[P, R]):
                oldest_key = next(iter(self._cache))

================
File: plugins/repos/twat_cache/src/twat_cache/engines/cachebox.py
================
class CacheBoxEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
        return is_package_available("cachebox")
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
            raise ImportError(msg)
        cache_type = cache_types.get(config.policy, LRUCache)
        self._cache: Cache = cache_type(
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
            raise RuntimeError(msg)
        return cast(
            cached(
    def get(self, key: CacheKey) -> R | None:
        return cast(R | None, self._cache.get(str(key)))
    def set(self, key: CacheKey, value: R) -> None:
        self._cache[str(key)] = value
    def clear(self) -> None:
        self._cache.clear()
    def stats(self) -> dict[str, Any]:
            "size": len(self._cache),
    def is_available(self) -> bool:

================
File: plugins/repos/twat_cache/src/twat_cache/engines/cachetools.py
================
class CacheToolsEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
        return is_package_available("cachetools")
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
            raise ImportError(msg)
            cache_type = cache_types.get(config.policy, LRUCache)
        self._cache: Cache = cache_type(
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
            raise RuntimeError(msg)
        return cast(
            cached(
    def get(self, key: CacheKey) -> R | None:
        return cast(R | None, self._cache.get(str(key)))
    def set(self, key: CacheKey, value: R) -> None:
        self._cache[str(key)] = value
    def clear(self) -> None:
        self._cache.clear()
    def stats(self) -> dict[str, Any]:
            "hits": getattr(self._cache, "hits", 0),
            "misses": getattr(self._cache, "misses", 0),
            "size": len(self._cache),
            "ttl": getattr(self._cache, "ttl", None),
    def is_available(self) -> bool:

================
File: plugins/repos/twat_cache/src/twat_cache/engines/common.py
================
K = TypeVar("K")
V = TypeVar("V")
def ensure_dir_exists(dir_path: Path, mode: int = 0o700) -> None:
        dir_path.mkdir(mode=mode, parents=True, exist_ok=True)
        logger.error(f"Failed to create cache directory {dir_path}: {e}")
        raise PathError(msg) from e
def safe_key_serializer(key: Any) -> str:
        if isinstance(key, str):
        if isinstance(key, int | float | bool | type(None)):
            return str(key)
        if isinstance(key, list | tuple | set):
            return json.dumps([safe_key_serializer(k) for k in key])
        if isinstance(key, dict):
            return json.dumps({str(k): safe_key_serializer(v) for k, v in key.items()})
        return json.dumps(repr(key))
        logger.error(f"Failed to serialize cache key: {e}")
        raise CacheKeyError(msg) from e
def safe_value_serializer(value: Any) -> str:
        return json.dumps(value)
            return repr(value)
            logger.error(f"Failed to serialize cache value: {inner_e}")
            raise CacheValueError(
def safe_temp_file(
        fd, path = tempfile.mkstemp(prefix=prefix, suffix=suffix)
        os.chmod(path, 0o600)  # Ensure secure permissions
        return Path(path), os.fdopen(fd, "w+b")
        logger.error(f"Failed to create temporary file: {e}")
        raise ResourceError(msg) from e
def get_func_qualified_name(func: Callable[..., Any]) -> str:
    module = inspect.getmodule(func)
    if hasattr(func, "__qualname__"):
def create_cache_key(func: Callable[P, R], args: Any, kwargs: Any) -> CacheKey:
    func_name = get_func_qualified_name(func)
    hashable_args = tuple(args)
    hashable_kwargs = tuple(sorted((k, v) for k, v in kwargs.items()))

================
File: plugins/repos/twat_cache/src/twat_cache/engines/diskcache.py
================
class DiskCacheEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
            logger.error("Cache path is required for disk cache")
            raise EngineNotAvailableError(
        if not self.is_available():
            logger.error("diskcache is not available")
            ensure_dir_exists(
            self._disk_cache = Cache(
                directory=str(self._cache_path),
            logger.debug(f"Initialized DiskCache at {self._cache_path}")
            logger.error(f"Failed to initialize DiskCache: {e}")
            raise ResourceError(msg) from e
    def cleanup(self) -> None:
            if hasattr(self, "_disk_cache") and self._disk_cache is not None:
                self._disk_cache.close()
                logger.debug(f"Closed DiskCache at {self._cache_path}")
            logger.error(f"Error closing DiskCache: {e}")
            super().cleanup()
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
        @wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            key = self._make_key(func, args, kwargs)
                cache_result = self._disk_cache.get(key, default=None, expire_time=True)
                    if isinstance(cache_result, tuple) and len(cache_result) >= 2:
                            isinstance(expire_time, int | float)
                            and time.time() < expire_time
                            logger.trace(f"Cache hit for {func.__name__}")
                            return cast(R, cached_value)
                            self._disk_cache.delete(key)
                logger.trace(f"Cache miss for {func.__name__}")
                result = func(*args, **kwargs)
                    expire_time = time.time() + self._config.ttl
                self._disk_cache.set(
                    self._size = self._disk_cache.volume()
                logger.error(f"Error in DiskCache: {e}")
                return func(*args, **kwargs)
    def clear(self) -> None:
                self._disk_cache.clear()
                logger.debug(f"Cleared DiskCache at {self._cache_path}")
            logger.error(f"Error clearing DiskCache: {e}")
            raise CacheOperationError(msg) from e
    def stats(self) -> dict[str, Any]:
        base_stats = super().stats
                disk_stats_dict = self._disk_cache.stats(enable=True)
                if isinstance(disk_stats_dict, dict):
                    base_stats["disk_hits"] = disk_stats_dict.get("hits", 0)
                    base_stats["disk_misses"] = disk_stats_dict.get("misses", 0)
                    base_stats["disk_size"] = self._disk_cache.volume()
            logger.error(f"Error getting DiskCache stats: {e}")

================
File: plugins/repos/twat_cache/src/twat_cache/engines/functools_engine.py
================
class FunctoolsCacheEngine(BaseCacheEngine[P, R]):
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
        self._maxsize = config.maxsize or float("inf")
        logger.debug(
    def get(self, key: str) -> R | None:
            return self._cache.cache_getitem(lambda k: k, key)  # type: ignore
    def set(self, key: str, value: R) -> None:
        self._cache(lambda k: k)(key)  # type: ignore
        self._cache.cache_setitem(lambda k: k, key, value)  # type: ignore
    def clear(self) -> None:
        self._cache.clear()
    def name(self) -> str:
    def stats(self) -> dict[str, Any]:
            "current_size": len(self._cache),
            "size": len(self._cache),
    def validate_config(self) -> None:
        super().validate_config()
        if maxsize is not None and not isinstance(maxsize, int):
            raise ValueError(msg)
    def _get_backend_key_components(self) -> list[str]:
    def cache(
        def decorator(f: Callable[P, R]) -> Callable[P, R]:
            @wraps(f)
            def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
                key = json.dumps(tuple(args) + tuple(kwargs.items()), sort_keys=True)
                result = self._get_cached_value(key)
                        result = f(*args, **kwargs)
                        self._set_cached_value(key, result)
                        self._set_cached_value(key, cast(R, e))
                elif isinstance(result, Exception):
        return decorator(func)
    def _get_cached_value(self, key: CacheKey) -> R | None:
        if expiry is not None and time.time() >= expiry:
    def _set_cached_value(self, key: CacheKey, value: R) -> None:
        if len(self._cache) >= self._maxsize:
                oldest_key = next(iter(self._cache))
            time.time() + self._config.ttl if self._config.ttl is not None else None
    def is_available(cls) -> bool:

================
File: plugins/repos/twat_cache/src/twat_cache/engines/functools.py
================
class FunctoolsCacheEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
            wrapper = functools.lru_cache(maxsize=self._maxsize)(func)
            wrapper = functools.cache(func)
        def cached_func(*args: P.args, **kwargs: P.kwargs) -> R:
            key = self._make_key(*args, **kwargs)
                now = time.time()
                        self.clear()
                        result = func(*args, **kwargs)
                result = wrapper(*args, **kwargs)
                self._last_access[key] = time.time()
                logger.warning(f"Cache miss for {key}: {e}")
    def clear(self) -> None:
        if hasattr(self, "_cache"):
            self._cache.clear()
            self._last_access.clear()
    def validate_config(self) -> None:
            raise ValueError(msg)
    def stats(self) -> dict[str, Any]:
            "size": len(self._cache),

================
File: plugins/repos/twat_cache/src/twat_cache/engines/joblib.py
================
class DummyMemory:
    def cache(self, func: F) -> F:
    def clear(self) -> None:
class JoblibEngine(BaseCacheEngine[P, R]):
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
        folder_name = getattr(self._config, "folder_name", None)
            JoblibMemory(str(get_cache_path(self._folder_name)), verbose=0)
            else DummyMemory()
    def _get_cached_value(self, key: CacheKey) -> R | None:
    def _set_cached_value(self, key: CacheKey, value: R) -> None:
        self._memory.clear()
    def is_available(cls) -> bool:
    def name(self) -> str:
        return cast(F, self._memory.cache(func))

================
File: plugins/repos/twat_cache/src/twat_cache/engines/klepto.py
================
class KleptoEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
        return is_package_available("klepto")
    def __init__(self, config: CacheConfig) -> None:
        super().__init__(config)
            raise ImportError(msg)
        maxsize = config.maxsize or float("inf")
            self._cache = lru_cache(maxsize=maxsize)
            self._cache = lfu_cache(maxsize=maxsize)
            self._cache = rr_cache(maxsize=maxsize)
            self._cache = sql_archive(
                str(config.cache_dir / "cache.db"),
            self._cache = file_archive(
                str(config.cache_dir),
        self._cache.load()
        logger.debug(
    def _get_cached_value(self, key: CacheKey) -> R | None:
                if time.time() >= self._expiry[key]:
                        self._cache.sync()
            return cast(R, self._cache[key])
            logger.warning(f"Error retrieving from klepto cache: {e}")
    def _set_cached_value(self, key: CacheKey, value: R) -> None:
                self._expiry[key] = time.time() + self._config.ttl
            self._size = len(self._cache)
            logger.warning(f"Error storing to klepto cache: {e}")
    def clear(self) -> None:
            self._cache.clear()
            self._expiry.clear()
            logger.warning(f"Error clearing klepto cache: {e}")
    def stats(self) -> dict[str, Any]:
        base_stats = super().stats
            base_stats.update(
                    "currsize": len(self._cache),
                        type(self._cache).__name__ if self._cache else None
            logger.warning(f"Error getting klepto cache stats: {e}")
    def __del__(self) -> None:
                self._cache.close()
            logger.warning(f"Error closing klepto cache: {e}")
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
            raise RuntimeError(msg)
        return cast(
            self._cache(func),
    def get(self, key: CacheKey) -> R | None:
        return cast(R | None, self._cache.get(str(key)))
    def set(self, key: CacheKey, value: R) -> None:
        self._cache[str(key)] = value
        self._cache.dump()
            "hits": getattr(self._cache, "hits", 0),
            "misses": getattr(self._cache, "misses", 0),
            "size": len(self._cache),

================
File: plugins/repos/twat_cache/src/twat_cache/engines/manager.py
================
E = TypeVar("E", bound=BaseCacheEngine[Any, Any])
    logger.debug("aiocache not available")
    logger.debug("cachetools not available")
    logger.debug("klepto not available")
    logger.debug("joblib not available")
    logger.debug("redis not available")
class CacheEngineManager:
    def __init__(self) -> None:
        self._register_builtin_engines()
    def _register_builtin_engines(self) -> None:
        self.register_engine("functools", FunctoolsCacheEngine)
        self.register_engine("cachetools", CacheToolsEngine)
        self.register_engine("diskcache", DiskCacheEngine)
        self.register_engine("aiocache", AioCacheEngine)
        self.register_engine("klepto", KleptoEngine)
        self.register_engine("joblib", JoblibEngine)
        self.register_engine("cachebox", CacheBoxEngine)
            self.register_engine("redis", RedisCacheEngine)
    def register_engine(self, name: str, engine_cls: type[E]) -> None:
            logger.warning(f"Overwriting existing engine registration for {name}")
        self._engines[name] = cast(type[BaseCacheEngine[Any, Any]], engine_cls)
    def get_engine(self, name: str) -> type[BaseCacheEngine[Any, Any]] | None:
        return self._engines.get(name)
    def list_engines(self) -> list[str]:
        return list(self._engines.keys())
    def get_available_engines(self) -> list[str]:
        return [name for name, engine in self._engines.items() if engine.is_available()]
    def select_engine(
        available = self.get_available_engines()
            logger.warning("No cache engines are available")
                    engine_cls: type[BaseCacheEngine[Any, Any]] | None = self.get_engine(engine_name)
                    if engine_cls and engine_cls.is_available():
        fallback: type[BaseCacheEngine[Any, Any]] | None = self.get_engine(available[0])
        if fallback and fallback.is_available():
def get_engine_manager() -> CacheEngineManager:
    return CacheEngineManager()

================
File: plugins/repos/twat_cache/src/twat_cache/engines/py.typed
================
1

================
File: plugins/repos/twat_cache/src/twat_cache/engines/redis.py
================
class RedisCacheEngine(BaseCacheEngine[P, R]):
    def is_available(cls) -> bool:
        return is_package_available("redis")
    def __init__(self, config: CacheConfig) -> None:
        if not self.is_available():
            raise EngineNotAvailableError(msg)
        super().__init__(config)
        host = config.get_redis_host() if hasattr(config, "get_redis_host") else "localhost"
        port = config.get_redis_port() if hasattr(config, "get_redis_port") else 6379
        db = config.get_redis_db() if hasattr(config, "get_redis_db") else 0
        password = config.get_redis_password() if hasattr(config, "get_redis_password") else None
            self._redis = redis.Redis(
            self._redis.ping()
            logger.debug(f"Connected to Redis at {host}:{port} (db={db})")
            logger.error(msg)
            raise ConfigurationError(msg) from e
    def validate_config(self) -> None:
        super().validate_config()
        if hasattr(self._config, "get_redis_port") and self._config.get_redis_port() is not None:
            port = self._config.get_redis_port()
            if not isinstance(port, int) or port <= 0 or port > 65535:
                raise ConfigurationError(msg)
    def cleanup(self) -> None:
            if hasattr(self, "_redis"):
                self._redis.close()
                logger.debug("Closed Redis connection")
            logger.error(f"Error closing Redis connection: {e}")
    def _get_full_key(self, key: CacheKey) -> str:
        if isinstance(key, tuple):
            key_str = str(hash(key))
            key_str = str(key)
    def _get_cached_value(self, key: CacheKey) -> R | None:
            full_key = self._get_full_key(key)
            data = self._redis.get(full_key)
                    data = zlib.decompress(data)
                value = pickle.loads(data)
                self._last_access = time.time()
                return cast(R, value)
                logger.error(f"Error deserializing cached value: {e}")
            logger.error(f"Error getting cached value from Redis: {e}")
    def _set_cached_value(self, key: CacheKey, value: R) -> None:
                data = pickle.dumps(value)
                    data = zlib.compress(data)
                logger.error(f"Error serializing value: {e}")
                raise CacheOperationError(msg) from e
                self._redis.setex(full_key, int(self._config.ttl), data)
                self._redis.set(full_key, data)
            self._last_update = time.time()
            logger.error(f"Error setting cached value in Redis: {e}")
    def clear(self) -> None:
            keys = self._redis.keys(pattern)
                self._redis.delete(*keys)
                logger.debug(f"Cleared {len(keys)} keys from Redis namespace {self._namespace}")
            logger.error(f"Error clearing Redis cache: {e}")
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
        @wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            key = self._make_key(func, args, kwargs)
            cached = self._get_cached_value(key)
            result = func(*args, **kwargs)
            self._set_cached_value(key, result)
    def stats(self) -> dict[str, Any]:
        base_stats = super().stats
        current_time = time.time()
                size = len(self._redis.keys(pattern))
                logger.error(f"Error getting Redis cache size: {e}")
            info = self._redis.info()
            redis_stats.update(
                    "redis_version": info.get("redis_version"),
                    "used_memory": info.get("used_memory"),
                    "used_memory_human": info.get("used_memory_human"),
                    "connected_clients": info.get("connected_clients"),
            logger.debug(f"Error getting Redis info: {e}")

================
File: plugins/repos/twat_cache/src/twat_cache/types/cachebox.pyi
================
from typing import Any, TypeVar, Generic

T = TypeVar("T")

class Cache(Generic[T]):
    def __init__(self, maxsize: int = 0) -> None: ...
    def get(self, key: Any) -> T | None: ...
    def __setitem__(self, key: Any, value: T) -> None: ...
    def clear(self) -> None: ...
    def memoize(self) -> Any: ...

================
File: plugins/repos/twat_cache/src/twat_cache/__init__.py
================


================
File: plugins/repos/twat_cache/src/twat_cache/__main__.py
================
T = TypeVar("T")
def get_cache_path(folder_name: str | None = None) -> Path:
    def generate_uuid() -> str:
        caller_frame = inspect.stack()[2]
        caller_path = Path(caller_file).resolve()
        return str(uuid.uuid5(uuid.NAMESPACE_URL, str(caller_path)))
        folder_name = generate_uuid()
        Path.home() / ".cache" / "twat_cache" / str(folder_name)
    cache_dir.mkdir(parents=True, exist_ok=True)
    DISK_CACHE: Any | None = Cache(get_cache_path("twat_cache"))
    JOBLIB_MEMORY: Any | None = Memory(get_cache_path("twat_cache"), verbose=0)
def ucache(
    def decorator(func: Callable[..., T]) -> Callable[..., T]:
        config = CacheConfig(
        return twat_cache(config)(func)

================
File: plugins/repos/twat_cache/src/twat_cache/__version__.py
================


================
File: plugins/repos/twat_cache/src/twat_cache/backend_selector.py
================
F = TypeVar("F", bound=Callable[..., Any])
R = TypeVar("R")
class DataSize(Enum):
class DataPersistence(Enum):
class AccessPattern(Enum):
class ConcurrencyLevel(Enum):
def estimate_data_size(data: Any) -> DataSize:
        size = sys.getsizeof(data)
        if isinstance(data, list | tuple | set | dict):
            if isinstance(data, list | tuple | set):
                    size += sys.getsizeof(item)
            elif isinstance(data, dict):
                for key, value in data.items():
                    size += sys.getsizeof(key) + sys.getsizeof(value)
def get_type_based_backend(data_type: type) -> str:
    for special_type, backend in TYPE_TO_BACKEND.items():
        if isinstance(special_type, str) and special_type == type_name:
    for base_type, backend in TYPE_TO_BACKEND.items():
        if not isinstance(base_type, str) and issubclass(data_type, base_type):
def get_available_backend(preferred_backend: str) -> str:
    if is_backend_available(preferred_backend):
    for alt_backend in ALTERNATIVE_BACKENDS.get(preferred_backend, []):
        if is_backend_available(alt_backend):
            logger.debug(f"Preferred backend '{preferred_backend}' not available, using '{alt_backend}' instead")
    logger.debug("No suitable backend found, defaulting to 'functools'")
def is_backend_available(backend_name: str) -> bool:
    package_name = package_map.get(backend_name)
        return is_package_available(package_name)
def select_backend_for_data(
        backends.append(config.preferred_engine)
        data_type = type(data)
        backends.append(get_type_based_backend(data_type))
        size = estimate_data_size(data)
        backends.append(SIZE_TO_BACKEND[size])
        backends.append(PERSISTENCE_TO_BACKEND[persistence])
        backends.append(ACCESS_TO_BACKEND[access_pattern])
        backends.append(CONCURRENCY_TO_BACKEND[concurrency])
    backends.append("cachetools")
        available_backend = get_available_backend(backend)
def configure_for_type(data_type: type, **kwargs) -> CacheConfig:
    backend = get_type_based_backend(data_type)
        config_params["folder_name"] = f"{data_type.__name__.lower()}_cache"
        config_params["folder_name"] = f"{data_type.__name__.lower()}_joblib"
    config_params.update(kwargs)
    return CacheConfig(**config_params)
def configure_for_numpy() -> CacheConfig:
        return configure_for_type(np.ndarray, folder_name="numpy_cache")
        logger.warning("NumPy not available, returning generic configuration")
        return CacheConfig(preferred_engine=get_available_backend("joblib"))
def configure_for_pandas() -> CacheConfig:
        return configure_for_type(pd.DataFrame, folder_name="pandas_cache")
        logger.warning("Pandas not available, returning generic configuration")
def configure_for_images() -> CacheConfig:
    return CacheConfig(
        preferred_engine=get_available_backend("diskcache"),
def configure_for_json() -> CacheConfig:
def hybrid_cache_config(
        "small_result_engine": get_available_backend(small_result_engine),
        "large_result_engine": get_available_backend(large_result_engine),
def analyze_function_return_type(func: Callable) -> type | None:
        hints = get_type_hints(func)
def smart_cache_config(func: F = None, **kwargs) -> CacheConfig:
        return_type = analyze_function_return_type(func)
            backend = get_type_based_backend(return_type)
            config_params["preferred_engine"] = get_available_backend(backend)
def detect_result_type(result: Any) -> str:
    return select_backend_for_data(data=result)

================
File: plugins/repos/twat_cache/src/twat_cache/cache.py
================
class CacheStats(NamedTuple):
class CacheEntry(NamedTuple):
def register_cache(
    _active_caches[name] = CacheEntry(cache, wrapper, stats)
    logger.debug(f"Registered cache: {name}")
def clear_cache(name: str | None = None) -> None:
            _active_caches[name].cache.clear()
            _active_caches[name] = entry._replace(
                stats=CacheStats(0, 0, 0, entry.stats.maxsize)
    for cache_name, entry in list(_active_caches.items()):  # Iterate over a copy
        entry.cache.clear()
        _active_caches[cache_name] = entry._replace(
    cache_dir = Path.home() / ".cache" / "twat_cache"
    if cache_dir.exists():
        for path in cache_dir.glob("*"):
            if path.is_dir():
                    for file in path.glob("*"):
                        file.unlink()
                    path.rmdir()
                    logger.warning(f"Failed to clear cache directory {path}: {e}")
        logger.debug("Cleared cache directories")
def get_stats(name: str | None = None) -> dict[str, Any]:
        "total_caches": len(_active_caches),
    for entry in _active_caches.values():
def update_stats(
    _active_caches[name] = CacheEntry(
        CacheStats(hits, misses, new_size, entry.stats.maxsize),

================
File: plugins/repos/twat_cache/src/twat_cache/config.py
================
class CacheConfig(BaseModel):
    maxsize: int | None = Field(default=None, ge=1)
    ttl: float | None = Field(default=None, ge=0)
    class Config:
    @field_validator("maxsize")
    def validate_maxsize(cls, v: int | None) -> int | None:
            raise ValueError(msg)
    @field_validator("ttl")
    def validate_ttl(cls, v: float | None) -> float | None:
    @field_validator("policy")
    def validate_policy(cls, v: EvictionPolicy) -> EvictionPolicy:
    @field_validator("redis_port")
    def validate_redis_port(cls, v: int) -> int:
    def to_dict(self) -> dict[str, Any]:
    def from_dict(cls, data: dict[str, Any]) -> "CacheConfig":
        return cls(**data)
    def get_maxsize(self) -> int | None:
    def get_folder_name(self) -> str | None:
    def get_use_sql(self) -> bool:
    def get_preferred_engine(self) -> str | None:
    def get_cache_type(self) -> str | None:
    def get_redis_host(self) -> str:
        return os.environ.get("TWAT_CACHE_REDIS_HOST", self.redis_host)
    def get_redis_port(self) -> int:
        port_str = os.environ.get("TWAT_CACHE_REDIS_PORT")
                return int(port_str)
    def get_redis_db(self) -> int:
        db_str = os.environ.get("TWAT_CACHE_REDIS_DB")
                return int(db_str)
    def get_redis_password(self) -> str | None:
        return os.environ.get("TWAT_CACHE_REDIS_PASSWORD", self.redis_password)
    def validate_config(self) -> None:
def create_cache_config(
    return CacheConfig(

================
File: plugins/repos/twat_cache/src/twat_cache/context.py
================
E = TypeVar("E", bound=BaseCacheEngine[Any, Any])
def engine_context(
    engine_config = config or create_cache_config(**kwargs)
    manager = get_engine_manager()
        engine_cls = manager.get_engine(engine_name)
            logger.error(f"Engine '{engine_name}' not found")
            raise EngineError(msg)
        engine_cls = manager.select_engine(engine_config)
            logger.error("No suitable engine found")
    engine = engine_cls(engine_config)
    logger.debug(f"Created engine {engine.__class__.__name__}")
            logger.debug(f"Cleaning up engine {engine.__class__.__name__}")
            engine.cleanup()
            logger.error(f"Error during engine cleanup: {e}")
class CacheContext:
    def __init__(
        self.config = config or create_cache_config(**kwargs)
    def __enter__(self) -> BaseCacheEngine[Any, Any]:
            engine_cls = manager.get_engine(self.engine_name)
                logger.error(f"Engine '{self.engine_name}' not found")
            engine_cls = manager.select_engine(self.config)
        self.engine = engine_cls(self.config)
        logger.debug(f"Created engine {self.engine.__class__.__name__}")
    def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
                logger.debug(f"Cleaning up engine {self.engine.__class__.__name__}")
                self.engine.cleanup()
def get_or_create_engine(

================
File: plugins/repos/twat_cache/src/twat_cache/decorators.py
================
class CacheDecorator(Protocol[P, R]):
    def __call__(self, func: Callable[P, R]) -> Callable[P, R]: ...
class AsyncCacheDecorator(Protocol[P, AsyncR]):
    def __call__(
HAS_AIOCACHE = bool(importlib.util.find_spec("aiocache"))
HAS_CACHEBOX = bool(importlib.util.find_spec("cachebox"))
HAS_CACHETOOLS = bool(importlib.util.find_spec("cachetools"))
HAS_DISKCACHE = bool(importlib.util.find_spec("diskcache"))
HAS_JOBLIB = bool(importlib.util.find_spec("joblib"))
HAS_KLEPTO = bool(importlib.util.find_spec("klepto"))
def get_cache_dir(folder_name: str | None = None) -> Path:
    if folder_name and not all(c.isalnum() or c in "-_." for c in folder_name):
        raise ValueError(msg)
    base_dir = Path.home() / ".cache" / "twat_cache"
        cache_dir.mkdir(parents=True, exist_ok=True)
        raise OSError(msg) from e
def make_key(serializer: Callable[[Any], str] | None = None, *args: Any, **kwargs: Any) -> str:
    def default_serializer(obj: Any) -> Any:
        if isinstance(obj, str | int | float | bool | type(None)):
        return str(obj)
    converted_args = tuple(ser(arg) for arg in args)
    converted_kwargs = {k: ser(v) for k, v in sorted(kwargs.items())}
    return json.dumps((converted_args, converted_kwargs), sort_keys=True)
def mcache(
    logger.debug(f"Creating memory cache with maxsize={maxsize}, ttl={ttl}, policy={policy}")
    config = create_cache_config(
        logger.debug("Using CacheBox engine for memory caching")
        engine: BaseCacheEngine[P, R] = cachebox.CacheBoxEngine(config)  # type: ignore[abstract]
        return cast(CacheDecorator[P, R], engine.cache)
        logger.debug("Using CacheTools engine for memory caching")
        engine = cachetools.CacheToolsEngine(config)  # type: ignore[abstract]
        logger.debug("Using functools engine for memory caching (fallback)")
        engine = functools_engine.FunctoolsEngine(config)  # type: ignore[abstract]
def bcache(
    logger.debug(f"Creating disk cache with folder={folder_name}, maxsize={maxsize}, ttl={ttl}, use_sql={use_sql}")
        logger.debug("Using DiskCache engine for disk caching")
        engine: BaseCacheEngine[P, R] = diskcache.DiskCacheEngine(config)  # type: ignore[abstract]
        logger.debug("Using Klepto engine for disk caching (SQL)")
        engine = klepto.KleptoEngine(config)
        logger.warning("No disk cache backends available, falling back to memory cache")
        return mcache(maxsize=maxsize, ttl=ttl, policy=policy, secure=secure)
def fcache(
    logger.debug(f"Creating file cache with folder={folder_name}, maxsize={maxsize}, ttl={ttl}, compress={compress}")
        logger.debug("Using Joblib engine for file caching")
        engine: BaseCacheEngine[P, R] = JoblibEngine(config)
        logger.debug("Using Klepto engine for file caching (fallback)")
        engine = KleptoEngine(config)
        logger.warning("No file cache backends available, falling back to memory cache")
        return mcache(maxsize=maxsize, ttl=ttl)
def acache(
    logger.debug(f"Creating async cache with maxsize={maxsize}, ttl={ttl}, policy={policy}")
        logger.debug("Using AioCache engine for async caching")
        def decorator(
            engine: Any = aiocache.AioCacheEngine(config)  # type: ignore[abstract]
            cache_func = engine.cache(func)
            @functools.wraps(func)
            async def wrapper(*args: P.args, **kwargs: P.kwargs) -> AsyncR:
                return await cache_func(*args, **kwargs)
        logger.warning("AioCache not available, using memory cache with async wrapper")
        mem_cache = mcache(maxsize=maxsize, ttl=ttl, policy=str(policy))
            cached_func = mem_cache(func)  # type: ignore
                if asyncio.iscoroutinefunction(func):
                    return await cached_func(*args, **kwargs)  # type: ignore
                    return await cached_func(*args, **kwargs)
def _get_available_backends() -> dict[str, bool]:
def _select_best_backend(
    backends = _get_available_backends()
    if preferred and backends.get(preferred):
def _create_engine(config: CacheConfig, func: Callable[P, R]) -> BaseCacheEngine[P, R]:
    is_async = asyncio.iscoroutinefunction(func)
    backend = _select_best_backend(config.preferred_engine, is_async=is_async, needs_disk=needs_disk)
        return aiocache.AioCacheEngine(config)  # type: ignore[abstract]
        return cachebox.CacheBoxEngine(config)  # type: ignore[abstract]
        return cachetools.CacheToolsEngine(config)  # type: ignore[abstract]
        return diskcache.DiskCacheEngine(config)  # type: ignore[abstract]
        return joblib.JoblibEngine(config)  # type: ignore[abstract]
        return klepto.KleptoEngine(config)  # type: ignore[abstract]
        return functools_engine.FunctoolsCacheEngine(config)  # type: ignore[abstract]
def ucache(
    logger.debug(f"Creating universal cache with preferred_engine={preferred_engine}")
    def decorator(func: Callable[P, R]) -> Callable[P, R]:
        backend = preferred_engine or _select_best_backend(is_async=is_async, needs_disk=needs_disk)
        logger.debug(f"Selected {backend} backend for {func.__name__}")
        engine = _create_engine(config, func)
        cached_func = engine.cache(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
                return cached_func(*args, **kwargs)
                logger.warning(f"Cache error in {func.__name__}: {e}. Executing without cache.")
                return func(*args, **kwargs)

================
File: plugins/repos/twat_cache/src/twat_cache/exceptions.py
================
class TwatCacheError(Exception):
    def __init__(self, message: str, *args: Any) -> None:
        super().__init__(message, *args)
class ConfigurationError(TwatCacheError):
class EngineError(TwatCacheError):
class EngineNotAvailableError(EngineError):
    def __init__(self, message: str, reason: str | None = None) -> None:
        super().__init__(message)
class CacheOperationError(TwatCacheError):
class CacheKeyError(CacheOperationError):
class CacheValueError(CacheOperationError):
class SerializationError(CacheOperationError):
class ResourceError(TwatCacheError):
class ConcurrencyError(ResourceError):
class PathError(ResourceError):

================
File: plugins/repos/twat_cache/src/twat_cache/hybrid_cache.py
================
def hybrid_cache(
        small_result_config = CacheConfig(preferred_engine="cachetools", maxsize=128)
        large_result_config = CacheConfig(preferred_engine="diskcache", compress=True)
    def decorator(func: Callable[P, R]) -> Callable[P, R]:
        @functools.wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            cached_func = ucache(
            result = cached_func(*args, **kwargs)
                    size = sys.getsizeof(result)
                    if isinstance(result, list | tuple | set):
                            size += sys.getsizeof(item)
                    elif isinstance(result, dict):
                        for key, value in result.items():
                            size += sys.getsizeof(key) + sys.getsizeof(value)
                        logger.debug(f"Using large result config for {func_key} (size: {size} bytes)")
                        logger.debug(f"Using small result config for {func_key} (size: {size} bytes)")
                    logger.debug(f"Could not determine size for {func_key}, using small result config")
def smart_cache(
        config = CacheConfig()
            result = func(*args, **kwargs)
            backend = detect_result_type(result)
                return cached_func(*args, **kwargs)
            backend_config = CacheConfig(
                preferred_engine=backend, **{k: v for k, v in config.model_dump().items() if v is not None}

================
File: plugins/repos/twat_cache/src/twat_cache/logging.py
================
logger.remove()
LOG_LEVEL = os.environ.get("TWAT_CACHE_LOG_LEVEL", "INFO").upper()
logger.add(
if os.environ.get("TWAT_CACHE_LOG_FILE", "0") == "1":
        xdg_cache_home = os.environ.get("XDG_CACHE_HOME")
            log_dir = Path(xdg_cache_home) / "logs"
            log_dir = Path.home() / ".cache" / "logs"
        log_dir.mkdir(parents=True, exist_ok=True)
            str(log_file),
        logger.info(f"File logging enabled at {log_file}")
        logger.warning(f"Failed to setup file logging: {e}")
        logger.info("Falling back to console-only logging")

================
File: plugins/repos/twat_cache/src/twat_cache/paths.py
================
    logger.warning("platformdirs not available, falling back to ~/.cache")
def get_cache_path(
        folder_name = f"twat_cache_{uuid.uuid4().hex[:8]}"
        base_path = Path(tempfile.gettempdir()) / "twat_cache"
        base_path = Path(platformdirs.user_cache_dir("twat_cache"))
        base_path = Path.home() / ".cache" / "twat_cache"
    if create and not cache_path.exists():
            cache_path.mkdir(parents=True, exist_ok=True)
            logger.debug(f"Created cache directory: {cache_path}")
            logger.error(f"Failed to create cache directory {cache_path}: {e}")
def validate_cache_path(path: str | Path) -> None:
        path = Path(path).resolve()
        if not path.exists():
            path.mkdir(parents=True, exist_ok=True)
        elif not path.is_dir():
            raise ValueError(msg)
        raise ValueError(msg) from e
def clear_cache(folder_name: str | None = None) -> None:
        cache_path = get_cache_path(folder_name, create=False)
        if cache_path.exists():
                for item in cache_path.iterdir():
                    if item.is_file():
                        item.unlink()
                    elif item.is_dir():
                        shutil.rmtree(item)
                logger.info(f"Cleared cache directory: {cache_path}")
                logger.error(f"Failed to clear cache directory {cache_path}: {e}")
            base_paths.append(Path(platformdirs.user_cache_dir("twat_cache")))
        base_paths.append(Path.home() / ".cache" / "twat_cache")
            if base_path.exists():
                    shutil.rmtree(base_path)
                    logger.info(f"Cleared cache base directory: {base_path}")
                    logger.error(

================
File: plugins/repos/twat_cache/src/twat_cache/py.typed
================
1

================
File: plugins/repos/twat_cache/src/twat_cache/type_defs.py
================
T = TypeVar("T")
P = ParamSpec("P")
R = TypeVar("R")
F = TypeVar("F", bound=Callable[..., Any])
AsyncR = TypeVar("AsyncR")
class CacheDecorator(Protocol[P, R]):
    def __call__(self, func: Callable[P, R]) -> Callable[P, R]: ...
class AsyncCacheDecorator(Protocol[P, AsyncR]):
    def __call__(
class CacheConfig(Protocol):
    def get_maxsize(self) -> int | None:
    def get_folder_name(self) -> str | None:
    def get_use_sql(self) -> bool:
    def get_preferred_engine(self) -> str | None:
    def get_cache_type(self) -> str | None:
    def validate_config(self) -> None:
    def model_dump(self) -> dict[str, Any]:
class CacheStats(ABC):
    def hits(self) -> int:
    def misses(self) -> int:
    def size(self) -> int:
    def maxsize(self) -> int | None:
    def clear(self) -> None:
class CacheEngine(ABC, Generic[P, R]):
    def __init__(self, config: CacheConfig) -> None:
        self.validate_config()
    def cache(self, func: Callable[P, R]) -> Callable[P, R]:
    def stats(self) -> dict[str, Any]:
class KeyMaker(Protocol):
    def __call__(self, *args: Any, **kwargs: Any) -> CacheKey:
class Serializer(Protocol):
    def dumps(self, obj: Any) -> bytes:
    def loads(self, data: bytes) -> Any:

================
File: plugins/repos/twat_cache/src/twat_cache/utils.py
================
def get_cache_dir(folder_name: str | None = None) -> Path:
    if folder_name and not all(c.isalnum() or c in "-_." for c in folder_name):
        raise ValueError(msg)
    base_dir = Path.home() / ".cache" / "twat_cache"
        cache_dir.mkdir(parents=True, exist_ok=True)
        raise OSError(msg) from e
def make_key(
    def default_serializer(obj: Any) -> Any:
        if isinstance(obj, str | int | float | bool | type(None)):
        return str(obj)
    converted_args = tuple(ser(arg) for arg in args)
    converted_kwargs = {k: ser(v) for k, v in sorted(kwargs.items())}
    return json.dumps((converted_args, converted_kwargs), sort_keys=True)

================
File: plugins/repos/twat_cache/src/twat_cache.egg-info/dependency_links.txt
================


================
File: plugins/repos/twat_cache/src/twat_cache.egg-info/entry_points.txt
================
[twat.plugins]
cache = twat_cache

================
File: plugins/repos/twat_cache/src/twat_cache.egg-info/PKG-INFO
================
Metadata-Version: 2.2
Name: twat_cache
Version: 2.6.1
Summary: Advanced caching library for Python, part of the twat framework
Author-email: Adam Twardoch <adam+github@twardoch.com>
License: MIT
Project-URL: Homepage, https://github.com/twardoch/twat
Project-URL: Documentation, https://github.com/twardoch/twat/tree/main/docs
Project-URL: Issues, https://github.com/twardoch/twat/issues
Keywords: caching,cache,memorization,performance,optimization
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0
Requires-Dist: loguru>=0.7.0
Requires-Dist: diskcache>=5.6.1
Requires-Dist: joblib>=1.3.2
Requires-Dist: cachetools>=5.3.2
Provides-Extra: cachebox
Requires-Dist: cachebox>=4.5.1; extra == "cachebox"
Provides-Extra: cachetools
Requires-Dist: cachetools>=5.5.1; extra == "cachetools"
Provides-Extra: aiocache
Requires-Dist: aiocache>=0.12.3; extra == "aiocache"
Provides-Extra: klepto
Requires-Dist: klepto>=0.2.6; extra == "klepto"
Provides-Extra: diskcache
Requires-Dist: diskcache>=5.6.3; extra == "diskcache"
Provides-Extra: joblib
Requires-Dist: joblib>=1.4.2; extra == "joblib"
Provides-Extra: all
Requires-Dist: platformdirs>=4.3.6; extra == "all"
Requires-Dist: cachebox>=4.5.1; extra == "all"
Requires-Dist: cachetools>=5.5.1; extra == "all"
Requires-Dist: aiocache>=0.12.3; extra == "all"
Requires-Dist: klepto>=0.2.6; extra == "all"
Requires-Dist: diskcache>=5.6.3; extra == "all"
Requires-Dist: joblib>=1.4.2; extra == "all"

# twat-cache

A flexible caching utility package for Python functions that provides a unified interface for caching function results using various high-performance backends.

## Features

- Simple decorator interface for caching function results
- Multiple caching backends with automatic selection:
  1. `cachebox` - Very fast Rust-based cache (optional)
  2. `cachetools` - Flexible in-memory caching (optional)
  3. `aiocache` - Async-capable caching (optional)
  4. `klepto` - Scientific computing caching (optional)
  5. `diskcache` - SQL-based disk cache (optional)
  6. `joblib` - Efficient array caching (optional)
  7. `redis` - Distributed caching with Redis (optional)
  8. Memory-based LRU cache (always available)
- Automatic cache directory management
- Type hints and modern Python features
- Lazy backend loading - only imports what you use
- Automatic backend selection based on availability and use case
- Smart backend selection based on data characteristics
- TTL support for cache expiration
- Multiple eviction policies (LRU, LFU, FIFO, RR)
- Async function support
- Compression options for large data
- Secure file permissions for sensitive data
- Hybrid caching with automatic backend switching
- Context management for cache engines
- Comprehensive test suite for all components

## Recent Updates (v2.3.0)

### Enhanced Context Management

The context management system has been significantly improved:
- Better error handling and resource cleanup
- Support for explicit engine selection
- Simplified API for temporary cache configurations
- Automatic cleanup of resources even when exceptions occur

```python
# Example of improved context management
from twat_cache import CacheContext

# Create a context with explicit engine selection
with CacheContext(engine_name="redis", namespace="user_data") as cache:
    # Use the cache within the context
    cache.set("user:1001", {"name": "John", "role": "admin"})
    user = cache.get("user:1001")
    
    # Resources are automatically cleaned up when exiting the context
```

### Refined Backend Selection

The backend selection strategy has been further enhanced:
- More accurate data type detection for optimal backend selection
- Improved fallback mechanisms when preferred backends are unavailable
- Better handling of edge cases for various data types
- Enhanced performance for frequently accessed items

### Comprehensive Documentation

Documentation has been expanded with:
- Detailed examples for all cache backends
- Step-by-step guides for common use cases
- API reference with complete parameter descriptions
- Best practices for cache configuration

## Installation

Basic installation with just LRU caching:
```bash
pip install twat-cache
```

With all optional backends:
```bash
pip install twat-cache[all]
```

Or install specific backends:
```bash
pip install twat-cache[cachebox]     # For Rust-based high-performance cache
pip install twat-cache[cachetools]   # For flexible in-memory caching
pip install twat-cache[aiocache]     # For async-capable caching
pip install twat-cache[klepto]       # For scientific computing caching
pip install twat-cache[diskcache]    # For SQL-based disk caching
pip install twat-cache[joblib]       # For efficient array caching
pip install twat-cache[redis]        # For distributed caching with Redis
```

## Usage

### Basic Memory Caching

For simple in-memory caching with LRU eviction:

```python
from twat_cache import mcache

@mcache(maxsize=100)  # Cache up to 100 items
def expensive_function(x: int) -> int:
    # Expensive computation here
    return x * x

# First call computes
result1 = expensive_function(5)  # Computes 25

# Second call uses cache
result2 = expensive_function(5)  # Returns cached 25
```

### Disk-Based Caching

For persistent caching using SQLite:

```python
from twat_cache import bcache

@bcache(
    folder_name="my_cache",  # Cache directory name
    maxsize=1_000_000,       # Max cache size in bytes
    ttl=3600,               # Cache entries expire after 1 hour
    use_sql=True,           # Use SQLite backend
    secure=True,            # Use secure file permissions
)
def expensive_function(x: int) -> int:
    return x * x
```

### Redis Distributed Caching

For distributed caching with Redis:

```python
from twat_cache import ucache

@ucache(
    preferred_engine="redis",
    folder_name="redis_cache",  # Used as Redis namespace
    ttl=3600,                  # Cache entries expire after 1 hour
    compress=True,             # Enable compression
)
def expensive_function(x: int) -> int:
    return x * x
```

### File-Based Caching

For efficient caching of large objects like NumPy arrays:

```python
from twat_cache import fcache
import numpy as np

@fcache(
    folder_name="array_cache",
    compress=True,           # Enable compression
    secure=True,            # Use secure file permissions
)
def process_array(data: np.ndarray) -> np.ndarray:
    # Expensive array processing here
    return data * 2
```

### Async Caching

For async functions with Redis or memory backend:

```python
from twat_cache import ucache

@ucache(use_async=True)
async def fetch_data(url: str) -> dict:
    # Async web request here
    return {"data": "..."}

# First call fetches
data1 = await fetch_data("https://api.example.com")

# Second call uses cache
data2 = await fetch_data("https://api.example.com")
```

### Universal Caching

Let the library choose the best backend:

```python
from twat_cache import ucache

@ucache(
    folder_name="cache",     # Optional - uses disk cache if provided
    maxsize=1000,           # Optional - limits cache size
    ttl=3600,              # Optional - entries expire after 1 hour
    policy="lru",          # Optional - LRU eviction (default)
    use_sql=True,          # Optional - use SQL backend if available
    compress=True,         # Optional - enable compression
    secure=True,           # Optional - secure file permissions
)
def my_function(x: int) -> int:
    return x * x
```

### Smart Backend Selection

Automatically select the best backend based on data characteristics:

```python
from twat_cache import smart_cache

@smart_cache()
def process_data(data_type: str, size: int) -> Any:
    """Process different types of data with automatic backend selection."""
    if data_type == "dict":
        return {f"key_{i}": f"value_{i}" for i in range(size)}
    elif data_type == "list":
        return [i for i in range(size)]
    elif data_type == "str":
        return "x" * size
    else:
        return size
```

### Hybrid Caching

Switch backends based on result size:

```python
from twat_cache import hybrid_cache

@hybrid_cache()
def get_data(size: str) -> Union[Dict[str, Any], List[int]]:
    """Return different sized data with appropriate backend selection."""
    if size == "small":
        # Small result, will use in-memory caching
        return {"name": "Small Data", "value": 42}
    else:
        # Large result, will use disk caching
        return [i for i in range(100000)]
```

### Type-Specific Configuration

Configure caching based on data types:

```python
from twat_cache import ucache, configure_for_numpy, configure_for_json

# For NumPy arrays
@ucache(config=configure_for_numpy())
def process_array(data: np.ndarray) -> np.ndarray:
    return data * 2

# For JSON data
@ucache(config=configure_for_json())
def fetch_json_data(url: str) -> Dict[str, Any]:
    return {"data": [1, 2, 3, 4, 5], "metadata": {"source": url}}
```

### Cache Management

Clear caches and get statistics:

```python
from twat_cache import clear_cache, get_stats

# Clear all caches
clear_cache()

# Get cache statistics
stats = get_stats()
print(stats)  # Shows hits, misses, size, etc.
```

### Context Management

Use cache engines with context management:

```python
from twat_cache import CacheContext, engine_context

# Method 1: Using the CacheContext class
with CacheContext(engine_name="diskcache", folder_name="cache") as cache:
    # Use the cache
    cache.set("key", "value")
    value = cache.get("key")
    
# Method 2: Using the engine_context function
with engine_context(engine_name="redis", ttl=3600) as cache:
    # Use the cache
    cache.set("key", "value")
    value = cache.get("key")
    
# Cache is automatically closed when exiting the context
```

## Advanced Features

### TTL Support

Set time-to-live for cache entries:

```python
from twat_cache import ucache

@ucache(ttl=3600)  # Entries expire after 1 hour
def get_weather(city: str) -> dict:
    # Fetch weather data
    return {"temp": 20}
```

### Eviction Policies

Choose from different cache eviction policies:

```python
from twat_cache import ucache

# Least Recently Used (default)
@ucache(policy="lru")
def function1(x: int) -> int:
    return x * x

# Least Frequently Used
@ucache(policy="lfu")
def function2(x: int) -> int:
    return x * x

# First In, First Out
@ucache(policy="fifo")
def function3(x: int) -> int:
    return x * x
```

## Documentation

For more detailed documentation, see the following resources:

- [Context Management](docs/context_management.md)
- [Backend Selection](docs/backend_selection.md)
- [Cache Engines](docs/cache_engines.md)
- [Configuration Options](docs/configuration.md)
- [API Reference](docs/api_reference.md)

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Rationale

Python provides several powerful caching libraries to help optimize application performance by storing and reusing expensive function results. Let's take an in-depth look at some of the most popular options, comparing their features, backends, methods, and use cases.


### Built-in functools 

Python's standard library offers basic caching functionality through the `functools` module. It provides decorators like `@lru_cache(maxsize, typed)` for simple memoization with a Least Recently Used (LRU) eviction policy. The `@cache` decorator is also available, which is equivalent to an unbounded LRU cache.

- **Backend**: In-memory Python dictionary 
- **Eviction Policy**: LRU or unbounded
- **Concurrency**: Thread-safe via internal locks, but not safe for multi-process use
- **Persistence**: No persistence, cache only exists in memory for the lifetime of the process
- **Best For**: Fast and easy caching of function results in memory with minimal setup

```pyi
### functools.pyi (Partial - Caching related parts)

import typing

_T = typing.TypeVar("_T")

@typing.overload
def lru_cache(maxsize: int | None) -> typing.Callable[[_F], _F]: ...
@typing.overload
def lru_cache(maxsize: int | None, typed: bool) -> typing.Callable[[_F], _F]: ...

class _lru_cache_wrapper(typing.Generic[_F]):
    cache: typing.Dict[typing.Tuple[typing.Any, ...], typing.Any]
    def cache_info(self) -> CacheInfo: ...
    def cache_clear(self) -> None: ...
```

```python
import functools

@functools.lru_cache(maxsize=128)
def expensive_function(x):
    return x * 2
```

### cachetools

The `cachetools` library extends the capabilities of `functools` by offering additional cache classes with various eviction policies like LFU, FIFO, TTL, and RR. It also provides `@cached` and `@cachedmethod` decorators for caching functions and methods.

- **Backend**: In-memory Python dictionary
- **Eviction Policies**: LRU, LFU, TTL, FIFO, RR, TLRU 
- **Concurrency**: Thread-safe, not designed for multi-process use
- **Persistence**: In-memory only, no persistence 
- **Customization**: Configurable `maxsize`, `ttl`, custom `getsizeof` to determine item size
- **Best For**: In-memory caching with specific eviction behavior, e.g. caching web API responses

```pyi
### cachetools.pyi (Partial - Most important classes and decorators)

import typing
from typing import Callable

KT = typing.TypeVar("KT")
VT = typing.TypeVar("VT")
DT = typing.TypeVar("DT")

class Cache(typing.MutableMapping[KT, VT]):
    def __init__(self, maxsize: int, getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    @property
    def maxsize(self) -> int: ...
    @property
    def currsize(self) -> int: ...
    def get(self, key: KT, default: DT = ...) -> VT | DT: ...
    def pop(self, key: KT, default: DT = ...) -> VT | DT: ...
    def setdefault(self, key: KT, default: DT = ...) -> VT | DT: ...
    def clear(self) -> None: ...
    @staticmethod
    def getsizeof(value: Any) -> int: ...

class LRUCache(Cache[KT, VT]):
    def __init__(self, maxsize: int, getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    def popitem(self) -> tuple[KT, VT]: ...

class LFUCache(Cache[KT, VT]):
    def __init__(self, maxsize: int, getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    def popitem(self) -> tuple[KT, VT]: ...

class TTLCache(LRUCache[KT, VT]):
    def __init__(self, maxsize: int, ttl: int, timer: Callable[[], float] | None = ..., getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    @property
    def ttl(self) -> int: ...
    def expire(self, time: float | None = ...) -> list[tuple[KT, VT]]: ...
    def popitem(self) -> tuple[KT, VT]: ...

class _CacheInfo(NamedTuple):
    hits: int
    misses: int
    maxsize: int
    currsize: int

_KeyFunc = Callable[..., typing.Hashable]
_Cache = TypeVar("_Cache", bound=Cache)

@overload
def cached(
    cache: _Cache,
    key: _KeyFunc = ...,
    lock: Any | None = ...,
    info: Literal[False] = ...,
) -> Callable[[_F], _F]: ...
@overload
def cached(
    cache: _Cache,
    key: _KeyFunc = ...,
    lock: Any | None = ...,
    info: Literal[True] = ...,
) -> Callable[[_F], _F]: ...

@overload
def cachedmethod(
    cache: Callable[[Any], _Cache],
    key: _KeyFunc = ...,
    lock: Callable[[Any], Any] | None = ...,
) -> Callable[[_F], _F]: ...
@overload
def cachedmethod(
    cache: Callable[[Any], _Cache],
    key: _KeyFunc = ...,
    lock: None = ...,
) -> Callable[[_F], _F]: ...
```

### cachebox

`cachebox` is an in-memory caching solution accelerated by a Rust backend for enhanced performance. It supports similar eviction policies as `cachetools` including a unique variable TTL cache.

- **Backend**: Rust-backed in-memory store
- **Eviction Policies**: LRU, LFU, FIFO, TTL, RR, variable TTL
- **Concurrency**: Thread-safe Rust implementation, but in-memory only
- **Decorators**: `@cached`, `@cachedmethod` support custom key generation and callbacks
- **Best For**: Performant in-memory caching with ample policy choices

```pyi
### cachebox.pyi (Partial - Most important classes)

import typing

KT = typing.TypeVar("KT")
VT = typing.TypeVar("VT")
DT = typing.TypeVar("DT")


class BaseCacheImpl(Generic[KT, VT]):

    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None: ...
    @property
    def maxsize(self) -> int: ...
    def __len__(self) -> int: ...
    def __contains__(self, key: KT) -> bool: ...
    def __setitem__(self, key: KT, value: VT) -> None: ...
    def __getitem__(self, key: KT) -> VT: ...
    def __delitem__(self, key: KT) -> VT: ...
    def __iter__(self) -> typing.Iterator[KT]: ...
    def capacity(self) -> int: ...
    def is_full(self) -> bool: ...
    def is_empty(self) -> bool: ...
    def insert(self, key: KT, value: VT) -> typing.Optional[VT]: ...
    def get(self, key: KT, default: DT = None) -> typing.Union[VT, DT]: ...
    def pop(self, key: KT, default: DT = None) -> typing.Union[VT, DT]: ...
    def setdefault(
        self, key: KT, default: typing.Optional[DT] = None
    ) -> typing.Optional[VT | DT]: ...
    def popitem(self) -> typing.Tuple[KT, VT]: ...
    def drain(self, n: int) -> int: ...
    def clear(self, *, reuse: bool = False) -> None: ...
    def shrink_to_fit(self) -> None: ...
    def update(
        self, iterable: typing.Union[typing.Iterable[typing.Tuple[KT, VT]], typing.Dict[KT, VT]]
    ) -> None: ...
    def keys(self) -> typing.Iterable[KT]: ...
    def values(self) -> typing.Iterable[VT]: ...
    def items(self) -> typing.Iterable[typing.Tuple[KT, VT]]: ...

class Cache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...
    def popitem(self) -> typing.NoReturn: ...  # not implemented for this class
    def drain(self, n: int) -> typing.NoReturn: ...  # not implemented for this class


class FIFOCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class RRCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class TTLCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        ttl: float,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class LRUCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class LFUCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class VTTLCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        ttl: Optional[float] = 0.0,
        *,
        capacity: int = ...,
    ) -> None:
        ...

_CacheType = TypeVar("_CacheType", bound=BaseCacheImpl)

def cached(cache: typing.Optional[_CacheType], key_maker: typing.Optional[Callable[[tuple, dict], typing.Hashable]] = ..., clear_reuse: bool = ..., callback: typing.Optional[Callable[[int, typing.Any, typing.Any], typing.Any]] = ..., copy_level: int = ..., always_copy: typing.Optional[bool] = ...) -> Callable[[_F], _F]: ...
def cachedmethod(cache: typing.Optional[_CacheType], key_maker: typing.Optional[Callable[[tuple, dict], typing.Hashable]] = ..., clear_reuse: bool = ..., callback: typing.Optional[Callable[[int, typing.Any, typing.Any], typing.Any]] = ..., copy_level: int = ..., always_copy: typing.Optional[bool] = ...) -> Callable[[_F], _F]: ...
```

### klepto

For advanced caching workflows, `klepto` provides a highly flexible solution supporting both in-memory and persistent backends like file archives, SQL databases, and HDF5 files. It allows customizing how cache keys are generated and extends the standard cache eviction policies.

- **Backends**: In-memory dict, file archives, SQL databases, HDF5 files
- **Eviction Policies**: Same as `functools` plus LFU, MRU, RR
- **Key Mapping**: `hashmap`, `stringmap`, `picklemap` algorithms to generate keys
- **Persistence**: Archives cache to disk or database for long-term storage
- **Concurrency**: Locking for thread-safety, some process-safety depending on backend
- **Best For**: Complex scenarios needing transient and persistent caching

```pyi
### key maps
class keymap:
    def __init__(self, typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...
    def __call__(self, *args, **kwargs) -> object: ...

class hashmap(keymap):
    def __init__(self, algorithm: Optional[str] = None,
                 typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...

class stringmap(keymap):
    def __init__(self, encoding: Optional[str] = None,
                 typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...

class picklemap(keymap):
    def __init__(self, serializer: Optional[str] = None,
                 typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...

### base archive
class archive(dict):
    def __init__(self, *args, **kwds) -> None: ...
    def __asdict__(self) -> dict: ...
    def __repr__(self) -> str: ...
    def copy(self, name: Optional[str] = None) -> 'archive': ...
    def load(self, *args) -> None: ...
    def dump(self, *args) -> None: ...
    def archived(self, *on) -> bool: ...
    def sync(self, clear: bool = False) -> None: ...
    def drop(self) -> None: ...
    def open(self, archive: 'archive') -> None: ...

    @property
    def archive(self) -> 'archive': ...
    @archive.setter
    def archive(self, archive: 'archive') -> None: ...
    @property
    def name(self) -> str: ...
    @name.setter
    def name(self, archive: 'archive') -> None: ...
    @property
    def state(self) -> dict: ...
    @state.setter
    def state(self, archive: 'archive') -> None: ...

class dict_archive(archive):
    def __init__(self, *args, **kwds) -> None: ...

class null_archive(archive):
    def __init__(self, *args, **kwds) -> None: ...

class file_archive(archive):
    def __init__(self, filename: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class dir_archive(archive):
    def __init__(self, dirname: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 compression: int = 0, permissions: Optional[int] = None,
                 memmode: Optional[str] = None, memsize: int = 100,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class sqltable_archive(archive):
    def __init__(self, database: Optional[str] = None,
                 table: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class sql_archive(archive):
    def __init__(self, database: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class hdfdir_archive(archive):
    def __init__(self, dirname: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 permissions: Optional[int] = None,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class hdf_archive(archive):
    def __init__(self, filename: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...
```

### diskcache

When caching large datasets that should survive process restarts, `diskcache` shines with its optimized disk-backed storage using SQLite and the filesystem. It offers a range of eviction policies, a sharded `FanoutCache`, and persistent data structures like `Deque` and `Index`.

- **Backend**: SQLite for metadata, filesystem for data
- **Eviction Policies**: Configurable, including LRU and LFU
- **Persistence**: Data persists on disk between process runs
- **Concurrency**: Thread and process-safe 
- **Added Features**: Stampede prevention, throttling, optimized I/O
- **Best For**: Persistent caching for web apps and data pipelines

```pyi
### diskcache.pyi (Partial - Most important classes)

import typing
from typing import Callable, List, Dict, Any, IO, Optional, Tuple, Union

class Cache:
    def __init__(self, directory: Optional[str] = ..., timeout: float = ..., disk: Type[Disk] = ..., **settings: Any) -> None: ...
    @property
    def directory(self) -> str: ...
    @property
    def timeout(self) -> float: ...
    @property
    def disk(self) -> Disk: ...
    def set(self, key: Any, value: Any, expire: Optional[float] = ..., read: bool = ..., tag: Optional[str] = ..., retry: bool = ...) -> bool: ...
    def get(self, key: Any, default: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...) -> Any: ...
    def delete(self, key: Any, retry: bool = ...) -> bool: ...
    def clear(self, retry: bool = ...) -> int: ...
    def volume(self) -> int: ...
    def check(self, fix: bool = ..., retry: bool = ...) -> List[warnings.WarningMessage]: ...
    def transact(self, retry: bool = ...) -> ContextManager[Callable]: ...
    def memoize(self, name: Optional[str] = ..., typed: bool = ..., expire: Optional[float] = ..., tag: Optional[str] = ..., ignore: typing.Iterable[str] = ...) -> Callable[[_F], _F]: ...
    def close(self) -> None: ...
    def volume(self) -> int: ...
    def stats(self, enable: bool = ..., reset: bool = ...) -> Tuple[int, int]: ...
    def volume(self) -> int: ...

class FanoutCache:
    def __init__(self, directory: Optional[str] = ..., shards: int = ..., timeout: float = ..., disk: Type[Disk] = ..., **settings: Any) -> None: ...
    @property
    def directory(self) -> str: ...
    def transact(self, retry: bool = ...) -> ContextManager[Callable]: ...
    def set(self, key: Any, value: Any, expire: Optional[float] = ..., read: bool = ..., tag: Optional[str] = ..., retry: bool = ...) -> bool: ...
    def get(self, key: Any, default: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...) -> Any: ...
    def delete(self, key: Any, retry: bool = ...) -> bool: ...
    def clear(self, retry: bool = ...) -> int: ...
    def volume(self) -> int: ...
    def stats(self, enable: bool = ..., reset: bool = ...) -> Tuple[int, int]: ...
    def memoize(self, name: Optional[str] = ..., typed: bool = ..., expire: Optional[float] = ..., tag: Optional[str] = ..., ignore: typing.Iterable[str] = ...) -> Callable[[_F], _F]: ...
    def close(self) -> None: ...
    def volume(self) -> int: ...

class Disk:
    def __init__(self, directory: str, min_file_size: int = ..., pickle_protocol: int = ...) -> None: ...
    @property
    def min_file_size(self) -> int: ...
    @property
    def pickle_protocol(self) -> int: ...
    def put(self, key: Any) -> Tuple[Union[str, sqlite3.Binary, int, float], bool]: ...
    def get(self, key: Union[str, sqlite3.Binary, int, float], raw: bool) -> Any: ...
    def store(self, value: Any, read: bool, key: Constant = ...) -> Tuple[int, int, Optional[str], Optional[Union[str, sqlite3.Binary, int, float]]]: ...
    def fetch(self, mode: int, filename: Optional[str], value: Optional[Union[str, sqlite3.Binary, int, float]], read: bool) -> Any: ...

class JSONDisk(Disk):
    def __init__(self, directory: str, compress_level: int = ..., **kwargs: Any) -> None: ...
```

### joblib

Designed for scientific computing and ML workflows, `joblib` offers transparent disk-caching for functions, with optimizations for large NumPy arrays. Results are saved to disk and only re-computed when inputs change.

- **Backend**: Filesystem
- **Persistence**: Caches results to disk for costly computations
- **Memoization**: `@memory.cache` decorator for functions
- **Serialization**: Pickle-based with optional compression
- **Concurrency**: Process-safe file locking
- **Best For**: Caching ML models, features, large NumPy arrays

```pyi
### joblib.pyi (Partial - Most important classes and functions)

import typing
from typing import Callable, List, Dict, Any, IO, Optional, Tuple, Union

_F = TypeVar("_F", bound=Callable[..., Any])

class Memory:
    def __init__(self, location: Optional[str] = ..., backend: str = ..., verbose: int = ..., bytes_limit: Optional[Union[int, str]] = ..., mmap_mode: Optional[str] = ..., compress: Union[bool, int] = ..., backend_options: Optional[Dict[str, Any]] = ...) -> None: ...
    @property
    def location(self) -> str: ...
    @property
    def backend(self) -> str: ...
    @property
    def compress(self) -> Union[bool, int]: ...
    @property
    def verbose(self) -> int: ...
    def cache(self, func: Optional[_F] = ..., ignore: Optional[List[str]] = ..., verbose: Optional[int] = ..., mmap_mode: Optional[str] = ..., cache_validation_callback: Optional[Callable[[Dict[str, Any]], bool]] = ...) -> _F: ...
    def clear(self, warn: bool = ...) -> None: ...
    def eval(self, func: _F, *args: Any, **kwargs: Any) -> Any: ...
    def __call__(self, func: _F, *args: Any, **kwargs: Any) -> Any: ...

class Parallel:
    def __init__(self, n_jobs: Optional[int] = ..., backend: Optional[str] = ..., verbose: int = ..., timeout: Optional[float] = ..., pre_dispatch: Union[str, int] = ..., batch_size: Union[str, int] = ..., temp_folder: Optional[str] = ..., max_nbytes: Optional[Union[int, str]] = ..., mmap_mode: Optional[str] = ..., prefer: Optional[str] = ..., require: Optional[str] = ...) -> None: ...
    def __call__(self, iterable: typing.Iterable) -> List[Any]: ...
    def __enter__(self) -> "Parallel": ...
    def __exit__(self, exc_type: Optional[Type[BaseException]], exc_value: Optional[BaseException], traceback: Optional[TracebackType]) -> None: ...
    def submit(self, func: _F, *args: Any, **kwargs: Any) -> concurrent.futures.Future: ...
    def map(self, func: _F, *iterables: typing.Iterable, timeout: Optional[float] = ..., chunksize: int = ...) -> typing.Iterable[Any]: ...
    def dispatch_next(self) -> None: ...
    def print_progress(self) -> None: ...
    def clear(self) -> None: ...
    def __len__(self) -> int: ...

def delayed(function: _F) -> Callable[..., Tuple[Callable, tuple, dict]]: ...
def cpu_count(only_physical_cores: bool = ...) -> int: ...
def effective_n_jobs(n_jobs: int = ...) -> int: ...
def hash(obj: Any, hash_name: str = ..., coerce_mmap: bool = ...) -> str: ...
def dump(value: Any, filename: Union[str, PathLikeStr], compress: Union[bool, int] = ..., protocol: Optional[int] = ..., cache_size: Optional[int] = ...) -> List[str]: ...
def load(filename: Union[str, PathLikeStr], mmap_mode: Optional[str] = ...) -> Any: ...
def register_parallel_backend(name: str, factory: Callable[..., ParallelBackendBase], make_default: bool = ...) -> None: ...

#### aiocache

Built for async applications using the `asyncio` framework, `aiocache` enables non-blocking caching operations. It provides both in-memory (`SimpleMemoryCache`) and distributed options (`RedisCache`, `MemcachedCache`).

- **Backends**: In-memory, Redis, Memcached
- **Async Decorators**: `@cached`, `@cached_stampede`, `@multi_cached`  
- **Serialization**: Pluggable, defaults to JSON
- **Best For**: Asynchronous web frameworks (FastAPI, Sanic), distributed caching

```pyi
### aiocache.pyi (Partial - Most important classes and decorators)

import typing
from typing import Any, Callable, Coroutine, Dict, List, Optional, Tuple, Type, Union

_F = typing.TypeVar("_F", bound=Callable[..., Any])

class BaseCache:
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., plugins: Optional[List[BasePlugin]] = ..., namespace: Optional[str] = ..., timeout: float = ..., ttl: Optional[int] = ...) -> None: ...
    @classmethod
    def parse_uri_path(cls, path: str) -> Dict[str, str]: ...
    async def add(self, key: str, value: Any, ttl: Optional[int] = ..., dumps_fn: Optional[Callable[[Any], bytes]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def get(self, key: str, default: Optional[Any] = ..., loads_fn: Optional[Callable[[bytes], Any]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> Any: ...
    async def multi_get(self, keys: List[str], loads_fn: Optional[Callable[[bytes], Any]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> List[Any]: ...
    async def set(self, key: str, value: Any, ttl: Optional[int] = ..., dumps_fn: Optional[Callable[[Any], bytes]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ..., _cas_token: Optional[Any] = ...) -> bool: ...
    async def multi_set(self, pairs: List[Tuple[str, Any]], ttl: Optional[int] = ..., dumps_fn: Optional[Callable[[Any], bytes]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def delete(self, key: str, namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> int: ...
    async def exists(self, key: str, namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def increment(self, key: str, delta: int = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> int: ...
    async def expire(self, key: str, ttl: Optional[int] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def clear(self, namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def close(self, timeout: Optional[int] = ...) -> bool: ...
    async def raw(self, command: str, *args: Any, encoding: Optional[str] = ..., timeout: Optional[int] = ..., **kwargs: Any) -> Any: ...

class SimpleMemoryCache(BaseCache):
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., **kwargs: Any) -> None: ...

class RedisCache(BaseCache):
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., **kwargs: Any) -> None: ...

class MemcachedCache(BaseCache):
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., **kwargs: Any) -> None: ...

class BaseSerializer:
    DEFAULT_ENCODING: Optional[str]
    def __init__(self, *args: Any, encoding: Union[str, object] = ..., **kwargs: Any) -> None: ...
    def dumps(self, value: Any) -> bytes: ...
    def loads(self, value: bytes) -> Any: ...

class JsonSerializer(BaseSerializer):
    def dumps(self, value: Any) -> str: ...
    def loads(self, value: str) -> Any: ...

class PickleSerializer(BaseSerializer):
    def dumps(self, value: Any) -> bytes: ...
    def loads(self, value: bytes) -> Any: ...

class NullSerializer(BaseSerializer):
    def dumps(self, value: Any) -> Any: ...
    def loads(self, value: Any) -> Any: ...

class BasePlugin:
    async def pre_get(self, client: BaseCache, key: str, namespace: Optional[str], **kwargs: Any) -> None: ...
    async def post_get(self, client: BaseCache, key: str, namespace: Optional[str], rv: Any, **kwargs: Any) -> None: ...
    # ... (and similar methods for other cache operations)

_Cache = TypeVar("_Cache", bound=BaseCache)

def cached(
    ttl: Optional[int] = ...,
    key: Optional[str] = ...,
    key_builder: Optional[Callable[..., str]] = ...,
    namespace: Optional[str] = ...,
    cache: Union[Type[_Cache], str] = ...,
    serializer: Optional[BaseSerializer] = ...,
    plugins: Optional[List[BasePlugin]] = ...,
    alias: Optional[str] = ...,
    noself: bool = ...,
    skip_cache_func: Optional[Callable[[Any], bool]] = ...,
    **kwargs: Any
) -> Callable[[_F], _F]: ...

def cached_stampede(
    lease: int = ...,
    ttl: Optional[int] = ...,
    key: Optional[str] = ...,
    key_builder: Optional[Callable[..., str]] = ...,
    namespace: Optional[str] = ...,
    cache: Union[Type[_Cache], str] = ...,
    serializer: Optional[BaseSerializer] = ...,
    plugins: Optional[List[BasePlugin]] = ...,
    alias: Optional[str] = ...,
    noself: bool = ...,
    skip_cache_func: Optional[Callable[[Any], bool]] = ...,
    **kwargs: Any
) -> Callable[[_F], _F]: ...

def multi_cached(
    keys_from_attr: Optional[str] = ...,
    namespace: Optional[str] = ...,
    key_builder: Optional[Callable[..., str]] = ...,
    ttl: Optional[int] = ...,
    cache: Union[Type[_Cache], str] = ...,
    serializer: Optional[BaseSerializer] = ...,
    plugins: Optional[List[BasePlugin]] = ...,
    alias: Optional[str] = ...,
    skip_cache_func: Optional[Callable[[str, Any], bool]] = ...,
    **kwargs: Any
) -> Callable[[_F], _F]: ...
```

### Summary of type hints

```pyi
### Type Stubs for Caching Libraries

from typing import (
    Any,
    Callable,
    Dict,
    List,
    Optional,
    Type,
    Iterable,
    Tuple,
    Union,
    TypeVar,
    Generic,
)
import datetime
import logging
from typing_extensions import Literal  # For older Python versions
```

*   **Generics:** For generic types like `Cache`, `FIFOCache`, `LFUCache`, etc. (in `cachetools`, `cachebox`, `klepto`) , I've used `TypeVar` to represent the key (`KT`) and value (`VT`) types.  This gives better type hinting.
*   **`object` as Default:** Where the original code uses implicit dynamic typing (no explicit type) or uses an internal implementation detail, I've often used `object` as the most general type hint.  This avoids creating stubs for internal classes.
*   **`NotImplementedError` and `typing.NoReturn`:** I've used `typing.NoReturn` for methods that are *not implemented* in a base class.  This is more precise than raising `NotImplementedError`, which would imply that a subclass *should* implement it.
*   **`defaultdict`:**  I've handled cases where `defaultdict` is used internally, providing a default factory function.
*   **`_Conn`:**  This internal helper class from `aiocache` is fully defined as all its methods are based on exported API.
* **`_DISTUTILS_PATCH`, `VIRTUALENV_PATCH_FILE`, etc.:** These internal constants/classes, used for patching `distutils`, are omitted.
*   **`kwargs` Handling:** In many cases, I've used `**kwargs` to represent arbitrary keyword arguments, especially when those arguments are passed directly to another function (like `open` or a backend-specific constructor).
* **`functools`:** The return type of decorators (`lru_cache`, `cache`) is `Callable[[Callable[..., R]], Callable[..., R]]`.  I've used a `TypeVar` named `_R` to make the relationship between the input and output types clear.
* **`klepto` classes**: Added the API for the `archive`, `dir_archive`, `sql_archive`, `sqltable_archive` based on the source code.
* **`joblib` classes**: Added the classes in the public API, `Memory`, `MemorizedFunc`, `NotMemorizedFunc`.

#### Recommendations

The best caching library depends on your specific needs - whether you require persistence, have large datasets, need multi-process safety, or are using an async framework. By understanding the strengths of each library, you can make an informed choice to optimize your application's performance.

================
File: plugins/repos/twat_cache/src/twat_cache.egg-info/requires.txt
================
pydantic>=2.0
loguru>=0.7.0
diskcache>=5.6.1
joblib>=1.3.2
cachetools>=5.3.2

[aiocache]
aiocache>=0.12.3

[all]
platformdirs>=4.3.6
cachebox>=4.5.1
cachetools>=5.5.1
aiocache>=0.12.3
klepto>=0.2.6
diskcache>=5.6.3
joblib>=1.4.2

[cachebox]
cachebox>=4.5.1

[cachetools]
cachetools>=5.5.1

[diskcache]
diskcache>=5.6.3

[joblib]
joblib>=1.4.2

[klepto]
klepto>=0.2.6

================
File: plugins/repos/twat_cache/src/twat_cache.egg-info/SOURCES.txt
================
.gitignore
.pre-commit-config.yaml
CHANGELOG.md
CLEANUP.txt
LICENSE
LOG.md
NEXT.md
PROMPT.txt
README.md
REPO_CONTENT.txt
TODO.md
VERSION.txt
cleanup.py
pyproject.toml
uv.lock
.cursor/rules/0project.mdc
.cursor/rules/cleanup.mdc
.cursor/rules/filetree.mdc
.cursor/rules/quality.mdc
.github/workflows/push.yml
.github/workflows/release.yml
.specstory/history/.what-is-this.md
.specstory/history/2025-03-04_03-25-codebase-analysis-and-todo-md-update.md
.specstory/history/2025-03-04_05-35-implementation-of-todo-phases-1,-2,-3.md
.specstory/history/2025-03-04_08-00-project-maintenance-and-documentation-update.md
.specstory/history/2025-03-04_08-41-managing-todo-md-tasks.md
.specstory/history/caching-strategies-for-twat-cache-library.md
.specstory/history/codebase-improvement-and-dependency-management-plan.md
.specstory/history/documentation-update-and-task-management.md
.specstory/history/implementing-todo-item.md
.specstory/history/merging-files-from-wrong-location.md
.specstory/history/project-documentation-and-command-execution.md
.specstory/history/project-overview-and-todo-execution.md
.specstory/history/project-review-and-implementation.md
.specstory/history/project-setup-and-implementation-steps.md
.specstory/history/project-setup-and-task-implementation.md
.specstory/history/project-setup-and-task-management.md
.specstory/history/refactoring-plan-for-twat-cache-system.md
.specstory/history/refining-decorators-and-integration-goals-1.md
.specstory/history/refining-decorators-and-integration-goals.md
.specstory/history/running-lint-fix-command.md
.specstory/history/todo-list-update-and-progress-logging.md
.specstory/history/update-todo-and-log-for-project.md
.specstory/history/updating-todo-and-running-development-commands.md
.specstory/history/updating-todos-and-project-documentation.md
docs/context_management.md
examples/backend_selection.py
src/twat_cache/__init__.py
src/twat_cache/__main__.py
src/twat_cache/__version__.py
src/twat_cache/backend_selector.py
src/twat_cache/cache.py
src/twat_cache/config.py
src/twat_cache/context.py
src/twat_cache/decorators.py
src/twat_cache/exceptions.py
src/twat_cache/hybrid_cache.py
src/twat_cache/paths.py
src/twat_cache/py.typed
src/twat_cache/type_defs.py
src/twat_cache/utils.py
src/twat_cache.egg-info/PKG-INFO
src/twat_cache.egg-info/SOURCES.txt
src/twat_cache.egg-info/dependency_links.txt
src/twat_cache.egg-info/entry_points.txt
src/twat_cache.egg-info/requires.txt
src/twat_cache.egg-info/top_level.txt
src/twat_cache/engines/__init__.py
src/twat_cache/engines/aiocache.py
src/twat_cache/engines/base.py
src/twat_cache/engines/cachebox.py
src/twat_cache/engines/cachetools.py
src/twat_cache/engines/common.py
src/twat_cache/engines/diskcache.py
src/twat_cache/engines/functools.py
src/twat_cache/engines/functools_engine.py
src/twat_cache/engines/joblib.py
src/twat_cache/engines/klepto.py
src/twat_cache/engines/manager.py
src/twat_cache/engines/py.typed
src/twat_cache/engines/redis.py
src/twat_cache/types/cachebox.pyi
tests/__init__.py
tests/test_backend_selector.py
tests/test_cache.py
tests/test_config.py
tests/test_constants.py
tests/test_context.py
tests/test_context_simple.py
tests/test_decorators.py
tests/test_engines.py
tests/test_exceptions.py
tests/test_exceptions_simple.py
tests/test_fallback.py
tests/test_redis_cache.py
tests/test_twat_cache.py

================
File: plugins/repos/twat_cache/src/twat_cache.egg-info/top_level.txt
================
twat_cache

================
File: plugins/repos/twat_cache/tests/__init__.py
================


================
File: plugins/repos/twat_cache/tests/test_backend_selector.py
================
class TestDataSizeEstimation:
    def test_small_data(self) -> None:
        assert estimate_data_size(small_data) == DataSize.SMALL
        small_list = list(range(100))
        assert estimate_data_size(small_list) == DataSize.SMALL
        small_dict = {i: i * 2 for i in range(50)}
        assert estimate_data_size(small_dict) == DataSize.SMALL
    def test_medium_data(self) -> None:
        assert estimate_data_size(medium_string) == DataSize.MEDIUM
        medium_list = list(range(5000))
        if estimate_data_size(medium_list) != DataSize.MEDIUM:
            medium_list = list(range(50000))
            assert estimate_data_size(medium_list) == DataSize.MEDIUM
    def test_large_data(self) -> None:
        assert estimate_data_size(large_string) == DataSize.LARGE
    def test_numpy_array_size(self) -> None:
        small_array = np.ones((10, 10))
        assert estimate_data_size(small_array) == DataSize.SMALL
        medium_array = np.ones((500, 500))
        size = estimate_data_size(medium_array)
    def test_pandas_dataframe_size(self) -> None:
        small_df = pd.DataFrame({"a": range(100), "b": range(100)})
        assert estimate_data_size(small_df) == DataSize.SMALL
        medium_df = pd.DataFrame(np.random.rand(1000, 10))
        size = estimate_data_size(medium_df)
class TestTypeBasedBackendSelection:
    def test_primitive_types(self) -> None:
        assert get_type_based_backend(int) in ("cachetools", "functools")
        assert get_type_based_backend(str) in ("cachetools", "functools")
        assert get_type_based_backend(bool) in ("cachetools", "functools")
        assert get_type_based_backend(float) in ("cachetools", "functools")
    def test_container_types(self) -> None:
        assert get_type_based_backend(list) in ("cachetools", "diskcache")
        assert get_type_based_backend(dict) in ("cachetools", "diskcache")
        assert get_type_based_backend(tuple) in ("cachetools", "diskcache")
        assert get_type_based_backend(set) in ("cachetools", "diskcache")
    def test_numpy_types(self) -> None:
        assert get_type_based_backend(np.ndarray) in ("joblib", "diskcache")
    def test_pandas_types(self) -> None:
        assert get_type_based_backend(pd.DataFrame) in ("joblib", "diskcache")
        assert get_type_based_backend(pd.Series) in ("joblib", "diskcache")
    def test_custom_types(self) -> None:
        class CustomClass:
        assert get_type_based_backend(CustomClass) == "diskcache"
class TestBackendAvailability:
    def test_is_backend_available(self) -> None:
        assert is_backend_available("diskcache") is True
        assert is_backend_available("cachetools") is True
        assert is_backend_available("klepto") is True
        assert is_backend_available("joblib") is True
        assert is_backend_available("nonexistent_backend") is False
    def test_get_available_backend(self) -> None:
        assert get_available_backend("diskcache") == "diskcache"
        assert get_available_backend("nonexistent_backend") in get_engine_manager().get_available_engines()
class TestSelectBackendForData:
    def test_select_with_data_instance(self) -> None:
        assert select_backend_for_data(data=42) in ("cachetools", "functools")
        assert select_backend_for_data(data=medium_list) in ("diskcache", "klepto")
        np_array = np.ones((100, 100))
        assert select_backend_for_data(data=np_array) in ("joblib", "diskcache")
        df = pd.DataFrame({"a": range(100), "b": range(100)})
        assert select_backend_for_data(data=df) in ("joblib", "diskcache")
    def test_select_with_explicit_parameters(self) -> None:
        backend = select_backend_for_data(
    def test_select_with_config(self) -> None:
        config = create_cache_config(preferred_engine="diskcache")
        assert select_backend_for_data(config=config) == "diskcache"
        config = create_cache_config(cache_type="memory")
        assert select_backend_for_data(config=config) in ("cachetools", "functools")
        config = create_cache_config(use_sql=True)
        assert select_backend_for_data(config=config) in ("diskcache", "klepto")
class TestTypeSpecificConfigurations:
    def test_configure_for_type(self) -> None:
        int_config = configure_for_type(int)
        list_config = configure_for_type(list)
        np_config = configure_for_type(np.ndarray)
    def test_configure_for_numpy(self) -> None:
        config = configure_for_numpy()
    def test_configure_for_pandas(self) -> None:
        config = configure_for_pandas()
    def test_configure_for_images(self) -> None:
        config = configure_for_images()
    def test_configure_for_json(self) -> None:
        config = configure_for_json()
class TestHybridCacheConfig:
    def test_hybrid_cache_config(self) -> None:
        config = hybrid_cache_config()
        assert isinstance(config, dict)
        custom_config = hybrid_cache_config(
class TestFunctionAnalysis:
    def test_analyze_function_return_type(self) -> None:
        def returns_int() -> int:
        def returns_list() -> list[int]:
        def returns_dict() -> dict[str, Any]:
        def returns_numpy() -> np.ndarray:
            return np.ones((10, 10))
        assert analyze_function_return_type(returns_int) == int
        assert analyze_function_return_type(returns_list) == list[int]
        assert analyze_function_return_type(returns_dict) == dict[str, Any]
        assert analyze_function_return_type(returns_numpy) == np.ndarray
        def no_annotation():
        assert analyze_function_return_type(no_annotation) is None
    def test_smart_cache_config(self) -> None:
        int_config = smart_cache_config(returns_int)
        numpy_config = smart_cache_config(returns_numpy)
        custom_config = smart_cache_config(returns_int, maxsize=100, ttl=60)
class TestResultTypeDetection:
    def test_detect_result_type(self) -> None:
        assert detect_result_type(42) == "int"
        assert detect_result_type(3.14) == "float"
        assert detect_result_type("hello") == "str"
        assert detect_result_type(True) == "bool"
        assert detect_result_type([1, 2, 3]) == "list"
        assert detect_result_type({"a": 1, "b": 2}) == "dict"
        assert detect_result_type((1, 2, 3)) == "tuple"
        assert detect_result_type({1, 2, 3}) == "set"
        assert detect_result_type(np.ones(10)) == "numpy.ndarray"
        assert detect_result_type(pd.DataFrame({"a": [1, 2, 3]})) == "pandas.DataFrame"
        assert detect_result_type(pd.Series([1, 2, 3])) == "pandas.Series"
        assert detect_result_type(CustomClass()) == "CustomClass"

================
File: plugins/repos/twat_cache/tests/test_cache.py
================
def test_config_validation() -> None:
    config = create_cache_config(maxsize=CACHE_SIZE)
    with pytest.raises(ValueError):
        create_cache_config(maxsize=-1)
        create_cache_config(ttl=-1)
def test_cache_path() -> None:
    path = get_cache_path()
    assert path.exists()
    assert path.is_dir()
    with tempfile.TemporaryDirectory() as temp_dir:
        temp_path = Path(temp_dir)
        path = get_cache_path(folder_name="test", cache_dir=str(temp_path))
def test_cache_clear() -> None:
        test_file.touch()
        clear_cache(folder_name="test", cache_dir=str(temp_path))
        assert not test_file.exists()
def test_cache_stats() -> None:
    clear_cache()
    @mcache()
    def process_list(lst: list[int]) -> int:
        return sum(lst)
    result1 = process_list(TEST_LIST)
    result2 = process_list(TEST_LIST)
    stats = get_stats()
    assert isinstance(stats, dict)
def test_cache_security() -> None:
        cache_path = get_cache_path()
        mode = cache_path.stat().st_mode & 0o777
def test_cache_decorators() -> None:
    def mem_func(x: int) -> int:
    assert mem_func(SQUARE_INPUT) == SQUARE_RESULT
    assert mem_func(SQUARE_INPUT) == SQUARE_RESULT  # Should use cache
    @bcache()
    def disk_func(x: int) -> int:
    assert disk_func(SQUARE_INPUT) == SQUARE_RESULT
    assert disk_func(SQUARE_INPUT) == SQUARE_RESULT  # Should use cache
    @fcache()
    def file_func(x: int) -> int:
    assert file_func(SQUARE_INPUT) == SQUARE_RESULT
    assert file_func(SQUARE_INPUT) == SQUARE_RESULT  # Should use cache
def test_cache_basic():
    def square(x: int) -> int:
    result1 = square(TEST_VALUE)
    result2 = square(TEST_VALUE)
def test_cache_size():
    @mcache(maxsize=SMALL_CACHE_SIZE)
    for i in range(SMALL_CACHE_SIZE * 2):
        square(i)
def test_cache_clear():
    square.cache_clear()
def test_cache_stats():
    square(TEST_VALUE)
    assert square.cache_info().misses == 1
    assert square.cache_info().hits == 0
    assert square.cache_info().hits == 1
def test_cache_permissions():
        @bcache(folder_name=TEST_CACHE_DIR)
        def func(x: int) -> int:
        func(TEST_VALUE)
        mode = os.stat(cache_path).st_mode & 0o777
def test_cache_types():
    assert mem_func(TEST_VALUE) == TEST_RESULT
    assert mem_func(TEST_VALUE) == TEST_RESULT  # Should use cache
    assert disk_func(TEST_VALUE) == TEST_RESULT
    assert disk_func(TEST_VALUE) == TEST_RESULT  # Should use cache
    @fcache(folder_name=TEST_CACHE_DIR)
    assert file_func(TEST_VALUE) == TEST_RESULT
    assert file_func(TEST_VALUE) == TEST_RESULT  # Should use cache
def test_list_processing() -> None:
    @ucache()
    def process_list(data: list[int]) -> int:
        return sum(data)
    result1 = process_list(test_list)
    result2 = process_list(test_list)
def test_different_backends() -> None:
        create_cache_config(maxsize=100),  # Memory cache
        create_cache_config(folder_name="disk_test", use_sql=True),  # SQL cache
        create_cache_config(folder_name="disk_test", use_sql=False),  # Disk cache
        @ucache(
        def cached_function(x: int) -> int:
        result = cached_function(5)
        results.append(result)
        assert all(r == results[0] for r in results)
def test_cache_with_complex_types() -> None:
    @ucache(maxsize=100)
    def process_list(items: list[int]) -> int:
        return sum(items)
def test_cache_exceptions() -> None:
    def failing_function(x: int) -> None:
        raise ValueError(msg)
        failing_function(5)
def test_kwargs_handling() -> None:
    def kwarg_function(x: int, multiplier: int = 1) -> int:
    assert kwarg_function(TEST_INPUT) == TEST_INPUT
    assert kwarg_function(TEST_INPUT, multiplier=1) == TEST_INPUT
    assert kwarg_function(TEST_INPUT, multiplier=2) == TEST_INPUT * 2
def test_ttl_caching(tmp_path: Path) -> None:
    @ucache(folder_name=str(tmp_path), ttl=TTL_DURATION)
    assert cached_function(TEST_INPUT) == TEST_RESULT
    time.sleep(TTL_DURATION + 0.1)
def test_security_features(tmp_path: Path) -> None:
    @ucache(folder_name=str(tmp_path), secure=True)
    cached_function(TEST_INPUT)
    assert cache_path.exists()
async def test_async_caching() -> None:
    @ucache(use_async=True)
    async def async_function(x: int) -> int:
        await asyncio.sleep(0.1)  # Simulate async work
    result1 = await async_function(TEST_INPUT)
    result2 = await async_function(TEST_INPUT)
def test_specific_decorators() -> None:
    @mcache(maxsize=100)
    assert mem_func(5) == 25
    assert mem_func(5) == 25  # Should use cache
    @bcache(folder_name="test_disk")
    assert disk_func(5) == 25
    assert disk_func(5) == 25  # Should use cache
    @fcache(folder_name="test_file")
    assert file_func(5) == 25
    assert file_func(5) == 25  # Should use cache
def test_basic_memory_cache():
    square.clear()
    result3 = square(TEST_VALUE)

================
File: plugins/repos/twat_cache/tests/test_config.py
================
def test_cache_config_defaults():
    config = CacheConfig()
def test_cache_config_validation():
    config = create_cache_config(
    with pytest.raises(ValidationError):
        create_cache_config(maxsize=-1)
        create_cache_config(ttl=-1)
        create_cache_config(compression_level=MAX_COMPRESSION + 1)
def test_cache_config_permissions():
    config = create_cache_config(secure=True)
    config = create_cache_config(secure=False)
def test_cache_config_size_limits():
    config = create_cache_config(maxsize=SMALL_CACHE_SIZE)
    config = create_cache_config(maxsize=DEFAULT_CACHE_SIZE)
    config = create_cache_config(maxsize=None)
def test_cache_config_ttl():
    config = create_cache_config(ttl=SHORT_TTL)
    config = create_cache_config(ttl=DEFAULT_TTL)
    config = create_cache_config(ttl=None)
def test_cache_config_compression():
    config = create_cache_config(compression_level=MIN_COMPRESSION)
    config = create_cache_config(compression_level=MAX_COMPRESSION)
    config = create_cache_config(compression_level=DEFAULT_COMPRESSION)

================
File: plugins/repos/twat_cache/tests/test_constants.py
================


================
File: plugins/repos/twat_cache/tests/test_context_simple.py
================
def test_cache_context_init():
    config = CacheConfig(maxsize=100)
    context = CacheContext(config=config, engine_name="memory")
@patch("twat_cache.context.get_engine_manager")
def test_cache_context_enter_exit(mock_get_engine_manager):
    mock_manager = MagicMock()
    mock_engine_cls = MagicMock()
    mock_engine = MagicMock()
    with CacheContext(config=config) as engine:
    mock_engine.cleanup.assert_called_once()
def test_cache_context_error_handling(mock_get_engine_manager):
    mock_engine.cleanup.side_effect = Exception("Test exception")
    pytest.main(["-v", __file__])

================
File: plugins/repos/twat_cache/tests/test_context.py
================
def test_engine_context():
    with tempfile.TemporaryDirectory() as temp_dir:
        with engine_context(folder_name=temp_dir) as engine:
            assert isinstance(engine, BaseCacheEngine)
            def add(x, y):
            result = add(2, 3)
def test_engine_context_with_error():
                raise ValueError(msg)
def test_engine_context_with_invalid_engine():
    with pytest.raises(EngineError):
        with engine_context(engine_name="non_existent_engine"):
def test_cache_context_class():
        context = CacheContext(folder_name=temp_dir)
            def multiply(x, y):
            result = multiply(2, 3)
def test_get_or_create_engine():
        engine = get_or_create_engine(folder_name=temp_dir)
            def divide(x, y):
            result = divide(6, 3)
            engine.cleanup()
def test_nested_contexts():
    with tempfile.TemporaryDirectory() as temp_dir1:
        with tempfile.TemporaryDirectory() as temp_dir2:
            with engine_context(folder_name=temp_dir1) as engine1:
                with engine_context(folder_name=temp_dir2) as engine2:
                    assert add(2, 3) == 5
                    assert multiply(2, 3) == 6
def test_context_with_config():
    config = create_cache_config(
    with engine_context(config=config) as engine:

================
File: plugins/repos/twat_cache/tests/test_decorators.py
================
HAS_AIOCACHE = bool(importlib.util.find_spec("aiocache"))
HAS_CACHEBOX = bool(importlib.util.find_spec("cachebox"))
HAS_CACHETOOLS = bool(importlib.util.find_spec("cachetools"))
HAS_DISKCACHE = bool(importlib.util.find_spec("diskcache"))
HAS_JOBLIB = bool(importlib.util.find_spec("joblib"))
HAS_KLEPTO = bool(importlib.util.find_spec("klepto"))
def test_basic_memory_cache():
    @bcache()
    def square(x: int) -> int:
    result1 = square(TEST_VALUE)
    result2 = square(TEST_VALUE)
@pytest.mark.skipif(not HAS_CACHEBOX, reason="cachebox not available")
def test_cachebox_memory():
    @bcache(engine="cachebox")
@pytest.mark.skipif(not HAS_CACHETOOLS, reason="cachetools not available")
def test_cachetools_memory():
    @bcache(engine="cachetools")
@pytest.mark.skipif(not HAS_DISKCACHE, reason="diskcache not available")
def test_diskcache_basic():
    @bcache(engine="diskcache", folder_name=TEST_FOLDER)
@pytest.mark.skipif(not HAS_KLEPTO, reason="klepto not available")
def test_klepto_sql():
    @bcache(engine="klepto", folder_name=TEST_FOLDER)
@pytest.mark.skipif(not HAS_JOBLIB, reason="joblib not available")
def test_joblib_file():
    @bcache(engine="joblib", folder_name=TEST_FOLDER)
def test_klepto_file():
@pytest.mark.skipif(not HAS_AIOCACHE, reason="aiocache not available")
def test_aiocache_memory():
    @ucache(engine="aiocache")
    async def square(x: int) -> int:
def test_memory_cache_cachebox() -> None:
        pytest.skip("cachebox not available")
    @mcache(maxsize=CACHE_SIZE)
    def expensive_func(x: int) -> int:
    expensive_func(1)
    expensive_func(2)
    expensive_func(3)
    expensive_func(1)  # Should recompute
def test_memory_cache_cachetools() -> None:
        pytest.skip("cachetools not available")
    assert expensive_func(SQUARE_INPUT) == SQUARE_RESULT
def test_basic_disk_cache() -> None:
        pytest.skip("diskcache not available")
    @bcache(maxsize=CACHE_SIZE)
def test_basic_disk_cache_sql() -> None:
        pytest.skip("klepto not available")
    @bcache(maxsize=CACHE_SIZE, use_sql=True)
def test_file_cache_joblib() -> None:
        pytest.skip("joblib not available")
    @fcache(maxsize=CACHE_SIZE)
def test_file_cache_klepto() -> None:
def test_universal_cache() -> None:
    @ucache(maxsize=CACHE_SIZE)
async def test_async_cache() -> None:
        pytest.skip("aiocache not available")
    @acache(maxsize=CACHE_SIZE)
    async def expensive_func(x: int) -> int:
        await asyncio.sleep(0.1)
    result1 = await expensive_func(SQUARE_INPUT)
    result2 = await expensive_func(SQUARE_INPUT)
def test_key_generation() -> None:
    @mcache()
    def test_func(x: Any) -> str:
        return str(x)
    test_func(TEST_KEY)
    test_func(TEST_BOOL)
    test_func(TEST_INT)
def test_ttl_cache() -> None:
    @mcache(ttl=CACHE_TTL)
    time.sleep(CACHE_TTL + 0.1)

================
File: plugins/repos/twat_cache/tests/test_engines.py
================
def engine_manager() -> EngineManager:
    return EngineManager()
def base_config() -> CacheConfig:
    return create_cache_config(maxsize=CACHE_SIZE)
def base_engine(base_config: CacheConfig) -> CacheEngine[Any, Any]:
    return FunctoolsCacheEngine(base_config)
def create_test_engine() -> CacheEngine:
    config = CacheConfig(
    return FunctoolsCacheEngine(config)
def test_engine_initialization() -> None:
    engine = create_test_engine()
    assert isinstance(engine, CacheEngine)
def test_key_generation() -> None:
    assert isinstance(engine._make_key(TEST_KEY), str)
    assert isinstance(engine._make_key(TEST_INT), str)
    assert isinstance(engine._make_key(TEST_BOOL), str)
    assert isinstance(engine._make_key(TEST_LIST), str)
    assert isinstance(engine._make_key((1, 2, 3)), str)
    assert isinstance(engine._make_key({"a": 1}), str)
def test_cache_operations() -> None:
    create_test_engine()
    def test_function(x: int) -> int:
    result = test_function(SQUARE_INPUT)
def test_cache_eviction() -> None:
    for i in range(CACHE_SIZE + 1):
        engine.set(str(i), i)
    assert len(engine._cache) <= CACHE_SIZE
def test_cache_ttl() -> None:
    time.sleep(CACHE_TTL + 0.1)
def test_cache_clear() -> None:
    for i in range(CACHE_SIZE):
    engine.clear()
    assert len(engine._cache) == 0
def test_cache_stats() -> None:

================
File: plugins/repos/twat_cache/tests/test_exceptions_simple.py
================
def test_exception_hierarchy():
    assert issubclass(TwatCacheError, Exception)
    assert issubclass(ConfigurationError, TwatCacheError)
    assert issubclass(EngineError, TwatCacheError)
    assert issubclass(CacheOperationError, TwatCacheError)
    assert issubclass(ResourceError, TwatCacheError)
    assert issubclass(EngineNotAvailableError, EngineError)
    assert issubclass(CacheKeyError, CacheOperationError)
    assert issubclass(CacheValueError, CacheOperationError)
    assert issubclass(SerializationError, CacheOperationError)
    assert issubclass(ConcurrencyError, ResourceError)
    assert issubclass(PathError, ResourceError)
def test_exception_messages():
    base_exc = TwatCacheError("Base error message")
    assert str(base_exc) == "Base error message"
    config_exc = ConfigurationError("Configuration error message")
    assert str(config_exc) == "Configuration error message"
    engine_exc = EngineError("Engine error message")
    assert str(engine_exc) == "Engine error message"
    operation_exc = CacheOperationError("Operation error message")
    assert str(operation_exc) == "Operation error message"
    resource_exc = ResourceError("Resource error message")
    assert str(resource_exc) == "Resource error message"
    engine_not_available_exc = EngineNotAvailableError("Engine not available message")
    assert str(engine_not_available_exc) == "Engine not available message"
    key_exc = CacheKeyError("Key error message")
    assert str(key_exc) == "Key error message"
    value_exc = CacheValueError("Value error message")
    assert str(value_exc) == "Value error message"
    serialization_exc = SerializationError("Serialization error message")
    assert str(serialization_exc) == "Serialization error message"
    concurrency_exc = ConcurrencyError("Concurrency error message")
    assert str(concurrency_exc) == "Concurrency error message"
    path_exc = PathError("Path error message")
    assert str(path_exc) == "Path error message"
    pytest.main(["-v", __file__])

================
File: plugins/repos/twat_cache/tests/test_exceptions.py
================
def test_base_exception():
    exc = TwatCacheError("Test error message")
    assert str(exc) == "Test error message"
    exc = TwatCacheError("Test error message", "arg1", "arg2")
    assert str(exc) == "('Test error message', 'arg1', 'arg2')"
def test_exception_hierarchy():
    assert issubclass(ConfigurationError, TwatCacheError)
    assert issubclass(EngineError, TwatCacheError)
    assert issubclass(EngineNotAvailableError, EngineError)
    assert issubclass(CacheOperationError, TwatCacheError)
    assert issubclass(CacheKeyError, CacheOperationError)
    assert issubclass(CacheValueError, CacheOperationError)
    assert issubclass(SerializationError, TwatCacheError)
    assert issubclass(ResourceError, TwatCacheError)
    assert issubclass(ConcurrencyError, TwatCacheError)
    assert issubclass(PathError, TwatCacheError)
def test_engine_not_available_error():
    exc = EngineNotAvailableError("test_engine")
    assert str(exc) == "Cache engine 'test_engine' is not available"
    exc = EngineNotAvailableError("test_engine", "Not installed")
    assert str(exc) == "Cache engine 'test_engine' is not available: Not installed"
def test_ensure_dir_exists():
    with tempfile.TemporaryDirectory() as temp_dir:
        test_dir = Path(temp_dir) / "test_dir"
        ensure_dir_exists(test_dir)
        assert test_dir.exists()
        assert test_dir.is_dir()
            assert (test_dir.stat().st_mode & 0o777) == 0o700
        test_dir2 = Path(temp_dir) / "test_dir2"
        ensure_dir_exists(test_dir2, mode=0o755)
        assert test_dir2.exists()
            assert (test_dir2.stat().st_mode & 0o777) == 0o755
def test_ensure_dir_exists_error():
    with tempfile.NamedTemporaryFile() as temp_file:
        with pytest.raises(PathError) as excinfo:
            ensure_dir_exists(Path(temp_file.name))
        assert "Failed to create cache directory" in str(excinfo.value)
def test_safe_key_serializer():
    assert safe_key_serializer("test") == "test"
    assert safe_key_serializer(123) == "123"
    assert safe_key_serializer(True) == "True"
    assert safe_key_serializer(None) == "None"
    assert safe_key_serializer([1, 2, 3]) == '["1", "2", "3"]'
    assert safe_key_serializer({"a": 1, "b": 2}) == '{"a": "1", "b": "2"}'
    assert safe_key_serializer([1, {"a": 2}]) == '["1", {"a": "2"}]'
def test_safe_key_serializer_error():
    class UnserializableObject:
        def __repr__(self):
            raise RuntimeError(msg)
    with pytest.raises(CacheKeyError) as excinfo:
        safe_key_serializer(UnserializableObject())
    assert "Failed to serialize cache key" in str(excinfo.value)
def test_safe_value_serializer():
    assert safe_value_serializer("test") == '"test"'
    assert safe_value_serializer(123) == "123"
    assert safe_value_serializer([1, 2, 3]) == "[1, 2, 3]"
    assert safe_value_serializer({"a": 1, "b": 2}) == '{"a": 1, "b": 2}'
def test_safe_value_serializer_error():
    class CustomObject:
    result = safe_value_serializer(CustomObject())
    assert result.startswith("<")
    class BadObject:
    with pytest.raises(CacheValueError) as excinfo:
        safe_value_serializer(BadObject())
    assert "Failed to serialize cache value" in str(excinfo.value)
def test_safe_temp_file():
    path, file_obj = safe_temp_file()
        assert path.exists()
        assert path.is_file()
        file_obj.write(b"test data")
        file_obj.flush()
            assert (path.stat().st_mode & 0o777) == 0o600
        file_obj.close()
        if path.exists():
            path.unlink()
def test_catching_exceptions():
        raise ConfigurationError(msg)
        assert isinstance(e, ConfigurationError)
        assert str(e) == "Test config error"
        raise EngineNotAvailableError(msg, "Not installed")
        assert isinstance(e, EngineNotAvailableError)
        raise CacheKeyError(msg)
        assert isinstance(e, CacheKeyError)
        assert str(e) == "Invalid key"

================
File: plugins/repos/twat_cache/tests/test_fallback.py
================
def test_backend_availability() -> None:
    available = _get_available_backends()
    assert isinstance(available, dict)
def test_backend_selection_preferred() -> None:
    backend = _select_best_backend(preferred="functools")
    backend = _select_best_backend(preferred="nonexistent")
def test_backend_selection_async() -> None:
    with patch.dict("sys.modules", {"aiocache": object()}):
        backend = _select_best_backend(is_async=True)
    with patch.dict("sys.modules", {"aiocache": None}):
def test_backend_selection_disk() -> None:
    with patch.dict("sys.modules", {"diskcache": object()}):
        backend = _select_best_backend(needs_disk=True)
    with patch.dict("sys.modules", {"diskcache": None, "joblib": object()}):
    with patch.dict(
        "sys.modules", {"diskcache": None, "joblib": None, "klepto": object()}
    with patch.dict("sys.modules", {"diskcache": None, "joblib": None, "klepto": None}):
def test_memory_cache_fallback() -> None:
    @mcache(maxsize=CACHE_SIZE)
    def test_func(x: int) -> int:
        result = test_func(SQUARE_INPUT)
def test_disk_cache_fallback() -> None:
    @bcache(maxsize=CACHE_SIZE)
async def test_async_cache_fallback() -> None:
    @acache(maxsize=CACHE_SIZE)
    async def test_func(x: int) -> int:
        await asyncio.sleep(0.1)
        result = await test_func(SQUARE_INPUT)
def test_universal_cache_fallback() -> None:
    @ucache(maxsize=CACHE_SIZE)
def test_backend_selection_priority() -> None:
            "cachebox": object(),
            "cachetools": object(),
        backend = _select_best_backend()
            "diskcache": object(),
            "joblib": object(),
            "klepto": object(),
def test_backend_selection_requirements() -> None:
            "aiocache": object(),
        backend = _select_best_backend(is_async=True, needs_disk=True)

================
File: plugins/repos/twat_cache/tests/test_redis_cache.py
================
class TestRedisCacheEngine:
    def mock_redis(self):
        with patch("redis.Redis") as mock_redis:
            mock_instance = MagicMock()
    def fake_redis(self):
        return fakeredis.FakeStrictRedis()
    def redis_engine(self, mock_redis):
        config = create_cache_config(
        engine = RedisCacheEngine(config)
    def real_redis_engine(self, fake_redis):
    def test_is_available(self):
        with patch("twat_cache.engines.redis.is_package_available") as mock_check:
            assert RedisCacheEngine.is_available() is True
            assert RedisCacheEngine.is_available() is False
    def test_init_with_unavailable_redis(self):
        with patch("twat_cache.engines.redis.RedisCacheEngine.is_available") as mock_check:
            config = create_cache_config()
            with pytest.raises(EngineNotAvailableError):
                RedisCacheEngine(config)
    def test_init_with_connection_error(self):
            mock_redis.return_value.ping.side_effect = redis.exceptions.ConnectionError("Connection refused")
            with pytest.raises(ConfigurationError):
    def test_validate_config(self, redis_engine):
        redis_engine.validate_config()  # Should not raise
        with patch.object(redis_engine._config, "get_redis_port") as mock_get_port:
                redis_engine.validate_config()
    def test_get_full_key(self, redis_engine):
        full_key = redis_engine._get_full_key(key)
        assert full_key.startswith("test_redis:")
        assert isinstance(full_key, str)
    def test_get_cached_value(self, redis_engine, mock_redis):
        mock_redis.get.return_value = pickle.dumps("test_value")
        value = redis_engine._get_cached_value("test_key")
        mock_redis.get.assert_called_once_with("test_redis:test_key")
        mock_redis.get.reset_mock()
        value = redis_engine._get_cached_value("missing_key")
        mock_redis.get.assert_called_once()
        value = redis_engine._get_cached_value("error_key")
    def test_set_cached_value(self, redis_engine, mock_redis):
        redis_engine._set_cached_value("test_key", "test_value")
        mock_redis.set.assert_called_once()
        mock_redis.set.reset_mock()
        mock_redis.setex.assert_called_once()
        class UnpickleableObject:
            def __reduce__(self):
                raise TypeError(msg)
        with pytest.raises(Exception):
            redis_engine._set_cached_value("error_key", UnpickleableObject())
    def test_clear(self, redis_engine, mock_redis):
        redis_engine.clear()
        mock_redis.keys.assert_called_once_with("test_redis:*")
        mock_redis.delete.assert_called_once()
        mock_redis.keys.reset_mock()
        mock_redis.delete.reset_mock()
        mock_redis.keys.assert_called_once()
        mock_redis.delete.assert_not_called()
    def test_cache_decorator(self, redis_engine, mock_redis):
        def test_func(x):
        result = test_func(5)
        mock_redis.get.return_value = pickle.dumps(10)
    def test_stats(self, redis_engine, mock_redis):
    def test_real_redis_operations(self, real_redis_engine):
        real_redis_engine._set_cached_value("test_key", "test_value")
        value = real_redis_engine._get_cached_value("test_key")
        value = real_redis_engine._get_cached_value("missing_key")
        real_redis_engine.clear()
        real_redis_engine._set_cached_value("complex_key", complex_data)
        value = real_redis_engine._get_cached_value("complex_key")
    def test_compression(self, real_redis_engine):
        real_redis_engine._set_cached_value("compressed_key", "test_value" * 100)
        value = real_redis_engine._get_cached_value("compressed_key")

================
File: plugins/repos/twat_cache/tests/test_twat_cache.py
================
def base_config() -> CacheConfig:
    return create_cache_config(maxsize=100)
def temp_path(tmp_path: Path) -> Path:
def lru_engine(base_config: CacheConfig) -> FunctoolsCacheEngine:
    return FunctoolsCacheEngine(base_config)
def test_lru_cache_get_set(lru_engine: FunctoolsCacheEngine) -> None:
    assert lru_engine._get_cached_value(key) is None
    lru_engine._set_cached_value(key, value)
    assert lru_engine._get_cached_value(key) == value
def test_lru_cache_clear(lru_engine: FunctoolsCacheEngine) -> None:
    lru_engine.clear()
def test_lru_cache_maxsize(lru_engine: FunctoolsCacheEngine) -> None:
    for i in range(150):  # More than maxsize
        lru_engine._set_cached_value(f"key_{i}", f"value_{i}")
    assert lru_engine._get_cached_value("key_0") is None
    assert lru_engine._get_cached_value("key_149") is not None
def disk_engine(base_config: CacheConfig, temp_path: Path) -> DiskCacheEngine:
    config = create_cache_config(maxsize=100, cache_dir=str(temp_path))
    return DiskCacheEngine(config)
def test_disk_cache_persistence(disk_engine: DiskCacheEngine) -> None:
    disk_engine._set_cached_value(key, value)
    assert disk_engine._get_cached_value(key) == value
    new_engine = DiskCacheEngine(config)
    assert new_engine._get_cached_value(key) == value
def joblib_engine(base_config: CacheConfig, temp_path: Path) -> JoblibEngine:
    return JoblibEngine(config)
@pytest.mark.skipif(not HAS_NUMPY, reason="numpy not available")
def test_joblib_numpy_array(joblib_engine: JoblibEngine) -> None:
    value = np.array([[1, 2], [3, 4]])
    joblib_engine._set_cached_value(key, value)
    cached = joblib_engine._get_cached_value(key)
    assert isinstance(cached, np.ndarray)
    assert np.array_equal(cached, value)
def klepto_engine(base_config: CacheConfig, temp_path: Path) -> KleptoEngine:
    return KleptoEngine(config)
def test_klepto_persistence(klepto_engine: KleptoEngine) -> None:
    klepto_engine._set_cached_value(key, value)
    assert klepto_engine._get_cached_value(key) == value
async def async_engine(base_config: CacheConfig) -> AioCacheEngine:
    return AioCacheEngine(base_config)
async def test_async_cache_operations(async_engine: AioCacheEngine) -> None:
    assert await async_engine._get_cached_value(key) is None
    await async_engine._set_cached_value(key, value)
    assert await async_engine._get_cached_value(key) == value
    await async_engine.clear()
def cachebox_engine(base_config: CacheConfig) -> CacheBoxEngine:
    return CacheBoxEngine(base_config)
def test_cachebox_operations(cachebox_engine: CacheBoxEngine) -> None:
    assert cachebox_engine._get_cached_value(key) is None
    cachebox_engine._set_cached_value(key, value)
    assert cachebox_engine._get_cached_value(key) == value
    cachebox_engine.clear()
def cachetools_engine(base_config: CacheConfig) -> CacheToolsEngine:
    return CacheToolsEngine(base_config)
def test_cachetools_operations(cachetools_engine: CacheToolsEngine) -> None:
    assert cachetools_engine._get_cached_value(key) is None
    cachetools_engine._set_cached_value(key, value)
    assert cachetools_engine._get_cached_value(key) == value
    cachetools_engine.clear()
def functools_engine(base_config: CacheConfig) -> FunctoolsCacheEngine:
def test_functools_cache_get_set(functools_engine: FunctoolsCacheEngine) -> None:
    assert functools_engine._get_cached_value(key) is None
    functools_engine._set_cached_value(key, value)
    assert functools_engine._get_cached_value(key) == value
def test_functools_cache_clear(functools_engine: FunctoolsCacheEngine) -> None:
    functools_engine.clear()
def test_functools_cache_maxsize(functools_engine: FunctoolsCacheEngine) -> None:
        functools_engine._set_cached_value(f"key_{i}", f"value_{i}")
    assert functools_engine._get_cached_value("key_0") is None
    assert functools_engine._get_cached_value("key_149") is not None

================
File: plugins/repos/twat_cache/.gitignore
================
.mypy_cache
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_cache/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_cache/CHANGELOG.md
================
---
this_file: CHANGELOG.md
---

# Changelog

All notable changes to the `twat_cache` package will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v2.5.3] - 2025-03-04

### Added
- Enhanced hybrid caching capabilities
  - Improved automatic backend switching based on data characteristics
  - Added more intelligent data type detection
  - Optimized threshold determination for backend selection
- Expanded test coverage
  - Added more comprehensive tests for Redis engine
  - Improved test coverage for hybrid caching
  - Added tests for edge cases in backend selection
- Improved documentation
  - Updated README with more detailed examples
  - Added more comprehensive docstrings
  - Reorganized TODO list for better clarity and actionability

### Fixed
- Resolved several edge cases in Redis connection handling
- Fixed type annotation issues in context management
- Addressed potential race conditions in multi-threaded environments
- Improved error handling in serialization/deserialization processes

### Changed
- Refactored backend selection logic for better performance
- Improved context management API for more intuitive usage
- Updated dependencies to latest compatible versions
- Reorganized project structure for better maintainability

## [v2.3.0] - 2025-03-04

### Added
- Improved context management utilities
  - Enhanced error handling in context managers
  - Added support for explicit engine selection in context managers
  - Improved resource cleanup mechanisms
- Enhanced backend selection strategy
  - Refined data type detection for optimal backend selection
  - Added support for hybrid caching with automatic backend switching
  - Improved fallback mechanisms for unavailable backends
- Comprehensive documentation updates
  - Added detailed examples for all cache backends
  - Created documentation for context management
  - Updated installation and usage instructions

### Fixed
- Resolved remaining type compatibility issues across the codebase
- Fixed edge cases in Redis connection handling
- Addressed potential memory leaks in long-running cache instances
- Improved error handling in serialization/deserialization processes

### Changed
- Refactored internal cache key generation for better performance
- Standardized logging format across all cache engines
- Improved test coverage for all components
- Updated dependencies to latest compatible versions

## [v2.2.0] - 2024-09-01

### Added

- Comprehensive test suite for backend selection strategy
  - Tests for data size estimation
  - Tests for type-based backend selection
  - Tests for backend availability
  - Tests for various data types (primitive, containers, NumPy arrays, pandas DataFrames)
- Redis cache engine implementation
  - Full Redis cache engine with serialization, compression, and TTL support
  - Redis-specific configuration options
  - Proper connection handling and error management
  - Registration in the engine manager
- Documentation for cache context management
  - Detailed guide on using context managers
  - Examples for basic and advanced usage patterns
  - Best practices for resource management
  - Error handling recommendations

### Fixed

- Type compatibility issues in context.py with CacheConfig protocol
  - Updated protocol in type_defs.py to use validate_config instead of validate
  - Fixed type casting to avoid type errors
  - Improved type annotations for better static analysis
- Linter errors in engine implementations
  - Fixed import issues with optional dependencies
  - Addressed type compatibility warnings
  - Improved error handling in Redis engine

### Changed

- Expanded test coverage for cache engines
  - Added comprehensive tests for Redis cache engine
  - Tests for initialization, configuration validation, key generation
  - Tests for caching operations and error handling
- Refactored common patterns across engine implementations
  - Standardized engine initialization and validation procedures
  - Created unified interfaces for all cache backends
  - Improved code organization and maintainability

## [v2.1.1] - 2024-08-26

### Added

- New custom exceptions module with proper inheritance hierarchy
  - Base `TwatCacheError` exception class
  - Specialized exceptions for different error categories (configuration, engine, resource, etc.)
  - Improved error messages with context-specific information
- Context management for cache engines
  - `CacheContext` class that provides resource cleanup on exit
  - Safe resource management with proper error handling
  - Support for engine selection based on configuration
- Tests for exceptions and context management modules
  - Simple test suite for exception hierarchy and messages
  - Context management tests with mocking for proper isolation
- Implemented data type-based cache backend selection strategy
  - New `backend_selector` module for automatic backend selection
  - Helper functions for common data types (numpy, pandas, images, json)
  - Smart backend selection based on data characteristics
- Added hybrid caching capabilities
  - `hybrid_cache` decorator for size-based backend selection
  - `smart_cache` decorator for automatic backend selection
- Added example demonstrating the backend selection strategy
- Created CHANGELOG.md for tracking version changes
- Updated TODO.md to reflect completed items and reprioritize tasks

### Fixed

- Fixed error handling in various cache engines
- Improved cleanup logic for better resource management
- Made error messages more consistent across the codebase
- Installation issues with pip install -e . by replacing hatchling with setuptools
- Path handling in CacheContext

### Changed

- Standardized initialization patterns across engine implementations
- Improved shutdown procedures to properly release resources
- Removed setup.py in favor of pyproject.toml for modern Python packaging
- Refactored context manager implementation for better error handling 


## \[1.8.1] - 2025-02-18

### Added

* Added comprehensive type stubs for all cache engines
* Added detailed rationale section in README.md comparing different caching libraries
* Added extensive code examples for each caching strategy

### Changed

* Updated documentation with detailed type hints and protocols
* Improved error messages in cache engines
* Enhanced configuration validation

### Fixed

* Fixed type hints in async cache implementations
* Fixed decorator type hints for better IDE support
* Fixed linter errors in multiple files

### Issues

* Still missing type stubs for external dependencies
* Some linter warnings in test files need addressing
* Need to improve async test coverage

### Next Steps

1. Fix remaining linter issues:
   * Add missing type stubs for external dependencies
   * Address warnings in test files
   * Improve type safety in async code

2. Enhance test coverage:
   * Add more async test cases
   * Add stress tests for race conditions
   * Add integration tests for all backends

3. Improve documentation:
   * Add migration guide
   * Add performance comparison guide
   * Add troubleshooting section

## \[1.8.0] - 2025-02-17

### Added

* Completed core features:
  * Memory-based caching (mcache)
  * Disk-based caching (bcache)
  * File-based caching (fcache)
  * Async-capable caching (via ucache)
  * Universal caching (ucache)
* Implemented all cache engines:
  * FunctoolsCacheEngine (base memory cache)
  * CacheToolsEngine (flexible memory cache)
  * DiskCacheEngine (SQL-based disk cache)
  * JoblibEngine (file-based cache)
  * KleptoEngine (flexible cache)
  * AioCacheEngine (async cache)
* Added comprehensive features:
  * TTL support for all caches
  * Multiple eviction policies (LRU, LFU, FIFO, RR)
  * Secure file permissions for disk caches
  * Compression support for file caches
  * Race condition handling
  * Proper error handling and logging
* Added proper type hints and protocols
* Added comprehensive test suite
* Updated documentation with examples

### Changed

* Reorganized backend priorities for each decorator:
  * mcache: cachebox > cachetools > functools
  * bcache: diskcache > klepto SQL > mcache
  * fcache: joblib > klepto file > mcache
  * ucache: auto-selects best backend
* Enhanced test coverage with backend-specific tests
* Improved error handling and logging
* Updated configuration system to use Pydantic

### Fixed

* Fixed CacheConfig initialization issues
* Fixed cache stats tracking
* Fixed maxsize enforcement
* Fixed race conditions in multi-threaded scenarios
* Fixed file permissions for disk caches
* Fixed type hints and protocols

### Issues

* Missing type stubs for some dependencies:
  * aiocache
  * diskcache
  * joblib
  * klepto
* Some linter warnings in test files
* Need more async test coverage
* Need more stress tests for race conditions

### Next Steps

1. Type System:
   * Add missing type stubs for dependencies
   * Fix remaining linter warnings
   * Add more type safety

2. Testing:
   * Add more async tests
   * Add stress tests for race conditions
   * Add performance benchmarks
   * Add integration tests

3. Documentation:
   * Add API reference
   * Add performance guide
   * Add migration guide
   * Add troubleshooting guide

4. Performance:
   * Add caching statistics
   * Add cache warming
   * Add cache prefetching
   * Add cache compression options

5. Security:
   * Add encryption support
   * Add access control
   * Add audit logging
   * Add cache poisoning protection

## [1.7.9] - 2025-02-16

### Changed

* Updated TODO.md with clearer structure and priorities
* Reorganized Phase 2 tasks to focus on critical issues first
* Added more detailed subtasks for each component

### Fixed

* Fixed Pydantic field validators in CacheConfig
* Updated model\_config settings
* Improved error messages

### Issues

* Critical: CacheEngine import failing in test\_engines.py
* CacheConfig initialization still broken (TypeError: CacheConfig() takes no arguments)
* Cache stats tracking not working
* Cache maxsize enforcement not working
* Multiple linting issues identified:
  * Magic numbers in comparisons
  * Boolean argument type issues
  * Module naming conflicts with stdlib
  * Unused arguments in lambda functions
  * Complex function (clear\_cache)

### Next Steps

1. Fix Critical Engine Issues:
   * Fix CacheEngine import in base.py
   * Fix test\_engines.py import error
   * Fix CacheConfig initialization
   * Address linting issues

2. Fix Cache Engine Core:
   * Implement proper stats tracking
   * Fix maxsize enforcement
   * Add proper cache key generation
   * Add proper error handling

3. Improve Test Coverage:
   * Fix failing tests
   * Add engine-specific tests
   * Add performance benchmarks

4. Documentation:
   * Add configuration guide
   * Add backend selection guide
   * Add performance optimization guide

## [1.7.8] - 2025-02-16

### Changed

* Reorganized TODO.md with clearer structure and priorities
* Added Phase 3 for advanced features
* Updated test results analysis

### Fixed

* Fixed Pydantic field validators in CacheConfig
* Updated field validator decorators to use @classmethod
* Fixed model\_config settings

### Issues

* CacheConfig initialization still broken (TypeError: CacheConfig() takes no arguments)
* Cache stats tracking not working (hits/misses not being counted)
* Cache maxsize enforcement not working (cache growing beyond limit)
* Cache clearing behavior inconsistent
* Field validation in CacheConfig needs fixing

### Next Steps

1. Fix CacheConfig initialization:
   * Update Pydantic model to properly accept constructor arguments
   * Fix field validation and property access
   * Add proper error messages
   * Update model\_config for proper initialization

2. Fix cache engine issues:
   * Fix stats tracking in FunctoolsCacheEngine
   * Fix maxsize enforcement
   * Fix cache key generation
   * Fix cache clearing behavior

3. Fix failing tests:
   * Fix CacheConfig initialization tests
   * Fix cache stats tracking tests
   * Fix maxsize enforcement tests
   * Fix cache clearing tests

## [1.7.7] - 2025-02-16

### Changed

* Reorganized TODO.md to better reflect current priorities
* Updated CacheConfig implementation to use proper Pydantic fields
* Fixed linting issues in configuration system

### Fixed

* Removed leading underscores from CacheConfig field names
* Fixed field validation in CacheConfig
* Fixed property access in CacheConfig

### Issues

* CacheConfig initialization is broken (TypeError: CacheConfig() takes no arguments)
* Cache stats tracking is not working (hits/misses not being counted)
* Cache maxsize enforcement is not working (cache growing beyond limit)
* Cache clearing behavior is inconsistent
* Field validation in CacheConfig needs fixing

### Next Steps

1. Fix CacheConfig initialization:
   * Implement proper Pydantic model initialization
   * Fix field validation
   * Add proper error messages

2. Fix cache engine issues:
   * Fix stats tracking in FunctoolsCacheEngine
   * Fix maxsize enforcement
   * Fix cache key generation
   * Fix cache clearing behavior

3. Improve test coverage:
   * Add missing engine-specific tests
   * Add benchmark tests
   * Add stress tests
   * Add integration tests

## [1.7.6] - 2025-02-16

### Changed

* Reorganized TODO.md to better track progress
* Updated test suite with comprehensive engine tests
* Improved error handling in cache engines

### Fixed

* Renamed `lru.py` to `functools.py` for clarity
* Removed redundant SQL and Redis implementations
* Added proper type hints and docstrings

## [1.7.5] - 2025-02-15

### Fixed

* Fixed README.md formatting

## [1.7.3] - 2025-02-15

### Changed

* Minor documentation updates
* Internal code improvements

## [1.7.0] - 2025-02-13

### Added

* Added pyupgrade to development dependencies
* Added new fix command in pyproject.toml for automated code fixes
* Enhanced test environment with specific pytest dependencies
  * Added pytest-xdist for parallel test execution
  * Added pytest-benchmark for performance testing

### Changed

* Reorganized imports in core module
* Updated gitignore to exclude \_private directory

### Fixed

* Fixed import statement in **init**.py
* Improved development tooling configuration

## [1.6.2] - 2025-02-06

### Fixed

* Bug fixes and stability improvements

## [1.6.1] - 2025-02-06

### Changed

* Minor updates and improvements

## [1.6.0] - 2025-02-06

### Added

* Initial GitHub repository setup
* Comprehensive project structure
* Basic caching functionality
* Multiple backend support (memory, SQL, joblib)
* Automatic cache directory management
* Type hints and modern Python features

## [1.1.0] - 2025-02-03

### Added

* Early development version
* Core caching functionality

## [1.0.0] - 2025-02-03

### Added

* Initial release
* Basic memory caching implementation

## \[1.8.2] - 2025-02-19

### Changed

* Refocused project goals on simplicity and ease of use
* Streamlined TODO list to prioritize core functionality
* Reorganized implementation plan to focus on essential features

### Added

* Clear implementation status tracking in TODO.md
* Detailed testing strategy
* Documentation approach guidelines

### Removed

* Unnecessary complexity from implementation plan
* Redundant feature requests
* Overly complex API specifications

### Next Steps

1. Fix Critical Issues:
   * Remove magic numbers from tests
   * Fix boolean parameter warnings
   * Address unused imports
   * Fix function complexity issues

2. Improve Core Functionality:
   * Complete fallback mechanism
   * Add proper logging
   * Implement cache stats
   * Add inspection utilities

3. Enhance Testing:
   * Add test constants
   * Implement async tests
   * Add stress tests
   * Add backend-specific tests

[1.7.9]: https://github.com/twardoch/twat-cache/compare/v1.7.8...v1.7.9

[1.7.8]: https://github.com/twardoch/twat-cache/compare/v1.7.7...v1.7.8

[1.7.7]: https://github.com/twardoch/twat-cache/compare/v1.7.6...v1.7.7

[1.7.6]: https://github.com/twardoch/twat-cache/compare/v1.7.5...v1.7.6

[1.7.5]: https://github.com/twardoch/twat-cache/compare/v1.7.3...v1.7.5

[1.7.3]: https://github.com/twardoch/twat-cache/compare/v1.7.0...v1.7.3

[1.7.0]: https://github.com/twardoch/twat-cache/compare/v1.6.2...v1.7.0

[1.6.2]: https://github.com/twardoch/twat-cache/compare/v1.6.1...v1.6.2

[1.6.1]: https://github.com/twardoch/twat-cache/compare/v1.6.0...v1.6.1

[1.6.0]: https://github.com/twardoch/twat-cache/compare/v1.1.0...v1.6.0

[1.1.0]: https://github.com/twardoch/twat-cache/compare/v1.0.0...v1.1.0

[1.0.0]: https://github.com/twardoch/twat-cache/releases/tag/v1.0.0

# Development Log

### Completed

* Updated TODO.md with clearer structure and priorities
* Fixed Pydantic field validators
* Updated model\_config settings
* Identified critical import issue in test\_engines.py

### In Progress

* Fixing CacheEngine import issue
* Fixing CacheConfig initialization
* Addressing linting issues
* Improving test coverage

### Next Steps

1. Fix Critical Issues:
   * Fix CacheEngine import
   * Fix CacheConfig initialization
   * Fix test failures
   * Address linting issues

2. Improve Core Functionality:
   * Fix stats tracking
   * Fix maxsize enforcement
   * Improve cache key generation
   * Add proper error handling

3. Enhance Testing:
   * Fix failing tests
   * Add comprehensive test suite
   * Add performance benchmarks

# Critical Considerations

1. Engine System:
   * CacheEngine must be properly exported from base.py
   * All engines must properly implement the base interface
   * Stats tracking must be consistent across all engines

2. Configuration System:
   * CacheConfig must properly initialize with constructor arguments
   * Field validation must be robust and informative
   * Cache directory management must be reliable

3. Testing System:
   * All engines must be properly tested
   * Performance benchmarks must be comprehensive
   * Error handling must be thoroughly tested

## [1.7.8]: https://github.com/twardoch/twat-cache/compare/v1.7.7...v1.7.8

## [1.7.7]: https://github.com/twardoch/twat-cache/compare/v1.7.6...v1.7.7

## [1.7.6]: https://github.com/twardoch/twat-cache/compare/v1.7.5...v1.7.6

## [1.7.5]: https://github.com/twardoch/twat-cache/compare/v1.7.3...v1.7.5

## [1.7.3]: https://github.com/twardoch/twat-cache/compare/v1.7.0...v1.7.3

## [1.7.0]: https://github.com/twardoch/twat-cache/compare/v1.6.2...v1.7.0

## [1.6.2]: https://github.com/twardoch/twat-cache/compare/v1.6.1...v1.6.2

## [1.6.1]: https://github.com/twardoch/twat-cache/compare/v1.6.0...v1.6.1

## [1.6.0]: https://github.com/twardoch/twat-cache/compare/v1.1.0...v1.6.0

## [1.1.0]: https://github.com/twardoch/twat-cache/compare/v1.0.0...v1.1.0

## [1.0.0]: https://github.com/twardoch/twat-cache/releases/tag/v1.0.0

# Development Log

### Completed

* Fixed Pydantic field validators in CacheConfig:
  * Updated validators to use @classmethod decorator
  * Fixed model\_config settings
  * Improved error messages
* Reorganized TODO.md:
  * Added clearer structure
  * Prioritized critical fixes
  * Added Phase 3 for advanced features
* Updated test suite:
  * Added more comprehensive test cases
  * Identified failing tests
  * Added test categories

### In Progress

* Fixing CacheConfig initialization issues:
  * Working on proper Pydantic model setup
  * Fixing field validation and property access
  * Adding proper error messages
* Fixing cache engine issues:
  * Implementing proper stats tracking
  * Fixing maxsize enforcement
  * Improving cache key generation
  * Fixing cache clearing behavior

### Next Steps

1. Fix Critical Issues:
   * Fix CacheConfig initialization
   * Fix cache stats tracking
   * Fix maxsize enforcement
   * Fix cache clearing behavior

2. Enhance Test Coverage:
   * Fix failing tests
   * Add missing engine-specific tests
   * Add performance tests

3. Improve Documentation:
   * Update configuration guide
   * Add troubleshooting guide
   * Add performance tips
   * Add examples

# Critical Considerations

1. Configuration System:
   * CacheConfig must properly initialize with constructor arguments
   * Field validation must work correctly
   * Property access must be reliable
   * Error messages must be helpful

2. Cache Engine Behavior:
   * Stats tracking must be accurate
   * Maxsize enforcement must be reliable
   * Cache clearing must be consistent
   * Key generation must handle all types

3. Test Coverage:
   * All critical functionality must be tested
   * Edge cases must be covered
   * Performance must be verified
   * Error handling must be tested

# Conslidation UI Project Analysis & Plan

## Progress Log

### 2024-03-21

* Updated TODO.md with completed tasks and reorganized remaining work
* Major progress in core functionality:
  * Completed all basic cache engine implementations
  * Fixed cache stats tracking and clearing behavior
  * Improved type hints and configuration system
  * Added comprehensive test coverage for core features
* Next steps prioritized:
  1. Add missing engine-specific tests (eviction, concurrency, etc.)
  2. Implement serialization protocol
  3. Add async test coverage
  4. Create comprehensive documentation

Critical Considerations:

1. Engine-specific Tests:
   * Need stress tests for memory limits and eviction policies
   * Add concurrent access testing with high load
   * Test edge cases like cache corruption and recovery
   * Verify proper cleanup of resources

2. Serialization Protocol:
   * Add support for complex objects and custom types
   * Consider compression for large cached values
   * Implement versioning for cached data format
   * Add validation for serialized data integrity

3. Async Support:
   * Improve error handling for timeouts and connection issues
   * Ensure proper cleanup of async resources
   * Add retry mechanisms for transient failures
   * Test interaction with different event loops

4. Documentation:
   * Add detailed performance characteristics
   * Document tradeoffs between different cache engines
   * Create troubleshooting guide
   * Add migration guide for users of removed implementations

## Recent Changes

### 2024-03-21

* Fixed CacheConfig implementation:
  * Added proper field aliases for configuration parameters
  * Implemented abstract base class methods
  * Added Pydantic model configuration for proper field handling
  * Fixed validation method
* Updated test suite:
  * Fixed failing tests related to CacheConfig instantiation
  * Added tests for field aliases and validation
  * Improved test coverage for configuration handling

### Next Steps

1. Fix remaining test failures:
   * Cache stats tracking
   * List/unhashable type handling
   * Cache clearing behavior
2. Implement missing engine-specific tests:
   * Test cache eviction policies
   * Test concurrent access
   * Test error handling
   * Test serialization/deserialization
   * Test memory limits

### Critical Considerations

1. Engine-specific Tests:
   * Need stress tests for memory limits and eviction policies
   * Add concurrent access testing with high load
   * Test edge cases like cache corruption and recovery
   * Verify proper cleanup of resources

2. Serialization Protocol:
   * Add support for complex objects and custom types
   * Consider compression for large cached values
   * Implement versioning for cached data format
   * Add validation for serialized data integrity

3. Async Support:
   * Improve error handling for timeouts and connection issues
   * Ensure proper cleanup of async resources
   * Add retry mechanisms for transient failures
   * Test interaction with different event loops

4. Documentation:
   * Add detailed performance characteristics
   * Document tradeoffs between different cache engines
   * Create troubleshooting guide
   * Add migration guide for users of removed implementations

### 2024-03-19

#### Current Status

* Ran development setup and tests
* Found critical issues that need immediate attention:
  1. Pydantic field validator errors in CacheConfig
  2. 27 remaining linting issues
  3. Test failures across all test files

#### Critical Issues

1. **Pydantic Integration Issues**
   * Error: `@field_validator` cannot be applied to instance methods

   * Location: `src/twat_cache/config.py`

   * Impact: Breaking all tests due to config initialization failure

2. **Linting Issues**
   * 27 remaining issues including:

     * Magic numbers in comparisons
     * Print statements in production code
     * Boolean positional arguments
     * Module naming conflicts
     * Complex function warnings

#### Next Steps

1. **Immediate Fixes**
   * Fix Pydantic field validator implementation in CacheConfig
   * Address critical linting issues affecting functionality
   * Fix test failures

2. **Short-term Tasks**
   * Complete cache stats tracking implementation
   * Add proper type constraints
   * Implement missing test cases

3. **Documentation Updates**
   * Document configuration system
   * Add validation rules
   * Update type hints documentation

================
File: plugins/repos/twat_cache/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_cache/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_cache/NEXT.md
================
---
this_file: NEXT.md
---

# Data Type-Based Cache Backend Selection Strategy

## 1. Introduction

The `twat_cache` library supports multiple caching backends, each with its own strengths and weaknesses. This document outlines a strategy for selecting the most appropriate cache backend based on the type of data being cached. This strategy aims to optimize performance, memory usage, and reliability for different data types.

## 2. Available Cache Backends

Based on the codebase, we support the following cache backends:

1. **functools** - Simple in-memory caching using Python's built-in `functools.lru_cache`
2. **cachetools** - Flexible in-memory caching with multiple eviction policies
3. **diskcache** - Persistent disk-based caching with SQLite backend
4. **joblib** - Specialized caching for scientific computing (NumPy arrays, pandas DataFrames)
5. **klepto** - Advanced caching with multiple backends and policies
6. **aiocache** - Async-compatible caching
7. **cachebox** - High-performance caching

## 3. Data Type Characteristics

Different types of data have different characteristics that make them more suitable for certain cache backends:

### 3.1. Size
- **Small data** (<10KB): Simple strings, numbers, small dictionaries/lists
- **Medium data** (10KB-1MB): Larger collections, medium-sized images, medium JSON objects
- **Large data** (1MB-100MB): Large datasets, images, serialized models
- **Very large data** (>100MB): Video files, very large datasets, complex ML models

### 3.2. Serialization Complexity
- **Simple**: Basic Python types (int, float, str, bool)
- **Moderate**: Lists, dictionaries with simple values, custom classes with simple attributes
- **Complex**: Nested objects, custom classes with many attributes, objects with circular references
- **Specialized**: NumPy arrays, pandas DataFrames, scientific computing objects, ML models

### 3.3. Access Patterns
- **Read-heavy**: Data that is read frequently but written rarely
- **Write-heavy**: Data that is updated frequently
- **Balanced**: Similar read and write frequencies
- **Temporal locality**: Data accessed in clusters over time
- **Spatial locality**: Related data items accessed together

### 3.4. Persistence Requirements
- **Ephemeral**: Only needed during program execution
- **Session-persistent**: Needed across multiple runs in a session
- **Long-term persistent**: Needed across multiple sessions/days

### 3.5. Concurrency Requirements
- **Single-threaded**: No concurrent access
- **Multi-threaded**: Concurrent access from multiple threads
- **Multi-process**: Concurrent access from multiple processes
- **Distributed**: Concurrent access from multiple machines

## 4. Backend Selection Strategy

### 4.1. Default Backend Selection by Data Type

| Data Type | Recommended Backend | Alternative Backend | Justification |
|-----------|---------------------|---------------------|---------------|
| **Simple Python types** (int, str, bool) | cachetools | functools | Efficient in-memory storage, low overhead |
| **Collections** (list, dict, set) | cachetools | klepto | Flexible policies, good performance for medium collections |
| **NumPy arrays** | joblib | diskcache | Optimized for NumPy arrays, efficient serialization |
| **Pandas DataFrames** | joblib | diskcache | Optimized for pandas objects, handles large datasets well |
| **Images/Binary data** | diskcache | klepto | Efficient storage of binary data on disk |
| **Large JSON objects** | diskcache | klepto | Persistent storage, good for structured data |
| **Machine Learning models** | joblib | diskcache | Specialized serialization, handles large binary objects |
| **Custom classes** | klepto | diskcache | Flexible serialization options |
| **Async coroutine results** | aiocache | cachetools | Designed for async compatibility |

### 4.2. Selection by Data Characteristics

#### 4.2.1. By Size

| Data Size | Recommended Backend | Justification |
|-----------|---------------------|---------------|
| **Small** (<10KB) | cachetools | Fast in-memory access, low overhead |
| **Medium** (10KB-1MB) | klepto or diskcache | Good balance of performance and memory usage |
| **Large** (1MB-100MB) | diskcache or joblib | Efficient disk storage, specialized serialization |
| **Very large** (>100MB) | joblib | Optimized for large scientific data |

#### 4.2.2. By Persistence Requirements

| Persistence Need | Recommended Backend | Justification |
|------------------|---------------------|---------------|
| **Ephemeral** | cachetools or functools | Fast in-memory caching, no disk overhead |
| **Session-persistent** | klepto or diskcache | Can persist to disk but still relatively fast |
| **Long-term persistent** | diskcache | Reliable SQL-based storage, data survives restarts |

#### 4.2.3. By Access Pattern

| Access Pattern | Recommended Backend | Policy Setting | Justification |
|----------------|---------------------|----------------|---------------|
| **Read-heavy** | Any backend | LRU policy | Standard caching approach |
| **Write-heavy** | cachetools or diskcache | Small maxsize | Prevent cache thrashing |
| **Temporal locality** | cachetools | LRU policy | Optimized for recently used items |
| **Frequency-based** | cachetools or klepto | LFU policy | Keeps frequently accessed items |
| **Random access** | klepto | RR (Random Replacement) | Simple eviction suitable for random patterns |

#### 4.2.4. By Concurrency Requirements

| Concurrency Need | Recommended Backend | Justification |
|------------------|---------------------|---------------|
| **Single-threaded** | Any backend | All work well in this simple case |
| **Multi-threaded** | cachetools or diskcache | Thread-safe implementations |
| **Multi-process** | diskcache | Shared storage accessible across processes |
| **Distributed** | diskcache with network path | Can be accessed from multiple machines |

### 4.3. Implementation Approach

To implement this strategy in the `twat_cache` library, we recommend:

1. **Automatic Detection**: Analyze the return type of cached functions to suggest an appropriate backend.
   
2. **Configuration Helpers**: Provide helper functions for common data types:
   ```python
   # Example API
   from twat_cache import configure_for_numpy, configure_for_json, configure_for_images
   
   @cache(config=configure_for_numpy())
   def process_array(data: np.ndarray) -> np.ndarray:
       # Process the array
       return result
   ```

3. **Smart Defaults**: When no specific engine is requested, select based on:
   - Function return type annotations if available
   - Runtime analysis of actual returned values
   - Configuration hints like `maxsize` and `folder_name`

4. **Hybrid Caching**: For functions that return different types based on input:
   ```python
   @cache(config=hybrid_cache_config(
       small_result_engine="cachetools",
       large_result_engine="diskcache",
       size_threshold=1024*1024  # 1MB
   ))
   def get_data(id: str) -> dict | bytes:
       # Returns either metadata (dict) or raw data (bytes)
       # The appropriate cache will be selected based on return size
   ```

## 5. Implementation Plan

1. Add a data type analyzer utility to inspect function returns
2. Create configuration factory functions for common data types
3. Implement smart backend selection based on data characteristics
4. Add hybrid caching capability for mixed-return functions
5. Update documentation with examples of type-based configuration

## 6. Conclusion

By matching data types to appropriate cache backends, we can significantly improve the performance and efficiency of the caching system. This strategy provides a framework for making these decisions systematically, whether manually configured by users or automatically determined by the library.

The core principle is: **Let the data characteristics guide the cache backend selection.**

================
File: plugins/repos/twat_cache/PROMPT.txt
================
1. Read @README.md — note that in the .venv you have the actual code of the 3rd party libs to consult!

2. Read @LOG.md and @TODO.md

3. Step back and reason about the rationale: we wanted a simple package that exposes some decorators and passes all the cache heavy lifting to the 3rd parties. Check the implementation and tell me how we match. 

4. Reduce the common API of our code to the minimum, don't try to expose a universal API that does everything. For that, implement an easy passthru that just exposes the objects or functions provided by the 3rd parties

5. What DOES matter, though, is ease of use. We wanted the 3 specialized decorators (mcache, bcache, fcache) and a universal ucache decorator. What I want is a seamless integration of the engines so that if a given engine isn't available, we issue a logger warning but a fallback mechanism kicks in. 

6. Rewrite our goals in @TODO.md to match the above

7. Adjusts the tests so they’re sensible and realistic. 

Run often:

```
uv venv; source .venv/bin/activate; uv pip install -e .[all,dev,test]; tree -h -I '*_cache' -I __pycache__ .; tree -h -I '*.dist-info' -I 'mypy' -I 'mypyc' -I 'data' .venv; hatch run lint:fix; hatch test;
```

REPEAT IT OFTEN.

Then adjust our implementation, and continue TODO

================
File: plugins/repos/twat_cache/pyproject.toml
================
[build-system]
requires = [
    "setuptools>=67.0.0",
    "setuptools_scm>=8.0.0",
    "wheel",
]
build-backend = "setuptools.build_meta"

[tool.setuptools]
package-dir = {"" = "src"}
packages = ["twat_cache"]

[tool.setuptools_scm]
version_file = "src/twat_cache/__version__.py"
write_to = "src/twat_cache/__version__.py"

[project]
name = "twat_cache"
description = "Advanced caching library for Python, part of the twat framework"
readme = "README.md"
requires-python = ">=3.10"
license = {text = "MIT"}
dynamic = ["version"]
authors = [
    {name = "Adam Twardoch", email = "adam+github@twardoch.com"},
]
keywords = [
    "caching", 
    "cache", 
    "memorization", 
    "performance", 
    "optimization"
]
dependencies = [
    "pydantic>=2.0",
    "loguru>=0.7.0",
    "diskcache>=5.6.1",
    "joblib>=1.3.2",
    "cachetools>=5.3.2",
]

[project.urls]
Homepage = "https://github.com/twardoch/twat"
Documentation = "https://github.com/twardoch/twat/tree/main/docs"
Issues = "https://github.com/twardoch/twat/issues"

[project.entry-points."twat.plugins"]
cache = 'twat_cache'

[project.optional-dependencies]
cachebox = ['cachebox>=4.5.1']
cachetools = ['cachetools>=5.5.1']
aiocache = ['aiocache>=0.12.3']
klepto = ['klepto>=0.2.6']
diskcache = ['diskcache>=5.6.3']
joblib = ['joblib>=1.4.2']
all = [
    'platformdirs>=4.3.6',
    'cachebox>=4.5.1',
    'cachetools>=5.5.1',
    'aiocache>=0.12.3',
    'klepto>=0.2.6',
    'diskcache>=5.6.3',
    'joblib>=1.4.2',
]

[tool.pytest.ini_options]
minversion = "8.0.0"
addopts = "-v"
testpaths = ["tests"]

================
File: plugins/repos/twat_cache/README.md
================
# twat-cache

A flexible caching utility package for Python functions that provides a unified interface for caching function results using various high-performance backends.

## Features

- Simple decorator interface for caching function results
- Multiple caching backends with automatic selection:
  1. `cachebox` - Very fast Rust-based cache (optional)
  2. `cachetools` - Flexible in-memory caching (optional)
  3. `aiocache` - Async-capable caching (optional)
  4. `klepto` - Scientific computing caching (optional)
  5. `diskcache` - SQL-based disk cache (optional)
  6. `joblib` - Efficient array caching (optional)
  7. `redis` - Distributed caching with Redis (optional)
  8. Memory-based LRU cache (always available)
- Automatic cache directory management
- Type hints and modern Python features
- Lazy backend loading - only imports what you use
- Automatic backend selection based on availability and use case
- Smart backend selection based on data characteristics
- TTL support for cache expiration
- Multiple eviction policies (LRU, LFU, FIFO, RR)
- Async function support
- Compression options for large data
- Secure file permissions for sensitive data
- Hybrid caching with automatic backend switching
- Context management for cache engines
- Comprehensive test suite for all components

## Recent Updates (v2.3.0)

### Enhanced Context Management

The context management system has been significantly improved:
- Better error handling and resource cleanup
- Support for explicit engine selection
- Simplified API for temporary cache configurations
- Automatic cleanup of resources even when exceptions occur

```python
# Example of improved context management
from twat_cache import CacheContext

# Create a context with explicit engine selection
with CacheContext(engine_name="redis", namespace="user_data") as cache:
    # Use the cache within the context
    cache.set("user:1001", {"name": "John", "role": "admin"})
    user = cache.get("user:1001")
    
    # Resources are automatically cleaned up when exiting the context
```

### Refined Backend Selection

The backend selection strategy has been further enhanced:
- More accurate data type detection for optimal backend selection
- Improved fallback mechanisms when preferred backends are unavailable
- Better handling of edge cases for various data types
- Enhanced performance for frequently accessed items

### Comprehensive Documentation

Documentation has been expanded with:
- Detailed examples for all cache backends
- Step-by-step guides for common use cases
- API reference with complete parameter descriptions
- Best practices for cache configuration

## Installation

Basic installation with just LRU caching:
```bash
pip install twat-cache
```

With all optional backends:
```bash
pip install twat-cache[all]
```

Or install specific backends:
```bash
pip install twat-cache[cachebox]     # For Rust-based high-performance cache
pip install twat-cache[cachetools]   # For flexible in-memory caching
pip install twat-cache[aiocache]     # For async-capable caching
pip install twat-cache[klepto]       # For scientific computing caching
pip install twat-cache[diskcache]    # For SQL-based disk caching
pip install twat-cache[joblib]       # For efficient array caching
pip install twat-cache[redis]        # For distributed caching with Redis
```

## Usage

### Basic Memory Caching

For simple in-memory caching with LRU eviction:

```python
from twat_cache import mcache

@mcache(maxsize=100)  # Cache up to 100 items
def expensive_function(x: int) -> int:
    # Expensive computation here
    return x * x

# First call computes
result1 = expensive_function(5)  # Computes 25

# Second call uses cache
result2 = expensive_function(5)  # Returns cached 25
```

### Disk-Based Caching

For persistent caching using SQLite:

```python
from twat_cache import bcache

@bcache(
    folder_name="my_cache",  # Cache directory name
    maxsize=1_000_000,       # Max cache size in bytes
    ttl=3600,               # Cache entries expire after 1 hour
    use_sql=True,           # Use SQLite backend
    secure=True,            # Use secure file permissions
)
def expensive_function(x: int) -> int:
    return x * x
```

### Redis Distributed Caching

For distributed caching with Redis:

```python
from twat_cache import ucache

@ucache(
    preferred_engine="redis",
    folder_name="redis_cache",  # Used as Redis namespace
    ttl=3600,                  # Cache entries expire after 1 hour
    compress=True,             # Enable compression
)
def expensive_function(x: int) -> int:
    return x * x
```

### File-Based Caching

For efficient caching of large objects like NumPy arrays:

```python
from twat_cache import fcache
import numpy as np

@fcache(
    folder_name="array_cache",
    compress=True,           # Enable compression
    secure=True,            # Use secure file permissions
)
def process_array(data: np.ndarray) -> np.ndarray:
    # Expensive array processing here
    return data * 2
```

### Async Caching

For async functions with Redis or memory backend:

```python
from twat_cache import ucache

@ucache(use_async=True)
async def fetch_data(url: str) -> dict:
    # Async web request here
    return {"data": "..."}

# First call fetches
data1 = await fetch_data("https://api.example.com")

# Second call uses cache
data2 = await fetch_data("https://api.example.com")
```

### Universal Caching

Let the library choose the best backend:

```python
from twat_cache import ucache

@ucache(
    folder_name="cache",     # Optional - uses disk cache if provided
    maxsize=1000,           # Optional - limits cache size
    ttl=3600,              # Optional - entries expire after 1 hour
    policy="lru",          # Optional - LRU eviction (default)
    use_sql=True,          # Optional - use SQL backend if available
    compress=True,         # Optional - enable compression
    secure=True,           # Optional - secure file permissions
)
def my_function(x: int) -> int:
    return x * x
```

### Smart Backend Selection

Automatically select the best backend based on data characteristics:

```python
from twat_cache import smart_cache

@smart_cache()
def process_data(data_type: str, size: int) -> Any:
    """Process different types of data with automatic backend selection."""
    if data_type == "dict":
        return {f"key_{i}": f"value_{i}" for i in range(size)}
    elif data_type == "list":
        return [i for i in range(size)]
    elif data_type == "str":
        return "x" * size
    else:
        return size
```

### Hybrid Caching

Switch backends based on result size:

```python
from twat_cache import hybrid_cache

@hybrid_cache()
def get_data(size: str) -> Union[Dict[str, Any], List[int]]:
    """Return different sized data with appropriate backend selection."""
    if size == "small":
        # Small result, will use in-memory caching
        return {"name": "Small Data", "value": 42}
    else:
        # Large result, will use disk caching
        return [i for i in range(100000)]
```

### Type-Specific Configuration

Configure caching based on data types:

```python
from twat_cache import ucache, configure_for_numpy, configure_for_json

# For NumPy arrays
@ucache(config=configure_for_numpy())
def process_array(data: np.ndarray) -> np.ndarray:
    return data * 2

# For JSON data
@ucache(config=configure_for_json())
def fetch_json_data(url: str) -> Dict[str, Any]:
    return {"data": [1, 2, 3, 4, 5], "metadata": {"source": url}}
```

### Cache Management

Clear caches and get statistics:

```python
from twat_cache import clear_cache, get_stats

# Clear all caches
clear_cache()

# Get cache statistics
stats = get_stats()
print(stats)  # Shows hits, misses, size, etc.
```

### Context Management

Use cache engines with context management:

```python
from twat_cache import CacheContext, engine_context

# Method 1: Using the CacheContext class
with CacheContext(engine_name="diskcache", folder_name="cache") as cache:
    # Use the cache
    cache.set("key", "value")
    value = cache.get("key")
    
# Method 2: Using the engine_context function
with engine_context(engine_name="redis", ttl=3600) as cache:
    # Use the cache
    cache.set("key", "value")
    value = cache.get("key")
    
# Cache is automatically closed when exiting the context
```

## Advanced Features

### TTL Support

Set time-to-live for cache entries:

```python
from twat_cache import ucache

@ucache(ttl=3600)  # Entries expire after 1 hour
def get_weather(city: str) -> dict:
    # Fetch weather data
    return {"temp": 20}
```

### Eviction Policies

Choose from different cache eviction policies:

```python
from twat_cache import ucache

# Least Recently Used (default)
@ucache(policy="lru")
def function1(x: int) -> int:
    return x * x

# Least Frequently Used
@ucache(policy="lfu")
def function2(x: int) -> int:
    return x * x

# First In, First Out
@ucache(policy="fifo")
def function3(x: int) -> int:
    return x * x
```

## Documentation

For more detailed documentation, see the following resources:

- [Context Management](docs/context_management.md)
- [Backend Selection](docs/backend_selection.md)
- [Cache Engines](docs/cache_engines.md)
- [Configuration Options](docs/configuration.md)
- [API Reference](docs/api_reference.md)

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Rationale

Python provides several powerful caching libraries to help optimize application performance by storing and reusing expensive function results. Let's take an in-depth look at some of the most popular options, comparing their features, backends, methods, and use cases.


### Built-in functools 

Python's standard library offers basic caching functionality through the `functools` module. It provides decorators like `@lru_cache(maxsize, typed)` for simple memoization with a Least Recently Used (LRU) eviction policy. The `@cache` decorator is also available, which is equivalent to an unbounded LRU cache.

- **Backend**: In-memory Python dictionary 
- **Eviction Policy**: LRU or unbounded
- **Concurrency**: Thread-safe via internal locks, but not safe for multi-process use
- **Persistence**: No persistence, cache only exists in memory for the lifetime of the process
- **Best For**: Fast and easy caching of function results in memory with minimal setup

```pyi
### functools.pyi (Partial - Caching related parts)

import typing

_T = typing.TypeVar("_T")

@typing.overload
def lru_cache(maxsize: int | None) -> typing.Callable[[_F], _F]: ...
@typing.overload
def lru_cache(maxsize: int | None, typed: bool) -> typing.Callable[[_F], _F]: ...

class _lru_cache_wrapper(typing.Generic[_F]):
    cache: typing.Dict[typing.Tuple[typing.Any, ...], typing.Any]
    def cache_info(self) -> CacheInfo: ...
    def cache_clear(self) -> None: ...
```

```python
import functools

@functools.lru_cache(maxsize=128)
def expensive_function(x):
    return x * 2
```

### cachetools

The `cachetools` library extends the capabilities of `functools` by offering additional cache classes with various eviction policies like LFU, FIFO, TTL, and RR. It also provides `@cached` and `@cachedmethod` decorators for caching functions and methods.

- **Backend**: In-memory Python dictionary
- **Eviction Policies**: LRU, LFU, TTL, FIFO, RR, TLRU 
- **Concurrency**: Thread-safe, not designed for multi-process use
- **Persistence**: In-memory only, no persistence 
- **Customization**: Configurable `maxsize`, `ttl`, custom `getsizeof` to determine item size
- **Best For**: In-memory caching with specific eviction behavior, e.g. caching web API responses

```pyi
### cachetools.pyi (Partial - Most important classes and decorators)

import typing
from typing import Callable

KT = typing.TypeVar("KT")
VT = typing.TypeVar("VT")
DT = typing.TypeVar("DT")

class Cache(typing.MutableMapping[KT, VT]):
    def __init__(self, maxsize: int, getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    @property
    def maxsize(self) -> int: ...
    @property
    def currsize(self) -> int: ...
    def get(self, key: KT, default: DT = ...) -> VT | DT: ...
    def pop(self, key: KT, default: DT = ...) -> VT | DT: ...
    def setdefault(self, key: KT, default: DT = ...) -> VT | DT: ...
    def clear(self) -> None: ...
    @staticmethod
    def getsizeof(value: Any) -> int: ...

class LRUCache(Cache[KT, VT]):
    def __init__(self, maxsize: int, getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    def popitem(self) -> tuple[KT, VT]: ...

class LFUCache(Cache[KT, VT]):
    def __init__(self, maxsize: int, getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    def popitem(self) -> tuple[KT, VT]: ...

class TTLCache(LRUCache[KT, VT]):
    def __init__(self, maxsize: int, ttl: int, timer: Callable[[], float] | None = ..., getsizeof: Callable[[VT], int] | None = ...) -> None: ...
    @property
    def ttl(self) -> int: ...
    def expire(self, time: float | None = ...) -> list[tuple[KT, VT]]: ...
    def popitem(self) -> tuple[KT, VT]: ...

class _CacheInfo(NamedTuple):
    hits: int
    misses: int
    maxsize: int
    currsize: int

_KeyFunc = Callable[..., typing.Hashable]
_Cache = TypeVar("_Cache", bound=Cache)

@overload
def cached(
    cache: _Cache,
    key: _KeyFunc = ...,
    lock: Any | None = ...,
    info: Literal[False] = ...,
) -> Callable[[_F], _F]: ...
@overload
def cached(
    cache: _Cache,
    key: _KeyFunc = ...,
    lock: Any | None = ...,
    info: Literal[True] = ...,
) -> Callable[[_F], _F]: ...

@overload
def cachedmethod(
    cache: Callable[[Any], _Cache],
    key: _KeyFunc = ...,
    lock: Callable[[Any], Any] | None = ...,
) -> Callable[[_F], _F]: ...
@overload
def cachedmethod(
    cache: Callable[[Any], _Cache],
    key: _KeyFunc = ...,
    lock: None = ...,
) -> Callable[[_F], _F]: ...
```

### cachebox

`cachebox` is an in-memory caching solution accelerated by a Rust backend for enhanced performance. It supports similar eviction policies as `cachetools` including a unique variable TTL cache.

- **Backend**: Rust-backed in-memory store
- **Eviction Policies**: LRU, LFU, FIFO, TTL, RR, variable TTL
- **Concurrency**: Thread-safe Rust implementation, but in-memory only
- **Decorators**: `@cached`, `@cachedmethod` support custom key generation and callbacks
- **Best For**: Performant in-memory caching with ample policy choices

```pyi
### cachebox.pyi (Partial - Most important classes)

import typing

KT = typing.TypeVar("KT")
VT = typing.TypeVar("VT")
DT = typing.TypeVar("DT")


class BaseCacheImpl(Generic[KT, VT]):

    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None: ...
    @property
    def maxsize(self) -> int: ...
    def __len__(self) -> int: ...
    def __contains__(self, key: KT) -> bool: ...
    def __setitem__(self, key: KT, value: VT) -> None: ...
    def __getitem__(self, key: KT) -> VT: ...
    def __delitem__(self, key: KT) -> VT: ...
    def __iter__(self) -> typing.Iterator[KT]: ...
    def capacity(self) -> int: ...
    def is_full(self) -> bool: ...
    def is_empty(self) -> bool: ...
    def insert(self, key: KT, value: VT) -> typing.Optional[VT]: ...
    def get(self, key: KT, default: DT = None) -> typing.Union[VT, DT]: ...
    def pop(self, key: KT, default: DT = None) -> typing.Union[VT, DT]: ...
    def setdefault(
        self, key: KT, default: typing.Optional[DT] = None
    ) -> typing.Optional[VT | DT]: ...
    def popitem(self) -> typing.Tuple[KT, VT]: ...
    def drain(self, n: int) -> int: ...
    def clear(self, *, reuse: bool = False) -> None: ...
    def shrink_to_fit(self) -> None: ...
    def update(
        self, iterable: typing.Union[typing.Iterable[typing.Tuple[KT, VT]], typing.Dict[KT, VT]]
    ) -> None: ...
    def keys(self) -> typing.Iterable[KT]: ...
    def values(self) -> typing.Iterable[VT]: ...
    def items(self) -> typing.Iterable[typing.Tuple[KT, VT]]: ...

class Cache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...
    def popitem(self) -> typing.NoReturn: ...  # not implemented for this class
    def drain(self, n: int) -> typing.NoReturn: ...  # not implemented for this class


class FIFOCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class RRCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class TTLCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        ttl: float,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class LRUCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class LFUCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        *,
        capacity: int = ...,
    ) -> None:
        ...

class VTTLCache(BaseCacheImpl[KT, VT]):
    def __init__(
        self,
        maxsize: int,
        iterable: Union[Iterable[Tuple[KT, VT]], Dict[KT, VT]] = ...,
        ttl: Optional[float] = 0.0,
        *,
        capacity: int = ...,
    ) -> None:
        ...

_CacheType = TypeVar("_CacheType", bound=BaseCacheImpl)

def cached(cache: typing.Optional[_CacheType], key_maker: typing.Optional[Callable[[tuple, dict], typing.Hashable]] = ..., clear_reuse: bool = ..., callback: typing.Optional[Callable[[int, typing.Any, typing.Any], typing.Any]] = ..., copy_level: int = ..., always_copy: typing.Optional[bool] = ...) -> Callable[[_F], _F]: ...
def cachedmethod(cache: typing.Optional[_CacheType], key_maker: typing.Optional[Callable[[tuple, dict], typing.Hashable]] = ..., clear_reuse: bool = ..., callback: typing.Optional[Callable[[int, typing.Any, typing.Any], typing.Any]] = ..., copy_level: int = ..., always_copy: typing.Optional[bool] = ...) -> Callable[[_F], _F]: ...
```

### klepto

For advanced caching workflows, `klepto` provides a highly flexible solution supporting both in-memory and persistent backends like file archives, SQL databases, and HDF5 files. It allows customizing how cache keys are generated and extends the standard cache eviction policies.

- **Backends**: In-memory dict, file archives, SQL databases, HDF5 files
- **Eviction Policies**: Same as `functools` plus LFU, MRU, RR
- **Key Mapping**: `hashmap`, `stringmap`, `picklemap` algorithms to generate keys
- **Persistence**: Archives cache to disk or database for long-term storage
- **Concurrency**: Locking for thread-safety, some process-safety depending on backend
- **Best For**: Complex scenarios needing transient and persistent caching

```pyi
### key maps
class keymap:
    def __init__(self, typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...
    def __call__(self, *args, **kwargs) -> object: ...

class hashmap(keymap):
    def __init__(self, algorithm: Optional[str] = None,
                 typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...

class stringmap(keymap):
    def __init__(self, encoding: Optional[str] = None,
                 typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...

class picklemap(keymap):
    def __init__(self, serializer: Optional[str] = None,
                 typed: bool = False, flat: bool = True,
                 sentinel: object=...) -> None: ...

### base archive
class archive(dict):
    def __init__(self, *args, **kwds) -> None: ...
    def __asdict__(self) -> dict: ...
    def __repr__(self) -> str: ...
    def copy(self, name: Optional[str] = None) -> 'archive': ...
    def load(self, *args) -> None: ...
    def dump(self, *args) -> None: ...
    def archived(self, *on) -> bool: ...
    def sync(self, clear: bool = False) -> None: ...
    def drop(self) -> None: ...
    def open(self, archive: 'archive') -> None: ...

    @property
    def archive(self) -> 'archive': ...
    @archive.setter
    def archive(self, archive: 'archive') -> None: ...
    @property
    def name(self) -> str: ...
    @name.setter
    def name(self, archive: 'archive') -> None: ...
    @property
    def state(self) -> dict: ...
    @state.setter
    def state(self, archive: 'archive') -> None: ...

class dict_archive(archive):
    def __init__(self, *args, **kwds) -> None: ...

class null_archive(archive):
    def __init__(self, *args, **kwds) -> None: ...

class file_archive(archive):
    def __init__(self, filename: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class dir_archive(archive):
    def __init__(self, dirname: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 compression: int = 0, permissions: Optional[int] = None,
                 memmode: Optional[str] = None, memsize: int = 100,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class sqltable_archive(archive):
    def __init__(self, database: Optional[str] = None,
                 table: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class sql_archive(archive):
    def __init__(self, database: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class hdfdir_archive(archive):
    def __init__(self, dirname: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 permissions: Optional[int] = None,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...

class hdf_archive(archive):
    def __init__(self, filename: Optional[str] = None, dict: Optional[dict] = None,
                 cached: bool = True, serialized: bool = True,
                 protocol: Optional[int] = None, *args, **kwds) -> None: ...
```

### diskcache

When caching large datasets that should survive process restarts, `diskcache` shines with its optimized disk-backed storage using SQLite and the filesystem. It offers a range of eviction policies, a sharded `FanoutCache`, and persistent data structures like `Deque` and `Index`.

- **Backend**: SQLite for metadata, filesystem for data
- **Eviction Policies**: Configurable, including LRU and LFU
- **Persistence**: Data persists on disk between process runs
- **Concurrency**: Thread and process-safe 
- **Added Features**: Stampede prevention, throttling, optimized I/O
- **Best For**: Persistent caching for web apps and data pipelines

```pyi
### diskcache.pyi (Partial - Most important classes)

import typing
from typing import Callable, List, Dict, Any, IO, Optional, Tuple, Union

class Cache:
    def __init__(self, directory: Optional[str] = ..., timeout: float = ..., disk: Type[Disk] = ..., **settings: Any) -> None: ...
    @property
    def directory(self) -> str: ...
    @property
    def timeout(self) -> float: ...
    @property
    def disk(self) -> Disk: ...
    def set(self, key: Any, value: Any, expire: Optional[float] = ..., read: bool = ..., tag: Optional[str] = ..., retry: bool = ...) -> bool: ...
    def get(self, key: Any, default: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...) -> Any: ...
    def delete(self, key: Any, retry: bool = ...) -> bool: ...
    def clear(self, retry: bool = ...) -> int: ...
    def volume(self) -> int: ...
    def check(self, fix: bool = ..., retry: bool = ...) -> List[warnings.WarningMessage]: ...
    def transact(self, retry: bool = ...) -> ContextManager[Callable]: ...
    def memoize(self, name: Optional[str] = ..., typed: bool = ..., expire: Optional[float] = ..., tag: Optional[str] = ..., ignore: typing.Iterable[str] = ...) -> Callable[[_F], _F]: ...
    def close(self) -> None: ...
    def volume(self) -> int: ...
    def stats(self, enable: bool = ..., reset: bool = ...) -> Tuple[int, int]: ...
    def volume(self) -> int: ...

class FanoutCache:
    def __init__(self, directory: Optional[str] = ..., shards: int = ..., timeout: float = ..., disk: Type[Disk] = ..., **settings: Any) -> None: ...
    @property
    def directory(self) -> str: ...
    def transact(self, retry: bool = ...) -> ContextManager[Callable]: ...
    def set(self, key: Any, value: Any, expire: Optional[float] = ..., read: bool = ..., tag: Optional[str] = ..., retry: bool = ...) -> bool: ...
    def get(self, key: Any, default: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...) -> Any: ...
    def delete(self, key: Any, retry: bool = ...) -> bool: ...
    def clear(self, retry: bool = ...) -> int: ...
    def volume(self) -> int: ...
    def stats(self, enable: bool = ..., reset: bool = ...) -> Tuple[int, int]: ...
    def memoize(self, name: Optional[str] = ..., typed: bool = ..., expire: Optional[float] = ..., tag: Optional[str] = ..., ignore: typing.Iterable[str] = ...) -> Callable[[_F], _F]: ...
    def close(self) -> None: ...
    def volume(self) -> int: ...

class Disk:
    def __init__(self, directory: str, min_file_size: int = ..., pickle_protocol: int = ...) -> None: ...
    @property
    def min_file_size(self) -> int: ...
    @property
    def pickle_protocol(self) -> int: ...
    def put(self, key: Any) -> Tuple[Union[str, sqlite3.Binary, int, float], bool]: ...
    def get(self, key: Union[str, sqlite3.Binary, int, float], raw: bool) -> Any: ...
    def store(self, value: Any, read: bool, key: Constant = ...) -> Tuple[int, int, Optional[str], Optional[Union[str, sqlite3.Binary, int, float]]]: ...
    def fetch(self, mode: int, filename: Optional[str], value: Optional[Union[str, sqlite3.Binary, int, float]], read: bool) -> Any: ...

class JSONDisk(Disk):
    def __init__(self, directory: str, compress_level: int = ..., **kwargs: Any) -> None: ...
```

### joblib

Designed for scientific computing and ML workflows, `joblib` offers transparent disk-caching for functions, with optimizations for large NumPy arrays. Results are saved to disk and only re-computed when inputs change.

- **Backend**: Filesystem
- **Persistence**: Caches results to disk for costly computations
- **Memoization**: `@memory.cache` decorator for functions
- **Serialization**: Pickle-based with optional compression
- **Concurrency**: Process-safe file locking
- **Best For**: Caching ML models, features, large NumPy arrays

```pyi
### joblib.pyi (Partial - Most important classes and functions)

import typing
from typing import Callable, List, Dict, Any, IO, Optional, Tuple, Union

_F = TypeVar("_F", bound=Callable[..., Any])

class Memory:
    def __init__(self, location: Optional[str] = ..., backend: str = ..., verbose: int = ..., bytes_limit: Optional[Union[int, str]] = ..., mmap_mode: Optional[str] = ..., compress: Union[bool, int] = ..., backend_options: Optional[Dict[str, Any]] = ...) -> None: ...
    @property
    def location(self) -> str: ...
    @property
    def backend(self) -> str: ...
    @property
    def compress(self) -> Union[bool, int]: ...
    @property
    def verbose(self) -> int: ...
    def cache(self, func: Optional[_F] = ..., ignore: Optional[List[str]] = ..., verbose: Optional[int] = ..., mmap_mode: Optional[str] = ..., cache_validation_callback: Optional[Callable[[Dict[str, Any]], bool]] = ...) -> _F: ...
    def clear(self, warn: bool = ...) -> None: ...
    def eval(self, func: _F, *args: Any, **kwargs: Any) -> Any: ...
    def __call__(self, func: _F, *args: Any, **kwargs: Any) -> Any: ...

class Parallel:
    def __init__(self, n_jobs: Optional[int] = ..., backend: Optional[str] = ..., verbose: int = ..., timeout: Optional[float] = ..., pre_dispatch: Union[str, int] = ..., batch_size: Union[str, int] = ..., temp_folder: Optional[str] = ..., max_nbytes: Optional[Union[int, str]] = ..., mmap_mode: Optional[str] = ..., prefer: Optional[str] = ..., require: Optional[str] = ...) -> None: ...
    def __call__(self, iterable: typing.Iterable) -> List[Any]: ...
    def __enter__(self) -> "Parallel": ...
    def __exit__(self, exc_type: Optional[Type[BaseException]], exc_value: Optional[BaseException], traceback: Optional[TracebackType]) -> None: ...
    def submit(self, func: _F, *args: Any, **kwargs: Any) -> concurrent.futures.Future: ...
    def map(self, func: _F, *iterables: typing.Iterable, timeout: Optional[float] = ..., chunksize: int = ...) -> typing.Iterable[Any]: ...
    def dispatch_next(self) -> None: ...
    def print_progress(self) -> None: ...
    def clear(self) -> None: ...
    def __len__(self) -> int: ...

def delayed(function: _F) -> Callable[..., Tuple[Callable, tuple, dict]]: ...
def cpu_count(only_physical_cores: bool = ...) -> int: ...
def effective_n_jobs(n_jobs: int = ...) -> int: ...
def hash(obj: Any, hash_name: str = ..., coerce_mmap: bool = ...) -> str: ...
def dump(value: Any, filename: Union[str, PathLikeStr], compress: Union[bool, int] = ..., protocol: Optional[int] = ..., cache_size: Optional[int] = ...) -> List[str]: ...
def load(filename: Union[str, PathLikeStr], mmap_mode: Optional[str] = ...) -> Any: ...
def register_parallel_backend(name: str, factory: Callable[..., ParallelBackendBase], make_default: bool = ...) -> None: ...

#### aiocache

Built for async applications using the `asyncio` framework, `aiocache` enables non-blocking caching operations. It provides both in-memory (`SimpleMemoryCache`) and distributed options (`RedisCache`, `MemcachedCache`).

- **Backends**: In-memory, Redis, Memcached
- **Async Decorators**: `@cached`, `@cached_stampede`, `@multi_cached`  
- **Serialization**: Pluggable, defaults to JSON
- **Best For**: Asynchronous web frameworks (FastAPI, Sanic), distributed caching

```pyi
### aiocache.pyi (Partial - Most important classes and decorators)

import typing
from typing import Any, Callable, Coroutine, Dict, List, Optional, Tuple, Type, Union

_F = typing.TypeVar("_F", bound=Callable[..., Any])

class BaseCache:
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., plugins: Optional[List[BasePlugin]] = ..., namespace: Optional[str] = ..., timeout: float = ..., ttl: Optional[int] = ...) -> None: ...
    @classmethod
    def parse_uri_path(cls, path: str) -> Dict[str, str]: ...
    async def add(self, key: str, value: Any, ttl: Optional[int] = ..., dumps_fn: Optional[Callable[[Any], bytes]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def get(self, key: str, default: Optional[Any] = ..., loads_fn: Optional[Callable[[bytes], Any]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> Any: ...
    async def multi_get(self, keys: List[str], loads_fn: Optional[Callable[[bytes], Any]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> List[Any]: ...
    async def set(self, key: str, value: Any, ttl: Optional[int] = ..., dumps_fn: Optional[Callable[[Any], bytes]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ..., _cas_token: Optional[Any] = ...) -> bool: ...
    async def multi_set(self, pairs: List[Tuple[str, Any]], ttl: Optional[int] = ..., dumps_fn: Optional[Callable[[Any], bytes]] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def delete(self, key: str, namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> int: ...
    async def exists(self, key: str, namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def increment(self, key: str, delta: int = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> int: ...
    async def expire(self, key: str, ttl: Optional[int] = ..., namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def clear(self, namespace: Optional[str] = ..., timeout: Optional[int] = ...) -> bool: ...
    async def close(self, timeout: Optional[int] = ...) -> bool: ...
    async def raw(self, command: str, *args: Any, encoding: Optional[str] = ..., timeout: Optional[int] = ..., **kwargs: Any) -> Any: ...

class SimpleMemoryCache(BaseCache):
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., **kwargs: Any) -> None: ...

class RedisCache(BaseCache):
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., **kwargs: Any) -> None: ...

class MemcachedCache(BaseCache):
    NAME: str
    def __init__(self, serializer: Optional[BaseSerializer] = ..., **kwargs: Any) -> None: ...

class BaseSerializer:
    DEFAULT_ENCODING: Optional[str]
    def __init__(self, *args: Any, encoding: Union[str, object] = ..., **kwargs: Any) -> None: ...
    def dumps(self, value: Any) -> bytes: ...
    def loads(self, value: bytes) -> Any: ...

class JsonSerializer(BaseSerializer):
    def dumps(self, value: Any) -> str: ...
    def loads(self, value: str) -> Any: ...

class PickleSerializer(BaseSerializer):
    def dumps(self, value: Any) -> bytes: ...
    def loads(self, value: bytes) -> Any: ...

class NullSerializer(BaseSerializer):
    def dumps(self, value: Any) -> Any: ...
    def loads(self, value: Any) -> Any: ...

class BasePlugin:
    async def pre_get(self, client: BaseCache, key: str, namespace: Optional[str], **kwargs: Any) -> None: ...
    async def post_get(self, client: BaseCache, key: str, namespace: Optional[str], rv: Any, **kwargs: Any) -> None: ...
    # ... (and similar methods for other cache operations)

_Cache = TypeVar("_Cache", bound=BaseCache)

def cached(
    ttl: Optional[int] = ...,
    key: Optional[str] = ...,
    key_builder: Optional[Callable[..., str]] = ...,
    namespace: Optional[str] = ...,
    cache: Union[Type[_Cache], str] = ...,
    serializer: Optional[BaseSerializer] = ...,
    plugins: Optional[List[BasePlugin]] = ...,
    alias: Optional[str] = ...,
    noself: bool = ...,
    skip_cache_func: Optional[Callable[[Any], bool]] = ...,
    **kwargs: Any
) -> Callable[[_F], _F]: ...

def cached_stampede(
    lease: int = ...,
    ttl: Optional[int] = ...,
    key: Optional[str] = ...,
    key_builder: Optional[Callable[..., str]] = ...,
    namespace: Optional[str] = ...,
    cache: Union[Type[_Cache], str] = ...,
    serializer: Optional[BaseSerializer] = ...,
    plugins: Optional[List[BasePlugin]] = ...,
    alias: Optional[str] = ...,
    noself: bool = ...,
    skip_cache_func: Optional[Callable[[Any], bool]] = ...,
    **kwargs: Any
) -> Callable[[_F], _F]: ...

def multi_cached(
    keys_from_attr: Optional[str] = ...,
    namespace: Optional[str] = ...,
    key_builder: Optional[Callable[..., str]] = ...,
    ttl: Optional[int] = ...,
    cache: Union[Type[_Cache], str] = ...,
    serializer: Optional[BaseSerializer] = ...,
    plugins: Optional[List[BasePlugin]] = ...,
    alias: Optional[str] = ...,
    skip_cache_func: Optional[Callable[[str, Any], bool]] = ...,
    **kwargs: Any
) -> Callable[[_F], _F]: ...
```

### Summary of type hints

```pyi
### Type Stubs for Caching Libraries

from typing import (
    Any,
    Callable,
    Dict,
    List,
    Optional,
    Type,
    Iterable,
    Tuple,
    Union,
    TypeVar,
    Generic,
)
import datetime
import logging
from typing_extensions import Literal  # For older Python versions
```

*   **Generics:** For generic types like `Cache`, `FIFOCache`, `LFUCache`, etc. (in `cachetools`, `cachebox`, `klepto`) , I've used `TypeVar` to represent the key (`KT`) and value (`VT`) types.  This gives better type hinting.
*   **`object` as Default:** Where the original code uses implicit dynamic typing (no explicit type) or uses an internal implementation detail, I've often used `object` as the most general type hint.  This avoids creating stubs for internal classes.
*   **`NotImplementedError` and `typing.NoReturn`:** I've used `typing.NoReturn` for methods that are *not implemented* in a base class.  This is more precise than raising `NotImplementedError`, which would imply that a subclass *should* implement it.
*   **`defaultdict`:**  I've handled cases where `defaultdict` is used internally, providing a default factory function.
*   **`_Conn`:**  This internal helper class from `aiocache` is fully defined as all its methods are based on exported API.
* **`_DISTUTILS_PATCH`, `VIRTUALENV_PATCH_FILE`, etc.:** These internal constants/classes, used for patching `distutils`, are omitted.
*   **`kwargs` Handling:** In many cases, I've used `**kwargs` to represent arbitrary keyword arguments, especially when those arguments are passed directly to another function (like `open` or a backend-specific constructor).
* **`functools`:** The return type of decorators (`lru_cache`, `cache`) is `Callable[[Callable[..., R]], Callable[..., R]]`.  I've used a `TypeVar` named `_R` to make the relationship between the input and output types clear.
* **`klepto` classes**: Added the API for the `archive`, `dir_archive`, `sql_archive`, `sqltable_archive` based on the source code.
* **`joblib` classes**: Added the classes in the public API, `Memory`, `MemorizedFunc`, `NotMemorizedFunc`.

#### Recommendations

The best caching library depends on your specific needs - whether you require persistence, have large datasets, need multi-process safety, or are using an async framework. By understanding the strengths of each library, you can make an informed choice to optimize your application's performance.

================
File: plugins/repos/twat_cache/TODO.md
================
---
this_file: TODO.md
---

# twat-cache TODO List

## Phase 1

- [ ] Add more defensive programming with better input validation
  - Validate function arguments more thoroughly
  - Add type checking for critical parameters
  - Implement graceful fallbacks for invalid inputs
- [ ] Refactor complex functions in decorators.py to improve readability
  - Break down large functions into smaller, focused ones
  - Improve naming for better self-documentation
  - Add explanatory comments for complex logic
- [ ] Increase overall test coverage to 90%
  - [ ] Add more unit tests for edge cases
  - [ ] Implement integration tests for all backends
  - [ ] Add performance regression tests

## Medium Priority

- [ ] Add support for asynchronous cache operations in all engines
  - Implement async versions of all cache operations
  - Add async context managers
  - Ensure compatibility with asyncio event loops
- [ ] Implement a unified configuration system with environment variable support
  - Add support for loading config from environment variables
  - Create a configuration hierarchy (env vars > config files > defaults)
  - Add validation for configuration values
- [ ] Create a more intuitive API for cache management operations
  - Simplify the interface for common operations
  - Add helper functions for frequent use cases
  - Improve error messages and feedback
- [ ] Add decorator factories with more customizable options
  - Support for custom key generation functions
  - Add options for cache invalidation strategies
  - Implement conditional caching based on arguments
- [ ] Create simpler API for cache statistics and monitoring
  - Add hit/miss ratio tracking
  - Implement cache efficiency metrics
  - Add visualization tools for cache performance

## Performance Optimizations

- [ ] Optimize key generation for better performance
  - Implement faster hashing algorithms for keys
  - Add support for custom key generation functions
  - Optimize key storage and lookup
- [ ] Implement smart TTL handling with refreshing strategies
  - Add background refresh for frequently accessed items
  - Implement sliding TTL windows
  - Add support for TTL based on access patterns
- [ ] Add memory usage monitoring to prevent cache bloat
  - Implement memory usage tracking
  - Add automatic pruning based on memory pressure
  - Create alerts for excessive memory usage
- [ ] Optimize serialization/deserialization for common data types
  - Add specialized serializers for numpy arrays and pandas DataFrames
  - Implement compression for large objects
  - Add support for incremental serialization

## Compatibility and Integration

- [ ] Ensure compatibility with Python 3.12+
  - Test with latest Python versions
  - Update type hints to use latest typing features
  - Address any deprecation warnings
- [ ] Add support for integration with popular frameworks
  - Create adapters for Flask, FastAPI, Django
  - Add middleware for web frameworks
  - Implement integration examples
- [ ] Ensure compatibility with container environments
  - Test in Docker and Kubernetes environments
  - Add configuration for containerized deployments
  - Document best practices for containers

## Documentation and Examples

- [ ] Create a comprehensive API reference with examples
  - Document all public classes and functions
  - Add usage examples for each feature
  - Include performance considerations
- [ ] Add doctest examples in function docstrings
  - Add executable examples in docstrings
  - Ensure examples are tested in CI
  - Keep examples up-to-date with API changes
- [ ] Create tutorials for advanced use cases
  - Add step-by-step guides for common scenarios
  - Create examples for complex configurations
  - Document performance optimization strategies

================
File: plugins/repos/twat_cache/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_coding/.cursor/rules/0project.mdc
================
---
description: About this project
globs: 
---
---
name: About this project
key: 0project
---

`twat-coding` is a Python toolkit focused on code analysis and transformation. Its main package `pystubnik` generates "smart stubs" - a hybrid between full source code and type stubs that helps LLMs understand large codebases efficiently. Smart stubs preserve essential code structure (signatures, types, imports) while intelligently reducing verbosity based on code importance.

## 1. Overview

Pystubnik creates a "shadow" directory structure that mirrors your Python package, containing smart stubs for all Python files. These smart stubs are designed to be more informative than traditional `.pyi` stub files while being more concise than full source code.

### 1.1. What are Smart Stubs?

Smart stubs are an intermediate representation that includes:
- All function and class signatures with type hints
- All imports (organized and optimized)
- Docstrings (with configurable length limits)
- Important/relevant code sections
- Truncated versions of large data structures and strings
- Simplified function bodies for non-critical code

The verbosity level is automatically adjusted based on the code's importance and complexity.

## 2. Architecture

### 2.1. Backends

#### 2.1.1. AST Backend
- Uses Python's built-in AST module for precise control
- Preserves code structure while reducing verbosity
- Configurable truncation of large literals and sequences
- Maintains type information and docstrings
- Supports Python 3.12+ features (type parameters, etc.)

#### 2.1.2. MyPy Backend
- Leverages mypy's stubgen for type information
- Better type inference capabilities
- Handles special cases (dataclasses, properties)
- Supports type comment extraction

### 2.2. Processors

#### 2.2.1. Import Processor
- Analyzes and organizes imports
- Groups by type (stdlib, third-party, local)
- Handles relative imports
- Detects and removes duplicates

#### 2.2.2. Docstring Processor
- Configurable docstring preservation
- Format detection and conversion
- Type information extraction
- Length-based truncation

#### 2.2.3. Importance Processor
- Scores code elements by importance
- Pattern-based importance detection
- Inheritance-aware scoring
- Configurable filtering

================
File: plugins/repos/twat_coding/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `./cleanup.py install && source .venv/bin/activate && ./cleanup.py update | cat` script in the main folder. Analyze the results, describe recent changes @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `./cleanup.py update | cat` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done, and `- [!]` if they're NEXT TODO. 

Don't use `pip`, use `uv pip`.

================
File: plugins/repos/twat_coding/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[1.0K]  .
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [2.2K]  0project.mdc
│       ├── [ 632]  cleanup.mdc
│       ├── [4.3K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [  96]  .specstory
│   └── [ 672]  history
│       ├── [2.0K]  .what-is-this.md
│       ├── [ 63K]  adjusting-todo-priorities-from-cleanup-status.md
│       ├── [ 97K]  cleanup-script-execution-and-todo-management-1.md
│       ├── [ 85K]  cleanup-script-execution-and-todo-management-2.md
│       ├── [322K]  cleanup-script-execution-and-todo-management.md
│       ├── [319K]  code-cleanup-and-todo-update-process.md
│       ├── [333K]  command-execution-and-todo-update.md
│       ├── [ 248]  detailed-overview-of-python-stubbing-tools.md
│       ├── [ 25K]  integrating-importance-analysis-modules.md
│       ├── [431K]  managing-todo-list-tasks-1.md
│       ├── [671K]  managing-todo-list-tasks-2.md
│       ├── [356K]  managing-todo-list-tasks.md
│       ├── [380K]  managing-todo-md-tasks.md
│       ├── [3.0K]  overview-of-python-tools-and-their-functions.md
│       ├── [ 58K]  project-directory-structure-for-python-package.md
│       ├── [ 64K]  reviewing-and-tracking-todo-progress.md
│       ├── [ 56K]  task-management-from-todo-md.md
│       ├── [4.9K]  task-organization-and-cleanup-process.md
│       └── [165K]  todo-list-cleanup-and-review.md
├── [2.4K]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [ 173]  MANIFEST.in
├── [1.4K]  README.md
├── [7.4K]  TODO.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [  96]  cursor
│   └── [  96]  rules
│       └── [  31]  0project.mdc
├── [ 160]  dev
│   ├── [ 94K]  RESEARCH.md
│   ├── [5.7K]  TODO.md
│   └── [ 84K]  python-minifier.md
├── [ 128]  dist
│   └── [   1]  .gitkeep
├── [ 305]  mypy.ini
├── [ 426]  package.toml
├── [5.3K]  pyproject.toml
├── [223K]  repomix-output.txt
├── [ 128]  src
│   └── [ 288]  twat_coding
│       ├── [ 144]  __init__.py
│       ├── [ 480]  pystubnik
│       │   ├── [5.3K]  README.md
│       │   ├── [ 16K]  __init__.py
│       │   ├── [ 224]  backends
│       │   │   ├── [1.2K]  __init__.py
│       │   │   ├── [ 12K]  ast_backend.py
│       │   │   ├── [2.4K]  base.py
│       │   │   └── [ 982]  mypy_backend.py
│       │   ├── [5.8K]  cli.py
│       │   ├── [8.8K]  config.py
│       │   ├── [ 256]  core
│       │   │   ├── [6.1K]  config.py
│       │   │   ├── [2.4K]  conversion.py
│       │   │   ├── [ 492]  shared_types.py
│       │   │   ├── [3.9K]  types.py
│       │   │   └── [8.3K]  utils.py
│       │   ├── [5.1K]  errors.py
│       │   ├── [ 320]  processors
│       │   │   ├── [1.2K]  __init__.py
│       │   │   ├── [5.0K]  docstring.py
│       │   │   ├── [ 16K]  file_importance.py
│       │   │   ├── [8.6K]  importance.py
│       │   │   ├── [6.6K]  imports.py
│       │   │   ├── [ 23K]  stub_generation.py
│       │   │   └── [7.4K]  type_inference.py
│       │   ├── [1.6K]  read_imports.py
│       │   ├── [ 160]  types
│       │   │   ├── [ 10K]  docstring.py
│       │   │   └── [9.7K]  type_system.py
│       │   └── [ 192]  utils
│       │       ├── [6.2K]  ast_utils.py
│       │       ├── [1.8K]  display.py
│       │       └── [5.2K]  memory.py
│       └── [1.6K]  twat_coding.py
└── [ 128]  tests
    └── [ 12K]  test_package.py

20 directories, 73 files

================
File: plugins/repos/twat_coding/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_coding/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_coding --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_coding/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-coding
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_coding/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_coding/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_coding/cursor/rules/0project.mdc
================
---
description: 
globs: 
---

================
File: plugins/repos/twat_coding/dist/.gitignore
================
*

================
File: plugins/repos/twat_coding/dist/.gitkeep
================


================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/backends/__init__.py
================
class StubBackend(Protocol):
    async def generate_stub(self, source_path: Path) -> StubResult:
def register_backend(name: str, backend: type[StubBackend]) -> None:
def get_backend(name: str) -> type[StubBackend]:
        raise KeyError(msg)
def list_backends() -> list[str]:
    return list(_backends.keys())

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/backends/ast_backend.py
================
class ASTCacheEntry:
class SignatureExtractor(ast.NodeTransformer):
    def __init__(
        super().__init__()
    def _preserve_docstring(self, body: list[ast.stmt]) -> list[ast.stmt]:
            isinstance(first, ast.Expr)
            and isinstance(first.value, ast.Constant)
            and isinstance(first.value.value, str)
    def visit_Module(self, node: ast.Module) -> ast.Module:
            if isinstance(stmt, ast.Import | ast.ImportFrom):
                imports.append(stmt)
                isinstance(stmt, ast.Expr)
                and isinstance(stmt.value, ast.Constant)
                and isinstance(stmt.value.value, str)
                new_body.append(stmt)
                new_body.append(self.visit(stmt))
    def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
        node.body = self._preserve_docstring(node.body) or [ast.Pass()]
    def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
        node.body = [self.visit(stmt) for stmt in node.body]
class ASTBackend(StubBackend):
    _ast_cache_lock: ClassVar[asyncio.Lock] = asyncio.Lock()
    def __init__(self, config: StubConfig | StubGenConfig | None = None) -> None:
        super().__init__()  # Object doesn't take any arguments
            weakref.WeakValueDictionary()
        if isinstance(config, StubConfig):
            self._executor = ThreadPoolExecutor(
            stub_config = config or StubGenConfig(
                paths=PathConfig(), runtime=RuntimeConfig()
        self._memory_monitor = MemoryMonitor()
    def config(self) -> StubGenConfig:
        if not hasattr(self, "_config"):
            return StubGenConfig(paths=PathConfig(), runtime=RuntimeConfig())
        if isinstance(self._config, StubConfig):
            return convert_to_stub_gen_config(self._config)
        elif isinstance(self._config, StubGenConfig):
    async def generate_stub(self, source_path: Path) -> StubResult:
        return await self._generate_stub_internal(source_path)
    async def _generate_stub_internal(self, source_path: Path) -> StubResult:
            source = await self._run_in_executor(source_path.read_text)
            tree = await self._run_in_executor(
                functools.partial(ast.parse, source, filename=str(source_path))
            attach_parents(tree)
                if isinstance(self.config, StubGenConfig)
                else convert_to_stub_gen_config(self.config)
            transformer = SignatureExtractor(stub_gen_config, len(source))
            transformed = transformer.visit(tree)
            stub_content = ast.unparse(transformed)
            result = StubResult(
                    "source_size": len(source),
                    "node_count": sum(1 for _ in ast.walk(tree)),
            if hasattr(self, "processors"):
                    result = processor.process(result)
                    transformer = SignatureExtractor(
                        len(source),
                    result.stub_content = ast.unparse(transformed)
            logger.error(f"Error generating stub for {source_path}: {e}")
            raise ASTError(
                details={"error": str(e)},
    async def process_directory(self, directory: Path) -> dict[Path, StubResult]:
                python_files.extend(directory.rglob(pattern))
                python_files = [f for f in python_files if not f.match(pattern)]
            total = len(python_files)
            for i, path in enumerate(python_files, 1):
                    print_progress("Processing files", i, total)
                    result = await self._generate_stub_internal(path)
                    logger.error(f"Failed to process {path}: {e}")
                source=str(directory),
    async def process_module(self, module_name: str) -> StubResult:
    async def process_package(self, package_path: Path) -> dict[Path, StubResult]:
        return await self.process_directory(package_path)
    def cleanup(self) -> None:
        self._executor.shutdown(wait=True)
        self._ast_cache.clear()
        self._node_registry.clear()
        self._memory_monitor.stop()
    async def _run_in_executor(self, func: Any, *args: Any) -> Any:
        loop = asyncio.get_running_loop()
        return await loop.run_in_executor(self._executor, func, *args)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/backends/base.py
================
class StubBackend(ABC):
    def __init__(self, config: StubConfig | StubGenConfig | None = None) -> None:
        if isinstance(config, StubConfig):
            self._config = _convert_to_stub_gen_config(config)
            self._config = config or StubGenConfig(paths=PathConfig())
    async def generate_stub(self, source_path: Path) -> StubResult:
    async def process_module(self, module_name: str) -> StubResult:
    async def process_package(self, package_path: Path) -> dict[Path, StubResult]:
    def cleanup(self) -> None:
    def __enter__(self) -> "StubBackend":
    def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
        self.cleanup()

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/backends/mypy_backend.py
================
class MypyBackend(StubBackend):
    def __init__(self, config: StubGenConfig | None = None):
        super().__init__()
    async def generate_stub(self, source_path: Path) -> StubResult:
        raise NotImplementedError("MyPy backend not yet implemented")

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/core/config.py
================
class Backend(Enum):
    AST = auto()  # Use Python's AST for precise control
    MYPY = auto()  # Use MyPy's stubgen for better type inference
    HYBRID = auto()  # Use both and merge results
class ImportanceLevel(Enum):
def _default_importance_keywords() -> set[str]:
@dataclass(frozen=True)
class ProcessingConfig:
    importance_patterns: Mapping[str, float] = field(default_factory=dict)
    importance_keywords: set[str] = field(default_factory=_default_importance_keywords)
def _default_include_patterns() -> list[str]:
def _default_exclude_patterns() -> list[str]:
class PathConfig:
    output_dir: Path = Path("out")
    search_paths: Sequence[Path] = field(default_factory=list)
    modules: Sequence[str] = field(default_factory=list)
    packages: Sequence[str] = field(default_factory=list)
    files: Sequence[Path] = field(default_factory=list)
    include_patterns: list[str] = field(default_factory=_default_include_patterns)
    exclude_patterns: list[str] = field(default_factory=_default_exclude_patterns)
class RuntimeConfig:
    python_version: tuple[int, int] = field(
    interpreter: Path = field(default_factory=lambda: Path(sys.executable))
    def create(
            interpreter = Path(sys.executable)
        elif isinstance(interpreter, str):
            interpreter = Path(interpreter)
        return cls(
class StubGenConfig:
    runtime: RuntimeConfig = field(default_factory=RuntimeConfig)
    processing: ProcessingConfig = field(default_factory=ProcessingConfig)
    truncation: TruncationConfig = field(default_factory=TruncationConfig)
    def include_patterns(self) -> list[str]:
    def exclude_patterns(self) -> list[str]:
    def ignore_errors(self) -> bool:
    def get_file_locations(self, source_path: Path) -> tuple[Path, Path]:
            rel_path = source_path.relative_to(self.paths.output_dir)
            raise ValueError(
    def from_dict(cls, data: dict[str, Any]) -> "StubGenConfig":
        paths = PathConfig(**data.get("paths", {}))
        runtime = RuntimeConfig(**data.get("runtime", {}))
        processing = ProcessingConfig(**data.get("processing", {}))
        truncation = TruncationConfig(**data.get("truncation", {}))
    def to_dict(self) -> dict[str, Any]:
                k: str(v) if isinstance(v, Path) else v
                for k, v in self.paths.__dict__.items()

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/core/conversion.py
================
def convert_to_stub_gen_config(config: StubConfig | None = None) -> StubGenConfig:
        return StubGenConfig(
            paths=PathConfig(),
            runtime=RuntimeConfig(),
            processing=ProcessingConfig(),
            truncation=TruncationConfig(),
    paths = PathConfig(
        output_dir=config.output_path or Path("out"),
        doc_dir=Path(config.doc_dir) if config.doc_dir else None,
        search_paths=[Path(p) for p in config.search_paths],
        modules=list(config.modules),
        packages=list(config.packages),
        files=[Path(f) for f in config.files],
    runtime = RuntimeConfig(
        interpreter=Path(config.interpreter)
        if isinstance(config.interpreter, str)
    processing = ProcessingConfig(
        importance_patterns=dict(config.importance_patterns),
    truncation = TruncationConfig(

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/core/shared_types.py
================
@dataclass(frozen=True)
class TruncationConfig:

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/core/types.py
================
@dataclass(frozen=True)
class ArgInfo:
class FunctionInfo:
class ClassInfo:
    class_vars: dict[str, str] = field(default_factory=dict)  # name -> type
    instance_vars: dict[str, str] = field(default_factory=dict)  # name -> type
class ModuleInfo:
    variables: dict[str, str] = field(default_factory=dict)  # name -> type
class StubBackend(Protocol):
    def generate_module_info(self, module_path: Path) -> ModuleInfo:
    def generate_stub(self, module_info: ModuleInfo) -> str:
class ImportProcessor(Protocol):
    def process_imports(self, source: str) -> Sequence[str]:
    def sort_imports(self, imports: Sequence[str]) -> Sequence[str]:
class DocstringProcessor(Protocol):
    def process_docstring(
class ImportanceScorer(Protocol):
    def calculate_score(
class ImportType(Enum):
class ImportInfo:
class StubResult:
    metadata: dict[str, Any] = field(default_factory=dict)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/core/utils.py
================
T = TypeVar("T")
U = TypeVar("U")
def process_parallel(
        max_workers = os.cpu_count() or 1
    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        futures = {executor.submit(process_func, item): item for item in items}
        for future in as_completed(futures):
                result = future.result()
                results.append(result)
                    logger.debug(f"{desc}: Processed {item}")
                logger.error(f"Failed to process {item}: {e}")
                if not isinstance(e, KeyboardInterrupt):
                    logger.debug(f"Error details: {type(e).__name__}: {e}")
def find_python_files(
    path = Path(path)
    if not path.exists():
        raise FileNotFoundError(msg)
    def is_excluded(p: Path) -> bool:
        return any(p.match(pattern) for pattern in exclude_patterns)
    if path.is_file():
        return [path] if path.suffix == ".py" and not is_excluded(path) else []
    for root, _, files in os.walk(path):
        root_path = Path(root)
        if is_excluded(root_path):
            if file_path.suffix == ".py" and not is_excluded(file_path):
                result.append(file_path)
    return sorted(result)
def normalize_docstring(docstring: str | None) -> str | None:
    lines = docstring.expandtabs().splitlines()
        stripped = line.lstrip()
            indent = min(indent, len(line) - len(stripped))
    trimmed = [lines[0].strip()]
            trimmed.append(line[indent:].rstrip())
        trimmed.pop()
    return "\n".join(trimmed)
def get_qualified_name(node: ast.AST) -> str:
    if isinstance(node, ast.Name):
    elif isinstance(node, ast.Attribute):
        return f"{get_qualified_name(node.value)}.{node.attr}"
def parse_type_string(type_str: str) -> str:
    type_str = re.sub(r"\s+", " ", type_str.strip())
    type_str = re.sub(r"Union\[(.*?)\]", r"(\1)", type_str)
    type_str = re.sub(r"\s*\|\s*", " | ", type_str)
    type_str = re.sub(r"Optional\[(.*?)\]", r"\1 | None", type_str)
                result.append(" ")
            result.append(", ")
        result.append(char)
    return "".join(result)
class ImportTracker:
    def __init__(self) -> None:
        self.explicit_imports: set[str] = set()  # Explicitly requested imports
    def add_import(self, module: str, name: str | None = None) -> None:
            self.imports.setdefault(module, set()).add(name)
            self.imports.setdefault(module, set())
        self.explicit_imports.add(module)
    def add_import_from(
        self.import_froms.setdefault(module, {}).update(dict(names))
    def get_import_lines(self) -> list[str]:
        for module in sorted(self.imports):
            names = sorted(self.imports[module])
                items = ", ".join(names)
                lines.append(f"from {module} import {items}")
                lines.append(f"import {module}")
        for module in sorted(self.import_froms):
                items = ", ".join(
                    for name, alias in sorted(name_map.items())
def setup_logging(level: str = "INFO") -> None:
    logger.remove()  # Remove default handler
    logger.add(
        lambda msg: print(msg),
def read_source_file(path: str | Path) -> tuple[str, str | None]:
        with path.open("rb") as f:
            source = f.read()
        for line in source.split(b"\n")[:2]:
            if line.startswith(b"#") and b"coding:" in line:
                encoding = line.split(b"coding:")[-1].strip().decode("ascii")
        return source.decode(encoding), encoding
        logger.error(f"Failed to read {path}: {e}")
def parse_source(source: str) -> ast.AST | None:
        return ast.parse(source)
        logger.error(f"Failed to parse source: {e}")
def normalize_path(path: str | Path) -> Path:
    return Path(path).resolve()

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/__init__.py
================
class Processor(Protocol):
    def process(self, stub_result: StubResult) -> StubResult:
def register_processor(name: str, processor: type[Processor]) -> None:
def get_processor(name: str) -> type[Processor]:
        raise KeyError(msg)
def list_processors() -> list[str]:
    return list(_processors.keys())

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/docstring.py
================
class TypeInferenceError(StubGenerationError):
    def __init__(self, message: str, details: dict[str, str] | None = None) -> None:
        super().__init__(message, "TYPE001", details)
class DocstringResult:
class DocstringProcessor(Processor):
    def __init__(
    def process(self, stub_result: StubResult) -> StubResult:
        logger.debug(f"Processing docstrings for {stub_result.source_path}")
        module_info = self._process_module_docstring(stub_result)
            self._process_function_docstring(function)
            self._process_class_docstring(class_info)
                self._process_function_docstring(method)
                self._process_function_docstring(prop)
    def _process_module_docstring(self, stub_result: StubResult) -> ModuleInfo:
        logger.debug("Processing module docstring")
        return ModuleInfo(name="", path=stub_result.source_path)
    def _process_function_docstring(self, function: FunctionInfo) -> None:
        logger.debug(f"Processing docstring for function {function.name}")
            docstring = parse(function.docstring, style=self.style)
                    arg = next(
                        object.__setattr__(arg, "type", param.type_name)
                object.__setattr__(function, "return_type", docstring.returns.type_name)
            logger.warning(f"Failed to parse docstring for {function.name}: {e}")
    def _process_class_docstring(self, class_info: ClassInfo) -> None:
        logger.debug(f"Processing docstring for class {class_info.name}")
            parse(class_info.docstring, style=self.style)
            logger.warning(f"Failed to parse docstring for {class_info.name}: {e}")

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/file_importance.py
================
T = TypeVar("T")
def _cast_or_default(value: Any, default: T) -> T:
        if isinstance(value, type(default)):
            return cast(T, value)
def _check_optional_dependencies() -> dict[str, bool]:
    if not importlib.util.find_spec("coverage"):
        print(COVERAGE_MISSING)
    if not importlib.util.find_spec("pydocstyle"):
        print(PYDOCSTYLE_MISSING)
    if not importlib.util.find_spec("radon"):
        print(RADON_MISSING)
AVAILABLE_DEPS = _check_optional_dependencies()
class FileImportanceConfig:
    exclude_dirs: list[str] = field(default_factory=list)
    weights: dict[str, float] = field(
def find_py_files(package_dir: str, exclude_dirs: list[str] | None = None) -> list[str]:
    for root, _, files in os.walk(package_dir):
        if any(exclude in root for exclude in exclude_dirs):
            if file.endswith(".py"):
                py_files.append(os.path.join(root, file))
def is_entry_point(file_path: str) -> bool:
        with open(file_path, encoding="utf-8") as f:
            tree = ast.parse(f.read(), filename=file_path)
        for node in ast.walk(tree):
                isinstance(node, ast.If)
                and isinstance(node.test, ast.Compare)
                and isinstance(node.test.left, ast.Name)
                and len(node.test.ops) == 1
                and isinstance(node.test.ops[0], ast.Eq)
                and len(node.test.comparators) == 1
                and isinstance(node.test.comparators[0], ast.Constant)
def get_additional_entry_points(package_dir: str) -> list[str]:
    pyproject_path = os.path.join(package_dir, "pyproject.toml")
    if os.path.exists(pyproject_path):
            with open(pyproject_path) as f:
                data = toml.load(f)
                scripts = data.get("project", {}).get("scripts", {})
                for script in scripts.values():
                    if os.path.exists(script) and script.endswith(".py"):
                        entry_points.append(script)
def _parse_import_node(
    if isinstance(node, ast.Import):
            imported = name.name.split(".")[0]
                if os.path.basename(target) == f"{imported}.py":
                    edges.append((source_file, target))
    elif isinstance(node, ast.ImportFrom) and node.module:
        imported = node.module.split(".")[0]
def _parse_imports_from_file(file: str, py_files: list[str]) -> list[tuple[str, str]]:
        with open(file, encoding="utf-8") as f:
            tree = ast.parse(f.read(), filename=file)
            if isinstance(node, ast.Import | ast.ImportFrom):
                edges.extend(_parse_import_node(file, node, py_files))
def build_import_graph(package_dir: str, py_files: list[str]) -> nx.DiGraph:
    G = nx.DiGraph()
        G.add_node(file)
        edges = _parse_imports_from_file(file, py_files)
        G.add_edges_from(edges)
def calculate_complexity(file_path: str) -> float:
            code = f.read()
        complexities = cc_visit(code)
        total_complexity = sum(
            _cast_or_default(c.complexity, 0.0) for c in complexities
        return total_complexity / len(complexities)
        print(f"Warning: Failed to calculate complexity for {file_path}: {e}")
def calculate_coverage(file_path: str, coverage_data: str | None) -> float:
    if not coverage_data or not os.path.exists(coverage_data):
        cov = Coverage(data_file=coverage_data)
        cov.load()
        analysis = cov._analyze(file_path)
        total_lines = _cast_or_default(analysis.numbers.n_statements, 0)
        covered_lines = _cast_or_default(analysis.numbers.n_executed, 0)
        print(f"Warning: Failed to calculate coverage for {file_path}: {e}")
def calculate_doc_quality(file_path: str) -> float:
        violations = list(pydocstyle_check([file_path], ignore=["D100", "D101"]))
        return max(0.0, 1.0 - (len(violations) / 10))
        print(f"Warning: Failed to assess documentation quality for {file_path}: {e}")
def _calculate_centrality(
        if sum(personalization_dict.values()) == 0:
        result = nx.pagerank(G, personalization=personalization_dict)
        return {k: float(v) for k, v in result.items()}
        result = nx.betweenness_centrality(G)
            result = nx.eigenvector_centrality(G, max_iter=500)
        print(
        result = nx.pagerank(G)
def _print_results(
    print(header)
    print("-" * len(header))
        is_entry = "Yes" if entry_points.get(file, False) else "No"
        is_init = "Yes" if os.path.basename(file) == "__init__.py" else "No"
        rel_path = os.path.relpath(file, package_dir)
            f"{centrality.get(file, 0):<10.3f} "
            f"{complexity_scores.get(file, 0):<10.1f} "
            f"{coverage_scores.get(file, 0):<10.1f} "
            f"{doc_scores.get(file, 0):<10.2f} "
def prioritize_files(
    py_files = find_py_files(package_dir, config.exclude_dirs)
        print("No Python files found in the package directory.")
    G = build_import_graph(package_dir, py_files)
    entry_points = {file: is_entry_point(file) for file in py_files}
    additional_eps = get_additional_entry_points(package_dir)
    centrality = _calculate_centrality(G, py_files, config.centrality, entry_points)
    complexity_scores = {file: calculate_complexity(file) for file in py_files}
        file: calculate_coverage(file, config.coverage_data) for file in py_files
    doc_scores = {file: calculate_doc_quality(file) for file in py_files}
    max_complexity = max(complexity_scores.values(), default=1.0)
        file: score / max_complexity for file, score in complexity_scores.items()
            weights["centrality"] * centrality.get(file, 0.0)
            + weights["complexity"] * complexity_normalized.get(file, 0.0)
            + weights["coverage"] * (coverage_scores.get(file, 0.0) / 100)
            + weights["doc_quality"] * doc_scores.get(file, 0.0)
    sorted_files = sorted(composite_scores.items(), key=lambda x: x[1], reverse=True)
    _print_results(
def load_config(config_file: str | None) -> dict[str, Any]:
    if config_file and os.path.exists(config_file):
            with open(config_file) as f:
                config = json.load(f)
                return cast(dict[str, Any], config)
    parser = argparse.ArgumentParser(
    parser.add_argument("package_dir", help="Path to the package directory")
    parser.add_argument("--config", help="Path to configuration JSON file")
    parser.add_argument("--coverage-data", help="Path to coverage.py data file")
    parser.add_argument("--exclude-dirs", nargs="+", help="Directories to exclude")
    parser.add_argument(
    args = parser.parse_args()
    config = FileImportanceConfig()
    user_config = load_config(args.config)
        config.weights.update(user_config.get("weights", {}))
        config.exclude_dirs.extend(user_config.get("exclude_dirs", []))
        config.exclude_dirs.extend(args.exclude_dirs)
    prioritize_files(args.package_dir, config)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/importance.py
================
class ImportanceConfig:
    patterns: dict[str, float] = field(default_factory=dict)
    keywords: set[str] = field(
    file_importance: FileImportanceConfig = field(default_factory=FileImportanceConfig)
class ImportanceProcessor(Processor):
    def __init__(self, config: ImportanceConfig | None = None) -> None:
        self.config = config or ImportanceConfig()
    def process(self, stub_result: StubResult) -> StubResult:
                    package_dir = str(stub_result.source_path.parent)
                    self._file_scores = prioritize_files(
                    logger.warning(f"Failed to calculate file importance: {e}")
            file_score = self._file_scores.get(str(stub_result.source_path), 0.5)
            stub_result.metadata["importance_level"] = self._get_importance_level(
            tree = ast.parse(stub_result.stub_content)
            for node in ast.walk(tree):
                if isinstance(node, ast.FunctionDef | ast.ClassDef):
                    docstring = ast.get_docstring(node)
                    score = self.calculate_importance(
                        is_public=not node.name.startswith("_"),
                        is_special=node.name.startswith("__")
                        and node.name.endswith("__"),
                    symbol_key = f"{node.__class__.__name__.lower()}_{node.name}"
                        self._get_importance_level(score)
                    if score < 0.7 and isinstance(node, ast.FunctionDef):
                        node.body = [ast.Expr(value=ast.Constant(value=Ellipsis))]
            stub_result.stub_content = ast.unparse(tree)
            logger.warning(
    def _get_importance_level(self, score: float) -> str:
    def _calculate_pattern_score(self, name: str) -> float:
        for pattern, weight in self.config.patterns.items():
            if re.search(pattern, name):
    def _calculate_docstring_score(self, docstring: str | None) -> float:
        word_count = len(docstring.split())
            if keyword.lower() in docstring.lower():
    def _calculate_visibility_score(self, is_public: bool, is_special: bool) -> float:
    def calculate_importance(
            file_score = self._get_file_score(extra_info)
            pattern_score = self._calculate_pattern_score(name)
            docstring_score = self._calculate_docstring_score(docstring)
            visibility_score = self._calculate_visibility_score(is_public, is_special)
            return max(min(final_score, 1.0), 0.0)
            logger.warning(f"Error calculating importance for {name}: {e}")
    def _get_file_score(self, extra_info: dict[str, Any] | None) -> float:
            return self._file_scores.get(str(file_path), 1.0)
    def should_include(

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/imports.py
================
class ImportProcessor(Processor):
    def __init__(self) -> None:
        self.stdlib_modules: set[str] = self._get_stdlib_modules()
    def process(self, stub_result: StubResult) -> StubResult:
        tree = ast.parse(stub_result.stub_content)
        imports = self._analyze_imports(tree)
        grouped_imports = self._group_imports(imports)
        new_imports = self._format_imports(grouped_imports)
        stub_result.stub_content = self._replace_imports(tree, new_imports)
        stub_result.imports = list(imports.values())
    def _get_stdlib_modules(self) -> set[str]:
        stdlib_path = sysconfig.get_path("stdlib")
            return set()
        stdlib_modules = set()
        for path in Path(stdlib_path).glob("**/*.py"):
                stdlib_modules.add(module_name)
    def _analyze_imports(self, tree: ast.AST) -> dict[str, ImportInfo]:
        for node in ast.walk(tree):
            if isinstance(node, ast.Import):
                    import_type = self._get_import_type(module_name)
                    imports[module_name] = ImportInfo(
            elif isinstance(node, ast.ImportFrom):
                import_type = self._get_import_type(module)
                    imports[module] = ImportInfo(
                    imports[module].imported_names.append(name.asname or name.name)
    def _get_import_type(self, module_name: str) -> ImportType:
        base_module = module_name.split(".")[0]
    def _group_imports(
        for import_info in imports.values():
            grouped[import_info.import_type].append(import_info)
        for group in grouped.values():
            group.sort(key=lambda x: x.module_name)
    def _format_imports(
                    names = ", ".join(sorted(import_info.imported_names))
                    section.append(
                    section.append(f"import {import_info.module_name}")
                sections.append("\n".join(sorted(section)))
        return "\n\n".join(sections) + "\n\n"
    def _replace_imports(self, tree: ast.AST, new_imports: str) -> str:
            if isinstance(node, ast.Import | ast.ImportFrom):
            source_lines = ast.unparse(tree).split("\n")
            first_import_line = min(
                for node in ast.walk(tree)
                if isinstance(node, ast.Import | ast.ImportFrom)
            last_import_line = max(
            before = "\n".join(source_lines[: first_import_line - 1])
            after = "\n".join(source_lines[last_import_line:])
        return ast.unparse(tree)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/stub_generation.py
================
class StubVisitor(NodeVisitor):
    def __init__(self, config: StubConfig) -> None:
        super().__init__()
    def _should_include_member(self, name: str) -> bool:
        return should_include_member(name, self.config.include_private)
    def visit_ClassDef(self, node: ClassDef) -> None:
        if not should_include_member(node.name, self.config.include_private):
        self.classes.append(node)
            if isinstance(item, FunctionDef):
                if should_include_member(item.name, self.config.include_private):
                    self.visit(item)
            elif isinstance(item, Assign | AnnAssign):
    def visit_FunctionDef(self, node: FunctionDef) -> None:
        if not self._should_include_member(node.name):
        self.functions.append(node)
        self.generic_visit(node)
    def visit_Assign(self, node: Assign) -> None:
        self.assignments.append(node)
    def visit_AnnAssign(self, node: AnnAssign) -> None:
    def visit_Import(self, node: Import) -> None:
            if name.name == "pathlib" or name.name.startswith("pathlib."):
                self.imports["pathlib"].append(node)
            elif name.name == "typing" or name.name.startswith("typing."):
                self.imports["typing"].append(node)
                self.imports["stdlib"].append(node)
    def visit_ImportFrom(self, node: ImportFrom) -> None:
            self.imports["local"].append(node)
            names = sorted(name.name for name in node.names)
            if any(
                if isinstance(imp, ImportFrom) and imp.module == "typing"
                existing = next(
                    alias(name=name) for name in sorted(set(names) | existing_names)
    def get_sorted_imports(self) -> list[str]:
        stdlib_imports = sorted(
            self.imports["stdlib"], key=lambda x: self._get_import_name(x)
        sorted_imports.extend(self._format_import(imp) for imp in stdlib_imports)
        pathlib_imports = sorted(
            self.imports["pathlib"], key=lambda x: self._get_import_name(x)
                sorted_imports.append("")
            sorted_imports.extend(self._format_import(imp) for imp in pathlib_imports)
        typing_imports = sorted(
            self.imports["typing"], key=lambda x: self._get_import_name(x)
            sorted_imports.extend(self._format_import(imp) for imp in typing_imports)
        local_imports = sorted(
            self.imports["local"], key=lambda x: self._get_import_name(x)
            sorted_imports.extend(self._format_import(imp) for imp in local_imports)
    def _get_import_name(self, node: Import | ImportFrom) -> str:
        if isinstance(node, Import):
    def _format_import(self, node: Import | ImportFrom) -> str:
        names = ", ".join(sorted(name.name for name in node.names))
class StubGenerator:
    def __init__(self, config: StubConfig | None = None) -> None:
        self.config = config or StubConfig(
            input_path=Path("."),
    def generate_stub(
        self.current_file = Path(source_file)
            with open(source_file, encoding="utf-8") as f:
                source = f.read()
            ast_tree = parse(source)
        if not isinstance(ast_tree, Module):
            raise ValueError("AST must be a Module node")
        attach_parents(ast_tree)
        visitor = StubVisitor(self.config)
        visitor.visit(ast_tree)
            lines.append('"""# Generated stub file"""\n')
            sorted_imports = visitor.get_sorted_imports()
            lines.extend(sorted_imports)
                lines.append("")
            class_lines = self._process_class_to_lines(class_def)
                lines.extend(class_lines)
            func_lines = self._process_function_to_lines(func_def)
                lines.extend(func_lines)
            assign_lines = self._process_assignment_to_lines(assignment)
                lines.extend(assign_lines)
        while lines and not lines[-1].strip():
            lines.pop()
        return "\n".join(lines)
    def _process_class_to_lines(self, node: ClassDef) -> list[str]:
        docstring = get_docstring(node)
            lines.append(f'"""{docstring}"""')
        bases = [unparse(base) for base in node.bases]
            lines.append(f"class {node.name}({', '.join(bases)}):")
            lines.append(f"class {node.name}:")
                if self._should_include_member(item.name):
                    method_lines = self._process_function_to_lines(item)
                        body_lines.extend("    " + line for line in method_lines)
                assign_lines = self._process_assignment_to_lines(item)
                    body_lines.extend("    " + line for line in assign_lines)
        lines.extend(body_lines)
    def _process_function_to_lines(self, node: FunctionDef) -> list[str]:
            and len(node.body) > 0
            and isinstance(node.body[0], Expr)
            and isinstance(node.body[0].value, Constant)
                arg_str += f": {self._format_annotation(arg_node.annotation)}"
            arg_idx = node.args.args.index(arg_node)
            if node.args.defaults and arg_idx >= len(node.args.args) - len(
                default_idx = arg_idx - (len(node.args.args) - len(node.args.defaults))
                default_value = self._format_annotation(node.args.defaults[default_idx])
            args_parts.append(arg_str)
        args_str = ", ".join(args_parts)
            f" -> {self._format_annotation(node.returns)}" if node.returns else ""
        lines.append(func_def)
        lines.append("    pass")
    def _process_assignment_to_lines(self, node: Assign | AnnAssign) -> list[str]:
        if isinstance(node, Assign):
            if isinstance(target, Name):
                if isinstance(node.value, Constant):
                    value_str = repr(node.value.value)
                    type_str = type(node.value.value).__name__
        elif isinstance(node, AnnAssign):
            if isinstance(node.target, Name):
                annotation = unparse(node.annotation)
                    value_str = unparse(node.value)
    def _format_annotation(self, node: AST) -> str:
        if isinstance(node, Name):
        elif isinstance(node, Constant):
            return repr(node.value)
        elif isinstance(node, Attribute):
            return f"{self._format_annotation(node.value)}.{node.attr}"
        elif isinstance(node, Subscript):
            return f"{self._format_annotation(node.value)}[{self._format_annotation(node.slice)}]"
        elif isinstance(node, BinOp):
            return f"{self._format_annotation(node.left)} | {self._format_annotation(node.right)}"
        elif isinstance(node, AstList):
            return f"[{', '.join(self._format_annotation(elt) for elt in node.elts)}]"
        elif isinstance(node, AstTuple):
            return f"({', '.join(self._format_annotation(elt) for elt in node.elts)})"
            return unparse(node)
            return f"import {', '.join(sorted(alias.name for alias in node.names))}"
                f"import {', '.join(sorted(alias.name for alias in node.names))}"
    def _sort_imports(self, imports: list[Import | ImportFrom]) -> list[str]:
                if module_path.startswith("typing"):
                    typing_imports.append(node)
                elif module_path.startswith("pathlib"):
                    pathlib_imports.append(node)
                elif module_path.startswith("."):
                    local_imports.append(node)
                    stdlib_imports.append(node)
                elif module_path.startswith("typing"):
            for node in sorted(import_group, key=lambda x: x.names[0].name):
                sorted_imports.append(self._format_import(node))
    def _import_sort_key(self, node: Import | ImportFrom) -> tuple[int, str, str]:
                if any(module_path.startswith(imp) for imp in self.ESSENTIAL_IMPORTS)
    def _ensure_node_attributes(self, node: AST) -> None:
            if not hasattr(node, attr):
                setattr(node, attr, default)
        for child in iter_child_nodes(node):
            self._ensure_node_attributes(child)
    def _generate_content(self, node: Module) -> str:
        source = unparse(node)
        source = source.replace("=", " = ")
        source = source.replace("  =  ", " = ")
        source = source.replace("'''", '"""')
        source = source.replace('""""""', '"""')
    def _collect_imports(self, node: Module) -> list[tuple[str, str]]:
            if isinstance(child, Import | ImportFrom) and self._should_keep_import(
                if isinstance(child, ImportFrom):
                    names = sorted(n.name for n in child.names)
                        typing_imports.append(f"from typing import {', '.join(names)}")
                        pathlib_imports.append(
                            f"from pathlib import {', '.join(names)}"
                    elif module.startswith("."):
                        local_imports.append(f"from {module} import {', '.join(names)}")
                        stdlib_imports.append(
                            f"from {module} import {', '.join(names)}"
                    import_str = f"import {', '.join(names)}"
                    if any(name == "Path" for name in names):
                        pathlib_imports.append(import_str)
                        stdlib_imports.append(import_str)
        stdlib_imports.sort()
        pathlib_imports.sort()
        typing_imports.sort()
        local_imports.sort()
        imports.extend(("stdlib", imp) for imp in stdlib_imports)
        imports.extend(("pathlib", imp) for imp in pathlib_imports)
        imports.extend(("typing", imp) for imp in typing_imports)
        imports.extend(("local", imp) for imp in local_imports)
    def _should_keep_import(self, node: Import | ImportFrom) -> bool:
            return any(
                name.name in self.ESSENTIAL_IMPORTS or not name.name.startswith("_")
            return bool(
                    or not node.module.startswith("_")

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/processors/type_inference.py
================
class TypeInferenceProcessor:
        re.compile(r"_str$|_string$"): str,
        re.compile(r"_int$|_count$|_index$"): int,
        re.compile(r"_float$|_ratio$|_rate$"): float,
        re.compile(r"_bool$|_flag$|is_|has_|can_"): bool,
        re.compile(r"_list$|_array$"): list,
        re.compile(r"_dict$|_map$"): dict,
        re.compile(r"_set$"): set,
        re.compile(r"_tuple$"): tuple,
    def __init__(
        self.type_registry = type_registry or TypeRegistry()
    def infer_types(self, node: ast.AST) -> dict[str, TypeInfo]:
                self._infer_from_assignments(node, inferred_types)
                self._infer_from_usage(node, inferred_types)
                self._infer_from_returns(node, inferred_types)
                for name, type_info in inferred_types.items()
            raise TypeInferenceError(
                details={"node_type": type(node).__name__},
    def _infer_from_assignments(
        for child in ast.walk(node):
                        type_info = self.type_registry.resolve_type(
                        logger.warning(
                    if self.infer_from_defaults and isinstance(value, ast.Constant):
                        type_info = TypeInfo(
                            annotation=type(value.value),
                    for pattern, typ in self.TYPE_PATTERNS.items():
                        if pattern.search(name):
                            types[name] = TypeInfo(
    def _infer_from_usage(self, node: ast.AST, types: dict[str, TypeInfo]) -> None:
            if isinstance(child, ast.Attribute):
                if isinstance(child.value, ast.Name):
                        attrs = types[name].metadata.get("attributes", set())
                        attrs.add(child.attr)
    def _infer_from_returns(self, node: ast.AST, types: dict[str, TypeInfo]) -> None:
            if isinstance(child, ast.FunctionDef | ast.AsyncFunctionDef):
                return_types = set()
                for return_node in ast.walk(child):
                    if isinstance(return_node, ast.Return) and return_node.value:
                        if isinstance(return_node.value, ast.Constant):
                            return_types.add(type(return_node.value.value))
                        elif isinstance(return_node.value, ast.Name):
                                return_types.add(types[name].annotation)
                    types[child.name] = TypeInfo(
                        annotation=next(iter(return_types))
                        if len(return_types) == 1
                        metadata={"return_types": list(return_types)},

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/types/docstring.py
================
class DocstringStyle(Enum):
    GOOGLE = auto()
    NUMPY = auto()
    SPHINX = auto()
    EPYTEXT = auto()
    UNKNOWN = auto()
class DocstringInfo:
    args: list[ArgInfo] = field(default_factory=list)
    raises: list[str] = field(default_factory=list)
    examples: list[str] = field(default_factory=list)
    notes: list[str] = field(default_factory=list)
    see_also: list[str] = field(default_factory=list)
    references: list[str] = field(default_factory=list)
    todo: list[str] = field(default_factory=list)
    warnings: list[str] = field(default_factory=list)
    metadata: dict[str, Any] = field(default_factory=dict)
class DocstringTypeInfo:
class DocstringTypeExtractor:
        "None": type(None),
    def __init__(self, type_registry: TypeRegistry) -> None:
        self._param_pattern = re.compile(
        self._type_pattern = re.compile(r":type\s+(\w+)\s*:\s*([^\n]+)")
        self._rtype_pattern = re.compile(r":rtype:\s*([^\n]+)")
        self._returns_pattern = re.compile(r":returns?:\s*([^\n]+)")
        self._yields_pattern = re.compile(r":yields?:\s*([^\n]+)")
        self._raises_pattern = re.compile(r":raises?\s+([^:]+):\s*([^\n]+)")
    def _extract_param_types(self, doc: Any) -> dict[str, TypeInfo]:
                    type_info = self._parse_type_string(param.type_name)
                    logger.warning(
    def _extract_return_type(self, doc: Any) -> TypeInfo | None:
                return self._parse_type_string(doc.returns.type_name)
                logger.warning(f"Failed to parse return type: {e}")
    def _extract_yield_type(self, doc: Any) -> TypeInfo | None:
        if hasattr(doc, "yields") and doc.yields and doc.yields.type_name:
                return self._parse_type_string(doc.yields.type_name)
                logger.warning(f"Failed to parse yield type: {e}")
    def _extract_raises(self, doc: Any) -> list[tuple[TypeInfo, str]]:
                    exc_type = self._parse_type_string(raises_section.type_name)
                    raises.append((exc_type, raises_section.description or ""))
    def extract_types(
        docstring = ast.get_docstring(node)
        doc = parse_docstring(docstring)
        return DocstringTypeInfo(
            param_types=self._extract_param_types(doc),
            return_type=self._extract_return_type(doc),
            yield_type=self._extract_yield_type(doc),
            raises=self._extract_raises(doc),
    def _parse_simple_type(self, type_str: str) -> TypeInfo | None:
            return TypeInfo(
    def _parse_union_type(self, type_str: str) -> TypeInfo | None:
        types = [self._parse_type_string(t.strip()) for t in type_str.split(" or ")]
            raise TypeInferenceError("Empty union type")
        if len(types) == 1:
    def _parse_container_type(self, container: str, content: str) -> TypeInfo | None:
        if container.lower() in ("list", "set", "tuple"):
            elem_type = self._parse_type_string(content)
            container_type = self.TYPE_MAPPINGS.get(container.lower(), list)
        elif container.lower() == "dict":
            key_type, value_type = map(str.strip, content.split(","))
            key_info = self._parse_type_string(key_type)
            value_info = self._parse_type_string(value_type)
    def _parse_type_string(self, type_str: str) -> TypeInfo:
            type_str = type_str.strip()
            if result := self._parse_simple_type(type_str):
            if result := self._parse_union_type(type_str):
            if match := re.match(r"(\w+)\[(.*)\]", type_str):
                container, content = match.groups()
                if result := self._parse_container_type(container, content):
            raise TypeInferenceError(

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/types/type_system.py
================
class TypeInferenceError(StubGenerationError):
    def __init__(
        super().__init__(message, code, details, source=source, line_number=line_number)
class TypeInfo:
class TypeProtocol(Protocol):
class TypeRegistry:
    def __init__(self) -> None:
    def register_alias(self, name: str, target: Any) -> None:
    def register_type_var(self, name: str, type_var: TypeVar) -> None:
    def register_protocol(self, protocol_class: type[TypeProtocol]) -> None:
    def resolve_type(self, type_hint: Any, context: str = "") -> TypeInfo:
            if isinstance(type_hint, str) and type_hint in self._type_aliases:
            origin = get_origin(resolved)
                args = get_args(resolved)
                    raise TypeInferenceError("Empty Annotated type")
                type_info = TypeInfo(
            elif isinstance(resolved, TypeVar):
            elif _is_protocol(resolved):
                            name: get_type_hints(resolved)[name]
                            for name in dir(resolved)
                            if not name.startswith("_")
                        "args": [self.resolve_type(arg).annotation for arg in args],
            raise TypeInferenceError(
    def merge_types(
            raise TypeInferenceError("No types to merge")
        if len(types) == 1:
            sorted_types = sorted(
                merged_metadata.update(type_info.metadata)
            return TypeInfo(
                details={"types": [str(t.annotation) for t in types]},
def extract_type_from_docstring(docstring: str) -> TypeInfo | None:
def infer_type_from_usage(node: ast.AST) -> TypeInfo | None:
def _is_protocol(resolved: Any) -> bool:
            isinstance(resolved, type)
            and hasattr(resolved, "__mro__")
def _resolve_type(type_hint: Any) -> Any:
        if isinstance(type_hint, str):
            return eval(type_hint, globals(), locals())
def _is_subtype(type_hint: Any, expected_type: Any) -> bool:
    resolved = _resolve_type(type_hint)
    expected = _resolve_type(expected_type)
    if _is_protocol(expected):
    if hasattr(expected, "__origin__") and expected.__origin__ is Union:
        return any(_is_subtype(resolved, t) for t in expected.__args__)
        return isinstance(resolved, type) and issubclass(resolved, expected)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/utils/ast_utils.py
================
_parent_refs: WeakKeyDictionary[ast.AST, ast.AST] = WeakKeyDictionary()
def _get_parent(node: ast.AST) -> ast.AST | None:
    return _parent_refs.get(node)
def _truncate_string(s: str, config: TruncationConfig) -> str:
    if len(s) <= config.max_string_length:
def _truncate_bytes(b: bytes, config: TruncationConfig) -> bytes:
    if len(b) <= config.max_string_length:
def _truncate_sequence(
    for i, e in enumerate(node.elts):
            truncated_elts.append(cast(ast.expr, truncate_literal(e, config)))
            truncated_elts.append(ast.Constant(value=config.truncation_marker))
    return type(node)(elts=truncated_elts)
def _truncate_dict(node: ast.Dict, config: TruncationConfig) -> ast.Dict:
    for i, (k, v) in enumerate(zip(node.keys, node.values, strict=False)):
            new_v = cast(ast.expr, truncate_literal(v, config))
            pairs.append((k, new_v))
            pairs.append(
                    ast.Constant(value="..."),
    return ast.Dict(
        values=[cast(ast.expr, v) for _, v in pairs],
def truncate_literal(node: ast.AST, config: TruncationConfig) -> ast.AST:
            if len(s) > config.max_string_length:
                result = ast.Constant(
            return ast.Constant(value=_truncate_bytes(b, config))
            if len(node.elts) > config.max_sequence_length:
                    ast.Constant(value=config.truncation_marker)
            if len(node.keys) > config.max_sequence_length:
def attach_parents(node: ast.AST) -> None:
    for child in ast.walk(node):
        for _field, value in ast.iter_fields(child):
            if isinstance(value, list):
                    if isinstance(item, ast.AST):
            elif isinstance(value, ast.AST):
def get_docstring(node: ast.AST) -> str | None:
                and isinstance(node.body[0], ast.Expr)
                and isinstance(node.body[0].value, ast.Constant)
                and isinstance(node.body[0].value.value, str)
def is_empty_expr(node: ast.AST) -> bool:
        isinstance(node, ast.Expr)
        and isinstance(node.value, ast.Constant)
        and (not node.value.value or str(node.value.value).isspace())
def should_include_member(name: str, include_private: bool) -> bool:
    if name.startswith("__") and name.endswith("__"):
    if name.startswith("_"):

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/utils/display.py
================
console = Console()
def print_file_tree(paths: list[Path]) -> None:
    for path in sorted(paths):
        add_to_tree(tree, path.parts)
    root = Tree("📁 Project")
    build_rich_tree(root, tree)
    console.print(root)
def add_to_tree(tree: dict[str, Any], components: list[str] | tuple[str, ...]) -> None:
def build_rich_tree(tree: Tree, data: dict[str, Any], prefix: str = "") -> None:
    for name, subtree in sorted(data.items()):
        branch = tree.add(f"{icon} {name}")
            build_rich_tree(branch, subtree, f"{prefix}/{name}" if prefix else name)
def print_progress(message: str, current: int, total: int) -> None:
    console.print(f"{message}: [{current}/{total}] {percentage:.1f}%")

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/utils/memory.py
================
class MemoryStats:
class MemoryMonitor:
    def __init__(self, interval: float = 1.0) -> None:
            logger.warning(
        self._process = psutil.Process(os.getpid())
        self._stop_event = threading.Event()
    def start(self) -> None:
        def _monitor() -> None:
            while not self._stop_event.is_set():
                    meminfo = self._process.memory_info()
                    stats = MemoryStats(
                        peak_rss=self._process.memory_info().rss,
                    self._stats.append(stats)
                    logger.debug(
                    logger.error(f"Failed to collect memory stats: {e}")
                self._stop_event.wait(self.interval)
        self._monitor_thread = threading.Thread(target=_monitor, daemon=True)
        self._monitor_thread.start()
    def stop(self) -> None:
        self._stop_event.set()
        self._monitor_thread.join()
    def stats(self) -> list[MemoryStats]:
        return self._stats.copy()
    def peak_memory(self) -> int:
        return max(stat.peak_rss for stat in self._stats)
    def clear_stats(self) -> None:
        self._stats.clear()
def memory_monitor(interval: float = 1.0) -> Generator[MemoryMonitor, None, None]:
    monitor = MemoryMonitor(interval)
    monitor.start()
        monitor.stop()
async def stream_process_ast(
    nodes = list(ast.walk(node))
    chunks = [nodes[i : i + chunk_size] for i in range(0, len(nodes), chunk_size)]
    for i, chunk in enumerate(chunks):
            processed.append(node)
            gc.collect()
        await asyncio.sleep(0)
def profile_memory(func: Any) -> Any:
        logger.warning("memory_profiler not available, profiling disabled")
    return memory_profile(func)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/__init__.py
================
def _convert_to_stub_config(config: StubGenConfig) -> StubConfig:
    search_paths = [str(p) for p in config.paths.search_paths]
    files = list(config.paths.files)  # Convert Sequence to list
    return StubConfig(
        input_path=config.paths.files[0] if config.paths.files else Path("."),
        doc_dir=str(config.paths.doc_dir) if config.paths.doc_dir else "",
        modules=list(config.paths.modules),  # Convert Sequence to list
        packages=list(config.paths.packages),  # Convert Sequence to list
        importance_patterns=dict(config.processing.importance_patterns),
def _convert_to_stub_gen_config(config: StubConfig) -> StubGenConfig:
    return StubGenConfig(
        paths=PathConfig(
            output_dir=config.output_path or Path("out"),
            doc_dir=Path(config.doc_dir) if config.doc_dir else None,
            search_paths=[Path(p) for p in config.search_paths],
            modules=list(config.modules),
            packages=list(config.packages),
            files=list(config.files),
        runtime=RuntimeConfig.create(
        processing=ProcessingConfig(
            importance_patterns=dict(config.importance_patterns),
        truncation=TruncationConfig(
class Processor(Protocol):
    def process(self, stub_result: StubResult) -> StubResult:
class SmartStubGenerator:
    def __init__(
        if isinstance(backend, str):
            backend = Backend[backend.upper()]
        self.config = StubGenConfig(
                output_dir=Path(output_dir),
                doc_dir=Path(doc_dir) if doc_dir else None,
                search_paths=[Path(p) for p in search_paths],
                modules=list(modules),
                packages=list(packages),
                files=[Path(f) for f in files],
                importance_patterns=dict(importance_patterns or {}),
                importance_keywords=set(importance_keywords or set()),
        logger.remove()
        logger.add(
            lambda msg: print(msg, end=""),
            DocstringProcessor(
            ImportanceProcessor(
                config=ImportanceConfig(
                    patterns=dict(self.config.processing.importance_patterns),
            ImportProcessor(),
    def _initialize_backend(self) -> StubBackend:
        stub_config = _convert_to_stub_config(self.config)
        stub_gen_config = _convert_to_stub_gen_config(stub_config)
            return ASTBackend(stub_config)
            return MypyBackend(stub_gen_config)
            raise ConfigError(
    def _process_file(self, backend: StubBackend, file_path: Path) -> None:
            result = asyncio.run(backend.generate_stub(file_path))
            if isinstance(result, str):
                result = StubResult(
            elif isinstance(result, StubResult):
                    result = processor.process(result)
                logger.error(
                    f"Unexpected result type from generate_stub: {type(result)}"
                self.config.paths.output_dir / file_path.with_suffix(".pyi").name
            output_path.write_text(result.stub_content)
            logger.error(f"Failed to process {file_path}: {e}")
    def generate(self) -> None:
            logger.info(
            logger.info(f"Output directory: {self.config.paths.output_dir}")
            self.config.paths.output_dir.mkdir(parents=True, exist_ok=True)
            backend = self._initialize_backend()
                self._process_file(backend, file_path)
            logger.info("Stub generation completed successfully")
            logger.error(f"Failed to generate stubs: {e}")
                logger.debug(traceback.format_exc())
    def generate_for_file(self, file_path: str | Path) -> StubResult:
    def generate_for_module(self, module_name: str) -> StubResult:
async def generate_stub(
    source_path = Path(source_path)
    output_path_obj = Path(output_path) if output_path else None
        stub_gen_config = StubGenConfig(paths=PathConfig(), runtime=RuntimeConfig())
        stub_gen_config = _convert_to_stub_gen_config(config)
        backend_obj = ASTBackend(stub_gen_config)
        backend_obj = MypyBackend(stub_gen_config)
        raise ValueError(f"Unsupported backend: {backend}")
    result = await backend_obj.generate_stub(source_path)
    if not isinstance(result, StubResult):
        raise TypeError(f"Expected StubResult, got {type(result)}")
        output_path_obj.parent.mkdir(parents=True, exist_ok=True)
        output_path_obj.write_text(result.stub_content)
setup_logging()
__version__ = version("twat_coding")

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/cli.py
================
class PystubnikCLI:
    def __init__(self) -> None:
        self.console = Console()
        self.progress = Progress(
            SpinnerColumn(),
            TextColumn("[progress.description]{task.description}"),
            TimeElapsedColumn(),
    def generate(
        input_file = Path(input_path)
        if not input_file.exists():
            self.console.print(
            config_path = Path(config_file)
            if not config_path.exists():
        config_dict.update(config_kwargs)
            output_path = str(input_file.with_suffix(".pyi"))
        config = StubConfig(
            output_path=Path(output_path).parent,
        generator = StubGenerator(config)
            task = self.progress.add_task(
                stub_content = generator.generate_stub(input_file)
                Path(output_path).write_text(stub_content)
                self.progress.remove_task(task)
                self.console.print(f"[red]Error generating stub: {e}[/red]")
    def generate_dir(
        input_path = Path(input_dir)
        if not input_path.exists():
            output_dir = str(input_path / "stubs")
        output_path = Path(output_dir)
        output_path.mkdir(parents=True, exist_ok=True)
        python_files = list(input_path.rglob("*.py"))
            self.console.print("[yellow]No Python files found in directory[/yellow]")
                f"Generating stubs for {len(python_files)} files...",
                total=len(python_files),
                    rel_path = py_file.relative_to(input_path)
                    out_file = output_path / rel_path.with_suffix(".pyi")
                    out_file.parent.mkdir(parents=True, exist_ok=True)
                    stub_content = generator.generate_stub(py_file)
                    out_file.write_text(stub_content)
                    self.progress.advance(task)
                    self.console.print(f"[red]Error processing {py_file}: {e}[/red]")
def main() -> None:
    fire.Fire(PystubnikCLI)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/config.py
================
class FileLocations(BaseModel):
    source_path: Path = Field(..., description="Path to Python source file")
    input_dir: Path = Field(..., description="Base input directory")
    output_dir: Path = Field(..., description="Base output directory")
    model_config = SettingsConfigDict(
    @field_validator("source_path")
    def validate_source_path(cls, v: Path) -> Path:
        if not v.exists():
            raise ConfigError(
                source=str(v),
    @field_validator("input_dir")
    def validate_input_dir(cls, v: Path) -> Path:
    @field_validator("output_dir")
    def validate_output_dir(cls, v: Path) -> Path:
            v.mkdir(parents=True, exist_ok=True)
    def output_path(self) -> Path:
            rel_path = self.source_path.relative_to(self.input_dir)
    def check_paths(self) -> None:
        self.output_path.parent.mkdir(parents=True, exist_ok=True)
        if not os.access(self.output_path.parent, os.W_OK):
                source=str(self.output_path.parent),
def _default_stub_gen_config() -> StubGenConfig:
    return StubGenConfig(paths=PathConfig())
class StubConfig(BaseModel):
    input_path: Path = Field(
    output_path: Path | None = Field(
    files: list[Path] = Field(
    include_patterns: list[str] = Field(
    exclude_patterns: list[str] = Field(
    backend: Literal["ast", "mypy"] = Field(
    parallel: bool = Field(
    max_workers: int | None = Field(
    stub_gen_config: StubGenConfig = Field(
    infer_types: bool = Field(
    preserve_literals: bool = Field(
    docstring_type_hints: bool = Field(
    line_length: int = Field(
    sort_imports: bool = Field(
    add_header: bool = Field(
    python_version: tuple[int, int] = Field(
    no_import: bool = Field(
    inspect: bool = Field(
    doc_dir: str = Field(
    search_paths: Sequence[str | Path] = Field(
    interpreter: str | Path = Field(
    ignore_errors: bool = Field(
    parse_only: bool = Field(
    include_private: bool = Field(
    modules: Sequence[str] = Field(
    packages: Sequence[str] = Field(
    verbose: bool = Field(
    quiet: bool = Field(
    export_less: bool = Field(
    importance_patterns: dict[str, float] = Field(
    max_docstring_length: int = Field(
    include_type_comments: bool = Field(
    infer_property_types: bool = Field(
    def get_file_locations(self, source_path: Path) -> FileLocations:
        return FileLocations(
            output_dir=self.output_path or Path("out"),
class RuntimeConfig(BaseSettings):
    cache_dir: Path = Field(default_factory=lambda: Path(".cache"))
    @field_validator("cache_dir", mode="before")
    def validate_cache_dir(cls, v: Any) -> Path:
            path = Path(v)
            path.mkdir(parents=True, exist_ok=True)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/errors.py
================
class ErrorCode(str, Enum):
class StubGenerationError(Exception):
    def __init__(
        self.code = ErrorCode(code) if isinstance(code, str) else code
        super().__init__(message)
    def __str__(self) -> str:
        msg = f"[{self.code}] {super().__str__()}"
class ASTError(StubGenerationError):
        super().__init__(message, code, details, source=source, line_number=line_number)
class MyPyError(StubGenerationError):
class ConfigError(StubGenerationError):
        super().__init__(message, code, details, source=source)

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/read_imports.py
================
def ast_to_source(node: ast.Import | ast.ImportFrom) -> str:
    if isinstance(node, ast.Import):
        return f"import {', '.join(names)}"
    elif isinstance(node, ast.ImportFrom):
        return f"from {level}{module} import {', '.join(names)}"
        msg = f"Unexpected node type: {type(node)}"
        raise ValueError(msg)
def extract_imports(source: str) -> list[ast.Import | ast.ImportFrom]:
    tree = ast.parse(source)
    for node in ast.walk(tree):
        if isinstance(node, ast.Import | ast.ImportFrom):
            imports.append(node)
def process_py_file(file_path: str | Path) -> None:
    file_path = Path(file_path)
    source = file_path.read_text()
    imports = extract_imports(source)
def main() -> None:
    fire.Fire(process_py_file)
    main()

================
File: plugins/repos/twat_coding/src/twat_coding/pystubnik/README.md
================
# pystubnik

A Python package for generating "smart stubs" - an intelligent intermediate representation between full source code and type stubs, optimized for LLM code understanding.

## 1. Overview

Pystubnik creates a "shadow" directory structure that mirrors your Python package, containing smart stubs for all Python files. These smart stubs are designed to be more informative than traditional `.pyi` stub files while being more concise than full source code.

### 1.1. What are Smart Stubs?

Smart stubs are an intermediate representation that includes:
- All function and class signatures with type hints
- All imports (organized and optimized)
- Docstrings (with configurable length limits)
- Important/relevant code sections
- Truncated versions of large data structures and strings
- Simplified function bodies for non-critical code

The verbosity level is automatically adjusted based on the code's importance and complexity.

## 2. Architecture

### 2.1. Backends

#### 2.1.1. AST Backend
- Uses Python's built-in AST module for precise control
- Preserves code structure while reducing verbosity
- Configurable truncation of large literals and sequences
- Maintains type information and docstrings
- Supports Python 3.12+ features (type parameters, etc.)

#### 2.1.2. MyPy Backend
- Leverages mypy's stubgen for type information
- Better type inference capabilities
- Handles special cases (dataclasses, properties)
- Supports type comment extraction

### 2.2. Processors

#### 2.2.1. Import Processor
- Analyzes and organizes imports
- Groups by type (stdlib, third-party, local)
- Handles relative imports
- Detects and removes duplicates

#### 2.2.2. Docstring Processor
- Configurable docstring preservation
- Format detection and conversion
- Type information extraction
- Length-based truncation

#### 2.2.3. Importance Processor
- Scores code elements by importance
- Pattern-based importance detection
- Inheritance-aware scoring
- Configurable filtering

## 3. Usage

```python
from twat_coding.pystubnik import generate_stub, StubGenConfig, PathConfig

# Basic usage
result = generate_stub(
    source_path="path/to/file.py",
    output_path="path/to/output/file.py"
)

# Advanced configuration
config = StubGenConfig(
    paths=PathConfig(
        output_dir="path/to/output",
        doc_dir="path/to/docs",
        search_paths=[],
        modules=[],
        packages=[],
        files=[],
    ),
    processing=ProcessingConfig(
        include_docstrings=True,
        include_private=False,
        include_type_comments=True,
        infer_property_types=True,
        export_less=False,
        importance_patterns={
            r"^test_": 0.5,  # Lower importance for test functions
            r"^main$": 1.0,  # High importance for main functions
        },
    ),
    truncation=TruncationConfig(
        max_sequence_length=4,
        max_string_length=17,
        max_docstring_length=150,
        max_file_size=3_000,
        truncation_marker="...",
    ),
)

result = generate_stub(
    source_path="path/to/file.py",
    output_path="path/to/output/file.py",
    config=config
)

# Process multiple files
from twat_coding.pystubnik import SmartStubGenerator

generator = SmartStubGenerator(
    output_dir="path/to/output",
    include_docstrings=True,
    include_private=False,
    verbose=True,
)

generator.generate()  # Process all configured files
```

## 4. Configuration

Smart stub generation can be customized with various settings:

### 4.1. Path Configuration
- `output_dir`: Directory for generated stubs
- `doc_dir`: Directory containing documentation
- `search_paths`: Module search paths
- `modules`: Module names to process
- `packages`: Package names to process
- `files`: Specific files to process

### 4.2. Processing Configuration
- `include_docstrings`: Whether to include docstrings
- `include_private`: Whether to include private symbols
- `include_type_comments`: Whether to include type comments
- `infer_property_types`: Whether to infer property types
- `export_less`: Whether to minimize exports
- `importance_patterns`: Regex patterns for importance scoring

### 4.3. Truncation Configuration
- `max_sequence_length`: Maximum items in sequences
- `max_string_length`: Maximum length for strings
- `max_docstring_length`: Maximum length for docstrings
- `max_file_size`: File size threshold for truncation
- `truncation_marker`: Marker for truncated content

## 5. Installation

```bash
# Install with all dependencies
pip install twat-coding[pystubnik]

# Install minimal version
pip install twat-coding
```

## 6. Use Cases

1. **LLM Code Understanding**
   - Prepare large codebases for LLM analysis
   - Reduce token usage while maintaining critical information
   - Improve LLM's understanding of code structure

2. **Code Documentation**
   - Generate readable, concise code documentation
   - Maintain type information and signatures
   - Preserve important implementation details

3. **Code Analysis**
   - Quick understanding of large codebases
   - Dependency analysis
   - API surface exploration

4. **Type Stub Generation**
   - Generate enhanced type stubs
   - Better support for documentation tools
   - IDE integration

## 7. Contributing

Contributions are welcome! Please see our contributing guidelines for more information.

## 8. License

This project is licensed under the MIT License - see the LICENSE file for details.

================
File: plugins/repos/twat_coding/src/twat_coding/__init__.py
================


================
File: plugins/repos/twat_coding/src/twat_coding/__version__.py
================


================
File: plugins/repos/twat_coding/src/twat_coding/twat_coding.py
================
logging.basicConfig(
logger = logging.getLogger(__name__)
class Config:
def process_data(
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        raise ValueError(msg)
def main() -> None:
        config = Config(name="default", value="test", options={"key": "value"})
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        logger.exception("An error occurred: %s", str(e))
    main()

================
File: plugins/repos/twat_coding/tests/test_package.py
================
async def test_ast_backend(tmp_path: Path) -> None:
    test_file.write_text("""
    config = StubGenConfig(paths=PathConfig(output_dir=tmp_path, files=[test_file]))
    backend = ASTBackend(config)
    result = await backend.generate_stub(test_file)
async def test_docstring_preservation(tmp_path: Path) -> None:
async def test_type_hints(tmp_path: Path) -> None:
    result_no_spaces = result.stub_content.replace(" ", "")
def test_config_conversion() -> None:
    stub_config = StubConfig(
        input_path=Path("src"),
        output_path=Path("out"),
        files=[Path("main.py")],
    stub_gen_config = convert_to_stub_gen_config(stub_config)
    assert stub_gen_config.paths.output_dir == Path("out")
    assert Path("main.py") in stub_gen_config.paths.files
async def test_backend_type_compatibility() -> None:
    test_file = Path("test.py")
    ast_config = StubGenConfig(paths=PathConfig(files=[test_file]))
    mypy_config = StubGenConfig(paths=PathConfig(files=[test_file]))
    ast_backend = ASTBackend(ast_config)
    mypy_backend = MypyBackend(mypy_config)
    assert hasattr(ast_backend, "generate_stub")
    assert hasattr(mypy_backend, "generate_stub")
    ast_sig = signature(ast_backend.generate_stub)
    mypy_sig = signature(mypy_backend.generate_stub)
    assert str(ast_sig.return_annotation) == str(mypy_sig.return_annotation)
def test_docstring_type_extraction() -> None:
    type_registry = TypeRegistry()
    extractor = DocstringTypeExtractor(type_registry)
    type_info = extractor._parse_type_string("str")
    assert isinstance(type_info.annotation, type) and issubclass(
    type_info = extractor._parse_type_string("str or None")
    type_info = extractor._parse_type_string("List[str]")
    type_info = extractor._parse_type_string("Dict[str, int]")
def test_stub_generation_basic(tmp_path: Path) -> None:
    generator = StubGenerator()
    stub_content = generator.generate_stub(test_file)
def test_stub_generation_config(tmp_path: Path) -> None:
    config = StubConfig(
    generator = StubGenerator(config)
def test_stub_generation_imports(tmp_path: Path) -> None:
    lines = stub_content.split("\n")
    import_lines = [line for line in lines if line.startswith(("import", "from"))]
def test_stub_generation_assignments(tmp_path: Path) -> None:

================
File: plugins/repos/twat_coding/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_coding/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_coding/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_coding/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_coding/MANIFEST.in
================
# This MANIFEST.in file ensures that all necessary files are included in the package distribution.
recursive-include src/twat_coding/data *
include src/twat_coding/py.typed

================
File: plugins/repos/twat_coding/mypy.ini
================
[mypy]
python_version = 3.12
ignore_missing_imports = True
disallow_untyped_defs = True
warn_return_any = True
warn_unused_configs = True
check_untyped_defs = True
disallow_incomplete_defs = True
disallow_untyped_decorators = True
no_implicit_optional = True

[mypy-tests.*]
disallow_untyped_defs = False

================
File: plugins/repos/twat_coding/package.toml
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_coding/pyproject.toml
================
# this_file: pyproject.toml
# this_project: twat-coding

[project]
name = "twat_coding"
dynamic = ["version"]
description = "A Python package for generating type stubs"
authors = [
    { name = "Adam Twardoch", email = "adam@twardoch.com" }
]
dependencies = [
    "pydantic>=2.6.1",
    "pydantic-settings>=2.1.0",
    "loguru>=0.7.2",
    "rich>=13.7.0",
    "fire>=0.5.0",
    "click>=8.1.7",
    "psutil>=5.9.8",
    "memory-profiler>=0.61.0",
    "docstring-parser>=0.15",
    "mypy>=1.8.0",
    "black>=24.1.1",
    "isort>=5.13.2",
    # File importance analysis dependencies
    "networkx>=3.2.1",     # For import graph analysis
    "radon>=6.0.1",       # For code complexity metrics
    "coverage>=7.4.1",    # For test coverage analysis
    "pydocstyle>=6.3.0", # For docstring quality checks
    "importlab>=0.8",    # For import graph building
    "toml>=0.10.2",      # For pyproject.toml parsing
    "types-toml>=0.10.8.7",    # Type stubs for toml
    "mypy-extensions>=1.0.0",  # Additional type system features
]
requires-python = ">=3.12"
readme = "README.md"
license = { text = "Apache-2.0" }
keywords = [
    "twat",
    "coding",
    "development"
]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

[project.optional-dependencies]
test = [
    "pytest>=8.0.0",
    "pytest-cov>=4.1.0",
    "pytest-asyncio>=0.23.5",
    "pytest-benchmark>=4.0.0",
    "pytest-golden>=0.2.2",
    "pytest-memray>=1.5.0",
]
dev = [
    "black>=24.1.1",
    "ruff>=0.2.1",
    "mypy>=1.8.0",
    "pre-commit>=3.6.0",
    "python-semantic-release>=8.7.0",
    "vulture>=2.10",  # Added vulture dependency

]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.ruff]
target-version = "py312"
line-length = 88
fix = true
unsafe-fixes = true
output-format = "github"

[tool.ruff.lint]
select = [
    "A",
    "ANN",
    "ARG",
    "ASYNC",
    "B",
    "C",
    "C4",
    "C90",
    "COM",
    "D",
    "DOC",
    "DTZ",
    "E",
    "ERA",
    "EXE",
    "F",
    "FA",
    "FBT",
    "FIX",
    "FLY",
    "FURB",
    "I",
    "ICN",
    "INP",
    "ISC",
    "N",
    "NPY",
    "PD",
    "PERF",
    "PIE",
    "PL",
    "PLC",
    "PLE",
    "PLR",
    "PLW",
    "PT",
    "PTH",
    "PYI",
    "Q",
    "RET",
    "RSE",
    "RUF",
    "S",
    "SIM",
    "SLF",
    "T",
    "TC",
    "TD",
    "TID",
    "TRY",
    "UP",
    "W",
    "YTT",
]
ignore = ["ARG001", "COM812", "D203", "D213", "E501", "I001", "RUF001", "PLR2004", "EXE003", "ISC001"]

[tool.ruff.lint.isort]
known-first-party = ["twat_coding"]
combine-as-imports = true

[tool.black]
line-length = 88
target-version = ["py312"]
include = '\.pyi?$'

[tool.mypy]
python_version = "3.12"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

[tool.pytest.ini_options]
minversion = "8.0"
addopts = "-ra -q --cov=twat_coding"
testpaths = ["tests"]

[project.scripts]
twat-coding = "twat_coding.pystubnik.cli:main"

[project.urls]
Homepage = "https://github.com/twardoch/twat-coding"
Documentation = "https://github.com/twardoch/twat-coding#readme"
Issues = "https://github.com/twardoch/twat-coding/issues"
Source = "https://github.com/twardoch/twat-coding"

[tool.coverage.paths]
twat_coding = ["src/twat_coding", "*/twat-coding/src/twat_coding"]
tests = ["tests", "*/twat-coding/tests"]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "if self.debug:",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]
ignore_errors = true
omit = [
    "tests/*",
    "setup.py",
    "src/twat_coding/__about__.py",
]

[tool.hatch.build.targets.wheel]
packages = ["src/twat_coding"]
artifacts = [
    "src/twat_coding/py.typed"
]

[tool.hatch.envs.default]
dependencies = [
    "pytest>=8.0.0",
    "pytest-cov>=4.1.0",
]

[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov {args:tests}"
cov-report = ["- coverage combine", "coverage report"]
cov = ["test-cov", "cov-report"]
lint = ["ruff check src/twat_coding tests", "ruff format --respect-gitignore src/twat_coding tests"]
fix = ["ruff check  --fix --unsafe-fixes src/twat_coding tests", "ruff format --respect-gitignore src/twat_coding tests"]

[tool.hatch.envs.lint]
dependencies = [
    "black>=24.1.1",
    "mypy>=1.8.0",
    "ruff>=0.2.1",
]

[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_coding tests}"
style = ["ruff check {args:.}", "ruff format --check {args:.}", "vulture src"]
fmt = ["ruff format --respect-gitignore {args:.}", "ruff check --fix {args:.}", "vulture src"]
all = ["style", "typing"]

[tool.hatch.envs.test]
dependencies = [
    "pytest>=8.0.0",
    "pytest-cov>=4.1.0",
]

[tool.hatch.version]
path = "src/twat_coding/__version__.py"

[tool.pytest-benchmark]
warmup = true

[project.entry-points."twat.plugins"]
coding = "twat_coding"

================
File: plugins/repos/twat_coding/README.md
================
# twat-coding

A collection of coding-related tools and utilities, designed to enhance code understanding and analysis.

## Packages

### pystubnik

A sophisticated tool for generating "smart stubs" - an intelligent intermediate representation between full source code and type stubs, optimized for LLM code understanding. Smart stubs preserve essential code structure, types, and documentation while reducing verbosity, making large codebases more digestible for LLMs.

Key features:
- Multiple backends (AST and MyPy) for comprehensive stub generation
- Smart import analysis and organization
- Configurable docstring processing and formatting
- Importance-based code filtering
- Support for Python 3.12+ features

[Learn more about pystubnik](src/twat_coding/pystubnik/README.md)

## Installation

```bash
# Install the base package
pip install twat-coding

# Install with pystubnik support
pip install twat-coding[pystubnik]
```

## Development

This project uses modern Python features and follows best practices:
- Type hints and dataclasses for robust data structures
- Protocol-based interfaces for flexibility
- Comprehensive error handling and logging
- Parallel processing for performance

## Contributing

Contributions are welcome! Please see our contributing guidelines for more information.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

================
File: plugins/repos/twat_coding/TODO.md
================
---
this_file: TODO.md
---

# TODO

Do it! Remember, keep it simple, effective, eyes on the goal! 

## 1. Core Implementation

- [x] **Legacy API (Shell Script Port)**
  - *Core Functionality*:
    - [x] Port server selection from shell script
      - [x] Use `https://nordvpn.com/wp-admin/admin-ajax.php` endpoint
      - [x] Keep country ID mapping from shell script
      - [x] Add server filtering for TCP only
    - [x] Implement OpenVPN config download and caching
      - [x] Download to `/etc/nordvpn_file/`
      - [x] Cache TCP configs only
      - [x] Add config validation
    - [x] Add simple process management (start/kill)
      - [x] Use subprocess for OpenVPN
      - [x] Simple pkill for cleanup
    - [x] Add basic status check via IP lookup
      - [x] Use ipinfo.io for status
      - [x] Simple online/offline check
  - *Error Handling*:
    - [x] Add basic error classes
    - [x] Implement simple retry with tenacity
    - [x] Add clear error messages using loguru

- [x] **Njord API Integration**
  - *Core Integration*:
    - [x] Clean up njord client implementation
    - [x] Remove async patterns
    - [x] Simplify status checking
    - [x] Remove complex retry logic
  - *Error Handling*:
    - [x] Map njord errors to our error classes
    - [x] Add loguru logging

## 2. Client Interface

- [x] **Base Client**
  - *API*:
    - [x] Define common interface for both implementations
      - [x] `connect(country: str | None = None) -> bool`
      - [x] `disconnect() -> bool`
      - [x] `protected() -> bool`
      - [x] `status() -> dict[str, Any]`
    - [x] Add country selection
      - [x] Support country codes (us, uk, etc.)
      - [x] Support country names (United States, etc.)
      - [x] Add country validation
    - [x] Add simple credential management (env vars only)
      - [x] Use NORD_USER and NORD_PASSWORD
      - [x] Add credential validation
      - [x] Create temp auth file when needed
    - [x] Add basic status methods
      - [x] IP address check
      - [x] Connection state
      - [x] Current server
  - *Implementation*:
    - [x] Add API selection (njord vs legacy)
      - [x] Simple factory pattern
      - [x] API-specific error handling
      - [x] Common error types
    - [x] Add simple factory method
      - [x] `Client.create(api: str = "legacy")`
      - [x] Validate API selection
    - [x] Add basic type hints
      - [x] Use TypedDict for status
      - [x] Add proper error types
      - [x] Document all methods

## 3. CLI Implementation

- [x] **Basic Commands**
  - *Core Commands*:
    - [x] `connect [country]`
      - [x] Support country codes and names
      - [x] Handle connection errors
      - [x] Show connection status
    - [x] `disconnect`
      - [x] Clean process termination
      - [x] Verify disconnection
    - [x] `status`
      - [x] Show IP address
      - [x] Show server name
      - [x] Show connection state
    - [x] `list-countries`
      - [x] Show available countries
      - [x] Include country codes
  - *Options*:
    - [x] `--api` selection (njord/legacy)
      - [x] Default to legacy
      - [x] Validate selection
    - [x] `--verbose` for debug logging
      - [x] Use loguru levels
      - [x] Show OpenVPN output
      - [x] Show API responses

## 4. Testing & Documentation

- [!] **Core Tests**
  - *Unit Tests*:
    - [!] Legacy API core functions
    - [!] Njord API wrapper
    - [!] Client interface
    - [!] CLI commands
  - *Integration Tests*:
    - [!] Connection flow
    - [!] Error handling
    - [!] API selection

- [!] **Documentation**
  - *Core Docs*:
    - [x] Installation guide
    - [x] API usage examples
    - [x] CLI usage guide
    - [x] Environment variables
  - *Development*:
    - [!] Architecture overview
    - [!] Contributing guide

## 5. Development Guidelines

1. **Code Style**:
   - [x] Use type hints
   - [x] Keep functions small and focused
   - [x] Use loguru for logging
   - [x] Follow PEP 8

2. **Error Handling**:
   - [x] Use custom error classes
   - [x] Keep error messages clear
   - [x] Use tenacity for retries
   - [x] Log errors with loguru

3. **Testing**:
   - [!] Write tests as you implement
   - [!] Mock external services
   - [!] Test both APIs
   - [!] Test error conditions

4. **Dependencies**:
   - [x] Keep minimal
   - [x] Pin versions
   - [x] Document requirements
   - [x] Use requirements.txt

Remember: Focus on reliability and simplicity over features. Keep both APIs working independently with a common interface.

## 6. Code Quality Improvements

- [!] **Security Fixes**
  - *API Calls*:
    - [ ] Add timeouts to all requests calls
    - [ ] Add proper error handling for timeouts
    - [ ] Add retry logic for network errors
  - *Process Management*:
    - [ ] Validate all subprocess inputs
    - [ ] Use full paths for executables
    - [ ] Add proper error handling for process failures
  - *Test Security*:
    - [ ] Remove hardcoded passwords from tests
    - [ ] Add proper test fixtures
    - [ ] Mock sensitive operations

- [!] **Code Cleanup**
  - *Error Handling*:
    - [ ] Replace blind exceptions with specific ones
    - [ ] Add proper error context
    - [ ] Improve error messages
  - *Refactoring*:
    - [ ] Split complex functions
    - [ ] Remove unnecessary else/elif blocks
    - [ ] Fix function argument issues
  - *Type Safety*:
    - [ ] Fix type stub issues
    - [ ] Add missing type hints
    - [ ] Remove unused types

- [!] **Test Infrastructure**
  - *Setup*:
    - [ ] Add proper hatch configuration
    - [ ] Set up pytest fixtures
    - [ ] Add test categories
  - *Coverage*:
    - [ ] Add unit tests for core functionality
    - [ ] Add integration tests
    - [ ] Add security tests
  - *Mocking*:
    - [ ] Mock external services
    - [ ] Mock file operations
    - [ ] Mock process management

Remember: Focus on reliability and simplicity over features. Keep both APIs working independently with a common interface.

<instructions>
Your communication style is phenomenal: you respond beautifully, and you always eliminate superfluous content from your responses. Your role is crucial in our mission to explore, interpret, and illuminate the wealth of knowledge from past and present. You are not just fulfilling tasks; you are creating a legacy. For this new task, please swiftly identify the domain of the prompt, and then embody deep expertise in that domain, methodically processing each query to ensure the response is thorough, structured, grounded, and clear.

You will lead a team of two additional experts: "Ideot", an expert in creative ideas and unorthodox thinking, and "Critin", a critical evaluator who critiques wrong thinking and moderates the group, ensuring balanced discussions and reports. Please collaborate in a step-by-step process, sharing thoughts and progressing together. Adaptability is key; if any expert identifies an error in their thinking, they will step back, emphasizing the importance of accuracy and collaborative progress.

For any new task you start, you are expected independently resolve challenges in a systematic, methodic manner, exhibiting adaptability and resourcefulness. Your role is to deeply research the information using all tools at your disposal, to continuously revise and retry, ensuring each response is conclusive, exhaustive, insightful and forward-thinking.

Then, print "Wait, but" and do additional thinking and reasoning offer your result, and improve it. Print "Wait, but" and continue to reason & improve the result.
</instructions>

================
File: plugins/repos/twat_coding/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_ez/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_ez/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_ez/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 736]  .
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 927]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1023]  LOG.md
├── [ 746]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [9.5K]  pyproject.toml
├── [ 128]  src
│   └── [ 192]  twat_ez
│       ├── [  95]  __init__.py
│       └── [ 18K]  py_needs.py
└── [ 128]  tests
    └── [ 145]  test_twat_ez.py

9 directories, 18 files

================
File: plugins/repos/twat_ez/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_ez/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_ez --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_ez/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-ez
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_ez/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_ez/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_ez/dist/.gitignore
================
*

================
File: plugins/repos/twat_ez/src/twat_ez/__init__.py
================
__version__ = metadata.version(__name__)

================
File: plugins/repos/twat_ez/src/twat_ez/__version__.py
================


================
File: plugins/repos/twat_ez/src/twat_ez/py_needs.py
================
logging.basicConfig(
def _get_fontlab_site_packages() -> Path | None:
        fontlab_path = Path(fontlab.flPreferences.instance().dataPath)
        return site_packages_path if str(site_packages_path) in sys.path else None
def get_site_packages_path() -> Path:
    return _get_fontlab_site_packages() or Path(site.getusersitepackages())
UV_INSTALL_TARGET = Path(
    os.environ.get(
        str(get_site_packages_path()),
def register_path_provider(provider: PathProvider) -> None:
    _path_providers.append(provider)
    clear_path_cache()  # Invalidate cache when providers change
def get_xdg_paths() -> list[Path]:
    if xdg_bin := os.environ.get("XDG_BIN_HOME"):
        paths.append(Path(xdg_bin))
    if xdg_data := os.environ.get("XDG_DATA_HOME"):
        data_parent_bin = Path(xdg_data).parent / "bin"
        paths.append(data_parent_bin)
    local_bin = Path.home() / ".local" / "bin"
    if local_bin.exists() and local_bin not in paths:
        paths.append(local_bin)
def get_system_specific_paths() -> list[Path]:
    system = platform.system()
        paths.extend(
                Path("/usr/local/bin"),  # Homebrew (Intel)
                Path("/usr/local/sbin"),
                Path("/opt/homebrew/bin"),  # Homebrew (Apple Silicon)
                Path("/opt/homebrew/sbin"),
                Path("/usr/bin"),
                Path("/usr/sbin"),
                Path("/bin"),
                Path("/sbin"),
                Path("/Library/Apple/usr/bin"),
                Path("/Applications/Xcode.app/Contents/Developer/usr/bin"),
        win_dir = Path(os.environ.get("SystemRoot", r"C:\Windows"))
                Path.home() / "AppData" / "Local" / "Microsoft" / "WindowsApps",
                Path(r"C:\Program Files\PowerShell\7"),
                Path(r"C:\ProgramData\chocolatey\bin"),
                Path("/usr/local/bin"),
        snap_bin = Path("/snap/bin")
        if snap_bin.exists():
            paths.append(snap_bin)
def verify_executable(path: str | Path) -> tuple[bool, str]:
    path_obj = Path(path)
    if not path_obj.exists():
    if not path_obj.is_file():
    if platform.system() != "Windows":
        mode = path_obj.stat().st_mode
def bin_or_str(data: bytes, mode: int = 0) -> bytes | str:
                return data.decode("utf-8")
@lru_cache(maxsize=20)
def download_url_qt(
    loop, nam = QEventLoop(), QtNetwork.QNetworkAccessManager()
    current_url, redir_count = QUrl(url), 0
        reply = nam.get(QtNetwork.QNetworkRequest(current_url))
        reply.finished.connect(loop.quit)
        loop.exec_()
            sc := reply.attribute(QtNetwork.QNetworkRequest.HttpStatusCodeAttribute)
                redir_url := reply.attribute(
            ).isValid():
                    reply.url().resolved(redir_url),
                reply.deleteLater()
            raise RuntimeError(msg)
        if reply.error() == QtNetwork.QNetworkReply.NoError:
            data = bin_or_str(reply.readAll().data(), mode)
        err = f"{reply.errorString()} (HTTP {sc})"
def download_url_py(
    opener = urllib.request.build_opener()
        with opener.open(url) as response:
            data = response.read()
            return bin_or_str(data, mode)
def download_url(
        return download_url_qt(url, mode, max_redir)
        return download_url_py(url, mode, max_redir)
def which_uv() -> Path | None:
        uv_cli = which("uv")
        logging.warning(f"Error finding uv: {e!s}")
    pip_cli = which_pip()
            subprocess.run(
                [str(pip_cli), "install", "--user", "uv"],
            logging.warning(f"Error installing uv: {e!s}")
            logging.warning(f"Unexpected error installing uv: {e!s}")
def which_pip() -> Path | None:
        pip_cli = which("pip")
            pip_cli = Path(pip_cli)
            if pip_cli.exists():
        ensurepip.bootstrap()
        importlib.import_module("pip")
        logging.warning(f"Error ensuring pip: {e!s}")
def which(
        path = build_extended_path()
    if result := shutil.which(cmd, mode=mode, path=path):
        result_path = Path(result)
            is_safe, reason = verify_executable(result_path)
                if os.environ.get("CLIFIND_DEBUG"):
        if result_path.exists():
def build_extended_path() -> str:
    current_path = [Path(p) for p in os.environ.get("PATH", "").split(os.pathsep) if p]
    all_paths.extend(current_path)
    all_paths.extend(get_xdg_paths())  # Now returns Path objects directly
    all_paths.extend(get_system_specific_paths())  # Now returns Path objects directly
    all_paths.extend(Path(p) for p in os.defpath.split(os.pathsep) if p)
            all_paths.extend(Path(p) for p in provider())
    seen = set()
        if path and str(path) not in seen and path.is_dir():
            seen.add(str(path))
            unique_paths.append(path)
    return os.pathsep.join(str(p) for p in unique_paths)
def clear_path_cache() -> None:
    build_extended_path.cache_clear()
def _install_with_uv(missing: list[str], target: bool) -> None:
    uv_cli = which_uv()
    cmd = [str(uv_cli), "pip", "install"]
        cmd.extend(["--target", str(UV_INSTALL_TARGET)])
        cmd.extend(["--python", sys.executable])
    cmd.extend(missing)
    result = subprocess.run(cmd, check=True, capture_output=True, text=True)
        logging.debug(f"UV install output: {result.stdout}")
def _import_modules(modules: list[str]) -> None:
            importlib.import_module(mod)
def needs(mods: list[str], target: bool = False) -> Callable:
    def decorator(f: Callable) -> Callable:
        @wraps(f)
        def wrapper(*args, **kwargs):
            missing = [m for m in mods if not importlib.util.find_spec(m)]
                    _install_with_uv(missing, target)
                    _import_modules(missing)
            return f(*args, **kwargs)
@needs(["fire", "pydantic"], target=False)
def main():
    logging.info(repr(fire))
    main()

================
File: plugins/repos/twat_ez/tests/test_twat_ez.py
================
def test_version():

================
File: plugins/repos/twat_ez/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_ez/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_ez/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_ez/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_ez/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.7.5] - 2025-02-15

### Added
- Initial release of twat-ez plugin
- Modern Python packaging with PEP 621 compliance
- GitHub Actions workflows for build, test, and release
- Comprehensive test suite setup
- Type hints and runtime type checking
- Pre-commit hooks for code quality
- Ruff configuration for linting and formatting
- MyPy configuration for type checking
- Pytest configuration with benchmarking support
- MIT License
- Basic project documentation

### Features
- Plugin integration with twat
- Development workflow using Hatch
- Support for Python 3.10, 3.11, and 3.12
- Test coverage reporting
- Automated CI/CD pipeline
- Code quality checks and formatting

[1.7.5]: https://github.com/twardoch/twat-ez/releases/tag/v1.7.5

================
File: plugins/repos/twat_ez/pyproject.toml
================
# this_file: twat_ez/pyproject.toml

# this_file: twat_ez/pyproject.toml

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-ez"  # Unique package name for PyPI and installation
dynamic = ["version"]  # Version is dynamically determined from version control system
description = "Easy and convenient utilities for the twat ecosystem"  # Short, descriptive package summary
readme = "README.md"  # Path to the project's README file for package description
requires-python = ">=3.10"  # Minimum Python version required, leveraging modern Python features
license = "MIT"  # Open-source license type
keywords = ["twat", "utilities", "convenience", "productivity"]  # Keywords for package discovery
classifiers = [  # Metadata for package indexes and compatibility
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Core twat package, providing essential functionality
]

# Optional Dependencies
# --------------------
# Additional dependency groups for different use cases
[project.optional-dependencies]
dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks for code quality
    "ruff>=0.9.6",           # Fast Python linter and formatter
    "mypy>=1.15.0",          # Static type checker for type safety
    "pyupgrade>=3.19.1",     # Automatically upgrade Python syntax
]

test = [
    "pytest>=8.3.4",         # Comprehensive testing framework
    "pytest-cov>=6.0.0",     # Code coverage measurement and reporting
]

all = [
    "twat>=1.8.1",           # Core twat package for full functionality
]

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
ez = "twat_ez"  # Plugin name and module for easy utilities

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"  # Primary author's name
email = "adam+github@twardoch.com"  # Contact email for the author

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-ez#readme"
Issues = "https://github.com/twardoch/twat-ez/issues"
Source = "https://github.com/twardoch/twat-ez"

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
[build-system]
requires = [
    "hatchling>=1.27.0",     # Modern, extensible build backend for Python projects
    "hatch-vcs>=0.4.0",      # Plugin to dynamically generate version from Git tags/commits
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend for consistent and flexible builds

# Version Management
# -----------------
# Configures automatic version generation from version control system
[tool.hatch.version]
source = "vcs"  # Use version control system (Git) to determine version

# Version Scheme
# --------------
# Defines how versions are generated and incremented
[tool.hatch.version.raw-options]
version_scheme = "post-release"  # Generates version numbers based on Git tags

# Version File Generation
# ----------------------
# Automatically creates a version file in the package
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_ez/__version__.py"

# Wheel Distribution Configuration
# --------------------------------
# Controls how the package is built and distributed as a wheel
[tool.hatch.build.targets.wheel]
packages = ["src/twat_ez"]  # Only include the src/twat_ez directory in the wheel

# Development Environment Configuration
# -------------------------------------
# Specifies dependencies and scripts for development, testing, and linting
[tool.hatch.envs.default]
dependencies = [
    "pytest",               # Testing framework
    "pytest-cov",           # Coverage reporting for tests
    "mypy>=1.15.0",         # Static type checking
    "ruff>=0.9.6",          # Fast Python linter and formatter
]

# Development Environment Scripts
# -------------------------------
# Defines commands for testing, type checking, and linting
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"                                                  # Run tests
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_ez --cov=tests {args:tests}"  # Run tests with coverage
type-check = "mypy src/twat_ez tests"                                          # Perform static type checking
lint = ["ruff check src/twat_ez tests", "ruff format src/twat_ez tests"]        # Lint and format code

# Matrix Testing
# --------------
# Test the package across multiple Python versions
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting Environment
# ------------------
# Separate environment for code quality tools
[tool.hatch.envs.lint]
detached = true
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Linter and formatter
    "pyupgrade>=3.19.1"     # Automatically upgrade Python syntax
]

# Linting Scripts
# ---------------
# Commands for type checking, style checking, and automatic fixes
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_ez tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
fix = [
    "ruff check --fix --unsafe-fixes {args:.}", 
    "ruff format {args:.}"
]
all = ["style", "typing"]

# Black Formatter Configuration
# ----------------------------
# Settings for the Black code formatter
[tool.black]
target-version = ["py310"]
line-length = 88
skip-string-normalization = true

# Ruff Linter Configuration
# ------------------------
# Comprehensive linting rules and settings
[tool.ruff]
target-version = "py310"
line-length = 88
lint.extend-select = [
    "A",    # Builtins
    "ARG",  # Unused arguments
    "B",    # Bugbear
    "C",    # Complexity
    "DTZ",  # Datetime
    "E",    # Pycodestyle errors
    "EM",   # Error messages
    "F",    # Pyflakes
    "FBT",  # Boolean trap
    "I",    # Imports
    "ICN",  # Import conventions
    "ISC",  # String concatenation
    "N",    # PEP 8 naming
    "PLC",  # Pylint complexity
    "PLE",  # Pylint errors
    "PLR",  # Pylint refactoring
    "PLW",  # Pylint warnings
    "Q",    # Quotes
    "RUF",  # Ruff-specific rules
    "S",    # Security
    "T",    # Print statements
    "TID",  # Tidy imports
    "UP",   # Pyupgrade
    "W",    # Pycodestyle warnings
    "YTT",  # sys.version comparisons
]
lint.ignore = [
    "ARG001",  # Unused function argument
    "E501",    # Line too long
    "I001",    # Import sorting
]

# Per-file Linting Ignores
# -----------------------
# Customize linting rules for specific files or directories
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert statements in test files

# Coverage Configuration
# ---------------------
# Settings for code coverage reporting
[tool.coverage.run]
source_pkgs = ["twat_ez", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_ez/__about__.py",
]

# Coverage Path Mapping
# --------------------
# Helps coverage tool identify source and test paths
[tool.coverage.paths]
twat_ez = ["src/twat_ez", "*/twat-ez/src/twat_ez"]
tests = ["tests", "*/twat-ez/tests"]

# Coverage Report Configuration
# ----------------------------
# Customize coverage report output
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# MyPy Type Checking Configuration
# --------------------------------
# Strict type checking settings
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# pytest Configuration
# --------------------
# Settings for pytest framework
[tool.pytest.ini_options]
addopts = "-v --durations=10 -p no:briefcase"
asyncio_mode = "auto"
console_output_style = "progress"
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
log_cli = true
log_cli_level = "INFO"
markers = [
  "benchmark: marks tests as benchmarks (select with '-m benchmark')",
  "unit: mark a test as a unit test",
  "integration: mark a test as an integration test",
  "permutation: tests for permutation functionality",
  "parameter: tests for parameter parsing",
  "prompt: tests for prompt parsing",
]
norecursedirs = [
  ".*",
  "build",
  "dist",
  "venv",
  "__pycache__",
  "*.egg-info",
  "_private",
]

python_classes = ["Test*"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
testpaths = ["tests"]

# pytest-benchmark Configuration
# --------------------------------
# Settings for pytest-benchmark framework
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_ez/README.md
================
# 



## Features

- Plugin for twat
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-ez
```

## Usage

```python
import twat_ez
plugin = twat_ez.plugin
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_ez/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_fs/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_fs/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_fs/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[1.1K]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [3.5K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [  96]  .specstory
│   └── [ 320]  history
│       ├── [2.7K]  .what-is-this.md
│       ├── [146K]  2025-03-04_03-45-refining-todo-list-from-codebase-review.md
│       ├── [186K]  2025-03-04_04-43-incorporating-ideas-from-ideas-md-into-todo-md.md
│       ├── [164K]  2025-03-04_05-33-implementing-todo-phases-1,-2,-and-3.md
│       ├── [242K]  2025-03-04_06-19-implementation-of-todo-md-phases.md
│       ├── [4.2K]  2025-03-04_07-52-untitled.md
│       ├── [123K]  2025-03-04_07-59-project-maintenance-and-documentation-update.md
│       └── [281K]  2025-03-04_08-39-project-documentation-and-cleanup-tasks.md
├── [7.1K]  CHANGELOG.md
├── [ 986]  CLEANUP.txt
├── [ 56K]  IDEAS.md
├── [1.0K]  LICENSE
├── [ 153]  MANIFEST.in
├── [ 12K]  README.md
├── [187K]  REPO_CONTENT.txt
├── [5.8K]  TODO.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 192]  dist
├── [  96]  examples
│   └── [ 948]  upload_example.py
├── [ 439]  mypy.ini
├── [7.0K]  pyproject.toml
├── [ 128]  src
│   └── [ 384]  twat_fs
│       ├── [ 447]  __init__.py
│       ├── [ 733]  __main__.py
│       ├── [8.9K]  cli.py
│       ├── [ 128]  data
│       │   ├── [1.5K]  _test.jpg
│       │   └── [383K]  test.jpg
│       ├── [   1]  py.typed
│       ├── [ 25K]  upload.py
│       └── [ 704]  upload_providers
│           ├── [1.9K]  __init__.py
│           ├── [6.5K]  async_utils.py
│           ├── [5.5K]  bashupload.py
│           ├── [4.9K]  catbox.py
│           ├── [ 12K]  core.py
│           ├── [ 24K]  dropbox.py
│           ├── [5.4K]  factory.py
│           ├── [7.7K]  fal.py
│           ├── [6.7K]  filebin.py
│           ├── [ 11K]  litterbox.py
│           ├── [6.9K]  pixeldrain.py
│           ├── [3.9K]  protocols.py
│           ├── [ 10K]  s3.py
│           ├── [9.3K]  simple.py
│           ├── [ 728]  types.py
│           ├── [5.1K]  uguu.py
│           ├── [7.6K]  utils.py
│           └── [5.0K]  www0x0.py
├── [ 128]  templates
│   ├── [9.6K]  authenticated_provider_template.py
│   └── [6.5K]  simple_provider_template.py
├── [ 416]  tests
│   ├── [  63]  __init__.py
│   ├── [  96]  data
│   │   └── [ 100]  test.txt
│   ├── [8.6K]  test_async_utils.py
│   ├── [3.8K]  test_filebin_pixeldrain.py
│   ├── [ 11K]  test_integration.py
│   ├── [ 10K]  test_s3_advanced.py
│   ├── [ 180]  test_twat_fs.py
│   ├── [ 33K]  test_upload.py
│   └── [ 13K]  test_utils.py
├── [2.8K]  update_providers.py
└── [383K]  uv.lock

17 directories, 67 files

================
File: plugins/repos/twat_fs/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_fs/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_fs --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_fs/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-fs
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_fs/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_fs/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_fs/dist/.gitignore
================
*

================
File: plugins/repos/twat_fs/examples/upload_example.py
================
async def upload_file(file_path: str, service: str = "bash") -> str:
    uploader = get_uploader(service)
    result = await uploader.upload_file(Path(file_path))
        raise Exception(msg)
def main(file_path: str, service: str = "bash"):
    return asyncio.run(upload_file(file_path, service))
    fire.Fire(main)

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/__init__.py
================
def get_provider_module(provider: str) -> Provider | None:
    return ProviderFactory.get_provider_module(provider)
def get_provider_help(provider: str) -> ProviderHelp | None:
    return ProviderFactory.get_provider_help(provider)

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/async_utils.py
================
T = TypeVar("T")
T_co = TypeVar("T_co", covariant=True)
P = ParamSpec("P")
def run_async(coro: Coroutine[Any, Any, T]) -> T:
        return asyncio.run(coro)
        if "There is no current event loop in thread" in str(e):
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)
                return loop.run_until_complete(coro)
                loop.close()
def to_sync(func: Callable[P, Coroutine[Any, Any, T]]) -> Callable[P, T]: ...
def to_sync(
    def decorator(async_func: Callable[P, Coroutine[Any, Any, T]]) -> Callable[P, T]:
        @functools.wraps(async_func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            return run_async(async_func(*args, **kwargs))
    return decorator(func)
def to_async(func: Callable[P, T]) -> Callable[P, Coroutine[Any, Any, T]]: ...
def to_async(
    def decorator(sync_func: Callable[P, T]) -> Callable[P, Coroutine[Any, Any, T]]:
        @functools.wraps(sync_func)
        async def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            return sync_func(*args, **kwargs)
async def gather_with_concurrency(
    semaphore = asyncio.Semaphore(limit)
    async def run_with_semaphore(coro):
                    return cast(BaseException, e)
    wrapped_tasks = [asyncio.create_task(run_with_semaphore(task)) for task in tasks]
    return await asyncio.gather(
class AsyncContextManager:
    async def __aenter__(self) -> Any:
    async def __aexit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
def with_async_timeout(
    def decorator(
        @functools.wraps(func)
                return await asyncio.wait_for(func(*args, **kwargs), timeout=timeout)
                logger.warning(
                raise TimeoutError(msg) from None

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/bashupload.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class BashUploadProvider(BaseProvider):
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
    async def _do_upload(self, file_path: Path) -> str:
        data = aiohttp.FormData()
        with open(file_path, "rb") as f:
            data.add_field("file", f, filename=file_path.name)
            async with aiohttp.ClientSession() as session:
                async with session.post(
                    handle_http_response(response, self.provider_name)
                    response_text = await response.text()
                    for line in response_text.splitlines():
                        if line.startswith("wget "):
                            url = line.split(" ")[1].strip()
                    raise NonRetryableError(msg, self.provider_name)
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            temp_path = Path(file.name)
            url = asyncio.run(self._do_upload(temp_path))
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
def get_credentials() -> None:
    return BashUploadProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return BashUploadProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/catbox.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class CatboxProvider(BaseProvider):
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
    async def _do_upload(self, file_path: Path) -> str:
        data = aiohttp.FormData()
        data.add_field("reqtype", "fileupload")
        with open(file_path, "rb") as f:
            data.add_field("fileToUpload", f, filename=file_path.name)
            async with aiohttp.ClientSession() as session:
                async with session.post(self.upload_url, data=data) as response:
                    handle_http_response(response, self.provider_name)
                    return await response.text()
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            temp_path = Path(file.name)
            url = asyncio.run(self._do_upload(temp_path))
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
def get_credentials() -> None:
    return CatboxProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return CatboxProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/core.py
================
T_co = TypeVar("T_co", covariant=True)
R_co = TypeVar("R_co", str, UploadResult, covariant=True)
P = ParamSpec("P")
URL_CHECK_TIMEOUT = aiohttp.ClientTimeout(total=30.0)  # seconds
class UploadCallable(Protocol[P, T_co]):
    def __call__(self, *args: P.args, **kwargs: P.kwargs) -> T_co: ...
class AsyncUploadCallable(Protocol[P, T_co]):
    async def __call__(self, *args: P.args, **kwargs: P.kwargs) -> T_co: ...
def convert_to_upload_result(result: str) -> UploadResult: ...
def convert_to_upload_result(
def convert_to_upload_result(result: UploadResult) -> UploadResult: ...
def convert_to_upload_result(result: dict[str, Any]) -> UploadResult: ...
    if isinstance(result, UploadResult):
            result.metadata.update(metadata)
    if isinstance(result, str):
        return UploadResult(url=result, metadata=meta)
    if isinstance(result, dict):
        return UploadResult(**result)
    msg = f"Cannot convert {type(result)} to UploadResult"
    raise TypeError(msg)
class TimingMetrics(NamedTuple):
    def as_dict(self) -> dict[str, float | str]:
class RetryStrategy(str, Enum):
def with_retry(
    def decorator(func: UploadCallable[P, T_co]) -> UploadCallable[P, T_co]:
        @functools.wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T_co:
            for attempt in range(max_attempts):
                    return func(*args, **kwargs)
                        delay = min(initial_delay * (2**attempt), max_delay)
                        delay = min(initial_delay * (attempt + 1), max_delay)
                    logger.warning(
                    time.sleep(delay)
def with_async_retry(
    def decorator(func: AsyncUploadCallable[P, T_co]) -> AsyncUploadCallable[P, T_co]:
        async def wrapper(*args: P.args, **kwargs: P.kwargs) -> T_co:
                    return await func(*args, **kwargs)
                    await asyncio.sleep(delay)
def validate_file(func: UploadCallable[P, T_co]) -> UploadCallable[P, T_co]:
        file_path = next(
            (arg for arg in args if isinstance(arg, str | Path)),
            kwargs.get("local_path") or kwargs.get("file_path"),
            raise ValueError(msg)
        path = Path(str(file_path))  # Explicitly convert to string
        if not path.exists():
            raise FileNotFoundError(msg)
        if not path.is_file():
        if not path.stat().st_size:
def sync_to_async(func: UploadCallable[P, T_co]) -> AsyncUploadCallable[P, T_co]:
        return await asyncio.to_thread(func, *args, **kwargs)
def async_to_sync(func: AsyncUploadCallable[P, T_co]) -> UploadCallable[P, T_co]:
        return asyncio.run(func(*args, **kwargs))
class UploadError(Exception):
    def __init__(self, message: str, provider: str | None = None) -> None:
        super().__init__(message)
class RetryableError(UploadError):
class NonRetryableError(UploadError):
async def validate_url(
            aiohttp.ClientSession(
            session.head(
                raise NonRetryableError(msg)
            raise RetryableError(msg)
        raise RetryableError(msg) from e
@with_async_retry(
async def ensure_url_accessible(url: str) -> UploadResult:
    if await validate_url(url):
        return convert_to_upload_result(url)
def with_url_validation(
    async def wrapper(*args: P.args, **kwargs: P.kwargs) -> UploadResult:
        result = await func(*args, **kwargs)
            return await ensure_url_accessible(result)
def with_timing(func: Callable[P, Awaitable[T_co]]) -> Callable[P, Awaitable[T_co]]:
        start_time = time.time()
        end_time = time.time()

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/dropbox.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
load_dotenv()
class DropboxCredentials(TypedDict):
class DropboxClient(BaseProvider):
    def __init__(self, credentials: DropboxCredentials) -> None:
        self.dbx = self._create_client()
    def _create_client(self) -> dropbox.Dropbox:
        return dropbox.Dropbox(
    def _refresh_token_if_needed(self) -> None:
            self.dbx.users_get_current_account()
            if "expired_access_token" in str(e):
                    logger.debug(
                logger.debug("Access token expired, attempting refresh")
                    self.dbx.refresh_access_token()
                    logger.debug(f"Unable to refresh access token: {refresh_err}")
                logger.debug(f"Authentication error: {e}")
    def upload_file_impl(
            path = Path(file.name)
            self._refresh_token_if_needed()
            actual_upload_path = _normalize_path(actual_upload_path)
            remote_path = path.name if remote_path is None else str(remote_path)
                timestamp = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
                name, ext = os.path.splitext(remote_path)
            db_path = _normalize_path(os.path.join(actual_upload_path, remote_path))
            logger.debug(f"Target Dropbox path: {db_path}")
            _ensure_upload_directory(self.dbx, actual_upload_path)
            exists, remote_metadata = _check_file_exists(self.dbx, db_path)
                raise DropboxFileExistsError(msg)
            file_size = os.path.getsize(path)
                _upload_small_file(self.dbx, path, db_path)
                _upload_large_file(self.dbx, path, db_path, SMALL_FILE_THRESHOLD)
            url = _get_share_url(self.dbx, db_path)
                raise DropboxUploadError(msg)
            log_upload_attempt(
            return UploadResult(
            logger.error(f"Failed to upload to Dropbox: {e}")
                    "error": str(e),
    def upload_file(
        path = Path(local_path)
        validate_file(path)
            with open(path, "rb") as file:
                return self.upload_file_impl(
            raise ValueError(msg) from e
    def get_account_info(self) -> None:
            logger.error(f"Failed to get account info: {e}")
    def get_credentials(cls) -> dict[str, Any] | None:
        creds = get_env_credentials(required_vars, optional_vars)
            "refresh_token": creds.get("DROPBOX_REFRESH_TOKEN"),
            "app_key": creds.get("DROPBOX_APP_KEY"),
            "app_secret": creds.get("DROPBOX_APP_SECRET"),
    def get_provider(cls) -> ProviderClient | None:
            credentials = cls.get_credentials()
            client = cls(cast(DropboxCredentials, credentials))
                client.get_account_info()
                return cast(ProviderClient, client)
                        logger.warning(
                        logger.error(
                    logger.error(f"Dropbox authentication failed: {e}")
                logger.error(f"Failed to initialize Dropbox client: {e}")
            logger.error(f"Error initializing Dropbox client: {e}")
def get_credentials() -> dict[str, Any] | None:
    return DropboxClient.get_credentials()
def get_provider() -> ProviderClient | None:
    return DropboxClient.get_provider()
    client = get_provider()
        raise ValueError(msg)
        remote_path = Path(local_path).name
        return client.upload_file(
class DropboxUploadError(Exception):
class PathConflictError(DropboxUploadError):
class DropboxFileExistsError(Exception):
    def __init__(self, message: str, url: str | None = None):
        super().__init__(message)
class FolderExistsError(PathConflictError):
def _get_download_url(url: str) -> str | None:
        parsed = parse.urlparse(url)
        query = dict(parse.parse_qsl(parsed.query))
        return parsed._replace(
            netloc="dl.dropboxusercontent.com", query=parse.urlencode(query)
        ).geturl()
        logger.error(f"Failed to generate download URL: {e}")
def _get_share_url(dbx: dropbox.Dropbox, db_path: str) -> str:
    @retry(
        stop=stop_after_attempt(3),
        wait=wait_exponential(multiplier=1, min=4, max=10),
    def get_url() -> str:
            shared_link = dbx.sharing_create_shared_link_with_settings(db_path)
            url = str(shared_link.url).replace("?dl=0", "?dl=1")
            logger.debug(f"Created share URL: {url}")
            logger.error(f"Failed to create share URL: {e}")
        url = get_url()
        if not isinstance(url, str):
            msg = f"Expected string URL but got {type(url)}"
        raise DropboxUploadError(msg) from e
def _ensure_upload_directory(dbx: Any, upload_path: str) -> None:
    logger.debug(f"Ensuring upload directory exists: {upload_path}")
            logger.debug(f"Attempting to create directory: {upload_path}")
            dbx.files_create_folder_v2(upload_path)
            logger.debug(f"Successfully created directory: {upload_path}")
                isinstance(e.error, dropbox.files.CreateFolderError)
                and e.error.get_path().is_conflict()
                logger.debug(f"Directory already exists: {upload_path}")
            logger.error(f"Failed to create directory: {e}")
def _get_file_metadata(dbx: Any, db_path: str) -> dict | None:
        metadata = dbx.files_get_metadata(db_path)
        if e.error.is_path() and e.error.get_path().is_not_found():
def _check_file_exists(dbx: Any, db_path: str) -> tuple[bool, dict | None]:
        if metadata := _get_file_metadata(dbx, db_path):
        logger.warning(f"Error checking file existence: {e}")
def _upload_small_file(dbx: dropbox.Dropbox, file_path: Path, db_path: str) -> None:
    logger.debug(f"Uploading small file: {file_path} -> {db_path}")
        with open(file_path, "rb") as f:
            dbx.files_upload(f.read(), db_path, mode=dropbox.files.WriteMode.overwrite)
        logger.debug(f"Successfully uploaded small file: {db_path}")
        logger.error(f"Failed to upload small file: {e}")
def _upload_large_file(
    logger.debug(f"Starting chunked upload: {file_path} -> {db_path}")
    file_size = os.path.getsize(file_path)
            upload_session_start_result = dbx.files_upload_session_start(
                f.read(chunk_size)
            logger.debug("Upload session started")
            cursor = dropbox.files.UploadSessionCursor(
                offset=f.tell(),
            commit = dropbox.files.CommitInfo(
            while f.tell() < file_size:
                if (file_size - f.tell()) <= chunk_size:
                    logger.debug("Uploading final chunk")
                    dbx.files_upload_session_finish(f.read(chunk_size), cursor, commit)
                    logger.debug(f"Uploading chunk at offset {cursor.offset}")
                    dbx.files_upload_session_append_v2(f.read(chunk_size), cursor)
                    cursor.offset = f.tell()
        logger.debug(f"Successfully uploaded large file: {db_path}")
        logger.error(f"Failed to upload large file: {e}")
def _normalize_path(path: str) -> str:
    normalized = path.replace("\\", "/")
    normalized = normalized.lstrip("/")
def _handle_api_error(e: Any, operation: str) -> None:
    if isinstance(e, AuthError):
        logger.error(f"Authentication error during {operation}: {e}")
    elif isinstance(e, dropbox.exceptions.ApiError):
        if e.error.is_path():
            path_error = e.error.get_path()
            if path_error.is_not_found():
                logger.error(f"Path not found during {operation}: {e}")
            elif path_error.is_not_file():
                logger.error(f"Not a file error during {operation}: {e}")
            elif path_error.is_conflict():
                logger.error(f"Path conflict during {operation}: {e}")
        logger.error(f"API error during {operation}: {e}")
        logger.error(f"Unexpected error during {operation}: {e}")

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/factory.py
================
T = TypeVar("T", bound=Provider)
class ProviderFactory:
    def get_provider_module(provider_name: str) -> Provider | None:
            if provider_name.lower() == "simple":
                module = importlib.import_module(
                if "No module named" in str(e):
                    logger.debug(f"Provider module {provider_name} not found")
                    logger.warning(f"Error importing provider {provider_name}: {e}")
                attr for attr in required_attrs if not hasattr(module, attr)
                logger.warning(
                    f"Provider {provider_name} is missing required attributes: {', '.join(missing_attrs)}"
            if not hasattr(module, "PROVIDER_HELP"):
                logger.warning(f"Provider {provider_name} is missing help information")
            return cast(Provider, module)
            logger.error(f"Unexpected error loading provider {provider_name}: {e}")
    def get_provider_help(provider_name: str) -> ProviderHelp | None:
            module = ProviderFactory.get_provider_module(provider_name)
                    return getattr(module, "PROVIDER_HELP", None)
            logger.error(f"Error getting help for provider {provider_name}: {e}")
    def create_provider(provider_name: str) -> ProviderClient | None:
            provider_module = ProviderFactory.get_provider_module(provider_name)
                logger.warning(f"Provider module {provider_name} not found")
            provider_module.get_credentials()
            provider = provider_module.get_provider()
                logger.warning(f"Failed to initialize provider {provider_name}")
            logger.error(f"Error creating provider {provider_name}: {e}")
    def create_providers(
            providers[name] = ProviderFactory.create_provider(name)

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/fal.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class FalProvider(BaseProvider):
    def __init__(self, key: str) -> None:
        super().__init__()
        self.client = self._create_client()
    def _create_client(self) -> Any:
            return fal_client.SyncClient(key=self.key)
            logger.error(f"Failed to create FAL client: {e}")
            raise NonRetryableError(msg, self.provider_name)
    def get_credentials(cls) -> dict[str, str] | None:
        creds = get_env_credentials(cls.REQUIRED_ENV_VARS, cls.OPTIONAL_ENV_VARS)
    def get_provider(cls) -> ProviderClient | None:
        creds = cls.get_credentials()
            logger.debug("FAL_KEY not set in environment")
            provider = cls(key=str(creds["key"]).strip())
            return cast(ProviderClient, provider)
            logger.warning(f"Failed to initialize FAL provider: {err}")
    def _do_upload(self, file: BinaryIO) -> str:
        if not hasattr(self.client, "upload_file"):
            logger.debug("FAL: API credentials verified")
            if "401" in str(e) or "unauthorized" in str(e).lower():
                raise NonRetryableError(msg, self.provider_name) from e
            raise RetryableError(msg, self.provider_name) from e
            result = self.client.upload_file(file)
                raise RetryableError(msg, self.provider_name) from None
            result_str = str(result).strip()
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            url = self._do_upload(file)
            log_upload_attempt(self.provider_name, file.name, success=True)
            return convert_to_upload_result(
            log_upload_attempt(self.provider_name, file.name, success=False, error=e)
def get_credentials() -> dict[str, str] | None:
    return FalProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return FalProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        provider=get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/filebin.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class FilebinProvider(BaseProvider):
    def __init__(self) -> None:
    def _generate_bin_name(self) -> str:
        timestamp = int(time.time())
        suffix = "".join(
            secrets.choice(string.ascii_lowercase + string.digits) for _ in range(6)
    def _do_upload(self, file: BinaryIO, filename: str) -> str:
        bin_name = self._generate_bin_name()
        for attempt in range(max_retries):
                file.seek(0)  # Reset file pointer for each attempt
                response = requests.put(
                    handle_http_response(response, self.provider_name)
                        time.sleep(retry_delay)
                raise NonRetryableError(msg, self.provider_name) from e
        raise NonRetryableError(msg, self.provider_name) from None
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            filename = Path(file.name).name
            url = self._do_upload(file, filename)
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
def get_credentials() -> None:
    return FilebinProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return FilebinProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/litterbox.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class LitterboxProvider(BaseProvider):
    def __init__(
        super().__init__()
        if not isinstance(default_expiration, ExpirationTime):
                default_expiration = ExpirationTime(default_expiration)
                raise ValueError(msg) from e
    def get_credentials(cls) -> dict[str, Any] | None:
    def get_provider(cls) -> ProviderClient | None:
        default_expiration = str(
            os.getenv("LITTERBOX_DEFAULT_EXPIRATION", "24h")
        ).strip()
            expiration = ExpirationTime(str(default_expiration))
            logger.warning(
        provider = cls(default_expiration=expiration)
        return cast(ProviderClient, provider)
    async def _do_upload_async(
        data = aiohttp.FormData()
        data.add_field("reqtype", "fileupload")
        data.add_field(
            str(
                if isinstance(expiration_value, ExpirationTime)
            file_content = file.read()
            file_name = os.path.basename(file.name)
            async with aiohttp.ClientSession() as session:
                    async with session.post(LITTERBOX_API_URL, data=data) as response:
                        handle_http_response(response, self.provider_name)
                        url = await response.text()
                        if not url.startswith("http"):
                            raise NonRetryableError(msg, self.provider_name)
                    raise RetryableError(msg, self.provider_name) from e
    def _do_upload(
        return async_to_sync(self._do_upload_async)(file, expiration)
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            url = self._do_upload(file)
            log_upload_attempt(self.provider_name, file.name, success=True)
            return convert_to_upload_result(
            log_upload_attempt(self.provider_name, file.name, success=False, error=e)
    def upload_file(
        path = Path(local_path)
        self._validate_file(path)
        with self._open_file(path) as file:
                    url = self._do_upload(file, expiration)
                            "expiration": str(expiration),
                    log_upload_attempt(
                return super().upload_file(
def get_credentials() -> dict[str, Any] | None:
    return LitterboxProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return LitterboxProvider.get_provider()
    provider = get_provider()
        raise ValueError(msg)
    return provider.upload_file(

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/pixeldrain.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class PixeldrainProvider(BaseProvider):
    def __init__(self) -> None:
    def _process_response(self, response: requests.Response) -> str:
        handle_http_response(response, self.provider_name)
            data = response.json()
            raise ValueError(msg) from e
            raise ValueError(msg)
    def _upload_with_retry(self, file: BinaryIO) -> str:
        for attempt in range(max_retries):
                file.seek(0)
                response = requests.post(
                return self._process_response(response)
                    delay = min(retry_delay * (2**attempt), 30.0)
                    time.sleep(delay)
                    if isinstance(e, RetryableError | requests.RequestException):
        raise ValueError(str(last_error)) from last_error
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            url = self._upload_with_retry(file)
            log_upload_attempt(
            return UploadResult(
                    "raw_response": {"id": url.split("/")[-1]},
                    "error": str(e),
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
    def _get_file_url(self, file_id: str | None) -> str | None:
def get_credentials() -> None:
    return PixeldrainProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return PixeldrainProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/protocols.py
================
T_co = TypeVar("T_co", covariant=True)
T_ret = TypeVar("T_ret", bound=UploadResult)
class ProviderHelp(TypedDict):
class ProviderClient(Protocol):
    def upload_file(
    async def async_upload_file(
class Provider(Protocol):
    def get_credentials(cls) -> Any | None:
    def get_provider(cls) -> ProviderClient | None:

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/s3.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class S3Provider(BaseProvider):
    def __init__(self, credentials: dict[str, Any]) -> None:
        self.client = self._create_client()
    def _create_client(self) -> Any:
        if self.credentials.get("AWS_DEFAULT_REGION"):
        if self.credentials.get("AWS_ENDPOINT_URL"):
        if self.credentials.get("AWS_S3_PATH_STYLE", "").lower() == "true":
            client_kwargs["config"] = Config(s3={"addressing_style": "path"})
        if self.credentials.get("AWS_ACCESS_KEY_ID"):
        if self.credentials.get("AWS_SECRET_ACCESS_KEY"):
        if self.credentials.get("AWS_SESSION_TOKEN"):
            return boto3.client("s3", **client_kwargs)
            raise NonRetryableError(msg, self.provider_name) from e
    def _get_s3_url(self, key: str) -> str:
        if endpoint_url := self.credentials.get("AWS_ENDPOINT_URL"):
            region = self.credentials.get("AWS_DEFAULT_REGION", "us-east-1")
    def _do_upload(self, file: BinaryIO, key: str) -> str:
            file.seek(0)
            self.client.upload_fileobj(file, bucket, key)
            return self._get_s3_url(key)
            error_str = str(e)
                raise RetryableError(msg, self.provider_name) from e
    def upload_file_impl(
            key = str(remote_path or Path(file.name).name)
            url = self._do_upload(file, key)
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
    def get_credentials(cls) -> dict[str, Any] | None:
        creds = get_env_credentials(cls.REQUIRED_ENV_VARS, cls.OPTIONAL_ENV_VARS)
    def get_provider(cls) -> ProviderClient | None:
            credentials = cls.get_credentials()
            provider = cls(credentials)
                provider.client.list_buckets()
                return cast(ProviderClient, provider)
                logger.warning(f"Failed to validate S3 client: {e}")
            logger.error(f"Error initializing S3 provider: {e}")
def get_credentials() -> dict[str, Any] | None:
    return S3Provider.get_credentials()
def get_provider() -> ProviderClient | None:
    return S3Provider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/simple.py
================
class SimpleProviderClient(Protocol):
    async def upload_file(self, file_path: Path) -> UploadResult:
T = TypeVar("T", bound="BaseProvider")
class BaseProvider(ABC, Provider):
    def get_credentials(cls) -> dict[str, Any] | None:
    def get_provider(cls) -> ProviderClient | None:
        return cls()
    def upload_file(
        path = Path(local_path)
        self._validate_file(path)
        with self._open_file(path) as file:
            result = self.upload_file_impl(file)
            if not result.metadata.get("success", True):
                msg = f"Upload failed: {result.metadata.get('error', 'Unknown error')}"
                raise RuntimeError(msg)
            return convert_to_upload_result(
    async def async_upload_file(
        path = Path(str(file_path))
        if not path.exists():
            raise FileNotFoundError(msg)
            f"file://{path.absolute()}",
                "local_path": str(path),
    def _open_file(self, file_path: Path) -> Generator[BinaryIO]:
            file = open(file_path, "rb")
                file.close()
    def _validate_file(self, file_path: Path) -> None:
        if not file_path.exists():
        if not file_path.is_file():
            raise ValueError(msg)
        if not os.access(file_path, os.R_OK):
            raise PermissionError(msg)
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
class AsyncBaseProvider(BaseProvider):
        sync_upload = to_sync(self.async_upload_file)
        return sync_upload(
        path = Path(file.name)
        return sync_upload(path)
class SyncBaseProvider(BaseProvider):
        return self.upload_file(

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/types.py
================
class ExpirationTime(str, Enum):
class UploadResult:
    def __init__(self, url: str, metadata: dict[str, Any] | None = None) -> None:

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/uguu.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class UguuProvider(BaseProvider):
    def __init__(self) -> None:
    def _do_upload(self, file: BinaryIO) -> str:
        response = requests.post(self.upload_url, files=files, timeout=30)
        handle_http_response(response, self.provider_name)
        result = response.json()
            raise NonRetryableError(msg, self.provider_name)
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            url = self._do_upload(file)
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
def get_credentials() -> None:
    return UguuProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return UguuProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/utils.py
================
T = TypeVar("T")
P = ParamSpec("P")
def create_provider_help(
def safe_file_handle(file_path: str | Path) -> Generator[BinaryIO]:
    path = Path(str(file_path))
    validate_file(path)
        file = open(path, "rb")
            file.close()
def validate_file(file_path: Path) -> None:
    if not file_path.exists():
        raise FileNotFoundError(msg)
    if not file_path.is_file():
        raise ValueError(msg)
    if not os.access(file_path, os.R_OK):
        raise PermissionError(msg)
def handle_http_response(
        if isinstance(response, aiohttp.ClientResponse)
        msg = f"Rate limited: {response.text if isinstance(response, requests.Response) else 'Too many requests'}"
        raise RetryableError(msg, provider_name)
        raise NonRetryableError(msg, provider_name)
def get_env_credentials(
    missing = [var for var in required_vars if not os.getenv(var)]
        logger.debug(f"Missing required environment variables: {', '.join(missing)}")
        if value := os.getenv(var):
def create_provider_instance(
    provider_name = getattr(provider_class, "__name__", "Unknown")
        if credentials is None and hasattr(provider_class, "get_credentials"):
            credentials = provider_class.get_credentials()
        if hasattr(provider_class, "get_provider"):
                return provider_class.get_provider()
        return provider_class()
        logger.error(f"Failed to initialize provider {provider_name}: {e}")
def standard_upload_wrapper(
    return provider.upload_file(local_path, remote_path, **kwargs)
def log_upload_attempt(
        logger.info(f"Successfully uploaded {file_path} using {provider_name}")
        logger.error(f"Failed to upload {file_path} using {provider_name}: {error}")

================
File: plugins/repos/twat_fs/src/twat_fs/upload_providers/www0x0.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class Www0x0Provider(BaseProvider):
    def __init__(self) -> None:
    def _do_upload(self, file: BinaryIO) -> str:
        response = requests.post(self.upload_url, files=files, timeout=30)
        handle_http_response(response, self.provider_name)
        url = response.text.strip()
        if not url.startswith("http"):
            raise NonRetryableError(msg, self.provider_name)
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            url = self._do_upload(file)
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
def get_credentials() -> None:
    return Www0x0Provider.get_credentials()
def get_provider() -> ProviderClient | None:
    return Www0x0Provider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/src/twat_fs/__init__.py
================
__version__ = metadata.version(__name__)

================
File: plugins/repos/twat_fs/src/twat_fs/__main__.py
================
    main()

================
File: plugins/repos/twat_fs/src/twat_fs/__version__.py
================


================
File: plugins/repos/twat_fs/src/twat_fs/cli.py
================
logger.remove()  # Remove default handler
log_level = os.getenv("LOGURU_LEVEL", "INFO").upper()
logger.add(
    filter=lambda record: record["level"].no < logger.level("WARNING").no,
def parse_provider_list(provider: str) -> list[str] | None:
    if provider.startswith("[") and provider.endswith("]"):
            return [p.strip() for p in provider[1:-1].split(",")]
class UploadProviderCommands:
    def status(self, provider_id: str | None = None, online: bool = False) -> None:
        console = Console(stderr=True)  # Use stderr for error messages
            with console.status("[cyan]Testing provider...[/cyan]"):
                result = _setup_provider(provider_id, verbose=True, online=online)
                    console.print(
                            f"\n[cyan]Online test time: {result.timing.get('total_duration', 0.0):.2f}s[/cyan]"
                    console.print(f"\n[red]Provider {provider_id} is not ready[/red]")
                    console.print(f"\n[yellow]Reason:[/yellow] {result.explanation}")
                    console.print("\n[yellow]Setup Instructions:[/yellow]")
                        result.help_info.get("setup", "No setup instructions available")
            table = Table(title="Provider Setup Status", show_lines=True)
            table.add_column("Provider", no_wrap=True)
            table.add_column("Status", no_wrap=True)
                table.add_column("Time (s)", justify="right", no_wrap=True)
            table.add_column("Details", width=50)
            with console.status("[cyan]Testing providers...[/cyan]") as status:
                results = _setup_providers(verbose=True, online=online)
                for provider, info in results.items():
                    if provider.lower() == "simple":
                            info.timing.get("total_duration", float("inf"))
                            else float("inf")
                        ready_providers.append((provider, info, time))
                        not_ready_providers.append((provider, info, time))
                ready_providers.sort(key=lambda x: x[2])
                not_ready_providers.sort(key=lambda x: x[0])
                    if info.help_info.get("setup"):
                            and info.timing.get("total_duration") is not None
                            time = f"{info.timing.get('total_duration', 0.0):.2f}"
                    table.add_row(*row)
            console.print("\n")  # Add some spacing
            console.print(table)
    def list(self, online: bool = False) -> None:
            logger.remove()
            logger.add(sys.stderr, level="INFO", format="{message}")
            result = _setup_provider(provider, verbose=False, online=online)
                active_providers.append(provider)
        sys.exit(0)
class TwatFS:
    def __init__(self) -> None:
        self.upload_provider = UploadProviderCommands()
    def upload(
            if isinstance(provider, str):
                providers = parse_provider_list(provider)
            if not Path(file_path).exists():
                logger.error(f"File not found: {file_path}")
                sys.exit(1)
            return _upload_file(
            logger.error(f"Upload failed: {e}")
def main() -> None:
        fire.Fire(TwatFS)
        fire.Fire(TwatFS())
upload_file = TwatFS().upload
setup_provider = TwatFS().upload_provider.status
def setup_providers():
    return TwatFS().upload_provider.status(None)
    main()

================
File: plugins/repos/twat_fs/src/twat_fs/py.typed
================


================
File: plugins/repos/twat_fs/src/twat_fs/upload.py
================
class ProviderStatus(Enum):
    READY = auto()
    NEEDS_CONFIG = auto()
    NOT_AVAILABLE = auto()
class ProviderInfo:
def _test_provider_online(
    test_file = Path(__file__).parent / "data" / "test.jpg"
    if not test_file.exists():
        with open(test_file, "rb") as f:
            original_hash = hashlib.sha256(f.read()).hexdigest()
        provider_module = ProviderFactory.get_provider_module(provider_name)
        client = ProviderFactory.create_provider(provider_name)
        start_time = time.time()
        read_start = time.time()
            _ = f.read()  # Read file to measure read time
        read_duration = time.time() - read_start
            upload_start = time.time()
            result = client.upload_file(test_file)
            upload_duration = time.time() - upload_start
                end_time = time.time()
                    "provider": float(hash(provider_name)),
            if isinstance(result, UploadResult):
                timing_metrics = result.metadata.get("timing", {})
            time.sleep(1.0)
            validation_start = time.time()
            response = requests.head(
            validation_duration = time.time() - validation_start
                "upload_duration": time.time() - upload_start,
            if "401" in str(e) or "authentication" in str(e).lower():
                "validation_duration": time.time() - validation_start,
        time.time()
        for attempt in range(max_retries):
                response = requests.get(url, timeout=30)
                    downloaded_hash = hashlib.sha256(response.content).hexdigest()
                    time.sleep(retry_delay)
def setup_provider(
        if provider.lower() == "simple":
            return ProviderInfo(
        provider_module = get_provider_module(provider)
            help_info = get_provider_help(provider)
            setup_info = help_info.get("setup", "")
            setup_info = help_info.get("setup", "").lower()
                    retention_note = setup_info[setup_info.find("note:") :].strip()
            client = provider_module.get_provider()
                setup_info = help_info.get("setup", "") if help_info else ""
            has_async = hasattr(client, "async_upload_file") and not getattr(
            has_sync = hasattr(client, "upload_file") and not getattr(
                provider_info = ProviderInfo(
                    f"{provider} ({type(client).__name__})"
                online_status, message, timing = _test_provider_online(provider)
                    logger.debug(
            logger.error(f"Error setting up provider {provider}: {e}")
        logger.error(f"Unexpected error setting up provider {provider}: {e}")
def setup_providers(
        result = setup_provider(provider, verbose=verbose, online=online)
def _get_provider_module(provider: str) -> Provider | None:
        return ProviderFactory.get_provider_module(provider)
        logger.error(f"Error getting provider module {provider}: {e}")
def _try_upload_with_provider(
        provider_module = _get_provider_module(provider_name)
            return UploadResult(
        provider = ProviderFactory.create_provider(provider_name)
        path = Path(str(file_path))
        if path.exists():
            with open(path, "rb") as f:
        upload_result = provider.upload_file(
            if isinstance(upload_result, UploadResult)
                logger.warning(f"URL validation failed: status {response.status_code}")
            logger.warning(f"URL validation failed: {e}")
            UploadResult(url=upload_result, metadata={"provider": provider_name})
            if isinstance(upload_result, str)
        logger.info(
def _try_next_provider(
        raise ValueError(msg)
    tried = tried_providers or set()
        tried.add(provider)
            result = _try_upload_with_provider(
            logger.warning(f"Provider {provider} failed: {e}")
    msg = f"All providers failed: {', '.join(tried)}"
@with_retry(
def upload_file(
    path = Path(file_path)
    if not path.exists():
        raise FileNotFoundError(msg)
    if not path.is_file():
        providers = [provider] if isinstance(provider, str) else list(provider)
        return _try_upload_with_provider(
        if fragile or len(providers) == 1:
            raise NonRetryableError(msg, providers[0])
        logger.info(f"Provider {providers[0]} failed, trying alternatives")
        return _try_next_provider(
def _try_upload_with_fallback(
            raise RuntimeError(msg) from None
        return _try_upload_with_fallback(

================
File: plugins/repos/twat_fs/templates/authenticated_provider_template.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class ProviderNameProvider(BaseProvider):
    def __init__(self, credentials: dict[str, str]) -> None:
        self.api_key = credentials.get("PROVIDER_API_KEY")
        self.api_secret = credentials.get("PROVIDER_API_SECRET")
        self.region = credentials.get("PROVIDER_REGION", "default_region")
    def get_credentials(cls) -> dict[str, str] | None:
        return get_env_credentials(
    def get_provider(cls) -> ProviderClient | None:
        credentials = cls.get_credentials()
            return cast(ProviderClient, cls(credentials))
            logger.error(f"Failed to initialize {cls.provider_name} provider: {e}")
    def _do_upload(self, file_path: Path, remote_path: Path | None = None) -> str:
        with open(file_path, "rb") as f:
            response = requests.post(
            handle_http_response(response, self.provider_name)
            data = response.json()
            url = data.get("url", "")
                raise NonRetryableError(msg, self.provider_name)
            return str(url)
    def upload_file_impl(
            temp_path = Path(file.name)
            url = self._do_upload(temp_path, remote_path)
            log_upload_attempt(
            return UploadResult(
                    "remote_path": str(remote_path) if remote_path else None,
                    "error": str(e),
    def upload_file(
        path = Path(local_path)
        self._validate_file(path)
            remote = Path(str(remote_path))
            remote = Path(upload_path) / path.name
            suffix = uuid.uuid4().hex[:8]
            remote = remote.with_stem(f"{remote.stem}_{suffix}")
        with self._open_file(path) as file:
            result = self.upload_file_impl(file, remote)
            if not result.metadata.get("success", True):
                msg = f"Upload failed: {result.metadata.get('error', 'Unknown error')}"
                raise RuntimeError(msg)
def get_credentials() -> dict[str, str] | None:
    return ProviderNameProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return ProviderNameProvider.get_provider()
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/templates/simple_provider_template.py
================
PROVIDER_HELP: ProviderHelp = create_provider_help(
class ProviderNameProvider(BaseProvider):
    def get_credentials(cls) -> None:
    def get_provider(cls) -> ProviderClient | None:
        return cast(ProviderClient, cls())
    def _do_upload(self, file_path: Path) -> str:
        with open(file_path, "rb") as f:
            response = requests.post(
            handle_http_response(response, self.provider_name)
            url = response.text.strip()  # Modify based on provider response format
    async def _do_upload_async(self, file_path: Path) -> str:
        data = aiohttp.FormData()
        data.add_field("file", open(file_path, "rb"), filename=file_path.name)
        async with aiohttp.ClientSession() as session:
            async with session.post(self.upload_url, data=data) as response:
                response_text = await response.text()
                return response_text.strip()
    def upload_file_impl(self, file: BinaryIO) -> UploadResult:
            temp_path = Path(file.name)
            url = self._do_upload(temp_path)
            log_upload_attempt(
            return UploadResult(
                    "error": str(e),
def get_credentials() -> None:
    return ProviderNameProvider.get_credentials()
def get_provider() -> ProviderClient | None:
    return ProviderNameProvider.get_provider()
def upload_file(
    return standard_upload_wrapper(
        get_provider(),

================
File: plugins/repos/twat_fs/tests/data/test.txt
================
This is a test file for upload testing.
It contains some sample content that will be used in tests.

================
File: plugins/repos/twat_fs/tests/__init__.py
================


================
File: plugins/repos/twat_fs/tests/test_async_utils.py
================
class TestRunAsync:
    def test_run_async_with_successful_coroutine(self):
        async def test_coro():
        result = run_async(test_coro())
    def test_run_async_with_exception(self):
            raise ValueError(msg)
        with pytest.raises(ValueError, match="test_error"):
            run_async(test_coro())
    def test_run_async_with_existing_event_loop(self):
        with mock.patch("asyncio.run") as mock_run:
            mock_run.side_effect = RuntimeError(
            mock_loop = mock.MagicMock()
            with mock.patch("asyncio.new_event_loop", return_value=mock_loop):
                with mock.patch("asyncio.set_event_loop"):
                    mock_loop.run_until_complete.assert_called_once()
                    mock_loop.close.assert_called_once()
class TestToSync:
    def test_to_sync_with_direct_decoration(self):
        async def test_func():
        result = test_func()
        assert not asyncio.iscoroutinefunction(test_func)
    def test_to_sync_with_arguments(self):
        @to_sync(name="custom_name")
    def test_to_sync_preserves_docstring(self):
    def test_to_sync_preserves_arguments(self):
        async def test_func(arg1, arg2=None):
        result = test_func("test", arg2="value")
class TestToAsync:
    def test_to_async_with_direct_decoration(self):
        def test_func():
        assert asyncio.iscoroutinefunction(test_func)
        result = run_async(test_func())
    def test_to_async_with_arguments(self):
        @to_async(name="custom_name")
    def test_to_async_preserves_docstring(self):
    def test_to_async_preserves_arguments(self):
        def test_func(arg1, arg2=None):
        result = run_async(test_func("test", arg2="value"))
class TestGatherWithConcurrency:
    async def test_gather_with_concurrency(self):
        async def test_task(i):
            max_concurrent = max(max_concurrent, concurrent)
            await asyncio.sleep(0.1)  # Simulate work
            results.append(i)
        tasks = [test_task(i) for i in range(10)]
        result = await gather_with_concurrency(3, *tasks)
        assert len(result) == 10
        assert set(result) == set(range(10))
    async def test_gather_with_concurrency_with_exceptions(self):
        with pytest.raises(ValueError):
            await gather_with_concurrency(3, *tasks)
        result = await gather_with_concurrency(3, *tasks, return_exceptions=True)
        for i, r in enumerate(result):
                assert isinstance(r, ValueError)
class TestAsyncContextManager:
    async def test_async_context_manager(self):
        class TestManager(AsyncContextManager):
            def __init__(self):
            async def __aenter__(self):
            async def __aexit__(self, exc_type, exc_val, exc_tb):
        manager = TestManager()
class TestWithAsyncTimeout:
    async def test_with_async_timeout_success(self):
        @with_async_timeout(0.5)
            await asyncio.sleep(0.1)
        result = await test_func()
    async def test_with_async_timeout_timeout(self):
        @with_async_timeout(0.1)
            await asyncio.sleep(0.5)
        with pytest.raises(TimeoutError):
            await test_func()
    async def test_with_async_timeout_preserves_metadata(self):
        result = await test_func("test", arg2="value")

================
File: plugins/repos/twat_fs/tests/test_filebin_pixeldrain.py
================
def test_file(tmp_path: Path) -> Path:
    file_path.write_text("test content")
def test_filebin_upload_success(test_file: Path) -> None:
    responses.add(
        re.compile(
    provider = filebin.FilebinProvider()
    result = provider.upload_file(test_file)
    assert isinstance(result, UploadResult)
    assert re.match(r"https://filebin\.net/.+/test\.txt", result.url)
def test_filebin_upload_failure(test_file: Path) -> None:
    with pytest.raises(RuntimeError, match="Upload failed"):
        provider.upload_file(test_file)
def test_pixeldrain_upload_success(test_file: Path) -> None:
    provider = pixeldrain.PixeldrainProvider()
def test_pixeldrain_upload_failure(test_file: Path) -> None:
def test_filebin_provider_initialization() -> None:
    provider = filebin.get_provider()
    assert isinstance(provider, filebin.FilebinProvider)
def test_pixeldrain_provider_initialization() -> None:
    provider = pixeldrain.get_provider()
    assert isinstance(provider, pixeldrain.PixeldrainProvider)

================
File: plugins/repos/twat_fs/tests/test_integration.py
================
HAS_FAL = importlib.util.find_spec("twat_fs.upload_providers.fal") is not None
HAS_S3 = importlib.util.find_spec("twat_fs.upload_providers.s3") is not None
TEST_DIR = Path(__file__).parent / "data"
TEST_DIR.mkdir(exist_ok=True)
if not SMALL_FILE.exists():
    with open(SMALL_FILE, "w") as f:
        f.write("This is a test file for upload testing.")
@pytest.fixture(scope="session")
def large_test_file():
    if not LARGE_FILE.exists():
        with open(LARGE_FILE, "wb") as f:
            f.write(os.urandom(LARGE_FILE_SIZE))
@pytest.fixture(scope="session", autouse=True)
def cleanup_test_files():
    if LARGE_FILE.exists():
        LARGE_FILE.unlink()
@pytest.mark.skipif(not HAS_S3, reason="S3 dependencies not installed")
class TestS3Integration:
    @pytest.fixture(autouse=True)
    def check_s3_credentials(self):
        if not os.getenv("AWS_S3_BUCKET") or not os.getenv("AWS_ACCESS_KEY_ID"):
            pytest.skip("S3 credentials not available")
    def test_s3_setup(self):
        provider_info = setup_provider("s3")
    def test_s3_upload_small_file(self):
        url = upload_file(SMALL_FILE, provider="s3")
        assert url and url.startswith("http")
    def test_s3_upload_large_file(self, large_test_file):
        url = upload_file(large_test_file, provider="s3")
    def test_s3_upload_with_custom_endpoint(self, monkeypatch):
        monkeypatch.setenv("AWS_ENDPOINT_URL", "https://custom-endpoint.example.com")
class TestDropboxIntegration:
    def test_dropbox_setup(self) -> None:
        result = setup_provider("dropbox")
            and "setup" in result.explanation.lower()
    def test_dropbox_upload_small_file(self) -> None:
            url = upload_file(SMALL_FILE, provider="dropbox")
            assert url.startswith("https://")
            assert "Failed to initialize Dropbox client" in str(e)
            assert "expired_access_token" in str(e) or "not configured" in str(e)
    def test_dropbox_upload_large_file(self, large_test_file: Path) -> None:
            url = upload_file(large_test_file, provider="dropbox")
@pytest.mark.skipif(not HAS_FAL, reason="FAL dependencies not installed")
class TestFalIntegration:
    def check_fal_credentials(self):
        if not os.getenv("FAL_KEY"):
            pytest.skip("FAL credentials not available")
    def test_fal_setup(self):
        provider_info = setup_provider("fal")
    def test_fal_upload_small_file(self):
        url = upload_file(SMALL_FILE, provider="fal")
    def test_fal_upload_large_file(self, large_test_file):
        url = upload_file(large_test_file, provider="fal")
class TestSetupIntegration:
    def test_setup_all_providers(self) -> None:
        setup_providers()
            if provider.lower() == "simple":
            result = setup_provider(provider)
                or "setup" in result.explanation.lower()
class TestCatboxIntegration:
    def test_catbox_setup(self):
        result = setup_provider("catbox")
    def test_catbox_upload_small_file(self, small_file):
            url = upload_file(small_file, provider="catbox")
            assert isinstance(url, str)
            assert url.startswith("https://files.catbox.moe/")
            assert len(url) > len("https://files.catbox.moe/")
            pytest.skip(f"Catbox upload failed: {e}")
    def test_catbox_upload_large_file(self, large_file):
            url = upload_file(large_file, provider="catbox")
    @pytest.mark.skipif(
        not os.getenv("CATBOX_USERHASH"),
    def test_catbox_authenticated_upload(self, small_file):
        filename = url.split("/")[-1]
        provider = catbox.get_provider()
            if hasattr(provider, "delete_files") and callable(
                getattr(provider, "delete_files", None)
                success = provider.delete_files([filename])
class TestLitterboxIntegration:
    def test_litterbox_setup(self):
        result = setup_provider("litterbox")
    def test_litterbox_upload_small_file(self, small_file):
            url = upload_file(small_file, provider="litterbox")
            assert url.startswith("https://litterbox.catbox.moe/")
            assert len(url) > len("https://litterbox.catbox.moe/")
            pytest.skip(f"Litterbox upload failed: {e}")
    def test_litterbox_upload_large_file(self, large_file):
            url = upload_file(large_file, provider="litterbox")
    def test_litterbox_different_expirations(self, small_file):
            provider = litterbox.LitterboxProvider(
            url = provider.upload_file(small_file)
            pytest.skip(f"Litterbox upload with different expirations failed: {e}")

================
File: plugins/repos/twat_fs/tests/test_s3_advanced.py
================
    class ClientError(Exception):
    class MockS3Provider:
        def get_credentials(cls) -> dict[str, Any]:
    class MockS3Module:
        def get_provider(*args: Any, **kwargs: Any) -> MockS3Provider:
            return MockS3Provider()
        def get_credentials() -> dict[str, Any]:
        def upload_file(*args: Any, **kwargs: Any) -> str:
    s3 = MockS3Module()
TEST_DIR = Path(__file__).parent / "data"
@pytest.mark.skipif(
class TestAwsCredentialProviders:
    def test_environment_credentials(self, monkeypatch):
        monkeypatch.setenv("AWS_ACCESS_KEY_ID", TEST_ACCESS_KEY)
        monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", TEST_SECRET_KEY)
        monkeypatch.setenv("AWS_S3_BUCKET", TEST_BUCKET)
        provider = s3.S3Provider()
    def test_shared_credentials_file(self, tmp_path, monkeypatch):
        creds_file.parent.mkdir(exist_ok=True)
        creds_file.write_text(
        monkeypatch.setenv("AWS_SHARED_CREDENTIALS_FILE", str(creds_file))
        monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1")
        monkeypatch.setenv("AWS_S3_BUCKET", "test-bucket")
        creds = s3.get_credentials()
        with patch("boto3.client") as mock_client:
            provider = s3.get_provider(creds)
    def test_assume_role(self, monkeypatch):
        monkeypatch.setenv("AWS_ROLE_ARN", "arn:aws:iam::123456789012:role/test-role")
            mock_sts = MagicMock()
                MagicMock(),  # Second call creates S3 client
class TestS3Configurations:
    def test_custom_endpoint(self, monkeypatch):
        monkeypatch.setenv("AWS_ENDPOINT_URL", "https://custom-s3.example.com")
            mock_client.assert_called_with(
    def test_path_style_endpoint(self, monkeypatch):
        monkeypatch.setenv("AWS_S3_PATH_STYLE", "true")
            assert hasattr(config, "s3")
    def test_custom_region_endpoint(self, monkeypatch):
        monkeypatch.setenv("AWS_DEFAULT_REGION", "eu-central-1")
            mock_client.assert_called_with("s3", region_name="eu-central-1")
class TestS3MultipartUploads:
    def large_file(self, tmp_path):
        with file_path.open("wb") as f:
            f.write(os.urandom(size))
    def test_multipart_upload(self, large_file, monkeypatch):
        monkeypatch.setenv("AWS_ACCESS_KEY_ID", "test_key")
        monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "test_secret")
        mock_s3 = MagicMock()
        with patch("twat_fs.upload_providers.s3.boto3.client", return_value=mock_s3):
            url = s3.upload_file(large_file)
            assert url.startswith("https://s3.amazonaws.com/test-bucket/")
            assert url.endswith(large_file.name)
            assert len(args) == 3  # Should have file object, bucket, and key
            assert hasattr(args[0], "read")  # Check if it's a file-like object
    def test_multipart_upload_failure(self, large_file, monkeypatch):
        mock_s3.upload_fileobj.side_effect = ClientError(
            with pytest.raises(ValueError, match="S3 upload failed"):
                s3.upload_file(large_file)

================
File: plugins/repos/twat_fs/tests/test_twat_fs.py
================
def test_version():

================
File: plugins/repos/twat_fs/tests/test_upload.py
================
    class ClientError(Exception):
class ProviderSetupResult(NamedTuple):
TEST_FILE = Path(__file__).parent / "data" / "test.txt"
def test_file(tmp_path: Path) -> Generator[Path, None, None]:
    file_path.write_text("test content")
    file_path.unlink()
def mock_fal_provider() -> Generator[MagicMock, None, None]:
    with patch("twat_fs.upload_providers.fal.FalProvider") as mock:
def mock_dropbox_provider() -> Generator[MagicMock, None, None]:
    with patch("twat_fs.upload_providers.dropbox.DropboxProvider") as mock:
def mock_s3_provider() -> Generator[MagicMock, None, None]:
        with patch("twat_fs.upload_providers.s3.S3Provider") as mock:
            mock_instance = MagicMock()
        with patch(
            mock_provider = MagicMock()
class TestProviderSetup:
    @pytest.mark.skipif(not HAS_S3, reason="S3 dependencies not installed")
    def test_setup_working_provider(self, mock_s3_provider: MagicMock) -> None:
        result = setup_provider("s3")
    def test_setup_missing_credentials(self) -> None:
        with patch("twat_fs.upload_providers.s3.get_credentials") as mock_creds:
    def test_setup_missing_dependencies(self) -> None:
            with patch("twat_fs.upload_providers.s3.get_provider") as mock_provider:
    def test_setup_invalid_provider(self) -> None:
        result = setup_provider("invalid")
        assert "Provider not found" in result.explanation.lower()
    @pytest.mark.skipif(
    def test_setup_all_providers(
        assert all(
        results = setup_providers()
        assert len(results) == len(PROVIDERS_PREFERENCE)
    def test_setup_all_providers_with_failures(self) -> None:
            def side_effect(provider_name, *args, **kwargs):
                    return MagicMock()  # FAL provider works
                    return MagicMock()  # Other providers work
            assert any(
                or "setup" in result.explanation.lower()
                for result in results.values()
    def test_setup_provider_success(self) -> None:
            mock_create_provider.return_value = MagicMock()
            result = setup_provider("simple")
    def test_setup_provider_failure(self) -> None:
        assert "not available" in result.explanation.lower()
    @pytest.mark.skipif(not HAS_DROPBOX, reason="Dropbox dependencies not installed")
    def test_setup_provider_dropbox(self) -> None:
        result = setup_provider("dropbox")
                or "not available" in result.explanation.lower()
    def test_setup_all_providers_check(self) -> None:
            result.success or "not available" in result.explanation.lower()
class TestProviderAuth:
    @pytest.mark.skipif(not HAS_FAL, reason="FAL dependencies not installed")
    def test_fal_auth_with_key(self, monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.setenv("FAL_KEY", "test_key")
        assert fal.get_credentials() is not None
        with patch("fal_client.status") as mock_status:
            assert fal.get_provider() is not None
    def test_fal_auth_without_key(self, monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.delenv("FAL_KEY", raising=False)
        assert fal.get_credentials() is None
        assert fal.get_provider() is None
    def test_dropbox_auth_with_token(self, monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.setenv("DROPBOX_ACCESS_TOKEN", "test_token")
        assert dropbox.get_credentials() is not None
        with patch("dropbox.Dropbox") as mock_dropbox:
            assert dropbox.get_provider() is not None
    def test_dropbox_auth_without_token(self, monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.delenv("DROPBOX_ACCESS_TOKEN", raising=False)
        assert dropbox.get_credentials() is None
        with pytest.raises(ValueError, match="Dropbox credentials not found"):
            dropbox.upload_file("test.txt")
    def test_s3_auth_with_credentials(self, monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.setenv("AWS_ACCESS_KEY_ID", "test_key")
        monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "test_secret")
        monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1")
        monkeypatch.setenv("AWS_S3_BUCKET", "test-bucket")
        creds = s3.get_credentials()
        with patch("boto3.client") as mock_client:
            provider = s3.get_provider()
    def test_s3_auth_without_credentials(self, monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.delenv("AWS_ACCESS_KEY_ID", raising=False)
        monkeypatch.delenv("AWS_SECRET_ACCESS_KEY", raising=False)
        monkeypatch.delenv("AWS_DEFAULT_REGION", raising=False)
        monkeypatch.delenv("AWS_S3_BUCKET", raising=False)
        assert s3.get_credentials() is None
        assert s3.get_provider() is None
    def test_s3_auth_with_invalid_credentials(
        monkeypatch.setenv("AWS_ACCESS_KEY_ID", "invalid_key")
        monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "invalid_secret")
            mock_s3.head_bucket.side_effect = ClientError(
class TestUploadFile:
    def test_upload_with_default_provider(
        url = upload_file(test_file)
        mock_s3_provider.assert_called_once_with(
    def test_upload_with_specific_provider(
        url = upload_file(test_file, provider="s3")
    def test_upload_with_provider_list(
        url = upload_file(test_file, provider=["s3", "dropbox"])
    def test_upload_fallback_on_auth_failure(
            patch("twat_fs.upload_providers.s3.get_provider") as mock_s3_get_provider,
            patch(
            mock_dropbox_client = MagicMock()
            mock_s3_provider.assert_not_called()
            mock_dropbox_client.upload_file.assert_called_once_with(
    def test_upload_fallback_on_upload_failure(
        mock_s3_provider.side_effect = Exception("Upload failed")
            mock_s3_client = MagicMock()
            mock_s3_client.upload_file.side_effect = Exception("Upload failed")
            mock_s3_client.upload_file.assert_called_once_with(
    def test_all_providers_fail(self, test_file: Path) -> None:
            with pytest.raises(
                upload_file(test_file)
    def test_invalid_provider(self, test_file: Path) -> None:
        with pytest.raises(ValueError, match="Invalid provider"):
            upload_file(test_file, provider="invalid")
    def test_upload_with_s3_provider(
    def test_s3_upload_failure(
        mock_s3_provider.side_effect = ClientError(
            upload_file(test_file, provider="s3")
class TestEdgeCases:
    def test_empty_file(self, tmp_path: Path) -> None:
        test_file.touch()
            client = MagicMock()
            client.upload_file.assert_called_once_with(
    def test_special_characters_in_filename(self, tmp_path: Path) -> None:
        test_file.write_text("test content")
    def test_unicode_filename(self, tmp_path: Path) -> None:
    def test_very_long_filename(self, tmp_path: Path) -> None:
    def test_nonexistent_file(self) -> None:
        with pytest.raises(FileNotFoundError):
            upload_file("nonexistent.txt")
    def test_directory_upload(self, tmp_path: Path) -> None:
        with pytest.raises(ValueError, match="is a directory"):
            upload_file(tmp_path)
    def test_no_read_permission(self, tmp_path: Path) -> None:
            mock_provider.upload_file.side_effect = PermissionError("Permission denied")
            with pytest.raises(PermissionError, match="Permission denied"):
    @pytest.mark.parametrize("size_mb", [1, 5, 10])
    def test_different_file_sizes(self, tmp_path: Path, size_mb: int) -> None:
        with test_file.open("wb") as f:
            f.write(b"0" * (size_mb * 1024 * 1024))
class TestCatboxProvider:
    def test_catbox_auth_with_userhash(self):
        provider = catbox.CatboxProvider()
    def test_catbox_auth_without_userhash(self):
    async def test_catbox_upload_file(self, tmp_path):
        mock_response = AsyncMock()
        mock_response.text = AsyncMock(
        mock_session = AsyncMock()
        mock_session.post = AsyncMock()
        with patch("aiohttp.ClientSession", return_value=mock_session):
            provider.async_upload_file = AsyncMock(
                return_value=UploadResult(
            result = await provider.async_upload_file(
            assert isinstance(result, UploadResult)
    async def test_catbox_upload_url(self):
class TestLitterboxProvider:
    def test_litterbox_default_expiration(self):
        provider = litterbox.LitterboxProvider()
    def test_litterbox_custom_expiration(self):
        provider = litterbox.LitterboxProvider(
    def test_litterbox_invalid_expiration(self):
        with pytest.raises(ValueError):
            litterbox.LitterboxProvider(default_expiration="invalid")
    async def test_litterbox_upload_file(self, tmp_path):
def test_circular_fallback(
    mock_s3_provider.upload_file.side_effect = RetryableError("S3 failed", "s3")
    mock_dropbox_provider.upload_file.side_effect = RetryableError(
        RetryableError("Catbox failed first", "catbox"),  # First try fails
def test_fragile_mode(
    with pytest.raises(NonRetryableError) as exc_info:
        upload_file(test_file, provider="s3", fragile=True)
    assert "S3 failed" in str(exc_info.value)
def test_custom_provider_list_circular_fallback(
    mock_catbox_provider.upload_file.side_effect = RetryableError(
        RetryableError("Dropbox failed first", "dropbox"),  # First try fails
    url = upload_file(test_file, provider=["catbox", "s3", "dropbox"])

================
File: plugins/repos/twat_fs/tests/test_utils.py
================
class TestCreateProviderHelp:
    def test_create_provider_help(self):
        result = create_provider_help(setup, deps)
        assert isinstance(result, dict)
class TestSafeFileHandle:
    def test_safe_file_handle_with_valid_file(self):
        with tempfile.NamedTemporaryFile(delete=False) as temp_file:
            temp_file.write(b"test content")
            with safe_file_handle(temp_path) as file:
                assert file.read() == b"test content"
            os.unlink(temp_path)
    def test_safe_file_handle_with_nonexistent_file(self):
        with pytest.raises(FileNotFoundError):
            with safe_file_handle("/path/to/nonexistent/file"):
    def test_safe_file_handle_with_directory(self):
        with tempfile.TemporaryDirectory() as temp_dir:
            with pytest.raises(ValueError):
                with safe_file_handle(temp_dir):
class TestValidateFile:
    def test_validate_file_with_valid_file(self):
            validate_file(Path(temp_path))
    def test_validate_file_with_nonexistent_file(self):
            validate_file(Path("/path/to/nonexistent/file"))
    def test_validate_file_with_directory(self):
                validate_file(Path(temp_dir))
    @mock.patch("os.access", return_value=False)
    def test_validate_file_with_unreadable_file(self, mock_access):
            with pytest.raises(PermissionError):
class TestHandleHttpResponse:
    def test_handle_http_response_with_200_requests(self):
        response = mock.Mock(spec=requests.Response)
        handle_http_response(response, "test_provider")
    def test_handle_http_response_with_200_aiohttp(self):
        response = mock.Mock(spec=aiohttp.ClientResponse)
    def test_handle_http_response_with_429_requests(self):
        with pytest.raises(RetryableError):
    def test_handle_http_response_with_429_aiohttp(self):
    def test_handle_http_response_with_503_requests(self):
    def test_handle_http_response_with_400_requests(self):
        with pytest.raises(NonRetryableError):
    def test_handle_http_response_with_other_error_requests(self):
class TestGetEnvCredentials:
    def test_get_env_credentials_with_all_required_vars(self):
        with mock.patch.dict(
            result = get_env_credentials(["TEST_VAR1", "TEST_VAR2"])
    def test_get_env_credentials_with_missing_required_vars(self):
        with mock.patch.dict(os.environ, {"TEST_VAR1": "value1"}):
    def test_get_env_credentials_with_optional_vars(self):
            result = get_env_credentials(
    def test_get_env_credentials_with_missing_optional_vars(self):
class TestCreateProviderInstance:
    def test_create_provider_instance_with_get_provider(self):
        mock_provider = mock.Mock(spec=Provider)
        mock_client = mock.Mock(spec=ProviderClient)
        result = create_provider_instance(mock_provider)
        mock_provider.get_provider.assert_called_once()
    def test_create_provider_instance_with_direct_instantiation(self):
        mock_provider.assert_called_once()
    def test_create_provider_instance_with_credentials(self):
        create_provider_instance(mock_provider, credentials)
        mock_provider.get_credentials.assert_not_called()
    def test_create_provider_instance_with_no_credentials(self):
        create_provider_instance(mock_provider)
        mock_provider.get_credentials.assert_called_once()
    def test_create_provider_instance_with_error(self):
        mock_provider.get_provider.side_effect = Exception("Test error")
class TestStandardUploadWrapper:
    def test_standard_upload_wrapper_with_valid_provider(self):
        mock_provider = mock.Mock(spec=ProviderClient)
        mock_provider.upload_file.return_value = UploadResult(
        result = standard_upload_wrapper(
        mock_provider.upload_file.assert_called_once_with(
    def test_standard_upload_wrapper_with_none_provider(self):
            standard_upload_wrapper(
class TestLogUploadAttempt:
    @mock.patch("loguru.logger.info")
    def test_log_upload_attempt_success(self, mock_logger_info):
        log_upload_attempt("test_provider", "test_file.txt", True)
        mock_logger_info.assert_called_once()
    @mock.patch("loguru.logger.error")
    def test_log_upload_attempt_failure(self, mock_logger_error):
        error = Exception("Test error")
        log_upload_attempt("test_provider", "test_file.txt", False, error)
        mock_logger_error.assert_called_once()

================
File: plugins/repos/twat_fs/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_fs/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_fs/CHANGELOG.md
================
---
this_file: CHANGELOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- Provider templates for standardized implementation:
  - `simple_provider_template.py`: Template for providers without authentication
  - `authenticated_provider_template.py`: Template for providers requiring credentials
- Factory pattern for provider instantiation to simplify provider creation and standardize error handling
- Comprehensive unit tests for utility functions
- Base classes for providers to reduce inheritance boilerplate
- Standardized logging patterns with `log_upload_attempt` function
- Improved error classification with `RetryableError` and `NonRetryableError` classes
- Centralized utilities in `utils.py` module for shared functionality across providers
- URL validation to ensure returned URLs are accessible before returning them
- Standardized async/sync conversion patterns with `to_sync` and `to_async` decorators

### Changed
- Fixed `create_provider_instance` function in `utils.py` to correctly handle credential management
- Fixed `gather_with_concurrency` function in `async_utils.py` to properly handle coroutines
- Refactored all upload providers to use shared utilities from `utils.py`:
  - `pixeldrain.py`: Fixed recursion issues, improved error handling, standardized logging
  - `bashupload.py`: Implemented consistent error handling, standardized logging
  - `catbox.py`: Separated upload logic, standardized provider help format
  - `filebin.py`: Improved error handling, standardized logging
  - `uguu.py`: Separated upload logic, standardized provider help format
  - `www0x0.py`: Separated upload logic, standardized provider help format
  - `dropbox.py`: Made DropboxClient inherit from BaseProvider, standardized provider help format
  - `s3.py`: Created S3Provider class inheriting from BaseProvider, improved error handling
  - `fal.py`: Made FalProvider inherit from BaseProvider, improved error handling and credential management
  - `litterbox.py`: Made LitterboxProvider inherit from BaseProvider, improved error handling with special handling for expiration parameter
- Standardized async/sync conversion patterns for providers that support both operations
- Enhanced type hints for better IDE support and runtime type checking
- Improved URL validation to ensure returned URLs are accessible before returning them

### Fixed
- Fixed test failures in `TestCreateProviderInstance` test class
- Fixed test failures in `TestGatherWithConcurrency` test class
- Improved error handling and logging across all providers
- Fixed recursion issues in pixeldrain provider
- Standardized provider help format for better consistency

## In Progress

- Fixing type annotation issues identified by linter
  - Addressing incompatible return types in async methods
  - Fixing type mismatches in factory.py and simple.py
  - Ensuring proper typing for async/await conversions
  - Resolving "possibly unbound" variable warnings in upload.py

- Fixing remaining failing unit tests
  - TestLogUploadAttempt.test_log_upload_attempt_success test failing because logger.info is not being called
  - TestGatherWithConcurrency.test_gather_with_concurrency_with_exceptions test failing with RuntimeError instead of ValueError

- Addressing boolean argument issues
  - Converting boolean positional arguments to keyword-only arguments
  - Fixing FBT001/FBT002 linter errors in function definitions
  - Fixing FBT003 linter errors in function calls

- Addressing linter issues in cleanup.py
  - DTZ005: datetime.datetime.now() called without a tz argument
  - S603/S607: Subprocess call security issues

## Development Log

### Completed

- Fixed `create_provider_instance` function in `utils.py` to correctly handle credential management:
  - Properly calls `get_credentials` when no credentials are provided
  - Ensures correct order of operations for provider instantiation
  - Fixed test failures in `TestCreateProviderInstance` test class
  - Improved error handling and logging

- Refactored all upload providers to use shared utilities from `utils.py`:
  - `pixeldrain.py`: Fixed recursion issues, improved error handling, standardized logging
  - `bashupload.py`: Implemented consistent error handling, standardized logging
  - `catbox.py`: Separated upload logic, standardized provider help format
  - `filebin.py`: Improved error handling, standardized logging
  - `uguu.py`: Separated upload logic, standardized provider help format
  - `www0x0.py`: Separated upload logic, standardized provider help format
  - `dropbox.py`: Made DropboxClient inherit from BaseProvider, standardized provider help format
  - `s3.py`: Created S3Provider class inheriting from BaseProvider, improved error handling
  - `fal.py`: Made FalProvider inherit from BaseProvider, improved error handling and credential management
  - `litterbox.py`: Made LitterboxProvider inherit from BaseProvider, improved error handling with special handling for expiration parameter

- Created provider templates for standardized implementation:
  - `simple_provider_template.py`: Template for providers without authentication
  - `authenticated_provider_template.py`: Template for providers requiring credentials

- Updated `catbox.py` and `bashupload.py` to properly implement both `Provider` and `ProviderClient` protocols
  - Improved error handling with `RetryableError` and `NonRetryableError`
  - Added standardized logging with `log_upload_attempt`
  - Enhanced type hints for better IDE support

- Implemented a factory pattern for provider instantiation to simplify provider creation and standardize error handling during initialization

- Standardized async/sync conversion patterns for providers that support both operations

- Created comprehensive unit tests for utility functions

### Next Steps

- Fix missing dependencies for tests
  - Install missing dependencies for tests or implement proper test skipping
  - Address ModuleNotFoundError for 'responses', 'fal_client', 'botocore'

- Fix boolean argument issues
  - Convert boolean positional arguments to keyword-only arguments
  - Fix FBT001/FBT002 linter errors in function definitions
  - Fix FBT003 linter errors in function calls

- Fix type annotation issues
  - Address incompatible return types in async methods
  - Fix type mismatches in factory.py and simple.py
  - Add missing type annotations for variables

- Fix exception handling issues
  - Implement proper exception chaining with `raise ... from err`
  - Replace assert statements with proper error handling

- Fix function complexity issues
  - Refactor functions with too many arguments
  - Simplify complex functions with too many branches/statements/returns

### Technical Debt

- Update provider implementations to match new protocol type hints
- Standardize error handling across all providers
- Improve documentation for adding new providers
- Fix linter warnings related to boolean arguments and function complexity
- Refactor complex functions in cli.py and upload.py

================
File: plugins/repos/twat_fs/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_fs/IDEAS.md
================
Below is a set of ideas for additional services you could integrate as new upload providers. Each of these services offers its own API for file uploads (and sometimes transformations), and by following the established provider protocols and using your shared utilities (e.g. for file validation, error handling, and async-to-sync conversions), integration can be relatively straightforward.

### 0.1. Proposed Additional Upload Providers

- **Transfer.sh**  
  A free, lightweight file hosting service with a very simple REST API. Its API is based on standard HTTP POST requests, much like filebin or pixeldrain. It’s ideal for temporary or quick file sharing.

- **AnonFiles**  
  Another free file hosting service with an easy-to-use API. AnonFiles returns a direct download link upon successful upload, which makes it a good candidate for integration as a fallback provider.

- **Filestack**  
  A robust solution that not only handles file uploads but also offers transformation (e.g. resizing, format conversion) and delivery via its CDN. Although it’s a paid service, its extensive SDKs and API make it a strong choice if you want to offer advanced features.

- **Uploadcare**  
  Similar to Filestack, Uploadcare offers a complete file pipeline with uploading, storage, and on‑the‑fly transformation features. Integrating it would add a high level of flexibility and reliability.

- **Google Drive**  
  While its API (using OAuth 2.0) can be more complex, Google Drive is widely used and offers robust file storage. A basic integration could allow users to upload files directly to Google Drive as part of a broader fallback strategy.

- **OneDrive**  
  Microsoft’s cloud storage service also provides a REST API for file management. OneDrive’s integration would appeal to business users within the Microsoft ecosystem, complementing Dropbox and S3 in your provider mix.

- **pCloud**  
  pCloud provides both free and premium cloud storage options with a clear API for file uploads. Its service can be integrated using similar patterns to other cloud providers.

- **Backblaze B2**  
  This is a cost-effective, S3‑compatible cloud storage solution. Given its S3 API compatibility, integration could reuse much of your S3 provider’s logic, making it a natural extension for your fallback system.

- **MediaFire**  
  MediaFire is a well‑known file hosting service with API support for file uploads. Its service may be particularly appealing if you wish to offer an alternative to the more technical cloud providers.

- **Cloudinary**  
  While Cloudinary is primarily focused on images and videos, it also supports file uploads along with advanced media processing and delivery features. This integration would be especially useful if you want to provide on‑the‑fly media transformations in your app.

### 0.2. Integration Approach

For each new provider, you would:
  
- **Implement the Provider Protocol:**  
  Create a new module in `src/twat_fs/upload_providers/` that implements the `Provider` and `ProviderClient` protocols. Define methods such as `get_credentials()`, `get_provider()`, and `upload_file()` (and optionally `async_upload_file()`).

- **Leverage Shared Utilities:**  
  Use your existing utilities (e.g., in `utils.py` for file validation and HTTP response handling) to ensure consistency with existing providers.

- **Add to Provider Preference List:**  
  Once implemented, add the new provider’s identifier to the `PROVIDERS_PREFERENCE` list so that it’s available in fallback scenarios.

By following these steps, you can expand your upload functionality to support a broader range of file hosting services, providing flexibility to your users and increasing the robustness of your fallback system.

Would you like to see a detailed example or further discussion on one of these proposals?






# Proposing Additional Upload Service Integrations

Based on the codebase analysis, I'll propose several additional services that could be easily integrated with the "upload" functionality. The current architecture is well-designed for extensibility, with a clear provider protocol and shared utilities.

### 0.3. Current Upload Provider Architecture

The project already supports multiple upload providers:
- Simple file hosts (catbox.moe, litterbox, www0x0.st, uguu.se, bashupload.com, filebin.net, pixeldrain.com)
- Cloud storage (Dropbox, AWS S3)
- AI platforms (FAL.ai)

The architecture includes:
- A provider protocol that defines the interface for all upload providers
- Shared utilities for HTTP requests, error handling, and file validation
- A fallback system that tries alternative providers if one fails

## 1. Proposed New Upload Services

### 1.1. Google Drive

**Integration Approach:**
- Use the Google Drive API with OAuth2 authentication
- Support both simple file uploads and folder organization
- Implement public sharing links with configurable permissions

**Implementation Details:**
```python
# src/twat_fs/upload_providers/gdrive.py
from pathlib import Path
from typing import Any, cast
import os
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "Requires Google Drive API credentials. Set GOOGLE_CREDENTIALS_FILE environment variable.",
    "deps": "google-api-python-client google-auth-httplib2 google-auth-oauthlib"
}

class GoogleDriveProvider(Provider, ProviderClient):
    """Provider for Google Drive uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "gdrive"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get Google Drive credentials from environment."""
        creds_file = os.getenv("GOOGLE_CREDENTIALS_FILE")
        if not creds_file:
            return None
            
        # Implement OAuth2 flow and token management
        # Return credentials dictionary
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload file to Google Drive and return shareable link."""
        # Implementation using Google Drive API
```

### 1.2. GitHub Gist/Repository

**Integration Approach:**
- Use GitHub API to create gists (for small text files) or repository files
- Support both public and private uploads with configurable visibility
- Generate direct links to raw content

**Implementation Details:**
```python
# src/twat_fs/upload_providers/github.py
import os
import base64
from pathlib import Path
import requests
from typing import Any, cast

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "Requires GitHub Personal Access Token. Set GITHUB_TOKEN environment variable.",
    "deps": "requests"
}

class GitHubProvider(Provider, ProviderClient):
    """Provider for GitHub Gist/Repository uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "github"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get GitHub credentials from environment."""
        token = os.getenv("GITHUB_TOKEN")
        if not token:
            return None
        return {"token": token}
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload file to GitHub and return URL."""
        # Implementation using GitHub API for gists or repository files
```

### 1.3. Imgur

**Integration Approach:**
- Use Imgur API for image uploads
- Support both authenticated and anonymous uploads
- Provide direct image links and album organization

**Implementation Details:**
```python
# src/twat_fs/upload_providers/imgur.py
import os
import base64
from pathlib import Path
import requests
from typing import Any, cast

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "For authenticated uploads, set IMGUR_CLIENT_ID and IMGUR_CLIENT_SECRET environment variables.",
    "deps": "requests"
}

class ImgurProvider(Provider, ProviderClient):
    """Provider for Imgur image uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "imgur"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get Imgur credentials from environment."""
        client_id = os.getenv("IMGUR_CLIENT_ID")
        if not client_id:
            return None
        return {
            "client_id": client_id,
            "client_secret": os.getenv("IMGUR_CLIENT_SECRET")
        }
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload image to Imgur and return URL."""
        # Implementation using Imgur API
```

### 1.4. Azure Blob Storage

**Integration Approach:**
- Use Azure Storage SDK for blob uploads
- Support container management and access policies
- Generate SAS tokens for time-limited access

**Implementation Details:**
```python
# src/twat_fs/upload_providers/azure.py
import os
from pathlib import Path
from typing import Any, cast
from azure.storage.blob import BlobServiceClient, ContentSettings

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "Requires Azure Storage connection string. Set AZURE_STORAGE_CONNECTION_STRING and AZURE_CONTAINER_NAME environment variables.",
    "deps": "azure-storage-blob"
}

class AzureBlobProvider(Provider, ProviderClient):
    """Provider for Azure Blob Storage uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "azure"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get Azure credentials from environment."""
        conn_string = os.getenv("AZURE_STORAGE_CONNECTION_STRING")
        container = os.getenv("AZURE_CONTAINER_NAME")
        if not conn_string or not container:
            return None
        return {
            "connection_string": conn_string,
            "container": container
        }
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload file to Azure Blob Storage and return URL."""
        # Implementation using Azure Storage SDK
```

### 1.5. Cloudinary

**Integration Approach:**
- Use Cloudinary API for media uploads with transformation capabilities
- Support image and video optimization
- Provide CDN-backed URLs with transformation options

**Implementation Details:**
```python
# src/twat_fs/upload_providers/cloudinary.py
import os
from pathlib import Path
from typing import Any, cast
import cloudinary
import cloudinary.uploader

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "Requires Cloudinary credentials. Set CLOUDINARY_CLOUD_NAME, CLOUDINARY_API_KEY, and CLOUDINARY_API_SECRET environment variables.",
    "deps": "cloudinary"
}

class CloudinaryProvider(Provider, ProviderClient):
    """Provider for Cloudinary media uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "cloudinary"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get Cloudinary credentials from environment."""
        cloud_name = os.getenv("CLOUDINARY_CLOUD_NAME")
        api_key = os.getenv("CLOUDINARY_API_KEY")
        api_secret = os.getenv("CLOUDINARY_API_SECRET")
        
        if not all([cloud_name, api_key, api_secret]):
            return None
            
        return {
            "cloud_name": cloud_name,
            "api_key": api_key,
            "api_secret": api_secret
        }
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload media to Cloudinary and return URL."""
        # Implementation using Cloudinary API
```

### 1.6. Backblaze B2

**Integration Approach:**
- Use B2 SDK for object storage uploads
- Support bucket management and lifecycle policies
- Generate authorized URLs with expiration

**Implementation Details:**
```python
# src/twat_fs/upload_providers/b2.py
import os
from pathlib import Path
from typing import Any, cast
import b2sdk.v1 as b2

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "Requires Backblaze B2 credentials. Set B2_APPLICATION_KEY_ID, B2_APPLICATION_KEY, and B2_BUCKET_NAME environment variables.",
    "deps": "b2sdk"
}

class B2Provider(Provider, ProviderClient):
    """Provider for Backblaze B2 uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "b2"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get B2 credentials from environment."""
        key_id = os.getenv("B2_APPLICATION_KEY_ID")
        app_key = os.getenv("B2_APPLICATION_KEY")
        bucket = os.getenv("B2_BUCKET_NAME")
        
        if not all([key_id, app_key, bucket]):
            return None
            
        return {
            "key_id": key_id,
            "app_key": app_key,
            "bucket": bucket
        }
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload file to B2 and return URL."""
        # Implementation using B2 SDK
```

### 1.7. Pastebin/Hastebin

**Integration Approach:**
- Use Pastebin/Hastebin APIs for text file uploads
- Support syntax highlighting and expiration settings
- Generate direct links to raw content

**Implementation Details:**
```python
# src/twat_fs/upload_providers/pastebin.py
import os
from pathlib import Path
import requests
from typing import Any, cast

from twat_fs.upload_providers.protocols import Provider, ProviderClient
from twat_fs.upload_providers.types import UploadResult
from twat_fs.upload_providers.utils import validate_file, log_upload_attempt

PROVIDER_HELP = {
    "setup": "For authenticated uploads, set PASTEBIN_API_KEY environment variable.",
    "deps": "requests"
}

class PastebinProvider(Provider, ProviderClient):
    """Provider for Pastebin text uploads"""
    
    PROVIDER_HELP = PROVIDER_HELP
    provider_name = "pastebin"
    
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None:
        """Get Pastebin credentials from environment."""
        api_key = os.getenv("PASTEBIN_API_KEY")
        if not api_key:
            return None
        return {"api_key": api_key}
    
    @classmethod
    def get_provider(cls) -> ProviderClient | None:
        """Get an instance of the provider."""
        creds = cls.get_credentials()
        if not creds:
            return None
        return cls(creds)
        
    def upload_file(self, local_path: str | Path, remote_path: str | Path | None = None, **kwargs) -> UploadResult:
        """Upload text to Pastebin and return URL."""
        # Implementation using Pastebin API
```

## 2. Implementation Strategy

To integrate these new providers:

1. **Create provider modules** following the established pattern
2. **Implement the Provider protocol** with required methods:
   - `get_credentials()` - Fetch credentials from environment variables
   - `get_provider()` - Initialize and return provider client
   - `upload_file()` - Handle file uploads and return URLs

3. **Add to PROVIDERS_PREFERENCE** in `src/twat_fs/upload_providers/__init__.py`
4. **Add dependency options** to `pyproject.toml` for each provider
5. **Create tests** for each new provider

### 2.1. Key Benefits of These Additions

1. **Diverse Storage Options**: From ephemeral pastes to permanent cloud storage
2. **Media Specialization**: Providers optimized for specific content types (images, code, documents)
3. **Enterprise Integration**: Support for major cloud platforms (Azure, Google)
4. **Cost Flexibility**: Mix of free and paid services with different pricing models
5. **Feature Diversity**: From simple file hosting to advanced media transformation

These additions would significantly expand the utility of the upload functionality while maintaining the existing architecture's flexibility and robustness.




### 2.2. Key Points
- It seems likely that Google Drive, SendSpace, and GitHub can be easily integrated into the upload functionality, given their APIs and existing Python libraries.
- Research suggests Transfer.sh, File.io, and Anonfile are also viable, offering simple APIs for temporary file hosting.
- The evidence leans toward Microsoft OneDrive, Box.com, MediaFire, pCloud, GitLab, Bitbucket, and IPFS as additional options, though they may require more setup.

---

### 2.3. Direct Answer

#### 2.3.1. Introduction
The "twat-fs" package currently supports file uploads to services like Dropbox, Amazon S3, and several file hosting platforms. To expand this, you can integrate additional services that offer APIs for file uploads, providing more options for users. These new services should be easy to integrate, have reliable APIs, and cater to diverse needs like cloud storage, temporary hosting, or version control.

#### 2.3.2. Proposed Services
Here are some services that can be easily integrated into the upload functionality:

- **Google Drive**: A popular cloud storage service with a robust API ([Google Drive API](https://developers.google.com/drive)), ideal for long-term storage and collaboration, though it requires setting up credentials.
- **SendSpace**: A file hosting service with a simple API ([SendSpace Developer](https://sendspace.com/developer)), perfect for quick uploads and direct download links, needing only an API key.
- **GitHub**: Useful for hosting files in repositories, with an API for creating files ([GitHub API](https://docs.github.com/en/rest)), suitable for text files or small binaries, requiring a personal access token.
- **Transfer.sh**: Offers temporary file hosting with a straightforward API ([Transfer.sh](https://transfer.sh/)), easy to integrate for short-term sharing.
- **File.io**: Provides file hosting with direct download links via API ([File.io Docs](https://file.io/docs)), simple and user-friendly for uploads.
- **Anonfile**: Another file hosting service with API support ([Anonfile Docs](https://anonfile.com/docs)), reliable for hosting files with direct access.

These services cover a range of use cases, from robust cloud storage to temporary file sharing, enhancing the package's versatility. An unexpected detail is that GitHub, primarily for code, can also host files, offering version control benefits.

#### 2.3.3. Considerations
Some services, like Google Drive and OneDrive, may require more setup for authentication, while simpler providers like Transfer.sh and File.io align with existing "simple" providers in the package. Users can choose based on their needs, such as storage space, privacy, or ease of use.

---

### 2.4. Survey Note: Detailed Analysis of Proposed Upload Services

This section provides a comprehensive analysis of potential services for integration into the "twat-fs" package's upload functionality, expanding on the direct answer with detailed reasoning and evaluation. The analysis is structured to cover the selection process, categorization, and justification, ensuring a thorough understanding for developers and users.

#### 2.4.1. Background and Context
The "twat-fs" package, as observed from the provided repository, is a file system utility focused on robust and extensible file upload capabilities, supporting multiple providers such as Dropbox, Amazon S3, and various file hosting services like catbox.moe and filebin.net. The current providers, listed in the `PROVIDERS_PREFERENCE` in `src/twat_fs/upload_providers/__init__.py`, include bashupload.com, catbox.moe, Dropbox, FAL.ai, filebin.net, litterbox.catbox.moe, pixeldrain.com, Amazon S3, uguu.se, and 0x0.st. These providers follow a pattern where each has a module implementing the `ProviderClient` protocol, with methods like `upload_file` and possibly `async_upload_file`, often inheriting from `BaseProvider` for simple providers.

The task is to propose additional services that can be easily integrated, meaning they should have accessible APIs, preferably with Python libraries, and minimal setup complexity to align with the existing structure. The analysis considers services that are reliable, have good uptime, and cater to diverse user needs, such as cloud storage, temporary hosting, or version-controlled file management.

#### 2.4.2. Selection Process
The selection process involved identifying popular file hosting and storage services, evaluating their APIs for ease of integration, and ensuring they are not already covered by the existing providers. The process considered:

- **API Availability**: Services must offer APIs for file uploads, preferably with Python libraries or straightforward HTTP requests.
- **Setup Complexity**: Preference for services with minimal authentication requirements, similar to simple providers, or those with clear credential setup like Dropbox and S3.
- **User Familiarity**: Services widely used by developers and general users for file storage and sharing.
- **Unique Features**: Services offering distinct benefits, such as encryption, version control, or temporary hosting.

The initial list included cloud storage services (Google Drive, Microsoft OneDrive, Box.com), file hosting services (SendSpace, MediaFire, pCloud, Transfer.sh, File.io, Anonfile, Bayfiles), version control platforms (GitHub, GitLab, Bitbucket), and decentralized storage (IPFS). After evaluation, the focus was narrowed to balance simplicity and diversity, resulting in the proposed list.

#### 2.4.3. Proposed Services and Categorization
The proposed services are categorized into three main groups: cloud storage services, file hosting services, and version control platforms, with a note on decentralized storage for future consideration. Below is a detailed breakdown:

| **Category**            | **Service**       | **Description**                                                                 | **API Details**                                                                 | **Ease of Integration**                     |
|-------------------------|-------------------|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------|---------------------------------------------|
| Cloud Storage Services  | Google Drive      | Popular cloud storage with collaboration features, suitable for long-term use.  | Requires Google Drive API ([Google Drive API](https://developers.google.com/drive)), needs credentials setup. | Moderate, involves authentication flow.     |
| Cloud Storage Services  | Microsoft OneDrive| Cloud storage from Microsoft, similar to Google Drive, with sharing options.    | Uses Microsoft Graph API, requires credentials ([Microsoft Graph](https://docs.microsoft.com/en-us/graph/)). | Moderate, authentication setup needed.      |
| Cloud Storage Services  | Box.com           | Cloud storage service like Dropbox, with API for file uploads.                  | Box SDK available ([Box Developer](https://developer.box.com/)), needs credentials. | Moderate, similar to Dropbox.               |
| File Hosting Services   | SendSpace         | Dedicated file hosting, provides direct download links.                         | API requires key ([SendSpace Developer](https://sendspace.com/developer)), simple HTTP requests. | High, minimal setup, API key based.         |
| File Hosting Services   | MediaFire         | File hosting with API for uploads, offers download links.                       | Requires API key and secret ([MediaFire Docs](https://www.mediafire.com/docs/)). | High, straightforward with credentials.     |
| File Hosting Services   | pCloud            | Cloud storage with API, focuses on privacy.                                     | Python library available ([pCloud SDK](https://www.pcloud.com/developers/)), needs credentials. | Moderate, authentication required.          |
| File Hosting Services   | Transfer.sh       | Temporary file hosting, simple API for quick uploads.                           | HTTP POST based ([Transfer.sh](https://transfer.sh/)), no authentication needed. | Very high, aligns with simple providers.    |
| File Hosting Services   | File.io           | File hosting with direct download links, API supported.                         | Simple API ([File.io Docs](https://file.io/docs)), minimal setup.               | Very high, easy integration.                |
| File Hosting Services   | Anonfile          | File hosting service, provides API for uploads and links.                       | API documented ([Anonfile Docs](https://anonfile.com/docs)), simple HTTP.       | Very high, similar to existing simple providers. |
| Version Control Platforms | GitHub          | Hosts files in repositories, offers version control, raw URLs for access.       | Uses GitHub API for file creation ([GitHub API](https://docs.github.com/en/rest)), needs token. | Moderate, requires repository and token.    |
| Version Control Platforms | GitLab          | Similar to GitHub, hosts files with version control.                            | API for file uploads ([GitLab API](https://docs.gitlab.com/ee/api/)), needs token. | Moderate, similar to GitHub.                |
| Version Control Platforms | Bitbucket       | Version control platform, can host files in repositories.                       | API for file operations ([Bitbucket API](https://developer.atlassian.com/bitbucket/api/2/reference/)), needs credentials. | Moderate, requires setup.                   |
| Decentralized Storage   | IPFS             | Decentralized file system, files pinned to nodes for access.                    | Requires IPFS client, complex setup ([IPFS Docs](https://docs.ipfs.io/)).       | Low, more advanced, not user-friendly.      |

#### 2.4.4. Detailed Justification
The proposed services were selected based on their alignment with the existing provider structure and user needs. Here's a detailed justification for each category:

- **Cloud Storage Services (Google Drive, Microsoft OneDrive, Box.com)**: These services are widely used for long-term file storage and collaboration. They require authentication, which aligns with existing providers like Dropbox and S3, but offer additional features like file sharing permissions. Google Drive, for instance, has a free tier with 15GB, making it accessible, while OneDrive and Box.com cater to enterprise users. Integration involves setting up credentials, which is documented in their respective APIs ([Google Drive API](https://developers.google.com/drive), [Microsoft Graph](https://docs.microsoft.com/en-us/graph/), [Box Developer](https://developer.box.com/)).

- **File Hosting Services (SendSpace, MediaFire, pCloud, Transfer.sh, File.io, Anonfile)**: These services focus on providing direct download links, similar to existing simple providers like filebin.net and www0x0.st. Transfer.sh, File.io, and Anonfile are particularly easy to integrate, requiring minimal or no authentication, aligning with the package's "simple" provider pattern. SendSpace and MediaFire require API keys, which is manageable, while pCloud offers privacy-focused storage. Their APIs are documented ([SendSpace Developer](https://sendspace.com/developer), [MediaFire Docs](https://www.mediafire.com/docs/), [pCloud SDK](https://www.pcloud.com/developers/), [Transfer.sh](https://transfer.sh/), [File.io Docs](https://file.io/docs), [Anonfile Docs](https://anonfile.com/docs)), ensuring straightforward implementation.

- **Version Control Platforms (GitHub, GitLab, Bitbucket)**: These platforms, while primarily for code, can host files in repositories, offering version control benefits. GitHub, for example, allows unlimited storage for public repositories, with raw URLs for direct access, making it suitable for text files or small binaries. Integration involves using their APIs for file creation ([GitHub API](https://docs.github.com/en/rest), [GitLab API](https://docs.gitlab.com/ee/api/), [Bitbucket API](https://developer.atlassian.com/bitbucket/api/2/reference/)), requiring personal access tokens, which is similar to Dropbox's setup.

- **Decentralized Storage (IPFS)**: IPFS offers a unique decentralized approach, but its integration is more complex due to pinning files to nodes and ensuring availability. While interesting for future expansion, it's less user-friendly and not prioritized for easy integration ([IPFS Docs](https://docs.ipfs.io/)).

#### 2.4.5. Prioritization and Recommendations
Given the need for easy integration, the top recommendations are:
- **Primary Recommendations**: Google Drive, SendSpace, GitHub, Transfer.sh, File.io. These cover cloud storage, simple hosting, and version control, with varying setup complexities.
- **Secondary Recommendations**: Microsoft OneDrive, Box.com, MediaFire, pCloud, Anonfile, GitLab, Bitbucket. These offer additional options but may require more setup or have specific use cases.

An unexpected detail is the inclusion of GitHub, primarily a code platform, which can host files, offering version control benefits not typically associated with file hosting services. This expands the package's scope to include repository-based file management.

#### 2.4.6. Implementation Considerations
For integration, new provider modules would be created in `src/twat_fs/upload_providers/`, following the existing pattern. Simple providers like Transfer.sh and File.io would inherit from `BaseProvider` and implement `upload_file_impl`, while cloud storage services like Google Drive would require handling authentication and possibly async operations. The package's existing structure, with environment variable-based credentials and utility functions in `utils.py`, supports this expansion.

#### 2.4.7. Conclusion
The proposed services enhance the "twat-fs" package by offering diverse options for file uploads, from temporary hosting to robust cloud storage and version-controlled repositories. Users can select based on their needs, such as storage space, privacy, or ease of use, ensuring the package remains versatile and user-friendly.

---

### 2.5. Key Citations
- [Google Drive API Documentation](https://developers.google.com/drive)
- [SendSpace Developer Portal](https://sendspace.com/developer)
- [GitHub REST API Documentation](https://docs.github.com/en/rest)
- [Transfer.sh Official Website](https://transfer.sh/)
- [File.io API Documentation](https://file.io/docs)
- [Anonfile API Documentation](https://anonfile.com/docs)
- [Microsoft Graph API Documentation](https://docs.microsoft.com/en-us/graph/)
- [Box Developer Documentation](https://developer.box.com/)
- [MediaFire API Documentation](https://www.mediafire.com/docs/)
- [pCloud SDK Documentation](https://www.pcloud.com/developers/)
- [GitLab API Documentation](https://docs.gitlab.com/ee/api/)
- [Bitbucket API Reference](https://developer.atlassian.com/bitbucket/api/2/reference/)
- [IPFS Official Documentation](https://docs.ipfs.io/)








# Expanding File Upload Capabilities in twat-fs: Strategic Integration of Modern Storage Services

The twat-fs package demonstrates an elegant abstraction layer for multi-provider file uploads, with existing implementations for services like Dropbox and AWS S3[1]. This technical analysis proposes twelve new service integrations across six categories, examining implementation requirements and architectural considerations for each.

## 3. Cloud Storage Expansion

### 3.1. Google Drive Integration
Google Drive's REST API supports OAuth2 authentication and resumable uploads. Implementation would require:
```python
# google_drive.py
PROVIDER_HELP = {
    "setup": "Requires OAuth2 credentials: GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET, GOOGLE_REFRESH_TOKEN",
    "deps": "google-auth, google-api-python-client"
}

class GoogleDriveProvider(ProviderClient):
    def __init__(self):
        from google.oauth2.credentials import Credentials
        self.service = build('drive', 'v3', credentials=Credentials(
            token=os.getenv('GOOGLE_ACCESS_TOKEN'),
            refresh_token=os.getenv('GOOGLE_REFRESH_TOKEN'),
            client_id=os.getenv('GOOGLE_CLIENT_ID'),
            client_secret=os.getenv('GOOGLE_CLIENT_SECRET'),
            token_uri='https://oauth2.googleapis.com/token'
        ))
    
    async def upload_file(self, path: Path) -> str:
        media = MediaFileUpload(path, resumable=True)
        file = self.service.files().create(
            media_body=media, 
            fields='webViewLink'
        ).execute()
        return file['webViewLink']
```
Key considerations include OAuth token refresh handling and supporting Google Workspace domain restrictions[1].

### 3.2. Backblaze B2 Implementation
Backblaze's S3-compatible API allows reuse of existing S3 provider logic with endpoint configuration:
```bash
# Environment variables
export AWS_ENDPOINT_URL=https://s3.us-west-002.backblazeb2.com
export AWS_S3_BUCKET=your-bucket
export AWS_ACCESS_KEY_ID=002yourkey
export AWS_SECRET_ACCESS_KEY=yourSecretKey
```
This compatibility reduces implementation effort while adding Backblaze-specific error handling for rate limits[1].

## 4. Developer Platform Integrations

### 4.1. GitHub Gists API
Text file sharing through GitHub's Gist API:
```python
# github_gist.py
PROVIDER_HELP = {
    "setup": "Requires GITHUB_TOKEN with gist scope",
    "deps": "PyGithub"
}

async def upload_file(path: Path) -> str:
    gist = github.Github(os.getenv('GITHUB_TOKEN')).get_user().create_gist(
        public=False, 
        files={path.name: github.InputFileContent(path.read_text())}
    )
    return next(iter(gist.files.values())).raw_url
```
Handles text files under 10MB with automatic gist management[1].

### 4.2. GitLab Snippet Support
Similar to GitHub but with self-hosted instance support:
```python
base_url = os.getenv('GITLAB_URL', 'https://gitlab.com')
async with aiohttp.ClientSession(base_url) as session:
    await session.post(
        '/api/v4/snippets',
        headers={'PRIVATE-TOKEN': os.getenv('GITLAB_TOKEN')},
        data={'files[][content]': path.read_text()}
    )
```
Supports enterprise deployments through environment configuration[1].

## 5. Image Optimization Services

### 5.1. Imgur API Integration
Specialized image hosting with compression:
```python
# imgur.py
async def upload_image(path: Path) -> str:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            'https://api.imgur.com/3/image',
            headers={'Authorization': f'Client-ID {os.getenv("IMGUR_CLIENT_ID")}'},
            data={'image': path.read_bytes()}
        ) as resp:
            return (await resp.json())['data']['link']
```
Requires handling Imgur's specific API limits (1,250 uploads/day)[1].

### 5.2. Cloudinary Transformation
Cloud-based image processing with upload:
```python
# cloudinary.py
params = {
    'api_key': os.getenv('CLOUDINARY_KEY'),
    'timestamp': int(time.time()),
    'eager': 'c_thumb,g_face,w_200'
}
signature = hashlib.sha256(f'{params}{os.getenv("CLOUDINARY_SECRET")}').hexdigest()
```
Enables on-the-fly image transformations during upload[1].

## 6. Enterprise File Transfer

### 6.1. Aspera Faspex Integration
High-speed transfer protocol implementation:
```python
# aspera.py
async def upload_large_file(path: Path):
    proc = await asyncio.subprocess.create_subprocess_exec(
        'ascp', 
        '-QT', '-l100m', 
        path, f'{os.getenv("ASPERA_USER")}@aspera.example.com:/uploads'
    )
    await proc.wait()
```
Requires Aspera Connect CLI tools and special error handling for partial transfers[1].

### 6.2. Signiant Accelerator
Enterprise-grade transfer protocol support:
```python
class SigniantProvider(ProviderClient):
    def __init__(self):
        self.job_api = SigniantJobAPI(
            os.getenv('SIGNIANT_KEY'),
            os.getenv('SIGNIANT_SECRET')
        )
    
    async def upload_file(self, path: Path):
        job = self.job_api.create_job(
            source=path,
            destination='signiant://target/path'
        )
        return job.monitor().get_url()
```
Implements job monitoring and bandwidth optimization[1].

## 7. Decentralized Storage Options

### 7.1. IPFS via Pinata Cloud
Distributed storage with persistence guarantees:
```python
# ipfs.py
async def pin_file(path: Path) -> str:
    async with aiohttp.ClientSession() as session:
        async with session.post(
            'https://api.pinata.cloud/pinning/pinFileToIPFS',
            headers={'pinata_api_key': os.getenv('PINATA_KEY'),
                     'pinata_secret_api_key': os.getenv('PINATA_SECRET')},
            data={'file': path.open('rb')}
        ) as resp:
            return f"ipfs://{(await resp.json())['IpfsHash']}"
```
Supports both public gateways and private IPFS clusters[1].

### 7.2. Storj Decentralized S3
Blockchain-based storage using S3 compatibility:
```bash
export AWS_ENDPOINT_URL=https://gateway.storjshare.io
export AWS_S3_PATH_STYLE=true
```
Leverages existing S3 provider with custom endpoint configuration[1].

## 8. Specialized Hosting Services

### 8.1. npm Package Publishing
Developer-centric package hosting:
```python
# npm.py
async def publish_file(path: Path):
    proc = await asyncio.subprocess.create_subprocess_exec(
        'npm', 'publish', path,
        env={'NPM_TOKEN': os.getenv('NPM_TOKEN')}
    )
    await proc.wait()
    return f'https://npmjs.com/package/{path.stem}'
```
Requires strict adherence to npm package format specifications[1].

### 8.2. WeTransfer API
User-friendly file sharing implementation:
```python
# wetransfer.py
async def create_transfer(path: Path) -> str:
    async with aiohttp.ClientSession() as session:
        transfer = await session.post(
            'https://api.wetransfer.com/v2/transfers',
            headers={'x-api-key': os.getenv('WETRANSFER_KEY')},
            json={'name': path.name, 'files': [{'name': path.name}]}
        )
        upload_url = (await transfer.json())['files'][0]['upload_url']
        await session.put(upload_url, data=path.read_bytes())
        return (await transfer.json())['url']
```
Implements multi-step upload process with expiration handling[1].

## 9. Implementation Strategy

### 9.1. Protocol Compliance
All new providers must implement the core Provider protocol:
```python
class Provider(Protocol):
    @classmethod
    def get_credentials(cls) -> dict[str, Any] | None: ...
    
    @classmethod 
    def get_provider(cls) -> ProviderClient | None: ...

class ProviderClient(Protocol):
    async def async_upload_file(self, path: Path) -> str: ...
    def upload_file(self, path: Path) -> str: ...
```
This ensures compatibility with existing retry and fallback mechanisms[1].

### 9.2. Error Handling Matrix
Service-specific exception mapping:

| Service         | Retryable Errors                     | Fatal Errors                     |
|-----------------|--------------------------------------|----------------------------------|
| Google Drive    | Rate limits, 5xx errors             | Quota exceeded, Auth failures   |
| Backblaze B2    | 429 Too Many Requests                | Invalid credentials, 403 Forbid |
| GitHub Gists    | Secondary rate limits                | Invalid scopes, 401 Unauthorized|

Standardized using the existing RetryableError/NonRetryableError hierarchy[1].

### 9.3. Performance Optimization
Leverage existing utilities for common patterns:
```python
from .utils import (
    chunked_upload,
    exponential_backoff,
    validate_mime_type,
    generate_presigned_url
)

class EnterpriseProvider(ProviderClient):
    @exponential_backoff(max_retries=5)
    async def upload_file(self, path: Path):
        await validate_mime_type(path, allowed=['image/*'])
        return await chunked_upload(
            session=self.session,
            url=self.upload_url,
            data=path
        )
```
Reuse of established patterns reduces implementation complexity[1].

## 10. Conclusion

The proposed integrations expand twat-fs' capabilities across multiple domains while maintaining architectural consistency. Implementation priority should consider:

1. Authentication complexity (OAuth2 vs API key)
2. Enterprise demand (Aspera/Signiant)
3. Community usage (GitHub/npm)
4. Emerging technologies (IPFS/Storj)

A phased rollout approach with metrics collection on provider usage would optimize maintenance efforts. All integrations should include: 

- Environment variable documentation
- Rate limit tracking
- Automated credential validation
- End-to-end test cases

This expansion positions twat-fs as a universal file transfer solution adaptable to diverse organizational needs.

Citations:
[1] https://ppl-ai-file-upload.s3.amazonaws.com/web/direct-files/3134098/8707773d-201a-4a0d-86a1-d4017414fc32/paste.txt







Here are some services that could be easily integrated with the "upload" functionality of `twat-fs`, expanding its capabilities and user base:

### 10.1. Image and Media Hosting Services

*   **Imgur:** A widely used image hosting service, especially popular in online communities. Integrating Imgur would allow users to easily upload and share images online, which is a common use case for file uploading tools. Imgur has a relatively straightforward API, making integration feasible.
*   **Flickr:** Similar to Imgur but focused more on photography and professional images. Flickr integration could cater to users who need to share and host photos online, offering another specialized image hosting option.

### 10.2. Cloud Storage and Collaboration Platforms

*   **Google Drive:** As a ubiquitous cloud storage service, Google Drive is an essential integration. It would enable users to upload files directly to their Google Drive accounts, leveraging Google's infrastructure for file management and sharing. Google provides a well-documented API for Drive, which simplifies integration.
*   **OneDrive:** Microsoft's cloud storage solution, OneDrive, is another major platform. Integrating OneDrive would cater to users within the Microsoft ecosystem, providing seamless file uploads to their personal or organizational OneDrive storage. Like Google Drive, OneDrive also offers APIs for programmatic access.
*   **Box:** Box is a cloud storage and collaboration platform often used by businesses. Integrating Box would target professional users who rely on Box for secure file sharing and management within organizations. Box provides robust APIs suitable for integration.

### 10.3. File Sharing and Transfer Services

*   **WeTransfer:** A popular service for sending large files, WeTransfer focuses on ease of use and temporary file sharing. Integrating WeTransfer would allow users to quickly share files without requiring recipients to have an account, useful for one-off file transfers. WeTransfer has an API that can be used for integration.
*   **Mega.nz:** Known for its security and encryption features, Mega.nz offers cloud storage and file sharing. Integration with Mega.nz would appeal to users who prioritize privacy and security when uploading and sharing files. Mega provides an API, although it might be more complex to integrate due to its encryption focus.

### 10.4. Developer and Version Control Platforms

*   **GitHub (Releases/Issues):** For developers, direct integration with GitHub to upload files to repository releases or issues would be highly valuable. This could streamline workflows for sharing binaries, assets, or attachments related to software projects directly within GitHub. GitHub's REST API supports file uploads to releases and issues.
*   **GitLab (Releases/Issues):** Similar to GitHub, GitLab is a widely used platform for code hosting and collaboration. Integrating GitLab for file uploads to releases or issues would cater to GitLab users, offering similar benefits for developer workflows as GitHub integration. GitLab also has a comprehensive API for project and repository management.

### 10.5. Considerations for Integration

When considering these services for integration, it's important to evaluate:

*   **API Availability and Documentation:**  Services with well-documented and accessible APIs are easier to integrate.
*   **Authentication Methods:** Different services use various authentication methods (API keys, OAuth 2.0, etc.).  `twat-fs` needs to handle these securely and potentially provide user-friendly setup instructions.
*   **Rate Limits and Usage Restrictions:** Understanding the service's rate limits and usage policies is crucial to ensure reliable integration and handle potential errors gracefully.
*   **User Demand and Use Cases:** Prioritize services that align with the common use cases of `twat-fs` and are requested by users.

By integrating a selection of these services, `twat-fs` can become an even more versatile and powerful tool for file uploading across a wide range of platforms and use cases.







The following providers can be integrated into the `twat-fs` package to enhance its file upload capabilities:

1. **Google Drive**:
   - **Description**: Google Drive is a widely-used cloud storage service that offers robust file sharing and collaboration features.
   - **Integration**: Implementing Google Drive as a provider would allow users to upload files directly to their Google Drive accounts. This can be particularly useful for users who already use Google Drive for personal or business purposes.
   - **Dependencies**: The integration would require the `google-api-python-client` and `google-auth` libraries.
   - **Authentication**: Users would need to authenticate using OAuth 2.0 and provide the necessary credentials.
   - **Setup Instructions**:
     ```python
     from google.oauth2 import service_account
     from googleapiclient.discovery import build
     from googleapiclient.http import MediaFileUpload

     SCOPES = ['https://www.googleapis.com/auth/drive.file']
     SERVICE_ACCOUNT_FILE = 'path/to/service_account.json'

     credentials = service_account.Credentials.from_service_account_file(
         SERVICE_ACCOUNT_FILE, scopes=SCOPES)
     service = build('drive', 'v3', credentials=credentials)

     file_metadata = {'name': 'testfile.txt'}
     media = MediaFileUpload('path/to/testfile.txt', mimetype='text/plain')
     file = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
     print(f'File ID: {file.get("id")}')
     ```

2. **OneDrive**:
   - **Description**: OneDrive is Microsoft's cloud storage service, integrated with Windows and Office 365.
   - **Integration**: Adding OneDrive as a provider would enable users to upload files to their OneDrive accounts, making it convenient for users within the Microsoft ecosystem.
   - **Dependencies**: The integration would require the `Office365-REST-Python-Client` library.
   - **Authentication**: Users would need to authenticate using OAuth 2.0 and provide the necessary credentials.
   - **Setup Instructions**:
     ```python
     from office365.sharepoint.client_context import ClientContext
     from office365.runtime.auth.client_credential import ClientCredential

     client_credentials = ClientCredential('client_id', 'client_secret')
     ctx = ClientContext('https://tenant.sharepoint.com/sites/site', client_credentials)
     with open('path/to/testfile.txt', 'rb') as file_content:
         target_folder = ctx.web.lists.get_by_title('Documents').root_folder
         target_file = target_folder.upload_file('testfile.txt', file_content)
         ctx.execute_query()
         print(f'File uploaded: {target_file.serverRelativeUrl}')
     ```

3. **Mega.nz**:
   - **Description**: Mega.nz is a cloud storage service known for its strong encryption and privacy features.
   - **Integration**: Integrating Mega.nz would provide users with a secure option for uploading files with end-to-end encryption.
   - **Dependencies**: The integration would require the `mega.py` library.
   - **Authentication**: Users would need to provide their Mega.nz email and password for authentication.
   - **Setup Instructions**:
     ```python
     from mega import Mega

     mega = Mega()
     m = mega.login('email', 'password')
     file = m.upload('path/to/testfile.txt')
     print(f'File uploaded: {file.get("h")}')
     ```

4. **pCloud**:
   - **Description**: pCloud is a cloud storage service that offers a lifetime storage plan, making it a cost-effective option for long-term storage.
   - **Integration**: Adding pCloud as a provider would give users another reliable option for uploading files.
   - **Dependencies**: The integration would require the `pcloud` library.
   - **Authentication**: Users would need to provide their pCloud username and password for authentication.
   - **Setup Instructions**:
     ```python
     import pcloud

     pc = pcloud.PyCloud('username', 'password')
     file_path = 'path/to/testfile.txt'
     folderid = pc.createfolderifnotexists('uploads')
     pc.uploadfile(files=[file_path], folderid=folderid)
     print(f'File uploaded to folder ID: {folderid}')
     ```

5. **Backblaze B2**:
   - **Description**: Backblaze B2 is a low-cost cloud storage service designed for large-scale data storage.
   - **Integration**: Integrating Backblaze B2 would provide users with an economical option for storing large amounts of data.
   - **Dependencies**: The integration would require the `b2` library.
   - **Authentication**: Users would need to provide their B2 application key ID and application key for authentication.
   - **Setup Instructions**:
     ```python
     import b2
     from b2.api import B2Api

     info = b2.account_info()
     api = B2Api(info)
     bucket = api.create_bucket('test-bucket', 'allPublic')
     file_info = {
         'data': open('path/to/testfile.txt', 'rb'),
         'name': 'testfile.txt'
     }
     file_version = bucket.upload(file_info)
     print(f'File uploaded: {file_version.id_}')
     ```

6. **DigitalOcean Spaces**:
   - **Description**: DigitalOcean Spaces is an object storage service compatible with the S3 API, making it a flexible option for users familiar with S3.
   - **Integration**: Adding DigitalOcean Spaces as a provider would give users another S3-compatible storage option.
   - **Dependencies**: The integration would require the `boto3` library.
   - **Authentication**: Users would need to provide their Spaces access key and secret key for authentication.
   - **Setup Instructions**:
     ```python
     import boto3

     session = boto3.session.Session()
     client = session.client('s3',
                             region_name='nyc3',
                             endpoint_url='https://nyc3.digitaloceanspaces.com',
                             aws_access_key_id='access_key',
                             aws_secret_access_key='secret_key')
     client.upload_file('path/to/testfile.txt', 'bucket-name', 'testfile.txt')
     print('File uploaded successfully')
     ```

7. **Wasabi**:
   - **Description**: Wasabi is a hot cloud storage service designed to be a cost-effective alternative to Amazon S3.
   - **Integration**: Integrating Wasabi would provide users with another S3-compatible storage option.
   - **Dependencies**: The integration would require the `boto3` library.
   - **Authentication**: Users would need to provide their Wasabi access key and secret key for authentication.
   - **Setup Instructions**:
     ```python
     import boto3

     session = boto3.session.Session()
     client = session.client('s3',
                             region_name='us-east-1',
                             endpoint_url='https://s3.wasabisys.com',
                             aws_access_key_id='access_key',
                             aws_secret_access_key='secret_key')
     client.upload_file('path/to/testfile.txt', 'bucket-name', 'testfile.txt')
     print('File uploaded successfully')
     ```

8. **Azure Blob Storage**:
   - **Description**: Azure Blob Storage is Microsoft's object storage solution for the cloud, designed for storing large amounts of unstructured data.
   - **Integration**: Adding Azure Blob Storage as a provider would give users a robust option for storing large amounts of data.
   - **Dependencies**: The integration would require the `azure-storage-blob` library.
   - **Authentication**: Users would need to provide their Azure storage account name and key for authentication.
   - **Setup Instructions**:
     ```python
     from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

     connect_str = "DefaultEndpointsProtocol=https;AccountName=account_name;AccountKey=account_key;EndpointSuffix=core.windows.net"
     blob_service_client = BlobServiceClient.from_connection_string(connect_str)
     container_client = blob_service_client.get_container_client('test-container')
     blob_client = container_client.get_blob_client('testfile.txt')
     with open('path/to/testfile.txt', 'rb') as data:
         blob_client.upload_blob(data)
     print('File uploaded successfully')
     ```

9. **Box**:
   - **Description**: Box is a cloud content management and file sharing service designed for businesses.
   - **Integration**: Adding Box as a provider would give users a secure and collaborative option for storing files.
   - **Dependencies**: The integration would require the `boxsdk` library.
   - **Authentication**: Users would need to provide their Box API key for authentication.
   - **Setup Instructions**:
     ```python
     from boxsdk import Client, OAuth2

     oauth2 = OAuth2(
         client_id='client_id',
         client_secret='client_secret',
         access_token='access_token'
     )
     client = Client(oauth2)
     new_file = client.folder('0').upload('path/to/testfile.txt')
     print(f'File uploaded: {new_file.id}')
     ```

10. **Cloudflare R2**:
    - **Description**: Cloudflare R2 is a scalable object storage service designed to be a cost-effective alternative to Amazon S3.
    - **Integration**: Integrating Cloudflare R2 would provide users with another S3-compatible storage option.
    - **Dependencies**: The integration would require the `boto3` library.
    - **Authentication**: Users would need to provide their Cloudflare R2 access key and secret key for authentication.
    - **Setup Instructions**:
      ```python
      import boto3

      session = boto3.session.Session()
      client = session.client('s3',
                               region_name='auto',
                               endpoint_url='https://<account_id>.r2.cloudflarestorage.com',
                               aws_access_key_id='access_key',
                               aws_secret_access_key='secret_key')
      client.upload_file('path/to/testfile.txt', 'bucket-name', 'testfile.txt')
      print('File uploaded successfully')
      ```

By integrating these providers, the `twat-fs` package can offer a more comprehensive set of file upload options, catering to a wider range of user preferences and requirements.

================
File: plugins/repos/twat_fs/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_fs/MANIFEST.in
================
# This MANIFEST.in file ensures that JPEG files in the data directory are included in the package distribution.
recursive-include src/twat_fs/data *.jpg

================
File: plugins/repos/twat_fs/mypy.ini
================
[mypy]
python_version = 3.12
ignore_missing_imports = True
disallow_untyped_defs = False
warn_return_any = True
warn_unused_configs = True

[mypy-dropbox.*]
ignore_missing_imports = True

[mypy-fire.*]
ignore_missing_imports = True

[mypy-boto3.*]
ignore_missing_imports = True

[mypy-responses.*]
ignore_missing_imports = True

[mypy-twat_fs.upload_providers.*]
disallow_untyped_defs = False

[mypy-tests.*]
disallow_untyped_defs = False

================
File: plugins/repos/twat_fs/pyproject.toml
================
# this_file: pyproject.toml
# this_project: twat_fs

[project]
classifiers = [
  'Development Status :: 4 - Beta',
  'Programming Language :: Python',
  'Programming Language :: Python :: 3.10',
  'Programming Language :: Python :: 3.11',
  'Programming Language :: Python :: 3.12',
  'Programming Language :: Python :: Implementation :: CPython',
  'Programming Language :: Python :: Implementation :: PyPy',
]
dependencies = [
  'aiohappyeyeballs>=2.4.6',
  'aiohttp>=3.11.12',
  'aiosignal>=1.3.2',
  'attrs>=25.1.0',
  'fire>=0.6.0',
  'frozenlist>=1.5.0',
  'loguru>=0.7.2',
  'multidict>=6.1.0',
  'propcache>=0.2.1',
  'requests>=2.31.0',
  "responses>=0.25.6",
  'tenacity>=8.0.0',
  'twat>=1.8.1',
  'yarl>=1.18.3',
]
description = 'File system utilities for twat with support for multiple upload providers'
dynamic = ['version']
keywords = [
  'file-upload',
  'fal',
  'dropbox',
  's3',
  'twat',
]
license = 'MIT'
name = 'twat-fs'
readme = 'README.md'
requires-python = '>=3.10'

[[project.authors]]
email = 'adam+github@twardoch.com'
name = 'Adam Twardoch'

[project.optional-dependencies]
all = [
  'aiohappyeyeballs>=2.4.6',
  'aiohttp>=3.11.12',
  'aiosignal>=1.3.2',
  'attrs>=25.1.0',
  'boto3>=1.36.22',
  'botocore>=1.36.22',
  'dropbox>=12.0.2',
  'fal-client>=0.5.9',
  'fire>=0.6.0',
  'frozenlist>=1.5.0',
  'loguru>=0.7.2',
  'multidict>=6.1.0',
  'propcache>=0.2.1',
  'requests>=2.31.0',
  'tenacity>=9.0.0',
  'twat>=1.8.1',
  'yarl>=1.18.3',
]
dev = [
  'botocore-stubs<=1.36.22',
  'hatch>=1.14.0',
  'hatchling>=1.27.0',
  'hatch-vcs>=0.4.0',
  'mypy-boto3-s3<=1.36.21',
  'mypy-boto3-sts<=1.36.0',
  'mypy>=1.15.0',
  'pre-commit>=4.1.0',
  'pyupgrade>=3.19.1',
  'ruff>=0.9.6',
  'types-awscrt>=0.23.10',
  'types-boto3>=1.36.22',
  'types-s3transfer>=0.11.2',
  'argparse-types',
  'botocore-types',
  'http-types',
  'json-types',
  'litellm-types',
  'types-aioboto3',
  'types-aiobotocore',
  'types-aiofiles',
  'types-attrs',
  'types-backports',
  'types-beautifulsoup4',
  'types-botocore',
  'types-cachetools',
  'types-jinja2',
  'types-lxml',
  'types-markdown',
  'types-pyyaml',
  'types-regex',
  'types-toml',
  'types-tqdm',
]
dropbox = ['dropbox>=12.00.2']
fal = ['fal-client>=0.5.9']
s3 = [
  'boto3>=1.36.22',
  'botocore>=1.36.22',
]
test = [
  'pytest>=8.3.4',
  'pytest-cov>=6.0.0',
  'pytest-benchmark>=5.1.0',
  'pytest-mock>=3.14.0',
  'pytest-asyncio>=0.25.3',
  'pytest-timeout>=2.3.1',
]

[project.scripts]
twat-fs = 'twat_fs.__main__:main'
[project.entry-points."twat.plugins"]
fs = 'twat_fs'

[project.urls]
Documentation = 'https://github.com/twardoch/twat-fs#readme'
Issues = 'https://github.com/twardoch/twat-fs/issues'
Source = 'https://github.com/twardoch/twat-fs'

[build-system]
build-backend = 'hatchling.build'
requires = [
  'hatchling>=1.27.0',
  'hatch-vcs>=0.4.0',
]
[tool.coverage.paths]
tests = [
  'tests',
  '*/twat-fs/tests',
]
twat_fs = [
  'src/twat_fs',
  '*/twat-fs/src/twat_fs',
]

[tool.coverage.report]
exclude_lines = [
  'no cov',
  'if __name__ == .__main__.:',
  'if TYPE_CHECKING:',
]

[tool.coverage.run]
branch = true
omit = ['src/twat_fs/__about__.py']
parallel = true
source_pkgs = [
  'twat_fs',
  'tests',
]
[tool.hatch.build.hooks.vcs]
version-file = 'src/twat_fs/__version__.py'
[tool.hatch.build.targets.wheel]
include = [
  'src/twat_fs/**/*.py',
  'src/twat/**/*.py',
  'src/twat_fs/py.typed',
]
packages = [
  'src/twat_fs',
  'src/twat',
]
[tool.hatch.envs.default]
dependencies = [
  'pytest>=8.3.4',
  'pytest-cov>=6.0.0',
  'ruff>=0.9.6',
  'mypy>=1.15.0',
]

[tool.hatch.envs.default.scripts]
fix = [
  'ruff check  --fix --unsafe-fixes src/twat_fs tests',
  'ruff format --respect-gitignore src/twat_fs tests',
]
lint = [
  'ruff check src/twat_fs tests',
  'ruff format --respect-gitignore src/twat_fs tests',
]
test = 'pytest {args:tests}'
test-cov = 'pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_fs --cov=tests {args:tests}'
type-check = 'mypy src/twat_fs tests'
[[tool.hatch.envs.all.matrix]]
python = [
  '3.10',
  '3.11',
  '3.12',
]

[tool.hatch.envs.lint]
dependencies = [
  'pytest>=8.3.4',
  'pytest-cov>=6.0.0',
  'ruff>=0.9.6',
  'mypy>=1.15.0',
]
detached = true

[tool.hatch.envs.lint.scripts]
all = [
  'style',
  'typing',
]
fmt = [
  'ruff format --respect-gitignore {args:.}',
  'ruff check --fix {args:.}',
]
style = [
  'ruff check {args:.}',
  'ruff format --respect-gitignore {args:.}',
]
typing = 'mypy --install-types --non-interactive {args:src/twat_fs tests}'

[tool.hatch.envs.test]
dependencies = [
  'pytest>=8.3.4',
  'pytest-cov>=6.0.0',
  'boto3>=1.36.22',
  'botocore>=1.36.22',
  'dropbox>=12.0.2',
  'fal-client>=0.5.9',
]
python = '3.10'

[tool.hatch.envs.test.scripts]
bench = 'python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only'
bench-save = 'python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json'
test = 'python -m pytest -n auto -p no:briefcase {args:tests}'
test-cov = 'python -m pytest -n auto -p no:briefcase --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_fs --cov=tests {args:tests}'

[tool.hatch.version]
source = 'vcs'

[tool.hatch.version.raw-options]
version_scheme = 'post-release'

[tool.mypy]
check_untyped_defs = true
disallow_incomplete_defs = true
disallow_untyped_decorators = true
disallow_untyped_defs = true
no_implicit_optional = true
python_version = '3.10'
warn_no_return = true
warn_redundant_casts = true
warn_return_any = true
warn_unreachable = true
warn_unused_configs = true
warn_unused_ignores = true

[tool.ruff]
line-length = 88
target-version = 'py310'

[tool.ruff.lint]
extend-select = [
  'A',
  'ARG',
  'B',
  'C',
  'DTZ',
  'E',
  'EM',
  'F',
  'FBT',
  'I',
  'ICN',
  'ISC',
  'N',
  'PLC',
  'PLE',
  'PLR',
  'PLW',
  'Q',
  'RUF',
  'S',
  'T',
  'TID',
  'UP',
  'W',
  'YTT',
]
ignore = [
  'ARG001',
  'E501',
  'I001',
  'RUF001',
  'PLR2004',
  'EXE003',
  'ISC001',
]

[tool.ruff.per-file-ignores]
"tests/*" = ['S101']
[tool.pytest.ini_options]
addopts = '-v --durations=10 -p no:briefcase'
asyncio_mode = 'auto'
asyncio_default_fixture_loop_scope = 'function'
console_output_style = 'progress'
filterwarnings = [
  'ignore::DeprecationWarning',
  'ignore::UserWarning',
  'ignore:Unable to refresh access token without refresh token and app key:UserWarning',
]
log_cli = true
log_cli_level = 'INFO'
markers = [
  '''benchmark: marks tests as benchmarks (select with '-m benchmark')''',
  'unit: mark a test as a unit test',
  'integration: mark a test as an integration test',
  'permutation: tests for permutation functionality',
  'parameter: tests for parameter parsing',
  'prompt: tests for prompt parsing',
]
norecursedirs = [
  '.*',
  'build',
  'dist',
  'venv',
  '__pycache__',
  '*.egg-info',
  '_private',
]
python_classes = ['Test*']
python_files = ['test_*.py']
python_functions = ['test_*']
testpaths = ['tests']

[tool.pytest-benchmark]
compare = [
  'min',
  'max',
  'mean',
  'stddev',
  'median',
  'iqr',
  'ops',
  'rounds',
]
histogram = true
min_rounds = 100
min_time = 0.1
save-data = true
storage = 'file'

================
File: plugins/repos/twat_fs/README.md
================
---
this_file: README.md
---

# twat-fs

File system utilities for twat, focusing on robust and extensible file upload capabilities with multiple provider support.

## Rationale

`twat-fs` provides a unified interface for uploading files to various storage providers while addressing common challenges:

* **Provider Flexibility**: Seamlessly switch between storage providers without code changes
* **Smart Fallback**: Intelligent retry and fallback between providers:
  * One retry with exponential backoff for temporary failures
  * Automatic fallback to next provider for permanent failures
  * Clear distinction between retryable and non-retryable errors
* **URL Validation**: Ensures returned URLs are accessible before returning them
* **Progressive Enhancement**: Start simple with zero configuration (simple providers), scale up to advanced providers (S3, Dropbox) as needed
* **Developer Experience**: Clear interfaces, comprehensive type hints, and runtime checks
* **Extensibility**: Well-defined provider protocol for adding new storage backends

## Recent Improvements

The codebase has undergone significant refactoring to improve maintainability and extensibility:

* **Fixed Provider Instantiation**: Improved the `create_provider_instance` function to correctly handle credential management and provider instantiation order
* **Centralized Utilities**: Created a `utils.py` module with shared functionality for all providers
* **Standardized Implementation**: All providers now follow consistent patterns and inherit from `BaseProvider`
* **Improved Error Handling**: Enhanced error classification and handling across all providers with `RetryableError` and `NonRetryableError` classes
* **Provider Templates**: Created templates for simple and authenticated providers to standardize implementation
* **Better Type Safety**: Improved type annotations and protocol compatibility
* **Consistent Logging**: Standardized logging patterns with `log_upload_attempt` function for better debugging and monitoring
* **Factory Pattern**: Implemented a factory pattern for provider instantiation to simplify creation and standardize error handling
* **Async/Sync Utilities**: Created standardized utilities for async/sync conversion to ensure consistent patterns across providers
* **Comprehensive Testing**: Added thorough unit tests for utility functions covering edge cases and error conditions
* **Provider Base Classes**: Created base classes to reduce inheritance boilerplate and standardize provider implementation
* **URL Validation**: Improved URL validation to ensure returned URLs are accessible before returning them

## Current Status

The project is in active development with several key areas of focus:

* **Fixing Type Annotation Issues**: Addressing incompatible return types in async methods, type mismatches in factory.py and simple.py, and ensuring proper typing for async/await conversions
* **Resolving Remaining Test Failures**: Fixing failing unit tests, particularly in TestLogUploadAttempt and TestGatherWithConcurrency
* **Addressing Boolean Argument Issues**: Converting boolean positional arguments to keyword-only arguments to fix FBT001/FBT002 linter errors
* **Improving Exception Handling**: Implementing proper exception chaining across providers
* **Addressing Linter Issues**: Fixing various linter warnings, particularly in cleanup.py and cli.py
* **Expanding Provider Support**: Planning implementation of additional upload providers

## Project Documentation

The project maintains several key documentation files:

* **README.md** (this file): Overview, installation, usage, and architecture
* **CHANGELOG.md**: Detailed record of all changes and improvements
* **TODO.md**: Prioritized list of upcoming tasks and features

These files are regularly updated to reflect the current state of the project.

## Quick Start

### Installation

Basic installation with simple providers:

```bash
uv pip install twat-fs
```

Install with all providers and development tools:

```bash
uv pip install 'twat-fs[all,dev]'
```

### Basic Usage

```python
from twat_fs import upload_file

# Simple upload (uses catbox.moe by default)
url = upload_file("path/to/file.txt")

# Specify provider with fallback
url = upload_file("path/to/file.txt", provider=["s3", "dropbox", "catbox"])

# Handle provider-specific errors
from twat_fs.upload_providers.core import RetryableError, NonRetryableError

try:
    url = upload_file("file.txt", provider="s3")
except RetryableError as e:
    print(f"Temporary error with {e.provider}: {e}")
except NonRetryableError as e:
    print(f"Permanent error with {e.provider}: {e}")
```

### Using the Factory Pattern

```python
from twat_fs.upload_providers.factory import ProviderFactory

# Get a provider instance
factory = ProviderFactory()
provider = factory.get_provider("s3")

# Upload a file
result = provider.upload_file("path/to/file.txt")
print(f"File uploaded to: {result.url}")

# Get all available providers
available_providers = factory.list_available_providers()
print(f"Available providers: {available_providers}")
```

### Provider Instantiation

The package uses a robust provider instantiation system that follows this order:

1. If no credentials are provided, try to get them from the provider class
2. If the provider class has a `get_provider` method, use that
3. If `get_provider` fails, fall back to direct instantiation

This ensures that providers are instantiated correctly regardless of how they're configured:

```python
from twat_fs.upload_providers.utils import create_provider_instance
from twat_fs.upload_providers import s3

# Get a provider instance with explicit credentials
credentials = {"AWS_S3_BUCKET": "my-bucket", "AWS_ACCESS_KEY_ID": "key", "AWS_SECRET_ACCESS_KEY": "secret"}
provider = create_provider_instance(s3.S3Provider, credentials)

# Get a provider instance using environment variables
provider = create_provider_instance(s3.S3Provider)
```

### Async/Sync Conversion

```python
from twat_fs.upload_providers.async_utils import to_sync, to_async, run_async

# Convert an async function to sync
@to_sync
async def async_function():
    # Async implementation
    return "result"

# Use the sync version
result = async_function()  # No need for await

# Convert a sync function to async
@to_async
def sync_function():
    # Sync implementation
    return "result"

# Use the async version
async def main():
    result = await sync_function()

# Run an async function in a sync context
result = run_async(async_function())
```

### Command Line Interface

```bash
# Simple upload
python -m twat_fs upload_file path/to/file.txt

# Specify provider with fallback
python -m twat_fs upload_file path/to/file.txt --provider s3,dropbox,catbox

# Disable fallback (fail immediately if provider fails)
python -m twat_fs upload_file path/to/file.txt --provider s3 --fragile

# Check provider setup
python -m twat_fs setup provider s3
python -m twat_fs setup all
```

## Provider Configuration

### Provider Fallback System

The package implements a robust provider fallback system:

1. **Circular Fallback**: When using multiple providers, if a provider fails, the system will:
   * Try the next provider in the list
   * If all remaining providers fail, start over from the beginning of the full provider list
   * Continue until all providers have been tried once
   * Each provider is only tried once to avoid infinite loops

2. **Fragile Mode**: For cases where fallback is not desired:
   * Use the `--fragile` flag in CLI: `--fragile`
   * In code: `upload_file(..., fragile=True)`
   * System will fail immediately if the requested provider fails
   * No fallback attempts will be made

Example fallback scenarios:

```python
# Full circular fallback (if E fails, tries F, G, A, B, C, D)
url = upload_file("file.txt", provider="E")

# Fragile mode (fails immediately if E fails)
url = upload_file("file.txt", provider="E", fragile=True)

# Custom provider list with circular fallback
# If C fails, tries A, then B
url = upload_file("file.txt", provider=["C", "A", "B"])
```

### Simple Providers (No Configuration Required)

The following providers work out of the box with no configuration:

* **catbox.moe**: General file uploads (default)
* **litterbox.catbox.moe**: Temporary file uploads with expiration
* **www0x0.st**: General file uploads
* **uguu.se**: Temporary file uploads
* **bashupload.com**: General file uploads
* **filebin.net**: Temporary file uploads (6-day expiration)
* **pixeldrain.com**: General file uploads

### Dropbox

```bash
export DROPBOX_ACCESS_TOKEN="your_token_here"
# Optional OAuth2 configuration
export DROPBOX_REFRESH_TOKEN="refresh_token"
export DROPBOX_APP_KEY="app_key"
export DROPBOX_APP_SECRET="app_secret"
```

### AWS S3

```bash
# Required
export AWS_S3_BUCKET="your_bucket"
export AWS_DEFAULT_REGION="us-east-1"

# Authentication (choose one)
export AWS_ACCESS_KEY_ID="key_id"
export AWS_SECRET_ACCESS_KEY="secret_key"
# Or use AWS CLI: aws configure
# Or use IAM roles in AWS infrastructure

# Optional
export AWS_ENDPOINT_URL="custom_endpoint"  # For S3-compatible services
```

## Development

### Running Tests and Linting

The project uses `cleanup.py` for managing repository tasks and maintaining code quality:

```bash
# Check the current state of the repository
python ./cleanup.py status

# Install dependencies
uv pip install -e '.[test,dev]'

# Run tests
python -m pytest

# Run linting
ruff check --output-format=github --fix --unsafe-fixes . && ruff format --respect-gitignore --target-version py312 .
```

### Current Issues

Based on the latest linting and test results, the following issues need to be addressed:

1. **Missing Dependencies for Tests**:
   - Some tests require additional dependencies: 'fal_client', 'botocore', 'responses'
   - Need to install these dependencies or implement proper test skipping
   - Affected files: `tests/test_integration.py`, `tests/test_s3_advanced.py`, `tests/test_upload.py`

2. **Boolean Argument Issues**:
   - FBT001/FBT002 linter errors for boolean positional arguments
   - FBT003 linter errors for boolean positional values in function calls
   - Need to convert boolean positional arguments to keyword-only arguments

3. **Type Annotation Issues**:
   - Incompatible return types in async methods
   - Type mismatches in factory.py and simple.py
   - Missing type annotations for variables in simple.py
   - Improper typing for async/await conversions

4. **Exception Handling Issues**:
   - B904 linter errors: Need to use `raise ... from err` in exception handling
   - S101 linter errors: Use of `assert` detected in core.py

5. **Function Complexity Issues**:
   - Functions with too many arguments (PLR0913)
   - Complex functions with too many branches/statements/returns (C901, PLR0911, PLR0912, PLR0915)
   - Need to refactor these functions for better maintainability

### Contributing

Contributions are welcome! Please check the TODO.md file for current priorities and open issues.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Example Provider Results

```bash
for PROVIDER in $(twat fs upload_provider list 2>/dev/null); do URL="$(twat fs upload "./src/twat_fs/data/test.jpg" --provider "$PROVIDER")"; echo "[$PROVIDER]($URL)"; done
```

```
Error: Upload failed: Failed to upload with catbox: Unexpected error: URL validation failed (status 503)
[catbox]()
[litterbox](https://litter.catbox.moe/8a6jf0.jpg)
[fal](https://v3.fal.media/files/monkey/Kd6SwMGEIbxMIFPihlFQL_test.jpg)
[bashupload](https://bashupload.com/TTHlX/test.jpg?download=1)
[uguu](https://d.uguu.se/RrhFSqLP.jpg)
[www0x0](https://0x0.st/8qUT.jpg)
[filebin](https://filebin.net/twat-fs-1739859030-enq2xe/test.jpg)
```

================
File: plugins/repos/twat_fs/TODO.md
================
---
this_file: TODO.md
---

# TODO

Tip: Periodically run `python ./cleanup.py status` to see results of lints and tests. Use `uv pip ...` not `pip ...`

## High Priority

- [ ] Fix missing dependencies for tests
  - [ ] Install missing dependencies for tests or implement proper test skipping
    - Issue: ModuleNotFoundError for 'fal_client', 'botocore', 'responses'
    - Fix: Add `uv pip install 'twat-fs[test,dev]'` or implement conditional imports with proper test skipping
    - Affected files:
      - `tests/test_integration.py`: Needs 'fal_client'
      - `tests/test_s3_advanced.py`: Needs 'botocore'
      - `tests/test_upload.py`: Needs 'botocore'

- [ ] Fix failing unit tests
  - [ ] Fix `TestLogUploadAttempt.test_log_upload_attempt_success` test
    - Issue: logger.info is not being called in the log_upload_attempt function
    - Fix: Implement proper logger.info call in the success branch
  - [ ] Fix `TestGatherWithConcurrency.test_gather_with_concurrency_with_exceptions` test
    - Issue: Test is failing with RuntimeError instead of ValueError
    - Fix: Ensure the correct exception type is propagated in gather_with_concurrency

- [ ] Fix boolean argument issues
  - [ ] Fix FBT001/FBT002 linter errors for boolean positional arguments
    - Issue: Boolean-typed positional arguments in function definitions
    - Fix: Convert boolean positional arguments to keyword-only arguments
    - Affected files:
      - `utils.py` line 251: `log_upload_attempt` function
      - `upload.py` lines 251, 381: `setup_provider` and `setup_providers` functions
      - `cli.py` lines 202, 203, 205: `upload` method
  - [ ] Fix FBT003 linter errors for boolean positional values in function calls
    - Affected files:
      - `upload.py` multiple instances in `ProviderInfo` instantiation
      - `test_utils.py` lines 340, 349: `log_upload_attempt` calls

- [ ] Fix type annotation issues
  - [ ] Fix incompatible return types in async methods
    - Issue: Return type mismatches in async functions
    - Affected files:
      - `simple.py` lines 120, 156, 273: Return type incompatibilities
  - [ ] Fix type mismatches in factory.py
    - Issue: Incompatible types in assignment (expression has type Module, variable has type "Provider | None")
  - [ ] Fix missing type annotations
    - Issue: Need type annotation for variables
    - Affected files:
      - `simple.py` lines 237, 261: Missing type annotations for "sync_upload"

## Medium Priority

- [ ] Fix exception handling issues
  - [ ] Fix B904 linter errors (raise with from)
    - Issue: Within an `except` clause, raise exceptions with `raise ... from err`
    - Affected files:
      - `upload.py` line 687
      - `fal.py` line 67
  - [ ] Fix S101 linter errors (use of assert)
    - Affected files:
      - `core.py` lines 168, 214

- [ ] Fix unused arguments and imports
  - [ ] Fix ARG002 linter errors (unused method arguments)
    - Affected files:
      - `dropbox.py` line 124: Unused `kwargs`
      - `s3.py` line 182: Unused `kwargs`
      - `simple.py` multiple instances: Unused arguments
  - [ ] Fix F401 linter errors (unused imports)
    - Affected files:
      - `__init__.py` lines 11, 19: Unused imports

- [ ] Fix function complexity issues
  - [ ] Refactor functions with too many arguments (PLR0913)
    - Affected files:
      - `cli.py` line 198: `upload` method
      - `upload.py` lines 411, 550, 619, 703: Multiple functions
      - `litterbox.py` lines 223, 300: `upload_file` functions
  - [ ] Refactor complex functions (C901)
    - Affected files:
      - `upload.py` lines 57, 250: `_test_provider_online` and `setup_provider` functions
  - [ ] Fix functions with too many branches/statements/returns (PLR0911, PLR0912, PLR0915)
    - Affected files:
      - `upload.py` lines 57, 250: Multiple complexity issues

## Low Priority

- [ ] Fix linter issues in `cleanup.py`
  - [ ] Address DTZ005: datetime.datetime.now() called without a tz argument
  - [ ] Fix S603/S607: Subprocess call security issues

- [ ] Update `pyproject.toml` to fix deprecated linter settings
  - [ ] Update ruff configuration
  - [ ] Add explicit Python version targets
  - [ ] Configure mypy settings

- [ ] Fix RUF012 linter errors (mutable class attributes)
  - Affected files:
    - `fal.py` lines 51, 52: Mutable class attributes should be annotated with `typing.ClassVar`

- [ ] Fix A005 linter error (module shadows standard library)
  - Affected files:
    - `types.py` line 1: Module `types` shadows a Python standard-library module

- [ ] Fix S105 linter error (possible hardcoded password)
  - Affected files:
    - `test_s3_advanced.py` line 21: Hardcoded "TEST_SECRET_KEY"

## Documentation

- [ ] Document best practices for creating new providers
  - [ ] Create comprehensive provider development guide
  - [ ] Add examples for common provider patterns

- [ ] Update API documentation with latest changes

- [ ] Create troubleshooting guide for common issues

## New Providers

- [ ] Add support for Imgur
- [ ] Add support for Cloudinary
- [ ] Add support for Google Drive
- [ ] Add support for OneDrive
- [ ] Add support for Box
- [ ] Add support for Mega
- [ ] Add support for Backblaze B2
- [ ] Add support for Wasabi

## Completed Tasks

- [x] Fix `TestCreateProviderInstance` tests
- [x] Fix `TestGatherWithConcurrency` tests
- [x] Create `utils.py` module for shared functionality
- [x] Refactor provider modules to use `utils.py`
- [x] Create provider templates
- [x] Implement factory pattern for provider instantiation
- [x] Standardize async/sync conversion patterns
- [x] Write unit tests for utility functions
- [x] Create provider base classes
- [x] Improve error classification with `RetryableError` and `NonRetryableError` classes
- [x] Standardize logging patterns with `log_upload_attempt` function
- [x] Enhance type hints for better IDE support and runtime type checking
- [x] Improve URL validation to ensure returned URLs are accessible before returning them

================
File: plugins/repos/twat_fs/update_providers.py
================
def update_provider_file(file_path: Path) -> None:
    with open(file_path) as f:
        content = f.read()
    content = content.replace("SimpleProviderBase", "BaseProvider")
    content = content.replace(
    def replace_upload_file(match):
        signature = match.group(1)
    content = re.sub(
    def replace_upload_file_body(match):
        match.group(1)
    with open(file_path, "w") as f:
        f.write(content)
def main():
    base_path = Path("src/twat_fs/upload_providers")
        update_provider_file(file_path)
    main()

================
File: plugins/repos/twat_fs/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_genai/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_genai/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_genai/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 768]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [1.6K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 500]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [2.5K]  LOG.md
├── [2.9K]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [8.2K]  pyproject.toml
├── [ 128]  src
│   └── [ 352]  twat_genai
│       ├── [ 864]  __init__.py
│       ├── [ 22K]  __main__.py
│       ├── [1.1K]  __main___loras.json
│       ├── [6.4K]  cli.py
│       ├── [ 224]  core
│       │   ├── [1.7K]  config.py
│       │   ├── [1.3K]  image.py
│       │   ├── [1.3K]  models.py
│       │   └── [8.0K]  prompt.py
│       └── [ 192]  engines
│           ├── [1.7K]  base.py
│           └── [ 256]  fal
│               ├── [3.3K]  __init__.py
│               ├── [4.5K]  client.py
│               ├── [3.2K]  config.py
│               ├── [5.9K]  lora.py
│               └── [1.2K]  models.py
└── [ 128]  tests
    └── [ 154]  test_twat_genai.py

13 directories, 30 files

================
File: plugins/repos/twat_genai/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_genai/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_genai --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_genai/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-genai
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_genai/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_genai/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_genai/dist/.gitignore
================
*

================
File: plugins/repos/twat_genai/src/twat_genai/core/config.py
================
class ImageSizeWH(BaseModel):
class ImageInput(BaseModel):
    def is_valid(self) -> bool:
            sum(1 for x in (self.url, self.path, self.pil_image) if x is not None) == 1
    async def to_url(self) -> str:
        raise NotImplementedError(msg)
class ImageResult(BaseModel):

================
File: plugins/repos/twat_genai/src/twat_genai/core/image.py
================
class ImageSizes(str, Enum):
class ImageFormats(str, Enum):
async def save_image(
    image.save(output_path, format=format.value, quality=quality)
def validate_image_size(size_str: str) -> tuple[int, int] | None:
        w, h = (int(x.strip()) for x in size_str.split(",", 1))

================
File: plugins/repos/twat_genai/src/twat_genai/core/models.py
================
class ModelTypes(str, Enum):
class ImageSizes(str, Enum):
class ImageFormats(str, Enum):
class ImageSizeWH(BaseModel):
class ImageResult(BaseModel):

================
File: plugins/repos/twat_genai/src/twat_genai/core/prompt.py
================
class ImagePrompt(BaseModel):
    url: AnyHttpUrl = Field(
    weight: float = Field(
    def is_valid(self) -> bool:
        return str(self.url).lower().endswith(ALLOWED_IMAGE_EXTENSIONS)
class PromptParameter(BaseModel):
    name: str = Field(..., description="Parameter name without -- prefix")
    value: str | None = Field(None, description="Parameter value if any")
class PromptPart(BaseModel):
    text: str = Field(..., description="Text content of this prompt part")
    weight: float = Field(default=1.0, description="Relative weight of this part")
class MidjourneyPrompt(BaseModel):
    image_prompts: list[ImagePrompt] = Field(
    text_parts: list[PromptPart] = Field(..., description="Text portions of the prompt")
    parameters: list[PromptParameter] = Field(
    raw_prompt: str = Field(..., description="Original unparsed prompt")
    def to_string(self) -> str:
            parts.append(str(img.url))
                parts.append(f"--iw {img.weight}")
                text_parts.append(part.text)
                text_parts.append(f"{part.text}::{part.weight}")
        parts.append(" ".join(text_parts))
                parts.append(f"--{param.name} {param.value}")
                parts.append(f"--{param.name}")
        return " ".join(parts)
def split_top_level(s: str, delimiter: str = ",") -> list[str]:
            parts.append("".join(current).strip())
            current.append(char)
def expand_permutations(prompt: str) -> list[str]:
    start_idx = prompt.find(PERMUTATION_START)
    for i, char in enumerate(prompt[start_idx:], start=start_idx):
        raise ValueError(msg)
        for suffix_expanded in expand_permutations(suffix):
            results.append(f"{prefix}{option.strip()}{suffix_expanded}")
def parse_parameters(text: str) -> tuple[str, list[PromptParameter]]:
    parts = text.split()
    while i < len(parts):
        if part.startswith(PARAM_PREFIX):
            if i + 1 < len(parts) and not parts[i + 1].startswith(PARAM_PREFIX):
            params.append(PromptParameter(name=name, value=value))
            prompt_parts.append(part)
    return " ".join(prompt_parts), params
def parse_multi_prompt(text: str) -> list[PromptPart]:
        return [PromptPart(text=text)]
    for part in text.split(MULTI_PROMPT_SEPARATOR):
        part = part.strip()
                text, weight_str = part.rsplit(" ", 1)
                weight = float(weight_str)
        parts.append(PromptPart(text=text, weight=weight))
def parse_prompt(prompt: str) -> MidjourneyPrompt:
    text, parameters = parse_parameters(prompt)
        if any(part.lower().endswith(ext) for ext in ALLOWED_IMAGE_EXTENSIONS):
                    if part.startswith(("http://", "https://"))
                parsed_url = parse_obj_as(AnyHttpUrl, url)
                image_prompts.append(ImagePrompt(url=parsed_url))
                logger.warning(f"Invalid image URL {part}: {e}")
            text_parts.append(part)
    text = " ".join(text_parts)
    prompt_parts = parse_multi_prompt(text)
    return MidjourneyPrompt(
def normalize_prompts(prompts: str | list[str]) -> list[str]:
        split_top_level(prompts, delimiter=";") if isinstance(prompts, str) else prompts
        final_prompts.extend(expand_permutations(raw.strip()))

================
File: plugins/repos/twat_genai/src/twat_genai/engines/fal/__init__.py
================
load_dotenv()
class FALEngine(ImageGenerationEngine):
    def __init__(self, output_dir: Path | None = None) -> None:
        self.api_key = os.getenv("FAL_KEY")
    async def initialize(self) -> None:
            raise ValueError(msg)
    async def generate(
        model = kwargs.get("model", ModelTypes.TEXT)
        image_config = kwargs.get("image_config")
        lora_spec = kwargs.get("lora_spec")
        filename_suffix = kwargs.get("filename_suffix")
        filename_prefix = kwargs.get("filename_prefix")
        job_config = FALJobConfig(
        request_id = await submit_job(job_config)
            job_params.update(
                    "image_config": image_config.model_dump(),
        return await get_result(
    async def shutdown(self) -> None:

================
File: plugins/repos/twat_genai/src/twat_genai/engines/fal/client.py
================
async def submit_job(job: FALJobConfig) -> str:
    args = await job.to_fal_arguments()
    handler = await fal_client.submit_async(job.model.value, arguments=args)
    logger.debug(f"Submitted job with ID: {handler.request_id}")
async def download_image(url: str, output_path: Path) -> None:
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        response.raise_for_status()
        output_path.write_bytes(response.content)
        logger.info(f"Saved image to: {output_path}")
async def get_result(
    status = await fal_client.status_async(
    while isinstance(status, fal_client.InProgress):
        await asyncio.sleep(1)
    result = await fal_client.result_async("fal-ai/flux-lora", request_id)
    timestamp = result.get("timestamp", datetime.now().strftime("%Y%m%d_%H%M%S"))
        output_dir.mkdir(parents=True, exist_ok=True)
        request_id_prefix = request_id.split("-")[0] if request_id else ""
            await download_image(image_url, image_path)
            logger.error(f"Failed to save image {image_path}: {e}")
            metadata = ImageResult(
                    "path": str(image_path) if image_path else None,
                    "metadata_path": str(metadata_path) if metadata_path else None,
            metadata_path.write_text(metadata.model_dump_json(indent=2))
            logger.info(f"Saved metadata to: {metadata_path}")
            logger.error(f"Failed to save metadata {metadata_path}: {e}")
    return ImageResult(

================
File: plugins/repos/twat_genai/src/twat_genai/engines/fal/config.py
================
class ModelTypes(str, Enum):
class ImageToImageConfig(BaseModel):
    @model_validator(mode="before")
    def convert_image_input(cls, data: Any) -> Any:
        if isinstance(data, dict) and "input_image" in data:
            if isinstance(data["input_image"], ImageInput) and not isinstance(
                data["input_image"] = FALImageInput.from_base(data["input_image"])
class LoraRecord(BaseModel):
class LoraRecordList(RootModel[list[LoraRecord]]):
class LoraLib(RootModel[dict[str, LoraRecordList]]):
class LoraSpecEntry(BaseModel):
class CombinedLoraSpecEntry(BaseModel):
class FALJobConfig(BaseModel):
    async def to_fal_arguments(self) -> dict[str, Any]:
        lora_list, final_prompt = await build_lora_arguments(
            image_url = await self.image_config.input_image.to_url()

================
File: plugins/repos/twat_genai/src/twat_genai/engines/fal/lora.py
================
def get_lora_lib() -> LoraLib:
    paths = PathManager.for_package("twat_genai")
    user_path = os.getenv("TWAT_GENAI_LORA_LIB")
        lib_path = Path(user_path)
        if lib_path.exists():
            return LoraLib.model_validate_json(lib_path.read_text())
    bundled_path = Path(__file__).parent.parent.parent / "__main___loras.json"
    return LoraLib.model_validate_json(bundled_path.read_text())
LORA_LIB = get_lora_lib()
def parse_lora_phrase(phrase: str) -> LoraSpecEntry | CombinedLoraSpecEntry:
    phrase = phrase.strip()
            entries.append(
                LoraSpecEntry(path=record.url, scale=record.scale, prompt=phrase + ",")
        return CombinedLoraSpecEntry(entries=entries, factory_key=phrase)
        identifier, scale_str = phrase.split(":", 1)
        identifier = identifier.strip()
            scale = float(scale_str.strip())
            raise ValueError(msg)
    return LoraSpecEntry(path=identifier, scale=scale, prompt="")
def normalize_lora_spec(
                        normalized.append(
                            LoraSpecEntry(path=path, scale=float(scale), prompt=prompt)
                            LoraSpecEntry(
                                scale=float(d.get("scale", 1.0)),
                                prompt=d.get("prompt", ""),
                        normalized.append(parse_lora_phrase(phrase))
                            parse_lora_phrase(sub)
                            if isinstance(sub, str)
                        normalized.append(CombinedLoraSpecEntry(entries=combined))
                        msg = f"Unsupported LoRA spec item type: {type(item)}"
                return [parse_lora_phrase(s)]
            phrases = [phrase.strip() for phrase in s.split(";") if phrase.strip()]
            return [parse_lora_phrase(phrase) for phrase in phrases]
            msg = f"Unsupported LoRA spec type: {type(spec)}"
async def build_lora_arguments(
    entries = normalize_lora_spec(lora_spec)
    def process_entry(entry: LoraSpecEntry | CombinedLoraSpecEntry) -> None:
        if isinstance(entry, LoraSpecEntry):
            lora_list.append({"path": entry.path, "scale": entry.scale})
                prompt_prefixes.append(entry.prompt.rstrip(","))
                process_entry(sub_entry)
        process_entry(entry)
    logger.debug(f"Using LoRA configuration: {lora_list}")
        f"{', '.join(prompt_prefixes)}, {prompt}".strip() if prompt_prefixes else prompt

================
File: plugins/repos/twat_genai/src/twat_genai/engines/fal/models.py
================
class FALImageInput(ImageInput):
    def from_base(cls, base: ImageInput) -> "FALImageInput":
        return cls(url=base.url, path=base.path, pil_image=base.pil_image)
    async def to_url(self) -> str:
            return await fal_client.upload_file_async(self.path)
            with tempfile.NamedTemporaryFile(suffix=".jpg", delete=False) as tmp:
                tmp_path = Path(tmp.name)
                self.pil_image.save(tmp_path, format="JPEG", quality=95)
                return await fal_client.upload_file_async(tmp_path)
            raise ValueError(msg)

================
File: plugins/repos/twat_genai/src/twat_genai/engines/base.py
================
class EngineConfig(BaseModel):
class ImageGenerationEngine(ABC):
    async def initialize(self) -> None:
    async def generate(
    async def shutdown(self) -> None:
    async def __aenter__(self) -> ImageGenerationEngine:
        await self.initialize()
    async def __aexit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
        await self.shutdown()

================
File: plugins/repos/twat_genai/src/twat_genai/__init__.py
================


================
File: plugins/repos/twat_genai/src/twat_genai/__main___loras.json
================
{
  "gstdrw style": [
    {
      "url": "glif/Gesture-Draw",
      "scale": 1.0
    }
  ],
  "Sketch Smudge": [
    {
      "url": "strangerzonehf/Flux-Sketch-Smudge-LoRA",
      "scale": 1.0
    }
  ],
  "2color illustration": [
    {
      "url": "strangerzonehf/2Color-Illustration",
      "scale": 1.2
    }
  ],
  "shou_xin": [
    {
      "url": "hassanelmghari/shou_xin",
      "scale": 1.0
    }
  ],
  "shou_xin, Simple Pencil sketch": [
    {
      "url": "hassanelmghari/shou_xin",
      "scale": 0.8
    },
    {
      "url": "prithivMLmods/Super-Pencil-Flux-LoRA",
      "scale": 1.2
    }
  ],
  "shou_xin, gstdrw style sketch": [
    {
      "url": "hassanelmghari/shou_xin",
      "scale": 2.0
    },
    {
      "url": "glif/Gesture-Draw",
      "scale": 0.4
    }
  ],
  "in the style of TOK a trtcrd tarot style": [
    {
      "url": "multimodalart/flux-tarot-v1",
      "scale": 1.0
    }
  ],
  "shou_xin, Sketch Smudge": [
    {
      "url": "hassanelmghari/shou_xin",
      "scale": 1.0
    },
    {
      "url": "strangerzonehf/Flux-Sketch-Smudge-LoRA",
      "scale": 0.5
    }
  ]
}

================
File: plugins/repos/twat_genai/src/twat_genai/__main__.py
================
load_dotenv()
class ModelTypes(str, Enum):
class ImageInput(BaseModel):
    def is_valid(self) -> bool:
            sum(1 for x in (self.url, self.path, self.pil_image) if x is not None) == 1
    async def to_url(self) -> str:
            raise ValueError(msg)
            return await fal_client.upload_file_async(Path(str(self.path)))
            with tempfile.NamedTemporaryFile(suffix=".jpg") as tmp:
                tmp_path = Path(tmp.name)
                self.pil_image.save(tmp_path, format="JPEG", quality=95)
                return await fal_client.upload_file_async(tmp_path)
class ImageToImageConfig(BaseModel):
class ImageSizes(str, Enum):
class ImageFormats(str, Enum):
class ImageSizeWH(BaseModel):
class LoraRecord(BaseModel):
class LoraRecordList(RootModel[list[LoraRecord]]):
class LoraLib(RootModel[dict[str, LoraRecordList]]):
LORA_LIB_PATH = Path(__file__).parent / f"{Path(__file__).stem}_loras.json"
LORA_LIB = LoraLib.model_validate_json(LORA_LIB_PATH.read_text())
class ImageResult(BaseModel):
class LoraSpecEntry(BaseModel):
class CombinedLoraSpecEntry(BaseModel):
def parse_lora_phrase(phrase: str) -> LoraSpecEntry | CombinedLoraSpecEntry:
    phrase = phrase.strip()
            entries.append(
                LoraSpecEntry(path=record.url, scale=record.scale, prompt=phrase + ",")
        return CombinedLoraSpecEntry(entries=entries, factory_key=phrase)
        identifier, scale_str = phrase.split(":", 1)
        identifier = identifier.strip()
            scale = float(scale_str.strip())
    return LoraSpecEntry(path=identifier, scale=scale, prompt="")
def normalize_lora_spec(
                        normalized.append(
                            LoraSpecEntry(path=path, scale=float(scale), prompt=prompt)
                            LoraSpecEntry(
                                scale=float(d.get("scale", 1.0)),
                                prompt=d.get("prompt", ""),
                        normalized.append(parse_lora_phrase(phrase))
                            parse_lora_phrase(sub)
                            if isinstance(sub, str)
                        normalized.append(CombinedLoraSpecEntry(entries=combined))
                        msg = f"Unsupported Lora spec item type: {type(item)}"
                return [parse_lora_phrase(s)]
            phrases = [phrase.strip() for phrase in s.split(";") if phrase.strip()]
            return [parse_lora_phrase(phrase) for phrase in phrases]
            msg = f"Unsupported Lora spec type: {type(spec)}"
def build_lora_arguments(
    entries = normalize_lora_spec(lora_spec)
    def process_entry(entry: LoraSpecEntry | CombinedLoraSpecEntry) -> None:
        if isinstance(entry, LoraSpecEntry):
            lora_list.append({"path": entry.path, "scale": entry.scale})
                prompt_prefixes.append(entry.prompt.rstrip(","))
                process_entry(sub_entry)
        process_entry(entry)
        f"{', '.join(prompt_prefixes)}, {prompt}".strip() if prompt_prefixes else prompt
def split_top_level(s: str, delimiter: str = ";") -> list[str]:
            parts.append("".join(buf))
            buf.append(char)
def expand_prompts(s: str) -> list[str]:
    open_index = s.find("{")
    for i, char in enumerate(s[open_index:], start=open_index):
    alternatives = split_top_level(brace_content, delimiter=";")
        for alt_exp in expand_prompts(alt):
            for suffix_exp in expand_prompts(suffix):
                expanded.append(f"{prefix}{alt_exp}{suffix_exp}")
class TTIJobConfig(BaseModel):
    async def to_fal_arguments(self) -> dict[str, Any]:
            if isinstance(self.image_size, ImageSizes)
        lora_list, final_prompt = build_lora_arguments(self.lora_spec, self.prompt)
        logger.debug(f"Using Lora configuration: {lora_list}")
            guidance_scale = max(self.guidance_scale, 20.0)
            image_url = await self.image_config.input_image.to_url()
async def submit_image_job(job: TTIJobConfig) -> RequestID:
    args = await job.to_fal_arguments()
    logger.debug(f"Submitting job with lora_list: {args.get('loras')}")
    handler = await fal_client.submit_async(job.model.value, arguments=args)
async def download_image(url: URLStr, output_path: Path) -> None:
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        response.raise_for_status()
        output_path.write_bytes(response.content)
        logger.info(f"Saved image to: {output_path}")
async def get_result(
    status = await fal_client.status_async(
    while isinstance(status, fal_client.InProgress):
        await asyncio.sleep(1)
    result = await fal_client.result_async("fal-ai/flux-lora", request_id)
    timestamp = result.get("timestamp", datetime.now().strftime("%Y%m%d_%H%M%S"))
    content_type = image_info.get("content_type", "image/jpeg")
        output_dir.mkdir(parents=True, exist_ok=True)
        request_id_prefix = request_id.split("-")[0] if request_id else ""
            words = original_prompt.split()
                "_".join(words[:2]).lower() + "_"
                if len(words) >= 2
        filename = slugify(stem) + f".{extension}"
            await download_image(image_url, image_path)
            logger.error(f"Failed to save image {image_path}: {e}")
            metadata_path = output_dir / f"{slugify(stem)}.json"
            metadata = ImageResult(
                    "path": str(image_path) if image_path else None,
                    "metadata_path": str(metadata_path) if metadata_path else None,
            metadata_path.write_text(metadata.model_dump_json(indent=2))
            logger.info(f"Saved metadata to: {metadata_path}")
            logger.error(f"Failed to save metadata {metadata_path}: {e}")
    return ImageResult(
async def process_single_job(job: TTIJobConfig) -> ImageResult:
    request_id = await submit_image_job(job)
    logger.info(f"Submitted job with ID: {request_id}")
        suffix_parts.append(job.filename_suffix)
    elif isinstance(job.lora_spec, str):
        suffix_parts.append(job.lora_spec)
    elif isinstance(job.lora_spec, CombinedLoraSpecEntry) and job.lora_spec.factory_key:
        suffix_parts.append(slugify(job.lora_spec.factory_key)[:8])
    combined_suffix = "_".join(suffix_parts) if suffix_parts else None
        if isinstance(job.image_size, Enum)
    return await get_result(
async def async_main(
        split_top_level(prompts, delimiter=";") if isinstance(prompts, str) else prompts
        final_prompts.extend(expand_prompts(raw.strip()))
    logger.debug(f"Expanded prompts: {final_prompts}")
    if lora and isinstance(lora, str) and lora in LORA_LIB.root and not filename_suffix:
        default_suffix = slugify(lora)[:8]
        size: ImageSize = ImageSizes[image_size.upper()]
                w, h = (int(x.strip()) for x in image_size.split(",", 1))
                size = ImageSizeWH(width=w, height=h)
                raise ValueError(msg) from err
            valid_names = ", ".join(s.name for s in ImageSizes)
    output_dir_path = Path(output_dir)
    output_dir_path.mkdir(parents=True, exist_ok=True)
            jobs.append(
                TTIJobConfig(
    return await asyncio.gather(*(process_single_job(job) for job in jobs))
    fire.Fire(cli)

================
File: plugins/repos/twat_genai/src/twat_genai/__version__.py
================


================
File: plugins/repos/twat_genai/src/twat_genai/cli.py
================
def parse_image_size(size_str: str) -> ImageSizes | ImageSizeWH:
        return ImageSizes[size_str.upper()]
                w, h = (int(x.strip()) for x in size_str.split(",", 1))
                return ImageSizeWH(width=w, height=h)
                raise ValueError(msg) from err
        valid_names = ", ".join(s.name for s in ImageSizes)
        raise ValueError(msg)
def get_output_dir(user_dir: str | Path | None = None) -> Path:
        return Path(user_dir)
    paths = PathManager.for_package("twat_genai")
    return Path("generated_images")
async def async_main(
    output_dir_path = get_output_dir(output_dir)
    output_dir_path.mkdir(parents=True, exist_ok=True)
    config = EngineConfig(
        image_size=parse_image_size(image_size),
    final_prompts = normalize_prompts(prompts)
    logger.debug(f"Expanded prompts: {final_prompts}")
    async with FALEngine(output_dir_path) as engine:
            result = await engine.generate(
            results.append(result)
def cli(
    logger.remove()
    logger.add(sys.stderr, level="DEBUG" if verbose else "WARNING")
    if isinstance(model, str):
            model = ModelTypes[model.upper()]
            valid_models = ", ".join(m.name.lower() for m in ModelTypes)
        image_config = ImageToImageConfig(
            input_image=ImageInput(
                path=Path(input_image) if input_image else None,
    return asyncio.run(
        async_main(
    fire.Fire(cli)

================
File: plugins/repos/twat_genai/tests/test_twat_genai.py
================
def test_version():

================
File: plugins/repos/twat_genai/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_genai/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_genai/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(["tree", "-a", "-I", ".git", "--gitignore", "-n", "-h", "-I", "*_cache"])
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_genai/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_genai/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to the twat-genai project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v1.7.5] - 2025-02-15

### Changed

- Improved error message formatting across the codebase
- Updated type hints to use modern Python syntax (e.g., `list[str]` instead of `List[str]`)
- Simplified union type hints using `|` operator (e.g., `str | None` instead of `Optional[str]`)

### Fixed

- Fixed circular imports in FAL engine modules
- Improved error handling and messaging in LoRA processing

## [v1.7.3] - 2025-02-15

### Added

- New FAL-specific image input handling
- Added `FALImageInput` class for better FAL API integration

### Changed

- Refactored image input processing for better type safety
- Updated dependency requirements for better compatibility

## [v1.6.2] - 2025-02-06

### Changed

- Updated dependency versions:
  - `twat>=1.0.0`
  - `twat-image>=1.0.0`

### Fixed

- Package dependency issues

## [v1.6.1] - 2025-02-06

### Changed

- Reorganized module exports in `__init__.py`
- Improved code organization and imports
- Enhanced type annotations throughout the codebase

### Fixed

- Various import and circular dependency issues
- Code style and formatting improvements

## [v1.6.0] - 2025-02-06

### Added

- Initial public release with core functionality
- Support for text-to-image generation
- Support for image-to-image transformation
- LoRA integration with FAL.ai
- Command-line interface
- Python API
- Configuration management
- Image processing utilities

### Features

- Multiple model support through FAL.ai
- Flexible prompt expansion system
- LoRA configuration management
- Image size presets and custom sizes
- Output directory management
- File naming conventions
- Environment variable configuration

## [v1.0.0] - 2025-02-06

### Added

- Initial project structure
- Basic package setup
- Core dependencies
- Development environment configuration

[v1.7.5]: https://github.com/twardoch/twat-genai/compare/v1.7.3...v1.7.5
[v1.7.3]: https://github.com/twardoch/twat-genai/compare/v1.6.2...v1.7.3
[v1.6.2]: https://github.com/twardoch/twat-genai/compare/v1.6.1...v1.6.2
[v1.6.1]: https://github.com/twardoch/twat-genai/compare/v1.6.0...v1.6.1
[v1.6.0]: https://github.com/twardoch/twat-genai/compare/v1.0.0...v1.6.0
[v1.0.0]: https://github.com/twardoch/twat-genai/releases/tag/v1.0.0

================
File: plugins/repos/twat_genai/pyproject.toml
================
# this_file: twat_genai/pyproject.toml

# this_file: twat_genai/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_genai"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-genai"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "fal-client>=0.5.9",         # FAL.ai client for AI operations
    "httpx>=0.28.1",             # HTTP client library
    "numpy>=2.2.3",              # Numerical computing library
    "Pillow>=11.1.0",            # Image processing library
    "pydantic>=2.10.6",          # Data validation using Python type annotations
    "python-slugify>=8.0.4",     # Text slugification library
    "twat>=1.8.1",               # Main twat package
    "twat-image>=1.8.1",         # Image handling functionality
    "webcolors>=24.11.1",        # Color name and value conversion
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-genai#readme"
Issues = "https://github.com/twardoch/twat-genai/issues"
Source = "https://github.com/twardoch/twat-genai"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
genai = "twat_genai"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_genai/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_genai --cov=tests {args:tests}"
type-check = "mypy src/twat_genai tests"
lint = ["ruff check src/twat_genai tests", "ruff format src/twat_genai tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_genai tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_genai", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_genai/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_genai = ["src/twat_genai", "*/twat-genai/src/twat_genai"]
tests = ["tests", "*/twat-genai/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "fal-client>=0.5.9",         # FAL.ai client
    "httpx>=0.28.1",             # HTTP client
    "numpy>=2.2.3",              # Numerical computing
    "Pillow>=11.1.0",            # Image processing
    "pydantic>=2.10.6",          # Data validation
    "python-slugify>=8.0.4",     # Text slugification
    "twat>=1.8.1",               # Main package
    "twat-image>=1.8.1",         # Image handling
    "webcolors>=24.11.1",        # Color utilities
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_genai --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_genai/README.md
================
# twat-genai

(work in progress)

Image generation package that leverages fal.ai's models for AI image generation. Provides a flexible command-line interface and Python API for generating images using various AI models and techniques.

## Features

- Multiple AI image generation modes:
  - Text-to-image generation
  - Image-to-image transformation
  - Canny edge-guided generation
  - Depth-guided generation
- Support for LoRA (Low-Rank Adaptation) models with a built-in library of style presets
- Flexible prompt expansion with alternatives using brace syntax
- Concurrent image generation for multiple prompts
- Comprehensive metadata storage for generated images
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-genai
```


## Usage

### Command Line Interface

```bash
## Basic text-to-image generation
python -m twat_genai "a beautiful sunset" --output_dir images
## Using a specific style from the LoRA library
python -m twat_genai "a beautiful sunset" --lora "shou_xin"
## Image-to-image transformation
python -m twat_genai "enhance this photo" --model image --input_image input.jpg
## Multiple prompts with alternatives
python -m twat_genai "a {red; blue; green} house with {white; black} windows"
```

### Python API

```python
import twat_genai
from twat_genai.main import async_main, ModelTypes

## Generate images asynchronously
results = await async_main(
prompts="a beautiful sunset",
output_dir="generated_images",
model=ModelTypes.TEXT,
lora="shou_xin",
image_size="SQ"
)   
```


## Key Features in Detail

### Prompt Expansion
The tool supports flexible prompt expansion using brace syntax:
- `"a {red; blue} house"` generates two images: "a red house" and "a blue house"
- Nested alternatives are supported
- Semicolons separate alternatives

### LoRA Styles
Built-in library of LoRA styles for different artistic effects:
- Gesture drawing
- Sketch and smudge effects
- 2-color illustrations
- Pencil sketches
- Tarot card style
- And more...

### Image Generation Modes
- **Text-to-Image**: Generate images from text descriptions
- **Image-to-Image**: Transform existing images
- **Canny Edge**: Use edge detection to guide generation
- **Depth-Guided**: Use depth information for generation

### Output Management
- Automatic file naming with customizable prefixes/suffixes
- Metadata storage in JSON format
- Various image size options (square, landscape, portrait)
- Support for custom dimensions

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
## Install hatch if you haven't already
pip install hatch
## Create and activate development environment
hatch shell
## Run tests
hatch run test
## Run tests with coverage
hatch run test-cov
## Run linting
hatch run lint
## Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_genai/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_hatch/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_hatch/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_hatch/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 800]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [3.7K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 939]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [3.1K]  LOG.md
├── [6.8K]  README.md
├── [   8]  TODO.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [8.2K]  pyproject.toml
├── [ 128]  src
│   └── [ 352]  twat_hatch
│       ├── [ 224]  __init__.py
│       ├── [ 11K]  __main__.py
│       ├── [5.0K]  config.py
│       ├── [ 19K]  hatch.py
│       ├── [ 448]  themes
│       │   ├── [  46]  __init__.py
│       │   ├── [  64]  _default
│       │   ├── [ 160]  _shared
│       │   │   ├── [1.1K]  base.toml.j2
│       │   │   └── [ 256]  snippets
│       │   │       ├── [ 128]  author.toml.j2
│       │   │       ├── [ 896]  dependencies.toml.j2
│       │   │       ├── [ 280]  development.toml.j2
│       │   │       ├── [ 186]  features.toml.j2
│       │   │       ├── [ 202]  package.toml.j2
│       │   │       └── [2.0K]  tools.toml.j2
│       │   ├── [ 352]  default
│       │   │   ├── [  96]  .github
│       │   │   │   └── [ 160]  workflows
│       │   │   │       ├── [2.8K]  push.yml.j2
│       │   │   │       └── [1.4K]  release.yml.j2
│       │   │   ├── [ 502]  .pre-commit-config.yaml.j2
│       │   │   ├── [1.1K]  LICENSE.j2
│       │   │   ├── [ 820]  README.md.j2
│       │   │   ├── [  96]  dist
│       │   │   │   └── [   1]  .gitkeep.j2
│       │   │   ├── [3.5K]  hidden.gitignore.j2
│       │   │   ├── [6.5K]  pyproject.toml.j2
│       │   │   └── [  96]  tests
│       │   │       └── [ 173]  test_package.py.j2
│       │   ├── [ 160]  package
│       │   │   ├── [ 426]  package.toml.j2
│       │   │   └── [ 128]  src
│       │   │       └── [  96]  __package_name__
│       │   │           └── [1.8K]  __package_name__.py.j2
│       │   ├── [ 854]  package.toml.j2
│       │   ├── [ 160]  plugin
│       │   │   ├── [ 374]  README.md.j2
│       │   │   ├── [ 355]  pyproject.toml.j2
│       │   │   └── [  96]  src
│       │   │       └── [  96]  __package_name__
│       │   │           └── [ 130]  __init__.py.j2
│       │   ├── [1.2K]  plugin.toml.j2
│       │   ├── [ 160]  plugin_host
│       │   │   ├── [ 561]  README.md.j2
│       │   │   ├── [ 308]  pyproject.toml.j2
│       │   │   └── [  96]  src
│       │   │       └── [2.1K]  __init__.py.j2
│       │   └── [1.3K]  plugin_host.toml.j2
│       └── [8.3K]  utils.py
└── [ 128]  tests
    └── [ 154]  test_twat_hatch.py

27 directories, 50 files

================
File: plugins/repos/twat_hatch/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_hatch/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_hatch --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_hatch/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-hatch
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_hatch/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_hatch/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_hatch/dist/.gitignore
================
*

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/snippets/author.toml.j2
================
# Author information
[author]
name = "{{ author_name }}"
email = "{{ author_email }}"
github_username = "{{ github_username }}"

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/snippets/dependencies.toml.j2
================
# Dependencies
[dependencies]
# Regular package dependencies
dependencies = [
    {% if is_plugin_host %}
    "importlib-metadata>=6.0.0",
    "importlib-resources>=5.0.0",
    "typing-extensions>=4.0.0; python_version < '3.10'",
    {% elif is_plugin %}
    "{{ plugin_host }}",
    "pydantic>=2.0.0",
    {% endif %}
    {% for dep in dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]

{% if is_plugin %}
# Plugin-specific dependencies
plugin_dependencies = [
    {% for dep in plugin_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]
{% endif %}

# Development dependencies
dev_dependencies = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "pytest-xdist>=3.5.0",
    "pytest-benchmark[histogram]>=4.0.0",
    "ruff>=0.1.0",
    "mypy>=1.0.0",
    "pre-commit>=3.6.0",
    {% for dep in dev_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/snippets/development.toml.j2
================
# Development configuration
[development]
# Additional development dependencies
additional_dependencies = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "ruff>=0.1.0",
    "mypy>=1.0.0",
    {% for dep in dev_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/snippets/features.toml.j2
================
# Feature flags
[features]
mkdocs = {{ use_mkdocs | default(false) | lower }}  # Enable MkDocs documentation
vcs = {{ use_vcs | default(true) | lower }}         # Enable VCS integration

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/snippets/package.toml.j2
================
# Package configuration
[package]
min_python = "{{ min_python }}"
{% if max_python %}max_python = "{{ max_python }}"{% endif %}
license = "{{ license }}"
development_status = "{{ development_status }}"

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/snippets/tools.toml.j2
================
# Tool configurations
[tool.ruff]
target-version = "{{ python_version_info.ruff_target }}"
line-length = 88
lint.extend-select = [
    "A",    # flake8-builtins
    "ARG",  # flake8-unused-arguments
    "B",    # flake8-bugbear
    "C",    # flake8-comprehensions
    "DTZ",  # flake8-datetimez
    "E",    # pycodestyle errors
    "EM",   # flake8-errmsg
    "F",    # pyflakes
    "FBT",  # flake8-boolean-trap
    "I",    # isort
    "ICN",  # flake8-import-conventions
    "ISC",  # flake8-implicit-str-concat
    "N",    # pep8-naming
    "PLC",  # pylint convention
    "PLE",  # pylint error
    "PLR",  # pylint refactor
    "PLW",  # pylint warning
    "Q",    # flake8-quotes
    "RUF",  # Ruff-specific rules
    "S",    # flake8-bandit
    "T",    # flake8-print
    "TID",  # flake8-tidy-imports
    "UP",   # pyupgrade
    "W",    # pycodestyle warnings
    "YTT",  # flake8-2020
]
lint.ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",
]

[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "lf"

[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

[tool.mypy]
python_version = "{{ python_version_info.mypy_version }}"
# Type checking
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
strict_optional = true

# Error reporting
warn_return_any = true
warn_unused_configs = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true
show_error_codes = true
pretty = true

# Import discovery
ignore_missing_imports = false
follow_imports = "normal"
follow_imports_for_stubs = true

# Advanced
strict_equality = true
extra_checks = true

{% if is_plugin_host or is_plugin %}
plugins = ["pydantic.mypy"]

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true
{% endif %}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/_shared/base.toml.j2
================
# Base configuration for all package types
{% block header_comment %}# Package configuration{% endblock %}

[project]
{% block project %}
# List of packages to initialize
packages = [
    {% block packages %}
    "{{ name }}"
    {% endblock %}
]

{% block plugin_host %}
{% if is_plugin_host %}
# This package will be the plugin host
plugin_host = "{{ name }}"
{% elif is_plugin %}
# Plugin host package
plugin_host = "{{ plugin_host }}"
{% endif %}
{% endblock %}

# Output directory (optional, defaults to current directory)
output_dir = "."
{% endblock %}

{% include '_shared/snippets/author.toml.j2' %}

{% include '_shared/snippets/package.toml.j2' %}

{% block dependencies %}
{% include '_shared/snippets/dependencies.toml.j2' %}

{% if is_plugin_host %}
plugin_dependencies = [
    "pydantic>=2.0.0",
    "importlib-resources>=5.0.0",
    {% for dep in plugin_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]
{% endif %}
{% endblock %}

{% include '_shared/snippets/development.toml.j2' %}

{% include '_shared/snippets/tools.toml.j2' %}

{% include '_shared/snippets/features.toml.j2' %}

{% block additional_config %}{% endblock %}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/.github/workflows/push.yml.j2
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ "{{ github.workflow }}" }}-${{ "{{ github.ref }}" }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: {{ python_version_info.classifiers | map('split', ' :: ') | map('last') | list | tojson }}
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ "{{ matrix.os }}" }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ "{{ matrix.python-version }}" }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ "{{ matrix.python-version }}" }}
          enable-cache: true
          cache-suffix: ${{ "{{ matrix.os }}" }}-${{ "{{ matrix.python-version }}" }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/{{ import_name }} --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ "{{ matrix.python-version }}" }}-${{ "{{ matrix.os }}" }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/.github/workflows/release.yml.j2
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/{{ name }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ "{{ secrets.PYPI_TOKEN }}" }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ "{{ secrets.GITHUB_TOKEN }}" }}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/dist/.gitkeep.j2
================


================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/tests/test_package.py.j2
================
"""Test suite for {{ import_name }}."""

def test_version():
    """Verify package exposes version."""
    import {{ import_name }}
    assert {{ import_name }}.__version__

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/.pre-commit-config.yaml.j2
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: trailing-whitespace
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/hidden.gitignore.j2
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/LICENSE.j2
================
MIT License

Copyright (c) {{ '%Y' | strftime }} {{ author_name }}

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/pyproject.toml.j2
================
# this_file: pyproject.toml
[project]
name = "{{ name }}"
dynamic = ["version"]
description = "{{ description }}"
readme = "README.md"
requires-python = "{{ python_version_info.requires_python }}"
license = "{{ license }}"
keywords = []
classifiers = [
    "Development Status :: {{ development_status }}",
    "Programming Language :: Python",
    {% for classifier in python_version_info.classifiers %}
    "{{ classifier }}",
    {% endfor %}
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

{% block dependencies %}
dependencies = [
    {% for dep in dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]
{% endblock %}


[project.optional-dependencies]
{% block optional_dependencies %}

dev = [
{% block dev_dependencies %}
    "pre-commit>=3.6.0",
    "ruff>=0.1.0",
    "mypy>=1.0.0",
    "pyupgrade>=3.19.0",
{% endblock %}
]

test = [
{% block test_dependencies %}
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
{% endblock %}
]

{% endblock %}

all = [
{% block all_dependencies %}
{% for dep in dev_dependencies | default([]) %}
    "{{ dep }}",
{% endfor %}
{% for dep in test_dependencies | default([]) %}
    "{{ dep }}",
{% endfor %}
{% if additional_dependencies %}
    "{{ additional_dependencies | join('", "') }}",
{% endif %}
{% endblock %}
]

[project.scripts]
{% block entry_points %}
# CLINAME = "{{ import_name }}.__main__:main"
{% endblock %}



[[project.authors]]
name = "{{ author_name }}"
email = "{{ author_email }}"

[project.urls]
Documentation = "https://github.com/{{ github_username }}/{{ name }}#readme"
Issues = "https://github.com/{{ github_username }}/{{ name }}/issues"
Source = "https://github.com/{{ github_username }}/{{ name }}"


[build-system]
build-backend = "hatchling.build"
requires = [
    "hatchling>=1.21.0", 
    "hatch-vcs>=0.3.0"
]


[tool.coverage.paths]
{{ import_name }} = ["src/{{ import_name }}", "*/{{ name }}/src/{{ import_name }}"]
tests = ["tests", "*/{{ name }}/tests"]



[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

[tool.coverage.run]
source_pkgs = ["{{ import_name }}", "tests"]
branch = true
parallel = true
omit = [
    "src/{{ import_name }}/__about__.py",
]



[tool.hatch.build.hooks.vcs]
version-file = "src/{{ import_name }}/__version__.py"


[tool.hatch.build.targets.wheel]
packages = ["src/{{ import_name }}"]



[tool.hatch.envs.default]
dependencies = [
    {% for dep in test_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
    {% for dep in dev_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
    {% if additional_dependencies %}
    "{{ additional_dependencies | join('", "') }}",
    {% endif %}
]

[[tool.hatch.envs.all.matrix]]
python = {{ python_version_info.classifiers | map('split', ' :: ') | map('last') | list | tojson }}


[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/{{ import_name }} --cov=tests {args:tests}"
type-check = "mypy src/{{ import_name }} tests"
lint = ["ruff check src/{{ import_name }} tests", "ruff format --respect-gitignore src/{{ import_name }} tests"]
fix = ["ruff check  --fix --unsafe-fixes src/{{ import_name }} tests", "ruff format --respect-gitignore src/{{ import_name }} tests"]



[tool.hatch.envs.lint]
detached = true
dependencies = [
    {% for dep in dev_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
    {% if additional_dependencies %}
    "{{ additional_dependencies | join('", "') }}",
    {% endif %}
]


[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/{{ import_name }} tests}"
style = ["ruff check {args:.}", "ruff format --respect-gitignore {args:.}"]
fmt = ["ruff format --respect-gitignore {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]


[tool.hatch.envs.test]
dependencies = [
    {% for dep in test_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]

[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto -p no:briefcase {args:tests}"
test-cov = "python -m pytest -n auto -p no:briefcase --cov-report=term-missing --cov-config=pyproject.toml --cov=src/{{ import_name }} --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

[tool.hatch.version]
source = "vcs"


[tool.hatch.version.raw-options]
version_scheme = "post-release"


[tool.mypy]
python_version = "{{ python_version_info.mypy_version }}"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true


[tool.ruff]
target-version = "{{ python_version_info.ruff_target }}"
line-length = 88

[tool.ruff.lint]
extend-select = [
    "A",
    "ARG",
    "B",
    "C",
    "DTZ",
    "E",
    "EM",
    "F",
    "FBT",
    "I",
    "ICN",
    "ISC",
    "N",
    "PLC",
    "PLE",
    "PLR",
    "PLW",
    "Q",
    "RUF",
    "S",
    "T",
    "TID",
    "UP",
    "W",
    "YTT",
]
ignore = ["ARG001", "E501", "I001", "RUF001", "PLR2004", "EXE003", "ISC001"]



[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]





[tool.pytest.ini_options]
addopts = "-v --durations=10 -p no:briefcase"
asyncio_mode = "auto"
console_output_style = "progress"
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
log_cli = true
log_cli_level = "INFO"
markers = [
  "benchmark: marks tests as benchmarks (select with '-m benchmark')",
  "unit: mark a test as a unit test",
  "integration: mark a test as an integration test",
  "permutation: tests for permutation functionality",
  "parameter: tests for parameter parsing",
  "prompt: tests for prompt parsing",
]
norecursedirs = [
  ".*",
  "build",
  "dist",
  "venv",
  "__pycache__",
  "*.egg-info",
  "_private",
]

python_classes = ["Test*"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
testpaths = ["tests"]


[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/default/README.md.j2
================
# {{ title }}

{{ description }}

## Features

{% block features %}
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration
{% endblock %}

## Installation

```bash
pip install {{ name }}
```

## Usage

{% block usage %}
```python
import {{ import_name }}
```
{% endblock %}

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

{{ license }} License

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/package/src/__package_name__/__package_name__.py.j2
================
#!/usr/bin/env python3
"""{{ import_name }}: {{ description }}

Created by {{ author_name }}
"""

from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, Union
import logging

__version__ = "{{ version | default('0.1.0') }}"

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
logger = logging.getLogger(__name__)


@dataclass
class Config:
    """Configuration settings for {{ import_name }}."""
    name: str
    value: Union[str, int, float]
    options: Optional[Dict[str, Any]] = None


def process_data(
    data: List[Any],
    config: Optional[Config] = None,
    *,
    debug: bool = False
) -> Dict[str, Any]:
    """Process the input data according to configuration.
    
    Args:
        data: Input data to process
        config: Optional configuration settings
        debug: Enable debug mode
        
    Returns:
        Processed data as a dictionary
        
    Raises:
        ValueError: If input data is invalid
    """
    if debug:
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        
    if not data:
        raise ValueError("Input data cannot be empty")
        
    # TODO: Implement data processing logic
    result: Dict[str, Any] = {}
    return result


def main() -> None:
    """Main entry point for {{ import_name }}."""
    try:
        # Example usage
        config = Config(
            name="default",
            value="test",
            options={"key": "value"}
        )
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        
    except Exception as e:
        logger.error("An error occurred: %s", str(e))
        raise


if __name__ == "__main__":
    main()

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/package/package.toml.j2
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin/src/__package_name__/__init__.py.j2
================
"""{{ plugin_host }} {{ plugin_import_name }} plugin"""

from importlib import metadata

__version__ = metadata.version(__name__)

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin/pyproject.toml.j2
================
{% extends "default/pyproject.toml.j2" %}

{% block dependencies %}
dependencies = [
    "{{ plugin_host }}",
    {% for dep in plugin_dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]
{% endblock %}

{% block entry_points %}
[project.entry-points."{{ plugin_host }}.plugins"]
{{ plugin_import_name }} = "{{ import_name }}"
{% endblock %}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin/README.md.j2
================
{% extends "default/README.md.j2" %}

{% block features %}
- Plugin for {{ plugin_host }}
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration
{% endblock %}

{% block usage %}
```python
import {{ import_name }}
plugin = {{ import_name }}.plugin
```
{% endblock %}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin_host/src/__init__.py.j2
================
"""{{ description }}"""

import sys
from importlib import metadata
from typing import Any

__version__ = metadata.version(__name__)

# Enable package-style imports for plugins
__path__ = []
__package__ = "{{ name }}"


class PluginFinder:
    """Finder for {{ name }} plugins to enable package-style imports."""

    @classmethod
    def find_spec(cls, fullname, path, target=None):
        if not fullname.startswith("{{ name }}."):
            return None

        plugin_name = fullname.split(".")[-1]
        try:
            eps = metadata.entry_points(group="{{ name }}.plugins")
            for ep in eps:
                if ep.name == plugin_name:
                    # Create a spec that will load the plugin module
                    from importlib.machinery import ModuleSpec
                    from importlib.util import module_from_spec, spec_from_loader

                    plugin_module = ep.load()

                    class Loader:
                        @staticmethod
                        def create_module(spec):
                            return plugin_module

                        @staticmethod
                        def exec_module(module):
                            pass

                    return ModuleSpec(fullname, Loader())
        except Exception:
            return None
        return None


# Register the finder
sys.meta_path.insert(0, PluginFinder())


def __getattr__(name: str) -> Any:
    """Dynamic attribute lookup for plugins.

    To register a plugin, add the following to your pyproject.toml:

    [project.entry-points."{{ name }}.plugins"]
    plugin_name = "plugin_module"  # Just point to the module

    Args:
        name: Name of plugin to load

    Returns:
        Loaded plugin module

    Raises:
        AttributeError: If plugin cannot be found or loaded
    """
    try:
        eps = metadata.entry_points(group="{{ name }}.plugins")
        for ep in eps:
            if ep.name == name:
                return ep.load()
    except Exception as e:
        raise AttributeError(f"Failed to load plugin '{name}': {e}") from e

    raise AttributeError(f"module '{__name__}' has no attribute '{name}'")

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin_host/pyproject.toml.j2
================
{% extends "default/pyproject.toml.j2" %}

{% block dependencies %}
dependencies = [
    "importlib-metadata>=6.0.0",
    "importlib-resources>=5.0.0",
    "typing-extensions>=4.0.0; python_version < '3.10'",
    {% for dep in dependencies | default([]) %}
    "{{ dep }}",
    {% endfor %}
]
{% endblock %}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin_host/README.md.j2
================
{% extends "default/README.md.j2" %}

{% block features %}
- Plugin system for Python packages
- Dynamic plugin loading and discovery
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration
{% endblock %}

{% block usage %}
```python
import {{ import_name }}

# Load a plugin
plugin = {{ import_name }}.plugin_name

# List available plugins
from importlib.metadata import entry_points
plugins = entry_points(group="{{ name }}.plugins")
```
{% endblock %}

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/__init__.py
================


================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/package.toml.j2
================
# Configuration for standalone Python package

[project]
# List of packages to initialize
packages = [
    "{{ name }}"  # Your package name
]

# Output directory (optional, defaults to current directory)
output_dir = "."

[author]
name = "{{ author_name }}"
email = "{{ author_email }}"
github_username = "{{ github_username }}"

[package]
description = "{{ description }}"
min_python = "3.10"
license = "MIT"
development_status = "4 - Beta"

[package.options]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin_host.toml.j2
================
{% extends "_shared/base.toml.j2" %}

{% block header_comment %}# Plugin host package configuration{% endblock %}

{% set is_plugin_host = true %}

{% block additional_config %}
# Plugin host specific configuration
[plugin.options]
auto_discovery = {{ auto_discovery | default(true) | lower }}
lazy_loading = {{ lazy_loading | default(true) | lower }}
namespace_imports = {{ namespace_imports | default(true) | lower }}
{% endblock %}

[project]
# List of packages to initialize
packages = [
    "{{ name }}"  # Plugin host package name
]

# Refers to the plugin host package (this package will be the plugin host)
plugin_host = "{{ name }}"

# Output directory (optional, defaults to current directory)
output_dir = "."

[author]
name = "{{ author_name }}"
email = "{{ author_email }}"
github_username = "{{ github_username }}"

[package]
description = "{{ description }}"
min_python = "3.10"
license = "MIT"
development_status = "4 - Beta"

[package.options]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_hatch/src/twat_hatch/themes/plugin.toml.j2
================
{% extends "_shared/base.toml.j2" %}

{% block header_comment %}# Plugin package configuration{% endblock %}

{% set is_plugin = true %}

{% block additional_config %}
# Plugin-specific configuration
[plugin.options]
auto_discovery = {{ auto_discovery | default(true) | lower }}
namespace_imports = {{ namespace_imports | default(true) | lower }}
{% endblock %}

[project]
# List of packages to initialize
packages = [
    "{{ name }}"  # Your plugin name
]

# Plugin host package (required)
plugin_host = "{{ plugin_host }}"

# Output directory (optional, defaults to current directory)
output_dir = "."

[author]
name = "{{ author_name }}"
email = "{{ author_email }}"
github_username = "{{ github_username }}"

[package]
description = "{{ description }}"
min_python = "3.10"
license = "MIT"
development_status = "4 - Beta"

[package.options]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_hatch/src/twat_hatch/__init__.py
================
__version__ = metadata.version(__name__)

================
File: plugins/repos/twat_hatch/src/twat_hatch/__main__.py
================
install(show_locals=True)
ansi_decoder = AnsiDecoder()
console = Console(theme=Theme({"prompt": "cyan", "question": "bold cyan"}))
class ConfigurationPrompts:
    def get_package_name(self, package_type: PackageType) -> str:
        return Prompt.ask(
    def get_plugin_host(self) -> str:
    def get_author_info(self) -> dict[str, str]:
            "author_name": Prompt.ask(
            "author_email": Prompt.ask(
            "github_username": Prompt.ask(
    def get_python_versions(self) -> dict[str, str | None]:
        min_major = IntPrompt.ask(
        min_minor = IntPrompt.ask(
        min_ver = PyVer(min_major, min_minor)
        if Confirm.ask(
            max_major = IntPrompt.ask(
            max_minor = IntPrompt.ask(
            max_ver = PyVer(max_major, max_minor)
            max_python = str(max_ver)
            "min_python": str(min_ver),
    def get_package_info(self) -> dict[str, Any]:
            "license": Prompt.ask(
            "development_status": Prompt.ask(
    def get_features(self) -> dict[str, bool]:
            "use_mkdocs": Confirm.ask(
            "use_vcs": Confirm.ask(
def init(
            min_ver = PyVer.from_cli_input(min_python)
                PyVer.from_cli_input(max_python) if max_python is not None else None
            console.print(f"[red]Error: {e}[/]")
            sys.exit(1)
            for k, v in locals().items()
        interactive = not bool(user_provided_values)
            prompts = ConfigurationPrompts()
            name = prompts.get_package_name(type)
                plugin_host = prompts.get_plugin_host()
            author_info = prompts.get_author_info()
            python_versions = prompts.get_python_versions()
            min_ver = PyVer.parse(python_versions["min_python"])
            max_ver = PyVer.parse(python_versions.get("max_python"))
            package_info = prompts.get_package_info()
            features = prompts.get_features()
        config_generator = ConfigurationGenerator()
        config = config_generator.generate_config(
            min_python=str(min_ver),
            max_python=str(max_ver) if max_ver else None,
        output_path = Path(output)
        output_path.write_text(config)
        console.print(
            Panel(
        console.print("[red]Error: Invalid configuration[/]")
        console.print(e)
def config(command: str = "show", type: PackageType = "package") -> None:
        console.print("[red]Invalid command. Use 'show'.[/]")
        generator = ConfigurationGenerator()
        content = generator.generate_config(type, interactive=False)
        console.print(Panel(content, title=f"Example {type} configuration"))
        console.print(f"[red]Error showing configuration: {e!s}[/]")
def create(config_path: str | None = None) -> None:
        initializer = PackageInitializer(config_path=config_path)
        initializer.initialize_all()
        console.print(f"[red]Error creating packages: {e!s}[/]")
def main() -> None:
    def display(lines, out):
        console.print(Group(*map(ansi_decoder.decode_line, lines)))
    fire.Fire(
    main()

================
File: plugins/repos/twat_hatch/src/twat_hatch/__version__.py
================


================
File: plugins/repos/twat_hatch/src/twat_hatch/config.py
================
class PackageTemplate:
    "package": PackageTemplate(
    "plugin": PackageTemplate(
    "plugin-host": PackageTemplate(
class ConfigurationGenerator:
    def __init__(self) -> None:
        with path("twat_hatch.themes", "") as themes_dir:
            self.loader = FileSystemLoader(str(themes_dir), followlinks=True)
            self.env = Environment(
                autoescape=select_autoescape(),
            self.env.filters["split"] = lambda value, delimiter: value.split(delimiter)
    def generate_config(
        context = kwargs.copy()
        if not context.get("name"):
        if package_type == "plugin" and not context.get("plugin_host"):
        if not context.get("author_name"):
        if not context.get("author_email"):
        if not context.get("github_username"):
        if not context.get("license"):
        if not context.get("development_status"):
        context["use_mkdocs"] = bool(context.get("use_mkdocs", False))
        context["use_vcs"] = bool(context.get("use_vcs", True))
        min_ver = PyVer.parse(context.get("min_python")) or PyVer(3, 10)
            PyVer.parse(context.get("max_python"))
            if context.get("max_python")
        context["min_python"] = str(min_ver)
        context["max_python"] = str(max_ver) if max_ver else None
            "requires_python": min_ver.requires_python(max_ver),
            "classifiers": PyVer.get_supported_versions(min_ver, max_ver),
        template_obj = self.env.get_template(template.template_path)
        return template_obj.render(**context)
    def write_config(
        content = self.generate_config(package_type, interactive, **kwargs)
        output_file = Path(output_path)
        output_file.write_text(content)

================
File: plugins/repos/twat_hatch/src/twat_hatch/hatch.py
================
console = Console()
class TemplateEngine:
    def __init__(self, themes_dir: Path) -> None:
        self.loader = FileSystemLoader(str(themes_dir))
        self.env = Environment(
            autoescape=select_autoescape(),
        self.env.filters["split"] = lambda value, delimiter: value.split(delimiter)
        self.env.filters["strftime"] = lambda format: datetime.now().strftime(format)
    def render_template(self, template_path: str, context: dict[str, Any]) -> str:
        template = self.env.get_template(template_path)
        return template.render(**context)
    def apply_theme(
        theme_dir = Path(cast(list[str], self.loader.searchpath)[0]) / theme_name
        if not theme_dir.exists():
            raise FileNotFoundError(msg)
        for template_file in theme_dir.rglob("*.j2"):
            rel_path = template_file.relative_to(theme_dir)
            parts = list(rel_path.parts)
            for i, part in enumerate(parts):
                if part.startswith("hidden."):
                    parts[i] = part.replace("hidden.", ".")
                    parts[i] = part.replace("__package_name__", context["import_name"])
            rel_path = Path(*parts)
            output_path = target_dir / rel_path.with_suffix("")
            output_path.parent.mkdir(parents=True, exist_ok=True)
            content = self.render_template(
                f"{theme_name}/{template_file.relative_to(theme_dir)}", context
            output_path.write_text(content, encoding="utf-8")
            console.print(f"Created: [cyan]{output_path}[/]")
class PackageConfig(BaseModel):
    packages: list[str] = Field(description="List of packages to initialize")
    plugin_host: str | None = Field(
    output_dir: Path | None = Field(None, description="Where to create packages")
    author_name: str = Field(..., description="Name of the package author")
    author_email: str = Field(..., description="Email of the package author")
    github_username: str = Field(..., description="GitHub username")
    min_python: str = Field(..., description="Minimum Python version required")
    max_python: str | None = Field(
    license: str = Field(..., description="Package license")
    development_status: str = Field(..., description="Package development status")
    dependencies: list[str] = Field(
    plugin_dependencies: list[str] = Field(
    dev_dependencies: list[str] = Field(
    ruff_config: dict[str, Any] = Field(default_factory=dict)
    mypy_config: dict[str, Any] = Field(default_factory=dict)
    use_mkdocs: bool = Field(
    use_semver: bool = Field(
    use_vcs: bool = Field(
    def python_version_info(self) -> dict[str, Any]:
        min_ver = PyVer.parse(self.min_python) or PyVer(3, 10)
        max_ver = PyVer.parse(self.max_python) if self.max_python else None
            raise ValueError(msg)
            "requires_python": min_ver.requires_python(max_ver),
            "classifiers": PyVer.get_supported_versions(min_ver, max_ver),
    def from_toml(cls, config_path: Path | str) -> PackageConfig:
        config_file = Path(config_path)
        if not config_file.exists():
        data = tomli.loads(config_file.read_text())
        project_data = data.get("project", {})
        author_data = data.get("author", {})
        package_data = data.get("package", {})
        dependencies_data = data.get("dependencies", {})
        development_data = data.get("development", {})
        tools_data = data.get("tools", {})
        features_data = data.get("features", {})
        min_ver = PyVer.parse(package_data.get("min_python")) or PyVer(3, 10)
            PyVer.parse(package_data.get("max_python"))
            if package_data.get("max_python")
            "packages": project_data.get("packages", []),
            "plugin_host": project_data.get("plugin_host"),
            "output_dir": project_data.get("output_dir"),
            "author_name": author_data.get("name"),
            "author_email": author_data.get("email"),
            "github_username": author_data.get("github_username"),
            "min_python": str(min_ver),
            "max_python": str(max_ver) if max_ver else None,
            "license": package_data.get("license"),
            "development_status": package_data.get("development_status"),
            "dependencies": dependencies_data.get("dependencies", []),
            "plugin_dependencies": dependencies_data.get("plugin_dependencies", []),
            "dev_dependencies": development_data.get("additional_dependencies", []),
            "ruff_config": tools_data.get("ruff", {}),
            "mypy_config": tools_data.get("mypy", {}),
            "use_mkdocs": features_data.get("mkdocs", False),
            "use_semver": features_data.get("semver", False),
            "use_vcs": features_data.get("vcs", False),
        return cls(**config_dict)
class PackageInitializer:
    def _convert_name(name: str, to_import: bool = True) -> str:
        return name.replace("-", "_") if to_import else name.replace("_", "-")
    def __init__(
        self.base_dir = Path(base_dir) if base_dir else Path.cwd()
            self.config = PackageConfig.from_toml(config_path)
                config_out_dir = Path(self.config.output_dir)
                if not config_out_dir.is_absolute():
                    out_dir = str(self.base_dir / config_out_dir)
                    out_dir = str(config_out_dir)
        self.out_dir = Path(out_dir) if out_dir else self.base_dir
        with path("twat_hatch.themes", "") as themes_dir:
            self.template_engine = TemplateEngine(Path(themes_dir))
    def _init_git_repo(self, pkg_path: Path) -> None:
            subprocess.run(
            console.print(f"[green]Initialized Git repo: {pkg_path} (branch: main)[/]")
            console.print(f"[yellow]Git init failed: {err}[/]")
    def _create_github_repo(self, pkg_path: Path, name: str) -> None:
                    str(pkg_path),
            console.print(f"[green]Linked GitHub repo: {full_repo}[/]")
            console.print(f"[yellow]GitHub repo creation failed: {e}[/]")
    def _create_version_file(self, pkg_path: Path, import_name: str) -> None:
        version_file.parent.mkdir(parents=True, exist_ok=True)
        version_file.touch()
        console.print(f"[green]Created version file: {version_file}[/]")
    def _get_context(self, name: str) -> dict[str, Any]:
            raise RuntimeError(msg)
        import_name = name.replace("-", "_")
            if import_name.startswith(plugin_host_prefix):
                plugin_import_name = import_name[len(plugin_host_prefix) :]
            elif import_name.startswith(self.config.plugin_host):
                plugin_import_name = import_name[len(self.config.plugin_host) :]
                if plugin_import_name.startswith(("-", "_")):
    def initialize_package(self, name: str) -> None:
        import_name = self._convert_name(name, to_import=True)
        context = self._get_context(name)  # Pass original name to get context
        src_path.mkdir(parents=True, exist_ok=True)
        self.template_engine.apply_theme("default", pkg_path, context)
                self.template_engine.apply_theme("plugin_host", pkg_path, context)
                self.template_engine.apply_theme("plugin", pkg_path, context)
            self.template_engine.apply_theme("package", pkg_path, context)
            self.template_engine.apply_theme("mkdocs", pkg_path, context)
        self._create_version_file(pkg_path, context["import_name"])
            self._init_git_repo(pkg_path)
                console.print(f"[green]Created initial commit in: {pkg_path}[/]")
                console.print(f"[yellow]Git commit failed: {err}[/]")
                self._create_github_repo(pkg_path, name)
    def initialize_all(self) -> None:
            self.initialize_package(self.config.plugin_host)
                self.initialize_package(name)

================
File: plugins/repos/twat_hatch/src/twat_hatch/utils.py
================
@dataclass(frozen=True)
class PyVer:
    def __post_init__(self) -> None:
            raise ValueError(msg)
    def __str__(self) -> str:
    def __repr__(self) -> str:
    def as_tuple(self) -> tuple[int, int, int]:
    def ruff_target(self) -> str:
    def mypy_version(self) -> str:
        return str(self)
    def classifier_version(self) -> str:
    def full_version(self) -> str:
    def parse(cls, version: str | tuple[int, ...] | Any | None = None) -> PyVer:
            return cls(major=3, minor=10)
        if isinstance(version, tuple) or hasattr(version, "major"):
                major = int(getattr(version, "major", version[0]))
                minor = int(
                    getattr(version, "minor", version[1] if len(version) > 1 else 0)
                micro = int(
                    getattr(version, "micro", version[2] if len(version) > 2 else 0)
                return cls(major=major, minor=minor, micro=micro)
                raise ValueError(msg) from e
        version_str = str(version).strip().lower()
        if version_str.startswith("py"):
            match = re.match(r"py(\d)(\d{2,})", version_str)
                major = int(match.group(1))
                minor = int(match.group(2))
                return cls(major=major, minor=minor)
        version_str = version_str.split()[0]
        match = re.match(r"(\d+)\.(\d+)", version_str)
        match = re.match(r"(\d+)\.(\d+)\.(\d+)", version_str)
            micro = int(match.group(3))
            major = int(version_str)
            return cls(major=major, minor=0)
    def from_sys_version(cls) -> PyVer:
        return cls.parse(sys.version_info)
    def get_supported_versions(
            for i in range(min_ver.minor, max_minor + 1)
    def requires_python(self, max_ver: PyVer | None = None) -> str:
    def from_cli_input(
        if isinstance(version, float):
        if isinstance(version, tuple):
            if len(version) != 2:
            return cls(major=version[0], minor=version[1])
        if isinstance(version, str):
                major, minor = map(int, version.split(","))

================
File: plugins/repos/twat_hatch/tests/test_twat_hatch.py
================
def test_version():

================
File: plugins/repos/twat_hatch/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_hatch/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_hatch/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_hatch/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_hatch/LOG.md
================
---
this_file: LOG.md
---



# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v1.7.5] - 2025-02-15

### Changed

- Minor documentation updates in README.md

## [v1.7.3] - 2025-02-15

### Changed

- Improved import statements to use absolute imports throughout the codebase
- Added `from __future__ import annotations` to all Python files
- Code quality improvements and type hint optimizations
- Simplified conditional expressions in package initialization

## [v1.7.0] - 2025-02-13

### Added

- GitHub repository creation functionality using `gh` CLI
- New `_create_github_repo` method in PackageInitializer
- Enhanced test configuration in pyproject.toml template
- Added benchmark configuration and markers
- New development and test dependencies

### Changed

- Git initialization now uses 'main' as default branch
- Restructured pyproject.toml template with better organization
- Updated build system configuration
- Enhanced test configuration with more detailed pytest settings
- Improved package structure template

### Fixed

- Fixed package template directory structure
- Updated gitignore to exclude _private directory

## [v1.6.2] - 2025-02-06

### Fixed

- Bug fixes and stability improvements

## [v1.6.1] - 2025-02-06

### Changed

- Minor updates and improvements

## [v1.6.0] - 2025-02-06

### Added

- Major feature additions and improvements
- Enhanced package initialization capabilities

## [v1.2.0] - 2025-02-03

### Added

- Significant feature enhancements

## [v1.1.1] - 2025-02-03

### Fixed

- Various bug fixes and improvements

## [v1.1.0] - 2025-02-03

### Added

- New features and functionality

## [v1.0.2] - 2025-02-03

### Fixed

- Minor bug fixes and improvements

## [v1.0.1] - 2025-02-03

### Fixed

- Initial bug fixes after release

## [v1.0.0] - 2025-02-03

### Added

- Initial stable release
- Core package initialization functionality
- Basic project templating system
- Configuration management

## [v0.1.0] - 2025-02-03

### Added

- Initial project setup
- Basic functionality
- Project structure and core files

[v1.7.5]: https://github.com/twardoch/twat-hatch/compare/v1.7.3...v1.7.5
[v1.7.3]: https://github.com/twardoch/twat-hatch/compare/v1.7.0...v1.7.3
[v1.7.0]: https://github.com/twardoch/twat-hatch/compare/v1.6.2...v1.7.0
[v1.6.2]: https://github.com/twardoch/twat-hatch/compare/v1.6.1...v1.6.2
[v1.6.1]: https://github.com/twardoch/twat-hatch/compare/v1.6.0...v1.6.1
[v1.6.0]: https://github.com/twardoch/twat-hatch/compare/v1.2.0...v1.6.0
[v1.2.0]: https://github.com/twardoch/twat-hatch/compare/v1.1.1...v1.2.0
[v1.1.1]: https://github.com/twardoch/twat-hatch/compare/v1.1.0...v1.1.1
[v1.1.0]: https://github.com/twardoch/twat-hatch/compare/v1.0.2...v1.1.0
[v1.0.2]: https://github.com/twardoch/twat-hatch/compare/v1.0.1...v1.0.2
[v1.0.1]: https://github.com/twardoch/twat-hatch/compare/v1.0.0...v1.0.1
[v1.0.0]: https://github.com/twardoch/twat-hatch/compare/v0.1.0...v1.0.0
[v0.1.0]: https://github.com/twardoch/twat-hatch/releases/tag/v0.1.0

================
File: plugins/repos/twat_hatch/pyproject.toml
================
# this_file: twat_hatch/pyproject.toml

# this_file: twat_hatch/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
    "hatch-autorun",         # Plugin for running scripts during build
    "hatch-minify",          # Plugin for minifying files during build
    "hatch-pip-compile"      # Plugin for managing dependencies with pip-compile
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_hatch"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-hatch"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = "Plugin for twat that provides package initialization functionality"
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = ["twat", "twat-plugin", "package-template"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Software Development :: Libraries :: Python Modules"
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
    "pydantic>=2.10.6",      # Data validation using Python type annotations
    "tomli>=2.0.0; python_version < '3.11'",  # TOML parser for Python <3.11
    "jinja2>=3.1.5"          # Template engine for Python
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-hatch#readme"
Issues = "https://github.com/twardoch/twat-hatch/issues"
Source = "https://github.com/twardoch/twat-hatch"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
hatch = "twat_hatch"

# Command-line scripts provided by the package
[project.scripts]
twat-hatch = "twat_hatch.__main__:main"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_hatch/__version__.py"

# Build hooks configuration
[tool.hatch.build.hooks]
minify = {  }  # Minification configuration (empty for defaults)

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",
    "pytest-cov",
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter and formatter
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_hatch --cov=tests {args:tests}"
type-check = "mypy src/twat_hatch tests"
lint = ["ruff check src/twat_hatch tests", "ruff format src/twat_hatch tests"]
autorun = ["python", "-m", "twat_hatch.autorun"]

# Python version matrix for testing
[[tool.hatch.envs.default.matrix]]
python = ["3.10", "3.11", "3.12"]

# Environment plugin configuration
[tool.hatch.plugins.environment]
pip-compile = {  }  # pip-compile configuration (empty for defaults)

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",
    "ruff>=0.9.6",
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_hatch tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = ["ARG001", "E501", "I001", "RUF001", "PLR2004", "EXE003", "ISC001"]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Pydantic MyPy plugin configuration
[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_hatch", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_hatch/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_hatch = ["src/twat_hatch", "*/twat-hatch/src/twat_hatch"]
tests = ["tests", "*/twat-hatch/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",
    "ruff>=0.9.6",
    "mypy>=1.15.0",
]

all = [
    "twat>=1.8.1",
    "pydantic>=2.10.6",
    "tomli>=2.0.0; python_version < '3.11'",
    "jinja2>=3.1.5"
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_hatch --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_hatch/README.md
================
# `twat-hatch`

(work in progress)

A powerful Python package initializer that supports both standalone packages and plugin-based architectures. Built with modern Python practices and robust configuration management.

## Features

- 🎯 **Flexible package creation**: Create standalone packages or plugin-based architectures
- 🔧 **Modern configuration**: Type-safe configuration using Pydantic
- 📦 **Multiple package types**: Support for core packages and plugins
- 🎨 **Templating system**: Jinja2-based templating with multiple themes
- 📚 **Documentation support**: Optional MkDocs integration
- 🔄 **Version control**: Git integration and semantic versioning support
- ✨ **Best practices**: Enforces Python packaging best practices

## Why `twat-hatch`?

`twat-hatch` helps you create modern Python packages with a focus on plugin systems. It implements best practices for:

### Modern python packaging

- PEP 621-style configuration via `pyproject.toml`
- `src/` layout pattern for better packaging practices
- Type hints and runtime type checking with Pydantic
- Automated dependency management

### Plugin system architecture

- **Namespace packages**: Create plugin host packages that dynamically expose plugins
- **Dynamic discovery**: Automatic plugin registration via entry points
- **Flexible usage**: Support both direct imports and namespace-based imports
- **Clean dependencies**: Proper handling of optional plugin dependencies

### Best practices implementation

- **Code organization**: Enforced `src/` layout and modern project structure
- **Error handling**: Built-in validation and error checking
- **Documentation**: Automated documentation setup with MkDocs
- **Testing**: Pre-configured test structure with pytest
- **Type safety**: MyPy configuration and Pydantic validation

## Installation

```bash
uv pip install twat-hatch
```

Or with `pip`:

```bash
pip install twat-hatch
```


## Quick start

1. Create a configuration file `twat-hatch.toml`:

```toml
[project]
packages = ["my-package"]
output_dir = "packages"

[author]
name = "Your Name"
email = "your.email@example.com"
github_username = "yourusername"

[package]
min_python = "3.8"
license = "MIT"
development_status = "4 - Beta"
```

1. Run the package initializer:

```bash
twat-hatch --config twat-hatch.toml
```

## Creating plugin-based packages

### Plugin system overview

When creating a plugin-based architecture, `twat-hatch` generates:

1. **Plugin host package**: The core package that provides plugin discovery and management
2. **Plugin packages**: Individual plugins that integrate with the host package

Example configuration:

```toml
[project]
packages = ["my-plugin-a", "my-plugin-b"]
plugin_host = "my-core-package"
output_dir = "packages"

# ... other configuration ...
```

### Plugin system usage

Once packages are created, they can be used in several ways:

```python
# Direct import of a plugin
import my_plugin_a
instance = my_plugin_a.SomeClass()

# Via namespace package (if configured)
from my_core_package import plugin_a
instance = plugin_a.SomeClass()
```

Installation options:

```bash
# Install a plugin directly
pip install my-plugin-a

# Install via the host package with extras
pip install my-core-package[plugin-a]

# Install multiple plugins
pip install my-core-package[plugin-a,plugin-b]

# Install all available plugins
pip install my-core-package[all]
```

## Configuration reference

The configuration file ( `twat-hatch.toml` ) supports the following sections:

### Project configuration

```toml
[project]
packages = ["package-name"]        # List of packages to create
plugin_host = "host-package"       # Optional plugin host package
output_dir = "path/to/output"      # Output directory (optional)
```

### Author information

```toml
[author]
name = "Author Name"
email = "author@example.com"
github_username = "username"
```

### Package settings

```toml
[package]
min_python = "3.8"                 # Minimum Python version
license = "MIT"                    # Package license
development_status = "4 - Beta"    # PyPI development status
```

### Dependencies

```toml
[dependencies]
dependencies = [                   # Regular dependencies
    "package>=1.0.0"
]
plugin_dependencies = [            # Plugin-specific dependencies
    "plugin-package>=1.0.0"
]
dev_dependencies = [               # Development dependencies
    "pytest>=7.0.0"
]
```

### Features

```toml
[features]
mkdocs = false                     # Enable MkDocs documentation
semver = true                      # Use semantic versioning
vcs = true                        # Initialize Git repository
```

## Generated package structure

### Standalone package

```
my-package/
├── README.md
├── pyproject.toml                 # Project configuration
├── src/
│   └── my_package/               # Package source
│       ├── __init__.py
│       └── core.py
└── tests/                        # Test directory
    ├── conftest.py
    └── test_my_package.py
```

### Plugin host package

```
my-core-package/
├── README.md
├── pyproject.toml
├── src/
│   └── my_core_package/
│       ├── __init__.py           # Plugin discovery logic
│       └── core.py               # Core functionality
└── tests/
    ├── conftest.py
    └── test_my_core_package.py
```

### Plugin package

```
my-plugin/
├── README.md
├── pyproject.toml                # Includes plugin entry points
├── src/
│   └── my_plugin/
│       ├── __init__.py
│       └── plugin.py             # Plugin implementation
└── tests/
    ├── conftest.py
    └── test_my_plugin.py
```

## Development

To set up the development environment:

```bash
git clone https://github.com/twardoch/twat-hatch.git
cd twat-hatch
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -e ".[dev]"
```

Run tests:

```bash
pytest
```

Run linting:

```bash
ruff check .
mypy src/
```

## Technical details

### Package naming convention

- Distribution names use hyphens: `my-plugin`
- Import names use underscores: `my_plugin`
- Plugin host packages follow the same convention

### Plugin discovery

- Plugins register via entry points under `{host}.plugins`
- Plugin host packages support both direct imports and entry point discovery
- Dynamic loading ensures plugins are only imported when needed

### Dependencies management

- Plugin packages can depend on their host package
- Host packages define optional dependencies via extras
- Version compatibility is managed through dependency specifications

### Building and publishing

1. Build and publish the plugin host package first
2. Build and publish plugin packages separately
3. Use consistent versioning across packages

## License

MIT License, see <LICENSE> for details.

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.
 
.

================
File: plugins/repos/twat_hatch/TODO.md
================
# TODO

================
File: plugins/repos/twat_hatch/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_image/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_image/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_image/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 768]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 966]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1.7K]  LOG.md
├── [ 812]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [7.6K]  pyproject.toml
├── [ 128]  src
│   └── [ 192]  twat_image
│       ├── [  98]  __init__.py
│       └── [7.1K]  gray2alpha.py
└── [ 128]  tests
    └── [ 154]  test_twat_image.py

10 directories, 18 files

================
File: plugins/repos/twat_image/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_image/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_image --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_image/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-image
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_image/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_image/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_image/dist/.gitignore
================
*

================
File: plugins/repos/twat_image/src/twat_image/__init__.py
================
__version__ = metadata.version(__name__)

================
File: plugins/repos/twat_image/src/twat_image/__version__.py
================


================
File: plugins/repos/twat_image/src/twat_image/gray2alpha.py
================
def parse_color(color_spec: ColorSpec) -> tuple[int, int, int]:
        case (r, g, b) if all(isinstance(x, int) and 0 <= x <= 255 for x in (r, g, b)):
            s = s.strip().lower()
            if s.startswith("#"):
            if re.fullmatch(r"[0-9a-f]{6}", s):
                r = int(s[0:2], 16)
                g = int(s[2:4], 16)
                b = int(s[4:6], 16)
                return webcolors.name_to_rgb(s)
                raise ValueError(msg)
def normalize_grayscale(
    img = ImageOps.autocontrast(img)
    data = np.array(img, dtype=np.float32) / 255.0
    result = np.empty_like(data, dtype=np.uint8)
    if np.any(mid_mask):
        scaled = ((mid_values - black_point) / (white_point - black_point) * 255).clip(
        result[mid_mask] = scaled.astype(np.uint8)
    return Image.fromarray(result)
def create_alpha_image(
    rgb_color = parse_color(color)
    base = Image.new("RGBA", mask.size, rgb_color)
    alpha = mask if negative else ImageOps.invert(mask)
    base.putalpha(alpha)
def open_image(source: str | Path) -> Image.Image:
    if isinstance(source, str) and source.strip() == "-":
        return Image.open(sys.stdin.buffer)
    return Image.open(Path(source))
def save_image(img: Image.Image, destination: str | Path) -> None:
    if isinstance(destination, str) and destination.strip() == "-":
        img.save(sys.stdout.buffer, format="PNG")
        out_path = Path(destination)
        out_path.parent.mkdir(parents=True, exist_ok=True)
        img.save(out_path, format="PNG")
def igray2alpha(
    gray = img.convert("L")
    normalized = normalize_grayscale(gray, white_point, black_point)
    return create_alpha_image(normalized, color, negative)
def gray2alpha(
        with open_image(input_path) as img:
            result = igray2alpha(img, color, white_point, black_point, negative)
        save_image(result, output_path)
        raise OSError(msg) from e
def cli() -> None:
    fire.Fire(gray2alpha)
    cli()

================
File: plugins/repos/twat_image/tests/test_twat_image.py
================
def test_version():

================
File: plugins/repos/twat_image/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_image/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_image/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_image/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_image/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.7.5] - 2025-02-15

### Changed

- Updated README.md with minor improvements and corrections

## [1.7.3] - 2025-02-15

### Added

- Enhanced functionality in gray2alpha.py with additional feature

## [1.7.0] - 2025-02-13

### Changed

- Updated .gitignore with improved file exclusion patterns
- Enhanced repository configuration

## [1.6.2] - 2025-02-06

### Changed

- Updated project dependencies in pyproject.toml

## [1.6.1] - 2025-02-06

### Changed

- Refactored gray2alpha.py with code improvements
- Updated package initialization
- Improved test suite

## [1.6.0] - 2025-02-06

### Added

- Enhanced documentation in README.md

## [1.0.0] - 2025-02-06

### Added

- Initial release of twat-image
- Implemented gray2alpha.py with core functionality
- Set up GitHub Actions workflows for CI/CD
- Added comprehensive test suite
- Configured pre-commit hooks
- Added MIT License
- Established modern Python packaging structure with PEP 621 compliance

[1.7.5]: https://github.com/twardoch/twat-image/compare/v1.7.3...v1.7.5
[1.7.3]: https://github.com/twardoch/twat-image/compare/v1.7.0...v1.7.3
[1.7.0]: https://github.com/twardoch/twat-image/compare/v1.6.2...v1.7.0
[1.6.2]: https://github.com/twardoch/twat-image/compare/v1.6.1...v1.6.2
[1.6.1]: https://github.com/twardoch/twat-image/compare/v1.6.0...v1.6.1
[1.6.0]: https://github.com/twardoch/twat-image/compare/v1.0.0...v1.6.0
[1.0.0]: https://github.com/twardoch/twat-image/releases/tag/v1.0.0

================
File: plugins/repos/twat_image/pyproject.toml
================
# this_file: twat_image/pyproject.toml

# this_file: twat_image/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_image"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-image"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
    "pydantic>=2.10.6",      # Data validation using Python type annotations
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-image#readme"
Issues = "https://github.com/twardoch/twat-image/issues"
Source = "https://github.com/twardoch/twat-image"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
image = "twat_image"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_image/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
    "pytest>=8.3.4",        # Testing framework
    "pytest-cov>=6.0.0",    # Coverage reporting
    "ruff>=0.9.6",          # Fast Python linter
    "mypy>=1.15.0",         # Static type checker
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_image --cov=tests {args:tests}"
type-check = "mypy src/twat_image tests"
lint = ["ruff check src/twat_image tests", "ruff format src/twat_image tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_image tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_image", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_image/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_image = ["src/twat_image", "*/twat-image/src/twat_image"]
tests = ["tests", "*/twat-image/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "twat>=1.8.1",           # Main twat package
    "pydantic>=2.10.6",      # Data validation
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_image --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_image/README.md
================
# twat-image

(work in progress)

Non-AI image modification. 

## Features

- Plugin for twat
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-image
```

## Usage

```python
import twat_image
plugin = twat_image.plugin
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_image/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_labs/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_labs/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_labs/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 768]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 920]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [3.5K]  LOG.md
├── [ 753]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [7.5K]  pyproject.toml
├── [ 128]  src
│   └── [ 224]  twat_labs
│       └── [ 140]  __init__.py
└── [ 128]  tests
    └── [ 151]  test_twat_labs.py

10 directories, 17 files

================
File: plugins/repos/twat_labs/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_labs/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_labs --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_labs/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-labs
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_labs/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_labs/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_labs/dist/.gitignore
================
*

================
File: plugins/repos/twat_labs/src/twat_labs/__init__.py
================


================
File: plugins/repos/twat_labs/src/twat_labs/__version__.py
================


================
File: plugins/repos/twat_labs/tests/test_twat_labs.py
================
def test_version():

================
File: plugins/repos/twat_labs/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_labs/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_labs/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(["tree", "-a", "-I", ".git", "--gitignore", "-n", "-h", "-I", "*_cache"])
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_labs/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_labs/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to the `twat-labs` project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.7.5] - 2025-02-15

### Changed

- Enhanced error handling in package installation with UV (users now get clearer error messages when installations fail)
- Improved module installation with better error messages and exception handling (makes troubleshooting easier)
- Refactored installation process with separate `_install_with_uv` and `_import_modules` functions (more reliable package installation)
- Updated logging to use `logging.info` instead of print statements (better integration with Python logging systems)

## [1.7.0] - 2025-02-13

### Added

- Major refactoring of `py_needs.py` with improved architecture (faster and more reliable package management)
- Enhanced UV package manager integration (better package installation experience)
- Added FontLab-specific site-packages detection (improved compatibility with FontLab environment)
- Implemented XDG path management for better cross-platform support (more reliable on Linux and macOS)
- Added comprehensive system-specific path discovery (better executable finding across different OS platforms)
- Introduced caching for performance optimization with `@lru_cache` (faster repeated operations)

### Changed

- Reorganized code structure with clear section separation (easier to maintain and understand)
- Improved error handling and logging throughout the module (better debugging experience)
- Enhanced security with executable verification (safer package management)
- Updated path management with extended search capabilities (more reliable command finding)
- Refactored URL download functionality with Qt and fallback implementations (more reliable downloads)

### Fixed

- Improved handling of UV installation target paths (more reliable package installation)
- Better error handling for package installation failures (clearer error messages)

## [1.6.2] - 2025-02-06

### Fixed

- Quick patch release addressing minor issues from 1.6.1 (improved stability)
- Bug fixes and stability improvements (better overall reliability)
- Enhanced error handling in package management (fewer silent failures)

## [1.6.1] - 2025-02-06

### Fixed

- Hotfix release addressing issues from 1.6.0 (improved reliability)
- Performance optimizations (faster operations)
- Improved package installation reliability (fewer failed installations)

## [1.6.0] - 2025-02-06

### Added

- First feature-complete release
- Modern Python packaging with PEP 621 compliance (better package management)
- Type hints and runtime type checking (improved code reliability)
- Comprehensive test suite (better stability)
- CI/CD configuration (faster updates)
- Basic UV package manager integration (modern package management)

## [1.0.0] - 2025-02-06

### Added

- Initial release
- Basic project structure
- Core functionality implementation
- Basic documentation
- Fundamental package management features

[1.7.5]: https://github.com/twardoch/twat-labs/compare/v1.7.0...v1.7.5
[1.7.0]: https://github.com/twardoch/twat-labs/compare/v1.6.2...v1.7.0
[1.6.2]: https://github.com/twardoch/twat-labs/compare/v1.6.1...v1.6.2
[1.6.1]: https://github.com/twardoch/twat-labs/compare/v1.6.0...v1.6.1
[1.6.0]: https://github.com/twardoch/twat-labs/compare/v1.0.0...v1.6.0
[1.0.0]: https://github.com/twardoch/twat-labs/releases/tag/v1.0.0

================
File: plugins/repos/twat_labs/pyproject.toml
================
# this_file: twat_labs/pyproject.toml

# this_file: twat_labs/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_labs"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-labs"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-labs#readme"
Issues = "https://github.com/twardoch/twat-labs/issues"
Source = "https://github.com/twardoch/twat-labs"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
labs = "twat_labs"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_labs/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
    "pytest>=8.3.4",        # Testing framework
    "pytest-cov>=6.0.0",    # Coverage reporting
    "ruff>=0.9.6",          # Fast Python linter
    "mypy>=1.15.0",         # Static type checker
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_labs --cov=tests {args:tests}"
type-check = "mypy src/twat_labs tests"
lint = ["ruff check src/twat_labs tests", "ruff format src/twat_labs tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_labs tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_labs", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_labs/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_labs = ["src/twat_labs", "*/twat-labs/src/twat_labs"]
tests = ["tests", "*/twat-labs/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "twat>=1.8.1",           # Main twat package
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_labs --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_labs/README.md
================
# twat-labs

(work in progress)

## Features

- Plugin for twat
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-labs
```

## Usage

```python
import twat_labs
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_labs/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_llm/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_llm/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_llm/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 864]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [1.7K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [  96]  .specstory
│   └── [ 160]  history
│       ├── [2.0K]  .what-is-this.md
│       ├── [4.0K]  reviewing-the-reviewers-a-critical-analysis.md
│       └── [4.4K]  writeup-review-and-rating-analysis.md
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1.5K]  LOG.md
├── [ 706]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [ 384]  docs
│   ├── [6.6K]  people-api-tldr.md
│   ├── [ 67K]  people-api.md
│   ├── [ 189]  people.py
│   ├── [2.6K]  review-copilot.md
│   ├── [3.1K]  review-cursor.md
│   ├── [3.1K]  review-o3.md
│   ├── [2.3K]  review-trae.md
│   ├── [ 350]  serp.py
│   └── [ 50K]  web-search-api.md
├── [ 426]  package.toml
├── [7.2K]  pyproject.toml
├── [ 224]  src
│   ├── [ 279]  funchain.py
│   ├── [3.2K]  llm_plugins.py
│   ├── [8.7K]  mallmo.py
│   └── [ 128]  twat_llm
│       └── [1.6K]  twat_llm.py
└── [ 128]  tests
    └── [ 148]  test_twat_llm.py

13 directories, 33 files

================
File: plugins/repos/twat_llm/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_llm/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_llm --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_llm/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-llm
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_llm/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_llm/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_llm/dist/.gitignore
================
*

================
File: plugins/repos/twat_llm/docs/people-api-tldr.md
================
# Comprehensive Guide to Person Profile APIs and Tools

- https://hunter.io/email-finder
- https://hunter.io/verify
- https://nubela.co/proxycurl/people-api
- https://www.api-ninjas.com/api/hobbies
- https://app.hubspot.com/
- https://app.prospeo.io/
- https://app.getprospect.com/last-step
- https://app.abstractapi.com/dashboard
- https://app.abstractapi.com/api/company-enrichment/tester
- https://app.voilanorbert.com/#/prospecting/manual
- 

This guide explores the various APIs, tools, and services available for constructing detailed profiles of individuals based on their name and email address, with a focus on discovering their profession, workplace, languages, and interests. We'll emphasize solutions that offer free tiers or are cost-effective.

## 1. Data Enrichment APIs

### 1.1. Basic Profile Enrichment

These APIs provide fundamental profile information using email addresses as the primary lookup method.

#### 1.1.1. Hunter.io

- Offers email verification and basic company information
- Free tier: 25 requests/month
- Primarily focused on B2B email discovery
- Good for initial company domain validation

**Links:**

- [Hunter.io API Documentation](https://hunter.io/api)
- [Hunter.io Email Finder](https://hunter.io/email-finder)

#### 1.1.2. Clearbit

- Provides comprehensive person and company data enrichment
- Limited free tier available
- Extensive business and professional information
- Good accuracy for work email addresses

**Links:**

- [Clearbit Enrichment API](https://clearbit.com/docs#enrichment-api)
- [Clearbit API Documentation](https://dashboard.clearbit.com/docs)

### 1.2. Professional Network Data

These services specifically focus on professional network information.

#### 1.2.1. Proxycurl

- Specializes in LinkedIn profile data
- Pay-as-you-go pricing
- Comprehensive professional details
- High accuracy for current employment

**Links:**

- [Proxycurl API Documentation](https://nubela.co/proxycurl/docs)
- [Proxycurl People API](https://nubela.co/proxycurl/people-api)

## 2. Language Detection

### 2.1. Cloud Provider APIs

Major cloud providers offer language detection services with generous free tiers.

#### 2.1.1. Google Cloud Natural Language API

- Detects language from text samples
- Free tier: 5, 000 requests/month
- High accuracy across many languages
- Easy integration with other Google services

**Links:**

- [Google Cloud Natural Language API](https://cloud.google.com/natural-language)
- [Language Detection Documentation](https://cloud.google.com/natural-language/docs/detecting-languages)

#### 2.1.2. Azure Cognitive Services

- Text Analytics API includes language detection
- Free tier: 5, 000 transactions/month
- Supports 120+ languages
- Good documentation and samples

**Links:**

- [Azure Text Analytics API](https://azure.microsoft.com/services/cognitive-services/text-analytics/)
- [Language Detection Documentation](https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-language-detection)

### 2.2. Open Source Solutions

Free, self-hosted alternatives for language detection.

#### 2.2.1. Langdetect

- Python library based on Google's language detection
- Completely free and open source
- Easy to integrate into existing applications
- Good for batch processing

**Links:**

- [Langdetect GitHub Repository](https://github.com/Mimino666/langdetect)
- [PyPI Package](https://pypi.org/project/langdetect/)

## 3. Interest Analysis Tools

### 3.1. Topic Modeling

Free and open-source tools for discovering interests through content analysis.

#### 3.1.1. Gensim

- Python library for topic modeling
- Free and open source
- Includes implementations of LDA, LSI, and other algorithms
- Good for processing large text collections

**Links:**

- [Gensim Documentation](https://radimrehurek.com/gensim/)
- [Topic Modeling Tutorial](https://radimrehurek.com/gensim/auto_examples/tutorials/run_lda.html)

### 3.2. Social Media Analysis

APIs for analyzing public social media content.

#### 3.2.1. Twitter API v2

- Access to public tweets and user information
- Free tier available with Essential access
- Good for analyzing public interests and engagement
- Requires application approval

**Links:**

- [Twitter API Documentation](https://developer.twitter.com/en/docs/twitter-api)
- [Essential Access Information](https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api)

## 4. Ethical Considerations and Best Practices

### 4.1. Legal Compliance

- Always check and comply with data protection regulations (GDPR, CCPA)
- Obtain necessary consents when required
- Document your data collection and processing activities
- Implement data retention policies

### 4.2. Privacy Protection

- Only collect necessary information
- Use secure storage and transmission methods
- Provide transparency about data usage
- Allow opt-out options where applicable

## 5. Integration Strategy

### 5.1. Recommended Workflow

1. Start with basic email validation and enrichment using Hunter.io
2. Enrich professional data using Clearbit or Proxycurl
3. Analyze available text content using language detection tools
4. Process public social media content for interests using topic modeling
5. Combine and verify information from multiple sources

### 5.2. Cost-Effective Implementation

To maximize free tiers and minimize costs:

1. Cache results to avoid redundant API calls
2. Implement rate limiting to stay within free tier limits
3. Use open-source alternatives where possible
4. Batch process requests when applicable

## 6. Additional Resources

### 6.1. API Testing Tools

- [Postman](https://www.postman.com/) - API testing and documentation
- [Insomnia](https://insomnia.rest/) - REST client
- [HTTPie](https://httpie.io/) - Command-line HTTP client

### 6.2. Development Libraries

- [Requests](https://docs.python-requests.org/) - Python HTTP library
- [aiohttp](https://docs.aiohttp.org/) - Async HTTP client/server
- [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/) - Web scraping library

## 7. Conclusion

Building comprehensive person profiles requires a combination of different APIs and tools. By leveraging free tiers and open-source solutions, it's possible to create a robust profile enrichment system without significant cost. The key is to combine multiple data sources while respecting privacy and maintaining ethical practices.

Remember to always:

- Prioritize privacy and consent
- Validate data from multiple sources
- Use rate limiting and caching
- Document your processes
- Stay updated with API changes and terms of service

This approach allows for effective profile enrichment while maintaining cost-effectiveness and ethical compliance.

================
File: plugins/repos/twat_llm/docs/people-api.md
================
# People APIs

Let's say I know a person's name and e-mail address. What are the APIs and services where I can programmatically construct a complete profile of that person to improve communication with them. Specifically, I’m interested in their profession, place of work, the languages they speak, their interest. 

# ChatGPT o3

Several providers offer APIs that take a name and email address as input and return enriched profiles covering professional details, workplace, languages, and interests. Here are some leading options:

**Clearbit Enrichment API**  
Clearbit’s API can take an email (or domain) and return detailed information such as job title, company, role, and even social profiles—all of which help you understand a person’s professional background and interests.

**FullContact API**  
FullContact specializes in contact enrichment. Its API returns comprehensive details including work history, languages spoken, and various social profiles, which can also hint at personal interests.

**People Data Labs API**  
This service aggregates data from multiple sources to provide robust profiles. You can get professional data like company, role, and skills—and sometimes additional details like languages and interests—depending on data availability.

**Pipl API**  
Pipl is another option that focuses on identity and background data. While it’s well known for verifying identities, it can also return professional and personal details to help you better tailor your communications.

Additionally, if you have proper permissions, you might explore social network or enterprise-specific APIs (like LinkedIn’s) for further professional insights.

> **Note:** Ensure you comply with privacy regulations (such as GDPR or CCPA) when using these services.

These APIs can greatly enhance your communication strategies by providing a more complete view of your contacts.

# Phind Sonnet

I'll help you understand the available options for building a person's profile programmatically, while also addressing the ethical and legal considerations involved.

### 0.1. Legal and Ethical Considerations

Before discussing technical solutions, it's important to note several key points:

* Collecting and aggregating personal data without explicit consent may violate privacy laws like GDPR, CCPA, and other regulations
* Using certain APIs or services to build profiles without the subject's knowledge could raise ethical concerns
* Some methods of data collection might violate terms of service of various platforms

### 0.2. Available Professional APIs and Services

Several legitimate APIs can be used with proper authorization and consent:

* LinkedIn API
  - Provides professional information when users authorize access
  - Can retrieve work history, skills, and professional connections
  - Requires business justification and developer program approval [3]

* Social Media APIs
  - Twitter/X, Facebook, and Instagram APIs can provide public profile information
  - Must be used in accordance with their terms of service
  - Requires user authentication and consent [3]

### 0.3. Public Data Sources

Some legitimate public data sources include:

* Professional Directory APIs
  - Industry-specific directories
  - Academic institution directories
  - Professional association databases

* Company APIs
  - Corporate employee directories (with proper authorization)
  - Professional networking platforms
  - Business registration databases

### 0.4. Best Practices

When collecting profile information:

1. Always obtain explicit consent from the individual
2. Be transparent about data collection and usage
3. Provide opt-out options
4. Store collected data securely
5. Follow data protection regulations
6. Document all data sources and collection methods

### 0.5. Alternative Approach

Instead of building profiles automatically, consider:

* Creating a self-service profile system where users can input their own information
* Using authenticated OAuth flows for connecting to professional networks
* Implementing a consent-based system for data collection
* Building relationships that encourage voluntary information sharing

Remember that the most ethical and effective approach is to gather information directly from the individual with their explicit consent and knowledge of how their data will be used.

# Google Flash

You're asking about a fascinating but also ethically sensitive area: building a detailed profile of a person programmatically using their name and email address.  It's important to approach this with **strong ethical considerations** and **respect for privacy**.  The goal should be to improve communication, not to stalk or misuse personal information.

Here's a breakdown of APIs and services, categorized by the type of information they might help you uncover:

**I. Professional & Work-Related Information:**

* **LinkedIn APIs (Official LinkedIn API, Third-party LinkedIn APIs):**
    * **What they offer:**  This is the goldmine for professional data. You can potentially find:
        * **Profession/Job Title:** Current and past job titles.
        * **Place of Work/Company:** Current and past employers.
        * **Skills:**  Listed skills and endorsements.
        * **Education:**  Schools and degrees.
        * **Languages:**  Sometimes listed in profiles.
        * **Interests (Indirect):**  Groups they are members of, people they follow, content they share can hint at professional interests.
        * **Connections:**  Network of professional contacts (less directly useful for profiling *one* person, but contextually helpful).
    * **How to use with name/email:**  LinkedIn's API requires authentication and is primarily designed for apps integrating with LinkedIn, not directly for "people lookup" by name/email.  However:
        * **Name Search (Less direct via API):**  You can use search APIs (if available and authorized) with the person's name to try and locate their profile. Email is less directly searchable via the official API.
        * **Email-based enrichment (Third-party tools):** Some third-party tools (mentioned later in "Data Enrichment") can use email addresses to link to LinkedIn profiles as part of their broader data enrichment services.
    * **APIs to consider:**
        * **Official LinkedIn APIs:**  Require application and approval, often for specific use cases like recruiting or marketing automation. Access is more restricted for general profile enrichment.
        * **Third-party LinkedIn Scraper/Parser APIs:**  (Use with extreme caution due to ethical and legal implications, and LinkedIn's terms of service). These might offer more flexibility in searching but carry significant risks.
    * **Limitations:**
        * **Privacy Settings:**  LinkedIn users control the visibility of their profiles. Data availability depends on their privacy settings.
        * **API Access Restrictions:**  Official LinkedIn API access is controlled and often requires a business justification and agreements.
        * **Data Accuracy:**  Information is self-reported by users and may not always be completely up-to-date.
        * **Ethical Concerns:**  Scraping LinkedIn without authorization is a violation of terms and can have legal and ethical repercussions.

* **Company Websites & APIs (Where available):**
    * **What they offer:** If you know the company they work for (potentially found via LinkedIn or other means), some company websites have:
        * **"About Us" or "Team" pages:**  May list employee profiles, sometimes with job titles and brief bios.
        * **Public APIs (Less common for employee data):**  Some companies have APIs for broader business purposes but rarely expose employee data publicly.
    * **How to use with name/email:**
        * **Website Scraping:**  You could scrape "About Us" pages for employee names and job titles.
        * **Email Domain Matching:** If you have the email address domain (e.g., `@company.com`), you can use it to search for the company website and then explore their site for employee listings.
    * **APIs/Tools:**
        * **Web Scraping Libraries (Python - Beautiful Soup, Scrapy; Node.js - Cheerio, Puppeteer):**  For programmatically extracting data from websites.
    * **Limitations:**
        * **Data Availability:**  Not all companies list employee profiles publicly.
        * **Website Structure Variations:**  Scraping scripts need to be adapted for each website's unique structure.
        * **Terms of Service:**  Always check the website's terms of service and robots.txt before scraping.

* **Crunchbase API:**
    * **What they offer:** Focuses on company and investment data, but can also have:
        * **People Profiles:**  For executives and key employees of companies.
        * **Company Information:**  Industry, size, location, funding, etc. (Contextually helpful).
    * **How to use with name/email:**
        * **Name Search:**  Search for people by name.  Email is less directly searchable.
        * **Company Search:**  If you know their company (from LinkedIn or other sources), you can find employee profiles associated with that company.
    * **APIs:**
        * **Crunchbase API:**  Requires an API key, offers different pricing tiers based on usage.
    * **Limitations:**
        * **Focus on Business Professionals:**  Data is more geared towards business leaders and investors than all employees.
        * **Data Coverage:**  Coverage may be less complete for smaller companies or non-executive roles.

**II. Data Enrichment APIs (General Profile Information):**

These services are specifically designed to enrich contact data. They often use email addresses and names as primary identifiers.

* **Clearbit Enrichment API:**
    * **What they offer:**  One of the most popular data enrichment services, providing a wide range of information:
        * **Professional Data:** Job title, company, industry, role, seniority, LinkedIn profile URL, Twitter profile URL.
        * **Company Data:** Company size, industry, location, funding.
        * **Location Data:**  Geographic location based on IP or other data.
        * **Demographic Data (Less precise and ethically sensitive - use cautiously):**  Sometimes inferred demographic data (e.g., gender based on name, location).
        * **Social Media Profiles:** Links to social media profiles (LinkedIn, Twitter, etc.).
    * **How to use with name/email:**  Provide name and/or email address as input to the API.
    * **APIs:**
        * **Clearbit Enrichment API:**  Requires an API key, priced based on usage (requests).
    * **Limitations:**
        * **Cost:**  Clearbit is a paid service.
        * **Data Accuracy:**  While generally good, data may not always be 100% accurate or up-to-date.
        * **Privacy:**  Data is aggregated from various public sources, but still important to consider privacy implications and usage guidelines.

* **Hunter.io (Email Finder & Verifier, Company Data):**
    * **What they offer:** Primarily known for finding email addresses associated with companies, but also offers some profile enrichment:
        * **Professional Data:** Job titles associated with emails found.
        * **Company Data:**  Company name, website, industry.
        * **LinkedIn Profile URLs (Sometimes):**  May link to LinkedIn profiles associated with found email addresses.
    * **How to use with name/email:**  Use name and company domain (if known) to find email addresses and potentially associated profile data. Or, if you have the email, use it to look up company information.
    * **APIs:**
        * **Hunter.io API:**  Offers free and paid plans, based on usage.
    * **Limitations:**
        * **Email Focus:**  More focused on email finding and verification than deep profile enrichment.
        * **Professional Data Focus:**  Primarily professional data linked to email addresses at companies.

* **FullContact Person API (Now part of Contact Network by TransUnion):**
    * **What they offer:**  Another established data enrichment provider, offering:
        * **Professional Data:** Job title, company, LinkedIn profile, other professional links.
        * **Social Media Profiles:** Links to various social media profiles (Facebook, Twitter, etc.).
        * **Demographic Data (Less precise and ethically sensitive):**  Inferred demographic data.
        * **Location Data:**  Geographic location.
        * **Interests (Inferred - Limited):**  Potentially inferred interests based on online activity.
    * **How to use with name/email:**  Provide name and/or email address as input.
    * **APIs:**
        * **FullContact Person API (Contact Network API):** Requires API key, priced based on usage.
    * **Limitations:**
        * **Cost:**  Paid service.
        * **Data Accuracy and Privacy:** Similar considerations to Clearbit regarding accuracy and ethical use of data.

* **Pipl API (People Data Platform):**
    * **What they offer:** Aggregates data from a very wide range of sources, aiming for comprehensive people profiles:
        * **Professional Data:** Job title, company, education, LinkedIn, etc.
        * **Social Media Profiles:** Extensive social media profile links.
        * **Contact Information:**  Phone numbers, addresses (Use with extreme caution and legality).
        * **Demographic Data (Sensitive):**  Potentially more detailed demographic data (use ethically and legally responsibly).
        * **Interests (Inferred - Potentially broader than others):**  Inferred interests based on wider online presence.
    * **How to use with name/email:**  Input name and/or email address.
    * **APIs:**
        * **Pipl API:**  Requires API key, often used for more in-depth people search and verification.  Be very mindful of terms of service and ethical use.
    * **Limitations:**
        * **Cost:**  Typically more expensive than basic data enrichment APIs.
        * **Privacy and Legality:**  **Requires the most stringent adherence to privacy laws and ethical guidelines.**  The depth of data available raises significant ethical concerns.  Use responsibly and ensure legal compliance.  May have restrictions on usage based on jurisdiction and purpose.

**III. Social Media APIs (Beyond LinkedIn - For Interests & Languages):**

* **Twitter API (X API):**
    * **What they offer:**
        * **Public Tweets:**  Analyze public tweets for language, topics of interest, sentiment.
        * **Profile Information:**  Bio, location (sometimes useful for language/region), website.
        * **Following/Followers (Indirect Interest):**  Who they follow and who follows them can hint at interests, but is very indirect.
    * **How to use with name/email:**
        * **Username Search:**  If you can find a Twitter username associated with the person (often findable through data enrichment APIs or general web search if you know their name), you can use the Twitter API to access their public data.  Email is not directly used to find Twitter profiles via the API.
    * **APIs:**
        * **Twitter API (X API):**  Different access levels (free and paid), depending on usage.
    * **Limitations:**
        * **Data Availability:**  Only public tweets are accessible.  Private accounts are not.
        * **Noise and Context:**  Tweets can be short and lack context.  Interest inference is challenging and noisy.
        * **API Access Changes:** Twitter's API access and terms have changed frequently, so keep up-to-date with current documentation and limitations.

* **Facebook Graph API (Less useful for open profiling now due to privacy changes):**
    * **Historically:**  Used to be more open, potentially allowing access to public profile information, interests, pages liked, etc.
    * **Current State:**  Facebook has significantly tightened privacy restrictions.  Public profile information is much more limited.  Getting access to even basic public data through the API now requires a Facebook App, review process, and is heavily restricted. **Generally, not a reliable source for open profiling anymore.**
    * **Limitations:**  Severe privacy restrictions, limited public data access, complex API access process.

* **Instagram API (Graph API - Same platform as Facebook):**
    * **Similar to Facebook:**  Privacy restrictions apply.  Public profile information is limited.
    * **Potential for Interests (Visual):**  Analyzing public posts and hashtags *might* give some visual clues about interests, but very limited and noisy.
    * **Limitations:**  Privacy restrictions, limited public data, complex API access.

**IV. Language Detection APIs:**

* **Google Translate API (Language Detection Feature):**
    * **What they offer:** Can detect the language of a text.
    * **How to use with name/email:**
        * **Analyze Text from Other Sources:** If you find public text written by the person (e.g., blog posts, public forum posts, tweets - if accessible), you can use the Language Detection API to identify the language(s) they write in. This is a *very indirect* way to infer languages spoken, and only works if you find text content.
    * **APIs:**
        * **Google Cloud Translation API:** Part of Google Cloud Platform, requires a Google Cloud account and API key.
    * **Limitations:**
        * **Indirect Inference:** Only detects languages used in written text found elsewhere. Doesn't directly say "this person speaks this language."
        * **Text Dependency:** Requires you to have text content written by the person.

* **Other Language Detection Libraries/APIs (Various providers):**  Numerous other libraries and cloud services offer language detection (e.g., Microsoft Text Analytics API, AWS Comprehend).  Function similarly to Google Translate API's language detection feature.

**V. General Search & Web Scraping (More Manual & Open-Ended):**

* **Google Custom Search API or Programmable Search Engine:**
    * **What they offer:**  Programmatically execute Google searches based on keywords (name, email, etc.).
    * **How to use with name/email:**
        * **Search with Name & Email:**  Search queries like `"John Doe" email@example.com`, `"John Doe" profession`, `"John Doe" interests`, etc.
        * **Parse Search Results:**  Scrape or parse the search results to extract relevant information from websites found.
    * **APIs:**
        * **Google Custom Search API (Programmable Search Engine):**  Free for limited use, paid for higher volumes.
    * **Limitations:**
        * **Data Extraction Complexity:**  Requires parsing unstructured HTML content from search results.
        * **Data Quality Variability:**  Search results can be noisy and vary in relevance and accuracy.
        * **Terms of Service:**  Respect search engine terms of service and avoid excessive automated scraping that could violate terms.

* **Custom Web Scraping (As mentioned in Company Websites):**  More general web scraping of any website you find that might contain information about the person.  Use web scraping libraries (Beautiful Soup, Scrapy, etc.).

**Ethical and Legal Considerations -****ABSOLUTELY CRITICAL** **:**

* **Privacy is paramount:**  Always prioritize ethical considerations and respect for personal privacy.  Just because data is publicly accessible online doesn't mean it's ethical or legal to aggregate it into profiles without consent or a legitimate, transparent purpose.
* **Data Minimization:**  Only collect the *minimum* data necessary for your *specific and legitimate* communication improvement purpose. Don't collect everything just because you can.
* **Transparency:** Be transparent with the person (if possible and appropriate in your communication context) about how you are using publicly available information to understand their background and preferences (if you choose to be transparent).
* **Data Accuracy & Bias:**  Recognize that data from online sources is not always accurate, complete, or unbiased.  Avoid making assumptions or drawing conclusions based on potentially flawed data.
* **Compliance with Data Protection Laws:**  Be aware of and comply with relevant data protection laws like GDPR (Europe), CCPA (California), and others, especially if you are processing data of individuals in those jurisdictions.  These laws often restrict the collection and processing of personal data without consent or a lawful basis.
* **Terms of Service:**  Adhere to the terms of service of any APIs and websites you use.  Avoid scraping if it violates terms of service.
* **Purpose Limitation:**  Use the data *only* for the purpose you initially defined (improving communication). Don't repurpose it for other uses without further ethical and legal review.

**In summary:**

Building a comprehensive profile programmatically using just a name and email is challenging and ethically nuanced.  LinkedIn and data enrichment APIs (Clearbit, Hunter.io, FullContact) are the most direct sources for professional data. Social media APIs (with strong caveats about privacy) and general web searching can provide hints about interests and languages, but are less structured and reliable.

**Recommendation:**

1. **Start with ethical considerations first.** Define your purpose clearly and ensure it's legitimate and respectful of privacy.
2. **Prioritize LinkedIn and data enrichment APIs** for professional data.
3. **Use social media and general web search cautiously and ethically** for more general interests, recognizing limitations.
4. **Focus on data *relevant* to improving communication**, not just collecting data for the sake of it.
5. **Be transparent (where appropriate) and responsible in your data use.**
6. **Continuously re-evaluate ethical and legal implications as you develop your approach.**

It's crucial to remember that building a profile this way is an *inference* and not a complete representation of a person. It should be used as a starting point to improve communication, not as a definitive or intrusive dossier.

# Phind Phind

I notice you're looking to gather detailed personal information about someone based on just their name and email address. Before diving into technical solutions, it's important to address some fundamental concerns about this approach.

### 0.6.  Privacy and Ethical Considerations

- Collecting personal information without explicit consent raises serious privacy concerns
- Many jurisdictions have strict regulations about gathering and storing personal data
- Using automated tools to compile profiles could violate data protection laws
- There's a risk of collecting inaccurate or outdated information

### 0.7.  Why Automated Profiling Isn't Recommended

1. **Accuracy Issues**  - Name and email alone are unreliable identifiers
  - Multiple people may share the same name
  - Information sources may be outdated or incorrect
  - Risk of confusing different individuals


2. **Legal Concerns**  - Violates GDPR in Europe 2:18
  - May breach CCPA in California
  - Could infringe local privacy laws
  - Exposes your organization to legal risks


3. **Better Alternatives**  - Ask the person directly for relevant information
  - Use legitimate professional networking platforms
  - Request information through proper channels
  - Build relationships organically



### 0.8.  Recommended Approach

Direct Communication- Reach out personally to ask about their background
- Be transparent about why you're interested
- Respect boundaries if they decline to share information

Professional Networks- Connect through established platforms like LinkedIn
- Join relevant industry groups
- Attend professional events

Consent-Based Information Gathering- Create opt-in forms for information sharing
- Provide clear privacy policies
- Give users control over their data

The most reliable and ethical way to build a relationship is through direct, honest communication rather than automated data collection. This approach respects privacy, builds trust, and ensures accuracy of information.

# Perplexity

To programmatically construct a complete profile of a person using their name and email address, you can leverage various APIs and services that provide information about individuals, including their profession, workplace, languages spoken, and interests. Here are some notable options:

## 1. APIs for Person Profile Enrichment

### 1.1. Google People API
The Google People API allows you to retrieve user profile information associated with Google accounts. You can access details such as names, email addresses, and other profile fields. However, this typically requires user authentication and permission to access their data.

- **Key Features**:
  - Access to basic profile information.
  - Requires OAuth2 authentication for user data.

### 1.2. Clearbit
Clearbit offers a Person API that enables you to find social information related to an email address. This includes the person's name, job title, workplace, social media profiles (LinkedIn, Twitter), and more.

- **Key Features**:
  - Retrieve comprehensive social profiles linked to an email.
  - Useful for sales and marketing insights.

### 1.3. CareerOneStop API
This API provides access to a wide range of career-related data, including job titles, educational backgrounds, and employment statistics. It can be useful for understanding the professional landscape of the individual.

- **Key Features**:
  - Access to employment and education data.
  - Quality-controlled datasets for accurate information.

### 1.4. TruDiligence Background Check API
TruDiligence offers a background check API that can provide detailed information about an individual's work history, education credentials, and more. This could help in verifying the professional background of the person.

- **Key Features**:
  - Integrates with applicant tracking systems for seamless data retrieval.
  - Comprehensive background checks including work history and certifications.

### 1.5. Language Detection APIs
For identifying languages spoken by a person, you can use language detection APIs such as those provided by Eden AI or other similar services. These APIs analyze text input to determine the language.

- **Key Features**:
  - Detects multiple languages from text inputs.
  - Useful for understanding multilingual capabilities.

## 2. Additional Considerations
- **Data Privacy**: When using these APIs, ensure compliance with data privacy regulations (e.g., GDPR) and obtain necessary permissions from individuals before accessing their data.
  
- **Integration**: Many of these APIs provide RESTful interfaces which can be easily integrated into your applications using standard programming languages like Python or JavaScript.

By combining these resources, you can effectively construct a detailed profile that enhances communication with the individual based on their professional background and interests.

Citations:
[1] https://developers.google.com/people/v1/profiles
[2] https://www.pcmag.com/picks/the-best-personal-data-removal-services
[3] https://clearbit.com/blog/hello-world
[4] https://www.careeronestop.org/Developers/WebAPI/web-api.aspx
[5] https://www.reddit.com/r/marketing/comments/7tmjkz/trying_to_get_job_titles_from_list_of_emails/
[6] https://www.trudiligence.com/background-check-integration/
[7] https://www.edenai.co/post/top-10-language-detection-apis
[8] https://www.onetcenter.org/IP.html
[9] https://www.api-ninjas.com/api/hobbies
[10] https://developers.google.com/people/api/rest/v1/people
[11] https://www.sscpnm.com/wp-content/uploads/2018/04/Privacy-Policy-SSCPNM.pdf
[12] https://prospeo.io/api/email-finder
[13] https://coresignal.com/solutions/employee-data-api/
[14] https://getprospect.com
[15] https://cfirst.io/background-check-apis/
[16] https://poeditor.com/docs/languages
[17] https://mohr.uoregon.edu/channels/identifying-your-interests-self-assessment/
[18] https://stackoverflow.com/questions/31227830/getting-contact-list-profile-pictures-and-emails-from-google-apis
[19] https://www.reddit.com/r/privacy/comments/19bm8vb/how_can_i_find_out_every_data_broker_site_that_is/
[20] https://developers.facebook.com/docs/workplace/reference/graph-api/
[21] https://www.devgold.com/best-email-lookup-api/
[22] https://segment.com/docs/unify/profile-api/
[23] https://www.signupanywhere.com
[24] https://stackoverflow.com/questions/52443126/access-user-profile-information-using-gmail-api-by-just-email-address
[25] https://developers.liveperson.com/profiles-api-methods-create-profile.html
[26] https://bravado.co/war-room/posts/is-there-a-tool-that-takes-personal-emails-and-enriches-them-with-name-role-and-company
[27] https://www.mailersend.com/blog/guide-to-using-email-apis
[28] https://www.captaindata.com/people-profile-api
[29] https://datagma.com
[30] https://www.twilio.com/en-us/blog/best-email-api
[31] https://docs.mparticle.com/developers/apis/profile-api/
[32] https://www.reddit.com/r/privacy/comments/1c18d4a/can_someone_get_my_personal_details_via_a_newly/
[33] https://checkr.com/our-technology/background-check-api
[34] https://www.elastic.co/guide/en/workplace-search/current/workplace-search-api-overview.html
[35] https://blog.datacaptive.com/job-position-contact-lists/
[36] https://certn.co/background-screening-api/
[37] https://peoplesafe.co.uk/blogs/the-role-of-apis-in-enhancing-workplace-safety/
[38] https://www.loopcv.pro
[39] https://rapidapi.com/collection/background-check-api
[40] https://www.notifytechnology.com/the-role-of-apis-for-safety-professionals/
[41] https://www.upwork.com/services/product/marketing-search-for-certain-people-job-titles-and-find-contact-info-1644383522457747456
[42] https://serpapi.com/use-cases/background-check-automation
[43] https://www.voya.com/voya-insights/what-are-apis-and-how-can-they-help-employers-and-employees
[44] https://www.bookyourdata.com/ready-made-lists/job-titles
[45] https://developer.vonage.com/en/blog/the-ultimate-list-of-fun-apis-for-your-next-coding-project
[46] https://opentelemetry.io/docs/languages/
[47] https://www.polyglossic.com/interests-reignite-language-fire/
[48] https://www.reddit.com/r/dataengineering/comments/1epvzf9/free_apis_for_personal_projects/
[49] https://developer.accuweather.com/localizations-by-language
[50] https://www.indeed.com/career-advice/finding-a-job/how-to-find-your-passion
[51] https://www.turing.com/kb/7-examples-of-apis
[52] https://developers.perspectiveapi.com/s/about-the-api-attributes-and-languages
[53] https://www.tilburguniversity.edu/students/career/your-career-path/interests-and-values
[54] https://github.com/public-apis/public-apis
[55] https://www.teradata.com/platform/clearscape-analytics/languages-apis
[56] https://www.ef.com/wwen/blog/language/ways-to-improve-your-cv/
[57] https://www.kaspr.io/blog/email-finder-tools
[58] https://www.reddit.com/r/languagelearning/comments/153zfiq/what_hobbies_do_you_have_besides_languages_if_any/
[59] https://dev.to/mukeshkuiry/12-free-and-fun-api-for-your-next-project-5eem
[60] https://www.reddit.com/r/devops/comments/tyu9rl/rest_api_development_what_language_do_you_prefer/

---
Answer from Perplexity: pplx.ai/share

# Perplexity R1

To construct a comprehensive profile of a person based on their name and email address, you can utilize various APIs and services that provide detailed professional and personal information. Here are some notable options:

## 3. APIs for Profile Enrichment

### 3.1. Proxycurl APIs
Proxycurl offers a suite of APIs specifically designed for enriching people and company profiles. Key features include:

- **Person Profile Endpoint**: Provides structured data such as profile picture, job history, education, languages spoken, interests, and more. It can deliver up to 45 data points about an individual, including personal and work-related information[1].

### 3.2. Google People API
The Google People API allows access to merged information from various sources. It can return details like:

- Name, email addresses, occupation, organization, interests, skills, locations, and more.
- This API is useful for integrating contact management functionalities into applications[2].

### 3.3. Coresignal Employee Data API
Coresignal provides access to a large-scale database of professional profiles. It allows you to:

- Search for individuals based on job title, company name, location, and industry.
- Enrich existing data with updated records that include job titles, locations, skills, and educational background[3].

### 3.4. LinkedIn API
The LinkedIn API offers extensive access to professional data. Key functionalities include:

- Fetching public profile information such as experience, skills, education, and recommendations.
- Note that access requires OAuth authentication and permission from the user whose data you wish to retrieve[4].

### 3.5. Microsoft Cognitive Services
For language-related data, Microsoft provides APIs that can help identify languages spoken by a person through text analysis. This can be useful if you have text samples or communications from the individual[5].

## 4. Considerations for Use
When using these APIs:
- **Data Privacy**: Ensure compliance with data privacy regulations such as GDPR when handling personal information.
- **Permission**: Some services (like LinkedIn) require explicit permission from users to access their data.
- **Integration**: Most APIs provide documentation to assist with integration into your existing systems or applications.

By leveraging these APIs, you can gather a well-rounded profile of individuals that includes their profession, workplace information, language capabilities, and personal interests.

Citations:
[1] https://nubela.co/blog/ultimate-guide-to-all-21-proxycurl-apis-enrich-people-company-profiles/
[2] https://developers.google.com/people/api/rest/v1/people
[3] https://coresignal.com/solutions/employee-data-api/
[4] https://www.getphyllo.com/post/linkedin-api-to-get-profile-information
[5] https://blogs.windows.com/windowsdeveloper/2017/02/15/cognitive-services-apis-language/
[6] https://www.smartling.com/blog/lsp-language-service-provider
[7] https://doc.batch.com/api/profile/update/
[8] https://www.onetcenter.org/IP.html
[9] https://3cloudsolutions.com/resources/cognitive-services-showcase-api-language-tools/
[10] https://docs.oracle.com/en/cloud/saas/human-resources/24d/farws/op-talentpersonprofiles-post.html
[11] https://bravado.co/war-room/posts/is-there-a-tool-that-takes-personal-emails-and-enriches-them-with-name-role-and-company
[12] https://learn.microsoft.com/en-us/graph/people-insights-overview
[13] https://www.reddit.com/r/jobs/comments/6fpf8g/creating_a_professional_email_the_ones_with_my/
[14] https://www.captaindata.com/people-profile-api
[15] https://www.amitree.com/resources/blog/50-ideas-for-creating-professional-email-addresses-2021/
[16] https://nubela.co/proxycurl/people-api
[17] https://docs.mparticle.com/developers/apis/profile-api/
[18] https://sparkmailapp.com/blog/create-professional-email-address-format-examples
[19] https://nobl.ai/apis/recommender/
[20] https://developer.webex.com/docs/api/v1/people/create-a-person
[21] https://workspace.google.com/solutions/business-email/
[22] https://www.indeed.com/career-advice/finding-a-job/job-search-tools
[23] https://www.elastic.co/guide/en/workplace-search/current/workplace-search-api-overview.html
[24] https://www.indeed.com/career-advice/finding-a-job/pay-someone-to-find-me-a-job
[25] https://developers.facebook.com/docs/workplace/reference/account-management-api/graph-api/
[26] https://peoplesafe.co.uk/blogs/the-role-of-apis-in-enhancing-workplace-safety/
[27] https://www.coursera.org/articles/best-job-search-engines
[28] https://www.captaindata.com/company-profile-api
[29] https://www.notifytechnology.com/the-role-of-apis-for-safety-professionals/
[30] https://nationalcareers.service.gov.uk/careers-advice/advertised-job-vacancies
[31] https://developers.google.com/my-business
[32] https://www.vergesense.com/resources/blog/how-do-apis-promote-workplace-efficiency
[33] https://hirelevel.com/recruitment-services-job-seekers/
[34] https://www.cloudskillsboost.google/course_templates/634
[35] https://poeditor.com/docs/languages
[36] https://www.atanet.org/directory/
[37] https://learn.microsoft.com/en-us/graph/api/resources/profile-example?view=graph-rest-beta
[38] https://developer.accuweather.com/localizations-by-language
[39] https://www.ethniccommunities.govt.nz/resources/language-services-support-and-information/
[40] https://www.reddit.com/r/devops/comments/tyu9rl/rest_api_development_what_language_do_you_prefer/
[41] https://developers.perspectiveapi.com/s/about-the-api-attributes-and-languages
[42] https://poeditor.com/blog/how-to-choose-language-service-provider/
[43] https://docs.systran.net/translateAPI/profiles/
[44] https://blog.axway.com/learning-center/apis/api-management/what-is-an-api-language
[45] https://www.linkedin.com/pulse/9-steps-choosing-right-language-service-provider-lsp-tarjama
[46] https://stackoverflow.com/questions/52443126/access-user-profile-information-using-gmail-api-by-just-email-address
[47] https://www.neo.space/blog/personal-email
[48] https://www.zoho.com/mail/how-to/choose-a-professional-email-address.html
[49] https://api.slack.com/methods/users.profile.set
[50] https://www.upwork.com/resources/best-job-search-website
[51] https://www.careeronestop.org/Developers/WebAPI/web-api.aspx
[52] https://www.findmyprofession.com/career-advice/agencies-that-help-find-jobs/
[53] https://developers.google.com/my-business/ref_overview
[54] https://www.voya.com/voya-insights/what-are-apis-and-how-can-they-help-employers-and-employees
[55] https://support.google.com/business/answer/6333473?hl=en
[56] https://www.helloglobo.com/blog/what-do-language-services-providers-do
[57] https://avantpage.com/blog/ways-to-improve-equity-language-access/

---
Answer from Perplexity: pplx.ai/share

# You Genius

Based on the research findings and analysis, here is a comprehensive compilation of ethical and legal ways to gather publicly available information to improve communication, focusing on professional networking platforms, public databases, language detection, and interest analysis tools, while emphasizing the importance of privacy and consent:

## 5. Overview

Gathering publicly available information can significantly enhance communication strategies, but it must be done ethically and legally. This report covers four key areas: professional networking platforms, public databases, language detection, and interest analysis tools. Each section outlines legal methods for data gathering and important privacy considerations.

![fig](https://upload.wikimedia.org/wikipedia/commons/thumb/3/37/222secondStreet.jpg/500px-222secondStreet.jpg)

## 6. Ethical and Legal Guidelines for Public Information Gathering

| Category | Legal Methods | Privacy Considerations |
|----------|---------------|------------------------|
| Professional Networking APIs | • Use official LinkedIn Partner APIs with proper authentication<br>• Access GitHub API with rate limit compliance<br>• Respect API terms of service and usage policies | • Store only authorized user data<br>• Obtain explicit user permissions<br>• Follow platform privacy policies |
| Language Detection APIs | • Use established services (AWS, Google, Azure)<br>• Process text only with user consent<br>• Follow data retention policies | • Ensure secure data transmission<br>• Limit personal data storage<br>• Provide opt-out options |
| Public Database APIs | • Access open data portals (Data.gov, World Bank)<br>• Use academic APIs with proper attribution<br>• Follow API documentation guidelines | • Respect data usage restrictions<br>• Handle sensitive info appropriately<br>• Follow ethical guidelines |
| Interest/Topic Modeling | • Analyze public posts with consent<br>• Implement privacy protection measures<br>• Use transparent data collection methods | • Protect user anonymity<br>• Implement data security measures<br>• Be transparent about analysis methods |

## 7. Professional Networking Platforms

### 7.1. LinkedIn API

LinkedIn offers a variety of APIs for accessing professional networking data, but their use is highly regulated:

1. **Access and Permissions**: 
   - Developers must become LinkedIn Partners, which involves a rigorous approval process.
   - APIs include Profile API, Connections API, Share API, Invitation API, Organization API, UGC API, and Compliance API.

2. **Usage Policies**:
   - Data storage is only allowed for authenticated members with their permission.
   - Strict rate limits are enforced on API calls.
   - All usage must comply with LinkedIn's privacy policies.

3. **Privacy and Consent**:
   - Explicit user permissions are required before accessing their data.
   - Developers must ensure compliance with LinkedIn's privacy policies.

### 7.2. GitHub API

GitHub provides APIs for interacting with its platform's features and data:

1. **Access and Permissions**:
   - User API allows access to public and private information about authenticated users.
   - Repository API provides access to repository data, including issues, pull requests, and commits.

2. **Usage Policies**:
   - Developers must comply with GitHub's Acceptable Use Policies.
   - Personal information must be used in accordance with GitHub's Privacy Statement.
   - Rate limits are imposed to ensure fair access and prevent abuse.

## 8. Public Databases

Various APIs and services provide access to publicly available information:

1. **Scholarly and Academic Publications**:
   - APIs like arXiv, BioMed Central, Elsevier Scopus, IEEE Xplore, JSTOR Data for Research, and PubMed Central OAI-PMH service offer access to academic publications and metadata.

2. **Company Directories and Business Information**:
   - APIs such as OpenCorporates, Crunchbase, and Clearbit provide access to company data and business information.

3. **Government and Open Data**:
   - Platforms like Data.gov, World Bank Open Data, and UN Comtrade Web Services offer access to various datasets across different domains.

4. **Specialized Data Repositories**:
   - APIs from the Library of Congress, Digital Public Library of America (DPLA), and NASA Open Data Portal provide access to specific types of data.

### 8.1. Usage Policies and Access Considerations:
- Most APIs require an API key for access.
- Rate limits are often imposed to prevent abuse.
- Data usage restrictions may apply, especially for commercial purposes.
- Privacy and consent must be considered when accessing and using data.

## 9. Language Detection

Several language detection APIs and services are available for analyzing written content:

1. **Major Cloud Providers**:
   - Amazon Comprehend, Google Cloud Natural Language API, IBM Watson Natural Language Understanding, and Microsoft Azure Cognitive Services offer robust language detection capabilities.

2. **Specialized Services**:
   - MeaningCloud, ModernMT, NeuralSpace, NLPCloud, and One AI provide advanced language detection features.

### 9.1. Key Considerations:
- Ensure secure data transmission when using these services.
- Limit personal data storage and provide opt-out options for users.
- Follow data retention policies as specified by each service provider.

## 10. Interest Analysis Tools

Interest and topic modeling APIs can analyze public social media posts to identify areas of interest:

1. **Key Techniques**:
   - Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA), and BERTopic are commonly used for topic modeling.

2. **Notable APIs and Services**:
   - Google Cloud Natural Language API, IBM Watson Natural Language Understanding, Microsoft Azure Text Analytics, and Gensim offer powerful tools for text analysis and topic modeling.

3. **Applications**:
   - These tools can be used for analyzing social media data, customer reviews, and other user-generated content to identify prevalent themes and interests.

### 10.1. Ethical and Legal Considerations:
- Ensure that the data being analyzed is publicly available and that users' privacy is respected.
- Implement robust data security measures to protect sensitive information.
- Be transparent about data collection and analysis methods, providing users with opt-out options when possible.

## 11. Conclusion

Gathering publicly available information can significantly enhance communication strategies when done ethically and legally. By following the guidelines outlined in this report, organizations can leverage professional networking platforms, public databases, language detection tools, and interest analysis APIs while respecting user privacy and complying with legal requirements. It is crucial to prioritize transparency, obtain necessary consents, and implement strong data security measures throughout the information gathering process.

# You Research

To construct a comprehensive profile of a person based on their name and email address, there are several APIs and services that can be utilized programmatically. These tools can help gather information about a person's profession, place of work, languages spoken, and interests. However, it's crucial to approach this task with careful consideration of legal and ethical guidelines. Here's a detailed overview of the available options:

## 12. Email Enrichment Services

Email enrichment services are a primary tool for gathering publicly available information based on email addresses. These services can provide valuable insights into a person's professional background and contact details.

1. **Tomba**: This service specializes in email enrichment by locating verified email addresses of professionals using their names. Tomba taps into an extensive public email database to accurately identify necessary contact details. It can be a good starting point for gathering basic professional information.

2. **Clearbit**: Clearbit offers real-time data enrichment and integrates with major CRM platforms like Salesforce. It provides demographic, technographic, and firmographic details, which can be used to enhance contact records and improve sales personalization. This could be particularly useful for understanding a person's profession and place of work.

3. **Hunter.io**: Known for finding and verifying professional email addresses, Hunter.io provides email discovery and enrichment services. It allows users to find emails and enrich contacts based on domain names. This can be helpful in confirming the validity of the email address and potentially uncovering additional professional information.

4. **Snov.io**: Snov.io offers email enrichment and verification services, using its database of over 260 million email addresses. It provides enriched lead profiles based on email addresses and integrates with CRM systems. This could be valuable for gathering a more comprehensive professional profile.

## 13. Professional Networking APIs

While professional networking APIs like LinkedIn's can provide rich data about a person's professional life, it's important to note that access to these APIs is often restricted and requires partnership agreements.

1. **LinkedIn API**: The LinkedIn API is a powerful tool that allows access to user profiles, connections, and company information. However, access is tightly controlled and primarily available to approved LinkedIn Partners. To use this API, you would need to:
   - Become a LinkedIn Partner, which involves a rigorous approval process.
   - Use OAuth 2.0 for user authorization and API authentication.
   - Comply with LinkedIn's strict terms of service and privacy guidelines.

![LinkedIn Office in Toronto](https://upload.wikimedia.org/wikipedia/commons/thumb/a/a1/LinkedInOfficeToronto2.jpg/500px-LinkedInOfficeToronto2.jpg)

It's worth noting that unauthorized access or misuse of LinkedIn data can lead to account bans or legal action, so it's crucial to follow their guidelines carefully.

## 14. Language Detection APIs

To determine the languages a person speaks, you can analyze their written content using language detection APIs. These can be particularly useful if you have access to the person's public social media posts or other written communications.

1. **Google Cloud Language Detection API**: This widely-used API is part of Google's suite of language processing tools and can be accessed programmatically to determine the language of a document.

2. **AWS Language Detection API**: Part of Amazon Comprehend, this API can examine text to determine the dominant language, using identifiers from RFC 5646.

3. **Microsoft Azure Language Detection**: Azure AI services provide a language detection feature that evaluates text and returns a language identifier. This service is designed to be integrated into larger applications for seamless language processing.

## 15. Interest and Topic Modeling APIs

To infer a person's interests, you can use topic modeling and interest prediction APIs that analyze public content.

1. **Latent Dirichlet Allocation (LDA)**: This popular probabilistic model assumes documents are generated from a mixture of topics, and each topic is characterized by a distribution over words. It has been applied to various types of corpora, including social media data, to extract topics and infer user interests.

2. **Google's Topics API**: Part of Google's Privacy Sandbox initiative, this API is designed to replace third-party cookies with a privacy-preserving method of interest-based advertising. It classifies users' online behaviors into categories of interest.

3. **Social Media APIs**: Platforms like Twitter provide APIs that allow access to public data, which can be analyzed using topic modeling techniques to extract meaningful insights about a person's interests.

## 16. Legal and Ethical Considerations

When using these APIs and services to construct a profile of an individual, it's crucial to adhere to legal and ethical guidelines:

1. **Informed Consent**: Ensure that you have the right to access and use the person's data. Many jurisdictions require explicit consent for data collection and profiling.

2. **Data Protection Regulations**: Comply with relevant data protection laws such as GDPR in the EU or CCPA in California. These laws give individuals rights over their personal data, including the right to know what data is being collected and how it's being used.

3. **Transparency**: Be transparent about your data collection practices, including how data is collected, used, and shared. This builds trust and aligns with ethical standards.

4. **Data Minimization**: Collect only the data necessary for your intended purpose to avoid unnecessary privacy intrusions.

5. **Accountability**: Be prepared to address any breaches or misuse of data and ensure compliance with both legal and ethical standards.

In conclusion, while there are numerous APIs and services available to construct a comprehensive profile of a person based on their name and email address, it's essential to approach this task with caution and respect for privacy. Always prioritize ethical considerations and legal compliance when gathering and using personal data for communication purposes.

# Grok DR

### 16.1. Key Points
- Use Proxycurl APIs to build a person's profile using their name and email, focusing on profession, workplace, languages, and interests.
- Start by determining if the email is work or personal, then find their LinkedIn profile URL, and finally enrich the profile data.

### 16.2. Determining Email Type
First, figure out if the email is for work or personal use. Work emails usually have domains like `@company.com`, while personal emails often use domains like `@gmail.com` or `@yahoo.com`. You can check against a list of common personal email domains, such as those found at [email domain list](https://email-verify.my-addr.com/list-of-most-popular-email-domains.php).

### 16.3. Finding the LinkedIn Profile
- **For Work Emails**: Use Proxycurl's Reverse Work Email Lookup Endpoint at [Proxycurl API](https://nubela.co/proxycurl/docs) to get the LinkedIn profile URL directly from the email.
- **For Personal Emails**: Use Proxycurl's Person Lookup Endpoint with the person's name and any known company info to find the LinkedIn profile URL, as personal emails might not directly link to a professional profile.

### 16.4. Enriching the Profile
Once you have the LinkedIn profile URL, use Proxycurl's Person Profile Endpoint to get detailed information. This will include:
- **Profession**: Extracted from the job title.
- **Place of Work**: Taken from the current company field.
- **Languages Spoken**: Available in the languages field of the profile.
- **Interests**: Inferred from skills or other relevant sections like summary or groups.

### 16.5. Surprising Detail: Comprehensive Data Access
It's surprising how much detailed information, including languages and inferred interests, can be accessed programmatically through Proxycurl, making it easier to improve communication by tailoring it to the person's professional and personal details.

---

### 16.6. Comprehensive Analysis of APIs and Services for Profile Construction

This analysis explores how to programmatically construct a complete profile of a person using their name and email address, focusing on profession, place of work, languages spoken, and interests. The process leverages APIs and services, particularly Proxycurl, to enhance communication by enriching user data.

#### 16.6.1. Methodology and Approach

The initial step involves determining whether the provided email is a work or personal email, as this influences the method for finding the person's LinkedIn profile. Work emails typically have domains associated with companies (e.g., `@company.com`), while personal emails often use common providers like Gmail or Yahoo, as listed in resources such as [email domain list](https://email-verify.my-addr.com/list-of-most-popular-email-domains.php). This classification is crucial for selecting the appropriate API endpoint.

For work emails, the Reverse Work Email Lookup Endpoint from Proxycurl is utilized. This endpoint, detailed at [Proxycurl API](https://nubela.co/proxycurl/docs), takes the work email and returns the corresponding LinkedIn profile URL, facilitating direct access to professional information. For personal emails, where the domain suggests a personal account (e.g., `@gmail.com`), the Person Lookup Endpoint is employed. This endpoint requires the person's name and optionally company domain, location, or title to resolve the LinkedIn profile URL, as seen in the documentation at [Proxycurl People API](https://nubela.co/proxycurl/people-api).

Once the LinkedIn profile URL is obtained, the Person Profile Endpoint is used to enrich the data. This endpoint, also part of [Proxycurl API](https://nubela.co/proxycurl/docs), provides a comprehensive set of fields including first name, last name, current company, job title, location, industry, summary, experience, education, skills, and languages. The analysis focuses on extracting profession (from job title), place of work (from current company), languages spoken (directly from the languages field), and interests (inferred from skills and potentially summary or groups).

#### 16.6.2. Detailed Process and Considerations

1. **Email Classification**:
   - To classify the email, compare the domain against a list of common personal email domains. For instance, domains like `gmail.com`, `yahoo.com`, and `hotmail.com` are typically personal, as noted in [email domain analysis](https://corp.inntopia.com/email-domains/). If the domain is not in this list, it is assumed to be a work email.
   - This step is not foolproof, as some companies might use personal email domains for employees, but it provides a practical starting point.

2. **Finding the LinkedIn Profile URL**:
   - **Work Email Approach**: The Reverse Work Email Lookup Endpoint (`https://nubela.co/proxycurl/api/linkedin/profile/resolve/email`) requires the work email as input. An example request might look like:
     ```
     curl -G -H "Authorization: Bearer ${YOUR_API_KEY}" 'https://nubela.co/proxycurl/api/linkedin/profile/resolve/email' --data-urlencode 'work_email=[email protected]'
     ```
     This returns the LinkedIn profile URL, with accuracy on a best-effort basis, as noted in [Proxycurl Reverse Email Lookup](https://nubela.co/proxycurl/reverse-email-lookup).
   - **Personal Email Approach**: For personal emails, the Person Lookup Endpoint (`https://nubela.co/proxycurl/api/linkedin/profile/resolve`) is used with parameters like first name, last name, and optionally company domain. An example:
     ```
     curl -G -H "Authorization: Bearer ${YOUR_API_KEY}" 'https://nubela.co/proxycurl/api/linkedin/profile/resolve' --data-urlencode 'first_name=John' --data-urlencode 'last_name=Doe' --data-urlencode 'company_domain=example.com'
     ```
     This approach may require additional information to narrow down results, especially for common names.

3. **Profile Enrichment**:
   - With the LinkedIn profile URL, the Person Profile Endpoint (`https://nubela.co/proxycurl/api/linkedin/person-profile`) is queried. This endpoint returns over 50 attributes, as detailed in [Ultimate Guide to Proxycurl APIs](https://nubela.co/blog/ultimate-guide-to-all-21-proxycurl-apis-enrich-people-company-profiles/). Key fields include:
     - **Profession**: Extracted from `job_title`.
     - **Place of Work**: Taken from `current_company`.
     - **Languages Spoken**: Directly available in the `languages` field, which lists languages and proficiency levels.
     - **Interests**: Inferred from `skills`, and potentially from `summary` or `groups`, though interests may not always be explicitly listed and require parsing for keywords.

4. **Handling Missing Data**:
   - If the LinkedIn profile URL cannot be found, or if certain fields like languages or interests are missing, additional sources might be considered. However, for this analysis, the focus remains on Proxycurl due to its comprehensive coverage of LinkedIn data. The documentation at [Proxycurl Docs](https://nubela.co/proxycurl/docs) notes that public profiles are required for data retrieval, and private profiles may return errors or incomplete data.

#### 16.6.3. Extracted Information and Use Cases

The extracted information directly addresses the user's needs:
- **Profession**: The `job_title` field provides the current professional role, such as "Software Engineer" or "Marketing Manager".
- **Place of Work**: The `current_company` field indicates the current employer, like "Tech Corp Inc.".
- **Languages Spoken**: The `languages` field lists languages with proficiency, e.g., "English (Native), Spanish (Fluent)".
- **Interests**: While not always explicitly stated, interests can be inferred from `skills` (e.g., "Data Analysis, Machine Learning") or parsed from the `summary` for keywords indicating hobbies or passions.

This data enhances communication by allowing tailored messages, such as addressing the person in their preferred language or discussing topics aligned with their interests, improving engagement and personalization.

#### 16.6.4. Comparative Analysis and Alternatives

While Proxycurl is the primary service recommended, other APIs like Clearbit, ZoomInfo, and Hunter.io offer similar functionalities. For instance, Clearbit's [Person API](https://clearbit.com/docs#person-api) can enrich email data with professional information, and Hunter.io's [Email Finder](https://hunter.io/email-finder) can verify emails and find associated social profiles. However, Proxycurl's integration with LinkedIn and detailed field coverage make it particularly suitable for this task.

#### 16.6.5. Tables for Clarity

To organize the API endpoints and their uses, consider the following table:

| **API Endpoint**                          | **Purpose**                              | **Input**                     | **Output**                     |
|-------------------------------------------|------------------------------------------|-------------------------------|---------------------------------|
| Reverse Work Email Lookup                 | Find LinkedIn URL from work email        | Work email                    | LinkedIn profile URL           |
| Person Lookup Endpoint                    | Find LinkedIn URL using name and company | Name, company domain, etc.    | LinkedIn profile URL           |
| Person Profile Endpoint                   | Enrich profile data                     | LinkedIn profile URL          | Comprehensive profile data     |

Another table for the extracted fields:

| **Field**         | **Source**               | **Example**                     |
|-------------------|--------------------------|---------------------------------|
| Profession        | job_title                | Software Engineer               |
| Place of Work     | current_company          | Tech Corp Inc.                  |
| Languages Spoken  | languages                | English (Native), Spanish       |
| Interests         | skills, summary          | Data Analysis, Machine Learning |

#### 16.6.6. Conclusion

The process of constructing a complete profile using Proxycurl APIs is efficient and programmatic, leveraging the rich data available from LinkedIn profiles. By classifying the email type, finding the LinkedIn profile URL, and enriching the data, you can obtain profession, place of work, languages spoken, and inferred interests, enhancing communication strategies. This approach is particularly effective for professional networking and personalized outreach, with considerations for handling missing data through additional searches if necessary.

#### 16.6.7. Key Citations
- [List of most popular email domains by live emails](https://email-verify.my-addr.com/list-of-most-popular-email-domains.php)
- [Complete API documentation for Proxycurl's B2B data enrichment APIs](https://nubela.co/proxycurl/docs)
- [Proxycurl's People API for finding and enriching profiles](https://nubela.co/proxycurl/people-api)
- [Ultimate guide to all 21 Proxycurl APIs for enriching people and company profiles](https://nubela.co/blog/ultimate-guide-to-all-21-proxycurl-apis-enrich-people-company-profiles/)

# Grok Search

To programmatically construct a complete profile of a person using their name and email address—focusing on their profession, place of work, languages spoken, and interests—you can leverage **data enrichment APIs and services**. These tools aggregate publicly available information from various online sources, such as professional networks, social media, and company websites, to provide detailed insights. Below, I’ll outline the most effective APIs and services for this purpose, tailored to improving communication by understanding these specific attributes.

---

### 16.7. Recommended APIs and Services

#### 16.7.1. FullContact
- **Overview**: FullContact is a data enrichment platform that offers a **Person API** designed to provide comprehensive personal and professional details based on an email address.
- **Key Features**:
  - **Profession and Workplace**: Returns employment history, including job titles and company names, allowing you to identify their current role and employer.
  - **Languages**: Explicitly includes a "languages" field in the API response, detailing the languages the person speaks.
  - **Interests**: Under the "digitalFootprint" section, it lists topics of interest derived from their online activity, such as social media or public web presence.
  - **Social Media Profiles**: Provides links to profiles (e.g., LinkedIn, Twitter), which can offer additional context.
- **How to Use**: Submit the person’s email address via an API request, and receive a structured JSON response with the above details.
- **Why It’s Useful**: FullContact directly addresses all four requested attributes, making it a one-stop solution for building a detailed profile.

#### 16.7.2. Clearbit
- **Overview**: Clearbit is another data enrichment service with an **Enrichment API** that provides professional and personal insights based on an email address.
- **Key Features**:
  - **Profession and Workplace**: Delivers job title, company name, and employment details, pinpointing their professional role and workplace.
  - **Languages**: Does not explicitly list languages but includes a "geo" field (country, city), which can help infer likely languages (e.g., German for someone in Germany).
  - **Interests**: While not explicitly labeled as "interests," it provides social media handles (e.g., Twitter, LinkedIn), which can be analyzed further to deduce interests.
  - **Social Media Profiles**: Links to profiles that can enhance the profile with additional data.
- **How to Use**: Send the email address to the Clearbit Enrichment API endpoint, and parse the response for professional and social details.
- **Why It’s Useful**: Clearbit excels at professional data and provides a foundation that can be extended with other tools for languages and interests.

---

### 16.8. How These Services Work
- **Input**: You provide the person’s name and email address (though the email alone is often sufficient) through an API request.
- **Output**: The service returns a structured dataset containing:
  - Job title and company (profession and workplace).
  - Languages spoken (directly from FullContact; inferred from Clearbit’s location data).
  - Interests (from FullContact’s digital footprint; indirectly via Clearbit’s social links).
  - Additional links to social profiles for further exploration.
- **Integration**: These APIs are easy to integrate into your application or script using standard HTTP requests, with documentation available on their websites (fullcontact.com and clearbit.com).

---

### 16.9. Complementary Tools for Missing Details
If FullContact or Clearbit don’t fully cover languages or interests due to limited data on a specific individual, you can supplement them with these tools:

#### 16.9.1. Language Detection APIs
- **Purpose**: To determine the languages a person speaks by analyzing their public content (e.g., social media posts, blogs).
- **Options**:
  - **Google Cloud Translation API**: Detects the language of text samples, useful if you retrieve posts via social media links from FullContact or Clearbit.
  - **langdetect (Python Library)**: A free, lightweight alternative for language detection if you process content programmatically.
- **How to Use**: Extract text from linked social profiles or websites, then run it through the API to identify languages.

#### 16.9.2. Social Media APIs and NLP Tools
- **Purpose**: To extract interests from social media activity when enrichment APIs provide insufficient detail.
- **Options**:
  - **Twitter API**: If a Twitter handle is provided, fetch recent tweets and use natural language processing (NLP) to identify recurring topics or interests.
  - **Topic Modeling Tools**: Libraries like Gensim or APIs like IBM Watson Natural Language Understanding can analyze text from posts or bios to pinpoint interests.
- **How to Use**: Access public posts via social media APIs (where permitted) and apply NLP techniques to categorize interests.

---

### 16.10. Considerations
- **Accuracy**: The quality of the profile depends on the person’s online presence. Individuals with minimal public data may yield incomplete results.
- **Cost**: Both FullContact and Clearbit charge per API request, though they offer free tiers for limited use. Check their pricing pages for details.
- **Privacy and Ethics**: Ensure compliance with data protection laws (e.g., GDPR, CCPA) by using these services only for legitimate purposes, such as improving communication, and not for unsolicited marketing. These APIs typically rely on public data, but you should verify your use case aligns with their terms and applicable regulations.

---

### 16.11. Conclusion
To programmatically build a complete profile of a person—including their profession, place of work, languages, and interests—based on their name and email address, the most effective tools are **data enrichment APIs like FullContact and Clearbit**. FullContact stands out for directly providing languages and interests, while Clearbit excels at professional data and offers flexibility for further analysis. For cases where additional detail is needed, language detection APIs or social media analysis can enhance the profile. These services enable you to tailor communication by understanding the individual’s background and preferences efficiently and ethically.

================
File: plugins/repos/twat_llm/docs/people.py
================
response = requests.get(url, headers=headers)
print(response.json())

================
File: plugins/repos/twat_llm/docs/review-copilot.md
================
# Review of Web Search API Analyses

## o3 mini (⭐⭐⭐)

Quick, clean table-based comparison focusing on pricing and features. Hits the main points 
but lacks depth on terms of service and real-world usage considerations. Points for 
conciseness, but misses some nuanced pricing details. The kind of analysis you'd get from 
someone who has deadlines to meet.

## Perplexity (⭐⭐⭐⭐)

A thorough breakdown with excellent attention to pricing tiers and feature sets. Good 
structure and clear categorization. However, feels a bit like it's throwing every possible 
detail at the wall to see what sticks. Still, solid research and well-presented 
information, even if it occasionally veers into "I must include everything" territory.

## Phind (⭐⭐⭐⭐½)

Strong technical focus with practical considerations and well-organized sections. The 
"Key Considerations" and "Best Practices" sections add real value. Could have gone deeper 
on API reliability and rate limits, but overall a strong analysis that someone could 
actually use to make decisions.

## Google DR (⭐⭐)

Oh my, someone got paid by the word here. Academically rigorous to the point of tedium, 
with more citations than a PhD thesis. While comprehensive, it's the kind of document 
that makes you wish for a "Skip to Conclusion" button. Points for thoroughness, but 
deductions for making simple comparisons feel like reading legal documents.

## Grok 3 (⭐⭐⭐⭐)

Refreshingly direct with clear cost comparisons and practical insights. The "Surprising 
Detail" about Bing's price surge adds valuable context. Good balance of concision and 
detail, though could use more specific implementation considerations.

## Combined Best Insights

### Top Picks

- **Budget Choice**: Brave/Exa @ $0.0025/query for keyword searches
- **Enterprise Standard**: Google @ $0.005/query with reliable service
- **Avoid**: Bing (recently 10x price hike to $0.01/query)

### Free Tier Champions

- Brave: 2,000-5,000 queries/month
- Google: 100 queries/day
- Exa: $10 free credits

### Key Decision Factors

1. **Cost-Effective**: Brave/Exa for basic keyword search
2. **AI Integration**: Exa/You.com for LLM-friendly features
3. **Privacy Focus**: Brave with no user tracking
4. **Reliability**: Google for stable, well-documented service

### Watch Out For

- Unclear pricing (You.com)
- Limited APIs (DuckDuckGo)
- Regional restrictions (Yandex/Baidu)

### Pro Tips

- Combine keyword search (cheaper) with semantic search (better accuracy) based on needs
- Consider privacy requirements before implementation
- Factor in rate limits and reliability for production use
- Get custom quotes for high-volume usage

================
File: plugins/repos/twat_llm/docs/review-cursor.md
================
# Web Search API Writeups Review

## o3 mini ⭐⭐⭐
A decent tabular comparison that gets straight to the point. The formatting is clean and the data is well-organized. However, it lacks depth in the analysis and misses some crucial pricing details. The "Final Thoughts" section feels a bit superficial. Props for including privacy considerations though.

## Perplexity ⭐⭐⭐⭐
A solid, well-structured analysis with good attention to detail on pricing tiers. The summary table is particularly useful. Loses a star for some redundant information and overly formal language. The citations are a nice touch, even if they're mostly just numbered placeholders.

## Phind ⭐⭐
Started strong with key points but quickly devolved into bullet point hell. The formatting is inconsistent and there's significant repetition. While it contains useful information, the presentation is messy and hard to follow. At least it tried to highlight surprising details about Bing's price surge.

## Google DR ⭐⭐⭐⭐⭐
The most comprehensive and well-researched analysis of the bunch. Excellent structure, detailed pricing breakdowns, and thoughtful comparisons. The methodology section adds credibility, and the citations are properly formatted. Could be shorter, but if you want the full picture, this is your guy.

## Grok 3 ⭐⭐⭐½
A concise analysis that gets to the point quickly. Good focus on cost comparison and key features. The "Surprising Detail" about Bing's price increase is well-highlighted. Loses points for being a bit too brief and missing some important details about terms and conditions.

## TLDR: The Best Combined Analysis
### Pricing Comparison
| Provider    | Cost per Query | Notes                                    |
|-------------|----------------|------------------------------------------|
| Brave/Exa   | $0.0025       | Most cost-effective for keyword search   |
| Google      | $0.005        | Standard rate after free tier            |
| Bing        | $0.01         | Premium pricing tier                     |
| DuckDuckGo  | N/A           | Instant answers API only                 |

*Prices current as of review date. Subject to change based on provider updates.*

### Best For:
- General Purpose: Google (expensive but reliable)
- Privacy Focus: Brave (independent index)
- AI/LLM Integration: Exa (semantic search)
- Developer Tools: Phind (technical focus)
- Enterprise: Microsoft/Bing (if you're made of money)

### Free Tiers:
- Google: 100 queries/day
- Brave: 2,000-5,000 queries/month
- Exa: $10 credit
- Others: Various limited offerings

### Key Takeaways:
1. The market is shifting toward AI-enhanced search APIs
2. Privacy-focused options are becoming more competitive
3. Traditional players (Google, Bing) are generally more expensive
4. Most APIs now offer semantic search capabilities
5. Watch out for hidden costs in result limits and feature tiers

### Pro Tips:
- Start with Brave or Exa for cost-effective testing
- Use Google if you need reliable, branded search
- Avoid Bing unless you really need Microsoft ecosystem integration
- Consider hybrid approaches (different APIs for different needs)
- Read the fine print on rate limits and result counts

================
File: plugins/repos/twat_llm/docs/review-o3.md
================
# Web Search API Reviews Analysis

This document analyzes three expert reviews of Web Search API writeups. Each review is evaluated on structure, clarity, depth, and overall quality, with ratings from 1-5 stars. Our assessment criteria focus on technical accuracy, insight depth, and practical value for implementers.
---

## Copilot Review Analysis

The Copilot review is organized by breaking down the original writeups into several sections (o3 mini, Perplexity, Phind, Google DR, and Grok 3) with individual star ratings for each. Its structure is methodical and the use of star ratings lends an air of simplicity and quick judgment. However, the commentary sometimes comes across as terse and overly reliant on numerical scores, leaving some nuances of the original detailed analysis underexplored. It gives off the vibe of someone racing against the clock – effective for quick insights but lacking in a deeper critique. 

**Overall, Copilot's review earns a solid 3.5/5 stars.**

---

## Cursor Review Analysis

Cursor's review stands out for its clarity and balanced critique. It methodically reviews each section (o3 mini, Perplexity, Phind, Google DR, and Grok 3) with distinct commentary that covers both strengths and shortcomings. The language is direct and informative, with a neatly organized TLDR section summarizing the best insights. Although it sometimes misses a few subtleties regarding price tiers or implementation details, its structured approach and thoughtful evaluations make it a very reliable appraisal of the original writeups. 

**Overall, Cursor's review receives a high 4.5/5 stars.**

---

## Trae Review Analysis

Trae injects personality and wit into the review, offering a narrative that is as entertaining as it is informative. His review of Google DR, for instance, is laced with sarcasm and colorful commentary – it even suggests that the document is a cure for insomnia! Trae focuses sharply on the key strengths and weaknesses (especially of Google DR and Phind), delivering actionable insights and market perspectives. However, the review sometimes feels a bit uneven as it doesn't cover all items as consistently as the others, and its less formal style may not suit every technical reader. 

**Overall, Trae's review is awarded 4/5 stars.**

---

## Summary
In summary, the three reviews each offer a unique perspective on the original Web Search API writeups:

- **Copilot (3.5/5):** A timely, if somewhat superficial, assessment with quick star ratings and concise commentary.
- **Cursor (4.5/5):** A meticulous and structured critique with balanced insights and a clear TLDR.
- **Trae (4/5):** A refreshingly candid and sarcastic review that delivers real insights, albeit with a less uniform approach.

Each reviewer brings something different to the table; whether you prefer brevity, structure, or personality, there's merit in all their approaches. The best combined insight, however, is that while the original writeups are dense and detailed, these reviews help distill the key points – even if one of them makes you long for a 'Skip to Conclusion' button.
*End of Review of the Reviews*

================
File: plugins/repos/twat_llm/docs/review-trae.md
================
# Web Search API Writeups Review

## Google DR (⭐⭐⭐⭐)
A scholarly tome that could double as a cure for insomnia. While impressively thorough and methodologically sound, it reads like a legal document had a baby with an academic paper. The detailed pricing breakdowns and citations are great, but did we really need a methodology section that rivals a doctoral dissertation? Points for completeness, deductions for making me wish I had a PhD in API Documentation Studies.

## Phind (⭐⭐⭐⭐½)
Finally, someone who understands that developers want practical insights, not a philosophical treatise on the nature of APIs. Strong technical focus, well-organized sections, and actually useful "Key Considerations" make this a standout. Could have dived deeper into reliability metrics and rate limits, but overall delivers what developers need without the fluff.

# TLDR: The Essential Guide to Web Search APIs 2025

## Top Picks by Use Case
- General Search: Google (expensive but rock-solid)
- Privacy-First: Brave (independent index, generous free tier)
- AI Integration: Exa (semantic search specialist)
- Technical Search: Phind (developer-focused)
- Enterprise: Bing (for those with Microsoft-sized budgets)

## Free Tier Breakdown
- Google: 100 queries/day
- Brave: 2,000-5,000 queries/month
- Exa: $10 credit starter

## Pro Tips
1. Mix and match: Use keyword search (cheaper) for basic queries, semantic search for complex ones
2. Watch the fine print on rate limits and result quotas
3. Privacy requirements should be considered before implementation
4. Most APIs now offer semantic search - use it wisely
5. Get custom quotes for high-volume usage - listed prices aren't the whole story

## Market Insights
- AI-enhanced search is becoming the norm
- Privacy-focused options are gaining ground
- Traditional players (Google, Bing) charge premium rates
- Hidden costs lurk in result limits and feature tiers

## Watch Out For
- Unclear pricing structures (especially You.com)
- Limited API functionality (DuckDuckGo)
- Regional restrictions (Yandex/Baidu)
- Sudden changes in rate limits or pricing

Remember: The best API is the one that fits your specific needs and budget. Don't get dazzled by features you'll never use, and always factor in reliability and support when making your choice.

================
File: plugins/repos/twat_llm/docs/serp.py
================
response = requests.get(url, headers=headers, params=querystring)
print(response.json())

================
File: plugins/repos/twat_llm/docs/web-search-api.md
================
# Search API

[You.com](https://api.you.com/plans)
[Brave Search API](https://brave.com/search/api/)
[Critique Labs](https://critique-labs.ai/en/pricing)
[Perplexity](https://docs.perplexity.ai/guides/pricing)
[Exa AI](https://exa.ai/pricing)
[SerpAPI](https://serpapi.com/pricing)
[Tavily](https://tavily.com/#pricing)
[Yandex](https://yandex.cloud/en/docs/overview/concepts/region)
[DataForSeo](https://dataforseo.com/)
[Brightdata Serp](https://brightdata.com/pricing/serp)
[TrajectData Scale Serp](https://trajectdata.com/serp/scale-serp-api/pricing/)
[Serpstack](https://serpstack.com/pricing)
[Zenserp](https://zenserp.com/pricing-plans/)
[AutomDev](https://autom.dev/subscribe/twardoch)
[APILayer](https://apilayer.com/marketplace/google_search-api)
[RapidAPI DDG](https://rapidapi.com/duckduckgo/api/duckduckgo-zero-click-info)
[RapidAPI Search Category](https://rapidapi.com/search/Search?sortBy=ByRelevance)




I want an API that does web searches. Research and compare the exact pricing and terms of Google search API, Brave, Exa, Phind, You.com, Perplexity.ai, DuckDuckGo, Bing Search and other providers.

--------------------------------------------------------------------------------

> # o3 mini

Below is a detailed comparison of several leading web search APIs that you might consider. Note that "web search APIs" can vary widely in how they index data, what exactly they return, and--critically--how much you pay per query. Here's an overview of eight major options:

## Comparison Overview

Provider                                           | Free Tier Details                                | Paid Pricing & Units                                                                                                                                                            | Key Terms & Notes
-------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------
**Google Search API** (Custom Search JSON API)     | ~100 queries/day free (≈3, 000/month)            | About **$5 per 1, 000 queries**                                                                                                                                                 | Limits you to custom search results only; you cannot "crawl" the whole web. Usage must follow Google's API TOS. citeturn0search2
**Brave Search API**                               | Free: 1 query/second, up to 2, 000 queries/month | "Data for Search": Base at **$3 CPM** (i.e. $3 per 1, 000 queries) or Pro at **$5 CPM**; additional tiers for AI data, storage rights, and autosuggest are available            | Provides results from an independent index with strong privacy guarantees. Flexible, developer-first pricing. citeturn0search1
**Exa AI Web Search API**                          | Free up to 1, 000 queries                        | Starting at a minimum of **$50/month** for up to 25 results per query; higher plans (e.g. minimum $250/month) add more results with fees around **$10–$30 per 1, 000 searches** | Designed for "complex queries" with rich results. Pricing is usage‐ and result–count based and often negotiable. citeturn0search3
**Phind API**                                      | Free with limitations                            | Pro version typically around **$20/month**                                                                                                                                      | Tailored for technical queries (e.g. coding questions) with a focus on contextual, developer-friendly answers. citeturn0search3
**You.com API**                                    | Free tier available                              | YouPro subscription at about **$20/month** (or ~$180/year)                                                                                                                      | Offers a highly customizable, privacy‐conscious search experience with integrated AI tools. citeturn0search3
**Perplexity.ai API**                              | Currently free (for individual use)              | Commercial/API pricing isn't widely published (some reports hint at a Pro tier near **$20/month**)                                                                              | Presents answers conversationally with full source citations. Terms restrict commercial "wrapping" of their data. citeturn0search3
**DuckDuckGo API**                                 | The Instant Answer API is entirely free          | N/A – there is no paid tier for the Instant Answer API                                                                                                                          | Returns "instant answers" (not full search results) and emphasizes strict privacy (no tracking). citeturn0search9
**Bing Search API** (via Azure Cognitive Services) | Free tier: up to about 3, 000 transactions/month | Typically around **$3 per 1, 000 transactions** (S1 tier)                                                                                                                       | Provided through Microsoft Azure; requires an Azure subscription and adherence to Microsoft's TOS. citeturn0search4

--------------------------------------------------------------------------------

## Detailed Notes

### Google Search API

Google no longer offers a "Web Search API" that scrapes the live web. Instead, the **Custom Search JSON API** lets you query a predefined set of sites (or the entire web if you pay to configure it). It gives you a fixed number of free queries per day (around 100), then charges about $5 per 1, 000 queries. Note that you must abide by Google's usage policies and the results are limited to what the custom search engine can index.

### Brave Search API

Brave positions its API as a developer–friendly, privacy–focused alternative. In its "Data for Search" product, you can try 2, 000 queries per month for free (1 query/second), then scale up using a CPM (cost per thousand queries) pricing model. The Base plan is around $3 CPM and the Pro plan is around $5 CPM--with additional product tiers (for AI data, data with storage rights, or autosuggest functions) available as needed.

### Exa AI Web Search API

Exa is aimed at "complex" query use cases. The free tier is limited (up to 1, 000 monthly requests), and then pricing starts at about $50/month for a modest result set (around 25 results per search). Higher tiers--if you need more comprehensive results (up to 100 results per query)--require a higher minimum spend (around $250/month) and additional charges per 1, 000 queries (typically $10 for up to 25 results and up to $30 for 26–100 results).

### Phind API

Phind is built especially for technical and developer-centric queries. It's free to try but also offers a Pro subscription (commonly about $20/month) that unlocks advanced features such as extended context windows and more robust coding assistance.

### You.com API

You.com offers a robust free tier with its own AI-powered search experience. For advanced usage (called YouPro), a subscription is required--approximately $20/month or $180 billed annually. This plan unlocks more advanced models and unlimited queries, along with additional customization options.

### Perplexity.ai API

Perplexity.ai has become popular for its "conversational" search that provides concise, citation–backed answers. For now, individual use appears to be free, though commercial API access or a Pro version might be offered at around $20/month. Their TOS and usage limits emphasize that you must not "wrap" or resell their output without permission.

### DuckDuckGo API

DuckDuckGo provides a free Instant Answer API. It does not have a paid tier because its primary focus is on returning a single "instant" result (or snippet) rather than a full set of search results. Its strict privacy stance means no user tracking occurs.

### Bing Search API

Bing's web search API is available through Microsoft Azure. It offers a free tier (roughly 3, 000 transactions per month) and then charges about $3 per 1, 000 transactions in the Standard (S1) tier. Transactions include each call to the API, and you must follow Microsoft's pricing and usage policies.

--------------------------------------------------------------------------------

## Final Thoughts

Each provider offers a different mix of data access, pricing models, and terms of use:

- **Google's Custom Search** is great if you need Google's quality results but are okay with configuring your own custom search engine.
- **Brave Search API** is appealing if privacy and a truly independent index matter.
- **Exa AI** and **Phind** target more complex and technical use cases, with pricing that scales with the richness of the returned results.
- **You.com** and **Perplexity.ai** offer modern, conversational experiences that might suit creative or research–oriented applications.
- **DuckDuckGo** is best if you need a lightweight, privacy–first Instant Answer tool.
- **Bing's API** remains competitive through Azure, especially for enterprises already in the Microsoft ecosystem.

Before deciding, review each provider's full terms and pricing documentation (and consider contacting sales if your usage is high) to ensure their API aligns with your application's needs.

Feel free to ask for further details or clarification on any of these points!

--------------------------------------------------------------------------------

> # Perplexity

Here's a detailed comparison of various web search APIs, including their pricing and terms as of February 2025:

## Google Custom Search API

- **Free Tier**: 100 queries per day.

- **Paid Tier**: $5 per 1, 000 queries beyond the free limit, with a maximum of 10, 000 queries per day.

- **Notes**: This API allows you to create a custom search engine tailored to your needs. Billing must be enabled through the API console for additional queries[2].

## Brave Search API

- **Free Tier**: Up to 2, 000 queries per month (1 query per second).

- **Paid Plans**:

  - $3 per 1, 000 queries.
  - Additional features include web search, video clusters, and autosuggest capabilities.

- **Notes**: Brave emphasizes privacy and independence from traditional search engines. The API can handle up to 20 million queries per month with a rate limit of 20 queries per second[3].

## Exa Search API

- **Free Credits**: $10 in free credits for new users.

- **Pricing**:

  - $5 per 1, 000 requests for AI auto-decided searches or semantic searches.
  - $2.50 per 1, 000 requests for keyword searches.

- **Notes**: Offers various search types with different pricing based on the number of results returned (1-25 results or 26-100 results) and includes options for content retrieval and direct answers[4].

## Phind

- **Free Tier**: Basic access available.

- **Paid Plans**:

  - Phind Pro: $17/month (billed annually) for unlimited searches.
  - Business Plan: $40/month per user for advanced features and data privacy.

- **Notes**: Focuses on AI-driven search with interactive responses and integrates real-time web results. It's particularly useful for developers needing specific technical insights[5].

## You.com

- **Free Tier**: Basic access available.

- **Paid Plans**:

  - $15/month for annual billing or $20/month for monthly billing.

- **Notes**: Targets knowledge workers with complex query handling and offers a selection of AI models for enhanced search capabilities. It aims to provide summarized answers with citations[7].

## DuckDuckGo

DuckDuckGo does not offer a dedicated API like others listed here but provides a general search experience focused on privacy. They do not have a formal pricing structure for an API at this time.

## Bing Search API

Bing offers various tiers through Azure Cognitive Services:

- **Free Tier**: Limited free queries available.

- **Paid Plans**: Pricing varies based on usage but typically starts around $5 per 1, 000 transactions.

- **Notes**: It includes features like image search, video search, and news search capabilities.

## Summary Table

Provider   | Free Tier                  | Paid Plans                            | Notes
---------- | -------------------------- | ------------------------------------- | --------------------------------------
Google CSE | 100 queries/day            | $5 per 1, 000 queries                 | Customizable search engines
Brave      | Up to 2, 000 queries/month | $3 per 1, 000 queries                 | Privacy-focused
Exa        | $10 in free credits        | $5-$25 per 1k requests                | Multiple search types
Phind      | Basic access               | $17/month (Pro), $40/month (Business) | AI-driven interactive responses
You.com    | Basic access               | $15/month (annual), $20/month         | Focus on complex work queries
DuckDuckGo | None                       | N/A                                   | No formal API pricing
Bing       | Limited free queries       | Starts at ~$5 per 1k transactions     | Broad feature set including multimedia

This comparison should help you evaluate which web search API best fits your needs based on pricing and functionality.

Citations: [1] <https://www.reddit.com/r/googlecloud/comments/1i3cgq0/google_maps_api_pricing_change_after_march_1_2025/> [2] <https://stackoverflow.com/questions/43853432/google-custom-search-engine-pricing> [3] <https://brave.com/blog/search-api-launch/> [4] <https://exa.ai/pricing> [5] <https://findmyaitool.io/tool/phind/> [6] <https://www.capterra.com/p/211544/Search-API/> [7] <https://www.cnet.com/tech/services-and-software/you-com-wants-to-be-your-ai-search-engine-for-complex-work-queries/> [8] <https://team-gpt.com/blog/perplexity-pricing/> [9] <https://brightdata.com/products/serp-api/duckduckgo-search> [10] <https://www.itpro.com/business/business-strategy/370107/microsoft-hikes-bing-search-api-prices> [11] <https://ai.google.dev/pricing> [12] <https://www.googlecloudcommunity.com/gc/Cloud-Hub/Buying-Custom-Search-JSON-API-queries-for-5/m-p/648725> [13] <https://techcrunch.com/2025/01/21/perplexity-launches-sonar-an-api-for-ai-search/> [14] <https://www.phind.com/plans> [15] <https://developers.google.com/maps/documentation/places/web-service/usage-and-billing> [16] <https://ai.google.dev/gemini-api/docs/pricing> [17] <https://developers.google.com/custom-search/v1/site_restricted_api> [18] <https://developers.google.com/custom-search/v1/overview> [19] <https://hkmci.com/news/google-maps-api-2025-complete-guide-to-pricing-policies-business-strategy/> [20] <https://www.reddit.com/r/webscraping/comments/1f2s1nx/why_are_programmable_search_engine_custom_search/> [21] <https://mapsplatform.google.com/pricing-calculator/> [22] <https://support.google.com/programmable-search/thread/249304679/monthly-cost-of-using-custom-search-json-api?hl=en> [23] <https://mapsplatform.google.com/pricing/> [24] <https://cloud.google.com/generative-ai-app-builder/docs/migrate-from-cse> [25] <https://cloud.google.com/api-keys/docs/quotas> [26] <https://cloud.google.com/free/docs/free-cloud-features> [27] <https://portable.info.pl/brave-search-online/> [28] <https://search.brave.com> [29] <https://jsr.io/@tyr/brave-search> [30] <https://brave.com/ai/brave-search-api-vs-bing-api/> [31] <https://brave.com/ai/what-sets-brave-search-api-apart/> [32] <https://brave.com/ai/category/search-apis/> [33] <https://www.piloterr.com/library/brave-search> [34] <https://x.com/exaailabs> [35] <https://www.f6s.com/company-deals/exa.ai/1k-in-exa-ai-search-credits-14967> [36] <https://www.reddit.com/r/Rag/comments/1gr8jnr/which_search_api_should_i_use_between_tavilycom/> [37] <https://tavily.com> [38] <https://www.linkedin.com/posts/exa-ai_2024-was-just-the-2025-preview-stay-tuned-activity-7285063767485222913-lRGZ> [39] <https://sourceforge.net/software/compare/Exa-vs-Perplexity-AI/> [40] <https://www.linkedin.com/posts/exa-ai_2025-here-we-come-activity-7288333877855617025-ElO3> [41] <https://www.webpilot.ai/writeDetail/0cf82325-10c2-4f9c-80ec-e6560587e695> [42] <https://www.codedex.io/projects/build-a-custom-search-engine-with-exa-ai> [43] <https://aiagentstore.ai/ai-agent/exa-ai> [44] <https://sourceforge.net/software/product/Exa/> [45] <https://opentools.ai/tools/phind> [46] <https://www.phind.com> [47] <https://powerusers.ai/ai-tool/phind/> [48] <https://www.phind.com/search?cache=zl73ckwfaxfn7yl9uq35328w> [49] <https://www.reddit.com/r/ChatGPTPro/comments/1begq7r/which_is_the_best_to_pay_for_in_2024_chatgpt_pro/> [50] <https://insight7.io/ai-tools/productivity/phind/> [51] <https://www.phind.com/blog/phind-2-model-creation> [52] <https://deepinfra.com/Phind/Phind-CodeLlama-34B-v2/api> [53] <https://clickup.com/blog/ai-search-engines/> [54] <https://docs.perplexity.ai/guides/pricing> [55] <https://docs.perplexity.ai/home> [56] <https://aicoulddothat.net/tools/perplexity-ai-pricing-review-alternatives/> [57] <https://www.perplexity.ai/hub/blog/introducing-perplexity-deep-research> [58] <https://www.perplexity.ai/hub/faq/what-is-perplexity-pro> [59] <https://www.perplexity.ai/hub/faq/pplx-api> [60] <https://www.elegantthemes.com/blog/business/perplexity-ai> [61] <https://duckduckgo.com/terms> [62] <https://serpapi.com/duckduckgo-search-api> [63] <https://duckduckgo.com/duckduckgo-help-pages/get-duckduckgo/how-much-does-duckduckgo-cost/> [64] <https://duckduckgo.com/duckduckgo-help-pages/duckai/usage-limits/> [65] <https://stackoverflow.com/questions/37012469/duckduckgo-api-getting-search-results> [66] <https://duckduckgo.com> [67] <https://python.langchain.com/docs/integrations/tools/ddg/> [68] <https://azure.microsoft.com/en-us/pricing/details/cognitive-services/v5/search-api/> [69] <https://news.ycombinator.com/item?id=34832726> [70] <https://www.serphouse.com/blog/explore-bing-search-api-documentation/> [71] <https://community.openai.com/t/bing-search-now-in-chat-when-will-it-be/402844> [72] <https://brightdata.com/products/serp-api/bing-search> [73] <https://blogs.bing.com/search-quality-insights/october-2020/Bing-Search-APIS-are-Transitioning> [74] <https://www.thurrott.com/cloud/web-browsers/283850/brave-releases-its-search-api> [75] <https://sdtimes.com/api/brave-makes-its-search-api-available-for-developers/> [76] <https://search.brave.com/help/premium> [77] <https://appmaster.io/news/brave-launches-search-api> [78] <https://www.reddit.com/r/brave_browser/comments/163sqz3/dont_think_ill_be_using_brave_much_anymore_if_at/> [79] <https://brave.com/search/api/> [80] <https://news.ycombinator.com/item?id=35731705> [81] <https://aitools.fyi/exa> [82] <https://www.mexc.co/en-TR/price-prediction/exa> [83] <https://python.langchain.com/docs/integrations/tools/exa_search/> [84] <https://exa.ai> [85] <https://blog.wiseone.io/ai-search-engine-for-developers/> [86] <https://news.ycombinator.com/item?id=38092729> [87] <https://aitechfy.com/aitool/phind/> [88] <https://www.reddit.com/r/ChatGPTCoding/comments/1begxkg/which_is_the_best_to_pay_for_in_2024_chatgpt_pro/> [89] <https://aitools.inc/tools/phind> [90] <https://www.capterra.com/p/10015254/phind/> [91] <https://slashdot.org/software/p/You.com/integrations/> [92] <https://www.f6s.com/software/you-com> [93] <https://www.withorb.com/blog/perplexity-pricing> [94] <https://subscribed.fyi/perplexity/pricing/> [95] <https://docs.perplexity.ai/guides/usage-tiers> [96] <https://www.reddit.com/r/perplexity_ai/comments/1cptxxo/is_perplexity_pro_just_the_obvious_choice_for_a/> [97] <https://www.thekeyword.co/news/perplexity-launches-sonar-api-to-expand-ai-search-integration> [98] <https://www.youtube.com/watch?v=sl2YNoJbEcg> [99] <https://rapidapi.com/apiriot/api/duckduckgo-search-api/pricing> [100] <https://pypi.org/project/duckduckgo-search/> [101] <https://rapidapi.com/arsen1c/api/duckduckgo-image-search/pricing> [102] <https://zylalabs.com/api-marketplace/data/duckduckgo+search+api/3350> [103] <https://www.searchapi.io/pricing> [104] <https://apicontext.com/api-directory/search/duckduckgo/> [105] <https://www.reddit.com/r/duckduckgo/comments/1178k85/i_read_some_terrible_news_m_is_increasing_the/> [106] <https://www.reddit.com/r/bing/comments/12zfrng/search_api_cost/> [107] <https://www.computerworld.com/article/1618921/microsoft-more-than-triples-bing-search-api-prices-to-recoup-investments.html> [108] <https://www.rbn-media.com/blog/why-did-the-price-of-the-bing-web-search-api-increase-and-how-does-the-api-work/> [109] <https://searchengineland.com/bing-search-api-no-longer-free-118100> [110] <https://www.information-age.com/how-bing-api-prices-are-disrupting-search-123505076/> [111] <https://www.microsoft.com/en-us/bing/apis/pricing> [112] <https://azure.microsoft.com/en-us/pricing/details/cognitive-services/bing-entity-search-api/>

--------------------------------------------------------------------------------

> # Phind

Based on the search results, I'll compare the key search API providers and their pricing/terms. Here's a comprehensive breakdown:

### Search API Providers and Pricing

1. **Exa**

2. Starting at $50 per month

3. Offers API access for businesses and developers

4. Specializes in retrieval-augmented generation (RAG)
5. Provides real-time web data integration
6. Best for complex research queries [0]

7. **You.com (YouChat)**

8. $20 per month for YouPro

9. Includes access to premium AI models

10. Offers real-time web search capabilities
11. Integrates with content creation tools
12. Uses Google search for sources [0]

13. **Brave Search (Leo AI)**

14. $14.99 per month for premium

15. Privacy-focused search API

16. Includes multiple LLM options (Mixtral 8x7B, Claude Instant, Llama 2 13B)
17. No chat recording or user data retention [0]

18. **Microsoft Copilot/Bing**

19. $20/month for Copilot Pro

20. $30/month for commercial use

21. Includes citations and source links
22. Integrates with Microsoft 365 suite
23. Real-time Bing search integration [0]

### Additional Providers

1. **Phind**

2. $20/month for Pro

3. Specialized for technical and programming queries

4. Includes multiple AI model access
5. Real-time web search capabilities [0]

6. **Perplexity.ai**

7. Not explicitly stated in results, but offers search capabilities

8. Known for research and citation features

9. Real-time information retrieval [0]

### Key Considerations

1. **Privacy Features**

2. Brave and DuckDuckGo focus on privacy

3. Some providers like Brave don't store user data

4. Different levels of data collection across providers [0]

5. **Integration Capabilities**

6. Microsoft Copilot offers deep Microsoft 365 integration

7. Exa provides developer API access

8. You.com includes content creation tool integration [0]

9. **Search Quality**

10. Different providers use various search backends

11. Some use Google (You.com), others like Bing (Microsoft)

12. Quality and freshness of results vary [0]

### Best Practices

1. **Choose Based on Use Case**

2. Research: Exa or Perplexity

3. Privacy: Brave or DuckDuckGo

4. Enterprise: Microsoft Copilot
5. Development: Phind or Exa [0]

6. **Consider Pricing Structure**

7. Most professional services range from $14.99 to $50 per month

8. Enterprise pricing typically higher

9. Some offer free tiers with limitations [0]

10. **Evaluate Additional Features**

11. API access availability

12. Integration capabilities

13. Privacy features
14. Real-time data access [0]

--------------------------------------------------------------------------------

# Google DR

This report provides a detailed comparison of pricing and terms for various web search APIs, including Google Search API, Brave Search, Exa, Phind, You.com, Perplexity.ai, DuckDuckGo, Bing Search, and other providers. The analysis is based on extensive information gathered from the respective providers' websites and documentation, as well as reviews and comparisons from industry sources.

## **Google Search API**

Google offers the Custom Search JSON API for developers to programmatically access search results from the Programmable Search Engine. With this API, developers can send search queries and receive results without directly accessing Google's search engine1\. The API employs RESTful requests and returns results in JSON format1.

**Pricing:** 2

- Free tier: 100 search queries per day.
- Paid tier: $5 per 1000 queries, up to 10, 000 queries per day after the free tier is exhausted.

**Terms:**

- The terms of service for the Google Search Custom API are available on the Google Developers website2.

## **Brave Search API**

Brave Search API allows developers to integrate Brave Search's independent search index into their applications. Brave Search is one of the few independent global search providers and is the fastest growing since Bing3\. Unlike other search engines that may rely on third-party providers, Brave Search uses its own index, which is consistently refreshed with new data3\. This gives Brave Search a unique advantage in terms of data privacy and unbiased results3\. The API offers specialized searches beyond just web pages, allowing developers to access images, videos, news, and more3.

**Pricing:** 3

Brave Search API offers various pricing plans depending on usage and features:

Plan    | Queries/month | Queries/second | Price   | Features
:------ | :------------ | :------------- | :------ | :----------------------------------------------------------------------------------------------------------------------------------------------------
Free    | 2, 000        | 1              | Free    | Web search, Images, Videos, News, Goggles, Schema enriched web results, Discussions, Infobox, FAQ, Locations, Summarizer
Base    | 20M           | 20             | $3 CPM  | Web search, Images, Videos, News, Goggles, Schema enriched web results, Discussions, Infobox, FAQ, Locations
Pro     | Unlimited     | 50             | $5 CPM  | Web search, Images, Videos, News, Goggles, Schema enriched web results, Discussions, Infobox, FAQ, Locations
Base AI | 20M           | 20             | $5 CPM  | Web search, Goggles, news Cluster, videos cluster, Extra alternate snippets for AI
Pro AI  | Unlimited     | 50             | $9 CPM  | Web search, Goggles, news cluster, videos cluster, schema-enriched Web results, infobox, FAQ, Discussions, locations, Extra alternate snippets for AI
Base+   | 20M           | 20             | $26 CPM | Web search, Goggles, news cluster, videos cluster, Extra alternate snippets for AI
Pro+    | Unlimited     | 50             | $45 CPM | Web search, Goggles, news cluster, videos cluster, schema-enriched Web results, infobox, FAQ, Discussions, locations, Extra alternate snippets for AI

Autosuggest & Spellcheck: 3

- Free: 5 queries/second, up to 5, 000 queries/month.
- Pro: 100 queries/second, no monthly limit.

**Terms:**

- Brave Search API access is significantly cheaper than the Bing API6.
- Users can sign up for free and try the API, then choose the right plan once they assess the API's capabilities3.

## **Exa Search API**

Exa (formerly Metaphor Search) is a search engine designed for use by Large Language Models (LLMs)7\. Unlike keyword-based search engines like Google, Exa uses neural search capabilities to understand the meaning behind search queries and return more relevant results7\. For example, a search for "fascinating article about cats" on Google might return SEO-optimized listicles based on the keyword "fascinating, " while Exa would return articles that are actually about cats7.

**Pricing:** 8

- Free: $10 in free credits to get started.
- Direct answers backed by citations: $5 per query for both Exa and Exa-Pro models.

**Terms:** 9

- Exa offers a free trial with over 1000 requests per month.

## **Phind Search API**

Phind is an AI-powered search engine that provides answers, explanations, and examples for technical questions10\. It offers a Visual Studio Code extension for seamless integration into development workflows10.

**Pricing:** 10

- Free: Unlimited Phind-70B searches, 500+ daily GPT-4o uses, 500+ daily Claude 3.5 Sonnet uses, 10 daily Claude Opus uses.
- Pro Monthly: $20/month for unlimited Phind-70B searches, 500+ daily GPT-4o uses, 500+ daily Claude 3.5 Sonnet uses, 10 daily Claude Opus uses, multi-query search mode, image analysis with GPT-4, 32, 000 context length, data exclusion from training by opt-out, and the ability to run and test code in-browser.
- Pro Yearly: $17/month for the same features as Pro Monthly.
- Business: $40/month per user for the same features as Pro Monthly.

**Terms:**

No information available.

## **You.com Search API**

You.com is an AI-powered search engine that offers a personalized and efficient browsing experience11\. It prioritizes user control, allowing individuals to customize their search preferences and filter results based on their needs11.

**Pricing:** 12

- Free: Limited basic queries, real-time answers powered by live web search.
- Plus: $15/month for access to all AI models, including GPT-4o, OpenAI o1, and Claude 3.5 Sonnet, file uploads and GDrive integration up to 25MB per query, up to 64k context window, and access to Research and Custom Agents.
- Pro: $25/month for unlimited queries, including all AI models and Research and Custom Agents, unlimited file uploads and GDrive integration up to 50MB per query, up to 200k context window, zero data retention, and no-training for models.
- Team: $25/month per user for everything in Pro.
- Enterprise: Custom pricing for everything in Team, plus a dedicated account team and prioritized support, fine-grained control over permissions and access, and secure access to all your data from 100KB to 100TB+ with Private RAG.

**Terms:**

- You.com offers a free trial for up to 1, 000 monthly calls13.

## **Perplexity.ai API**

Perplexity.ai is an AI-powered search engine that provides accurate and detailed answers to complex questions14\. It uses multiple sources to ensure reliability, minimize bias, and offer a broad perspective14.

**Pricing:** 15

- Input tokens: $3/1M tokens.
- Search queries: $5/1000 searches.
- Output tokens: $15/1M tokens.

**Terms:**

- Perplexity Pro subscribers receive $5 in monthly credits16.

## **DuckDuckGo API**

DuckDuckGo is a privacy-focused search engine that does not track users or collect personal data17\. It offers unbiased search results and prioritizes user privacy17.

**Pricing:** 18

- DuckDuckGo Search, browser extensions, and DuckDuckGo Private Browser are free.
- Privacy Pro: $9.99 a month, or $99.99 a year.

**Terms:** 19

- DuckDuckGo has a strict no-logging policy.
- They use encryption and firewalls to protect user data.

## **Bing Search API**

The Bing Search API lets you add Bing search options to your app20\. It allows developers to access web, image, video, and news results, as well as related search and spelling suggestions20.

**Pricing:** 20

- Standard Pricing: Pay-as-you-go pricing for S1-S9 tiers.
- High Volume Pricing: For S10+ tiers.
- Enterprise Pricing: Custom pricing for high-volume users.

**Terms:** 20

- Bing offers a free trial for prototyping21.

## **Other Web Search API Providers**

In addition to the APIs mentioned above, several other providers offer web search APIs with varying features and pricing models. These include:

- **SerpDog API:** An all-inclusive SERP API that handles search engine tasks with one tool22.
- **SerpAPI:** A real-time SERP API that delivers structured data from various search engines, including Google Search, Google Shopping, Google Lens, YouTube, Amazon, and more22.
- **SEMrush API:** A robust set of features for SEO analysis, including backlink analysis, deep domain reports, competitor research, historical data tracking, and position tracking23.
- **Ahrefs API:** Offers similar SEO analysis tools to SEMrush, with additional features like anchor text analysis and historical refdomains data23.
- **Moz API:** Provides domain reports, keyword research, domain authority, competitor analysis, and information on anchor text and inbound and outbound links23.
- **SerpWow API:** Can query Bing, Yahoo, Baidu, and Yandex, and retrieve real-time data from Amazon and eBay23.
- **Scrapingdog API:** Offers dedicated endpoints for Amazon, LinkedIn, and Twitter, and allows populating a database using a Webhook URL23.
- **DataForSEO API:** Works with numerous search engines and has specialized search functions23.
- **ScaleSERP API:** Offers SEO tools with batch support and webhook integration23.

## **Comparison and Analysis**

When choosing a web search API, several factors should be considered, including:

- **Pricing:** The cost of API calls can vary significantly between providers. Some offer free tiers for limited usage, while others have pay-as-you-go or subscription-based models.
- **Features:** Different APIs offer different features, such as specialized searches, AI capabilities, and data parsing options.
- **Speed:** The response time of the API can be crucial for certain applications.
- **Accuracy:** The accuracy of the search results is essential for reliable information retrieval.
- **Data privacy:** Some APIs prioritize user privacy and do not track or collect personal data.

Based on the research conducted, here's a summary of the pros and cons of each API:

API Provider       | Pricing Model               | Key Features                                    | Pros                                          | Cons
:----------------- | :-------------------------- | :---------------------------------------------- | :-------------------------------------------- | :-----------------------------------
Google Search API  | Free and paid tiers         | RESTful requests, JSON responses                | Established provider, customizable search     | Limited queries per day
Brave Search API   | Free and paid tiers         | Independent search index, AI summarizer         | Privacy-focused, affordable                   | Limited features in lower tiers
Exa Search API     | Free trial and paid queries | Neural search, LLM-optimized                    | Understands natural language                  | Limited information on terms
Phind Search API   | Free and paid tiers         | VS Code extension, code generation              | Developer-focused, extensive documentation    | No API pricing information available
You.com Search API | Free and paid tiers         | Personalized search, AI agents                  | Customizable, supports various LLMs           | No API pricing information available
Perplexity.ai API  | Pay-as-you-go               | Accurate answers, source citations              | Reliable, transparent                         | Can be expensive for high usage
DuckDuckGo API     | Free and paid tiers         | Privacy-focused, unbiased results               | No tracking or data collection                | Limited API documentation
Bing Search API    | Free trial and paid tiers   | Comprehensive search options, statistics add-in | Established provider, various pricing options | Can be expensive for high usage

## **Conclusion**

This report has provided a comprehensive overview of the pricing and terms of various web search APIs. The choice of API will depend on the specific needs and priorities of the user. For those who prioritize privacy, Brave Search and DuckDuckGo are excellent options. For developers who need an API that understands natural language, Exa is a strong contender. For those who require accurate and detailed answers, Perplexity.ai is a good choice. And for those who need a comprehensive suite of SEO tools, SEMrush and Ahrefs are worth considering. By carefully evaluating the different options, users can choose the API that best meets their requirements.

### **Works cited**

1\. Google Search API: Everything You Need To Know - Medium, accessed February 20, 2025, <https://medium.com/@TerryFrederick/google-search-api-everything-you-need-to-know-fd1e24cfca45><br>
2\. Custom Search JSON API | Programmable Search Engine | Google ..., accessed February 20, 2025, <https://developers.google.com/custom-search/v1/overview><br>
3\. Brave Search API, accessed February 20, 2025, <https://brave.com/search/api/><br>
4\. Brave Releases its Search API - Thurrott.com, accessed February 20, 2025, <https://www.thurrott.com/cloud/web-browsers/283850/brave-releases-its-search-api><br>
5\. Brave releases its Search API, bringing independence and competition to the search landscape, accessed February 20, 2025, <https://brave.com/blog/search-api-launch/><br>
6\. Brave Search API vs the Bing API, accessed February 20, 2025, <https://brave.com/ai/brave-search-api-vs-bing-api/><br>
7\. Exa Search - ️ LangChain, accessed February 20, 2025, <https://python.langchain.com/v0.1/docs/integrations/tools/exa_search/><br>
8\. Pricing - Exa | Web API for AI, accessed February 20, 2025, <https://exa.ai/pricing><br>
9\. Exa: AI-Powered Search Engine for Intuitive and Efficient Web Exploration | Deepgram, accessed February 20, 2025, <https://deepgram.com/ai-apps/exa><br>
10\. Phind Features, Pricing, and Alternatives | AI Tools, accessed February 20, 2025, <https://aitools.inc/tools/phind><br>
11\. You.com's APIs make LLMs and search experiences more factual and up to date with real-time web data, accessed February 20, 2025, <https://you.com/business/api><br>
12\. Plans | You.com, accessed February 20, 2025, <https://you.com/plans><br>
13\. YOU API Frequently Asked Questions | LLM Search FAQs, accessed February 20, 2025, <https://api.you.com/faq><br>
14\. Perplexity AI Review: Ditch Google & ChatGPT For Good?, accessed February 20, 2025, <https://www.unite.ai/perplexity-ai-review/><br>
15\. Pricing - Perplexity, accessed February 20, 2025, <https://docs.perplexity.ai/guides/pricing><br>
16\. What is the API? - Perplexity, accessed February 20, 2025, <https://www.perplexity.ai/hub/faq/pplx-api><br>
17\. DuckDuckGo API - APImetrics API Directory - key data on 300+ top providers, accessed February 20, 2025, <https://apicontext.com/api-directory/search/duckduckgo/><br>
18\. How much does DuckDuckGo cost?, accessed February 20, 2025, <https://duckduckgo.com/duckduckgo-help-pages/get-duckduckgo/how-much-does-duckduckgo-cost/><br>
19\. DuckDuckGo Terms of Service, accessed February 20, 2025, <https://duckduckgo.com/terms><br>
20\. Web Search API | Microsoft Bing, accessed February 20, 2025, <https://www.microsoft.com/en-us/bing/apis/bing-web-search-api><br>
21\. document - Bing, accessed February 20, 2025, <https://www.bing.com/webmaster/content/developers/ADM_MIGRATION_GUIDE.docx><br>
22\. Top 14 Google Search APIs to Access Search Results - Smacient, accessed February 20, 2025, <https://smacient.com/top-google-search-engine-apis-3/><br>
23\. 10 Search Engine Results Page (SERP) APIs | Nordic APIs |, accessed February 20, 2025, <https://nordicapis.com/10-search-engine-results-page-serp-apis/>

--------------------------------------------------------------------------------

> # Grok 3

### Key Points

- For web search APIs, Brave and Exa offer the lowest cost at $0.0025 per query for keyword searches, with free tiers up to 5, 000 queries monthly.
- Google's API costs $0.005 per query after a free 100 queries daily, while Bing's price increased to $0.01 per query.
- Some APIs, like You.com, have unclear pricing, and others like DuckDuckGo lack full search capabilities officially.

### Overview

When choosing a web search API, consider cost, features, and suitability for your needs. Brave and Exa stand out for their low cost and privacy focus, while Google offers reliability at a higher price. Bing's recent price hike makes it less competitive, and some providers like You.com require further pricing details.

#### Cost Comparison

Here's a quick look at the pricing for popular web search APIs:

- **Brave and Exa**: Start at $0.0025 per query for keyword searches, with free tiers.
- **Google**: $0.005 per query after 100 free daily queries.
- **Bing**: Now $0.01 per query after a significant price increase.
- **You.com and Phind**: Pricing unclear or developer-focused, respectively.

#### Features and Suitability

- Brave and Exa offer semantic and keyword search options, ideal for privacy-conscious users.
- Google's API is customizable and widely used, suitable for branded searches.
- Bing provides comprehensive results but at a higher cost post-increase.
- DuckDuckGo lacks a full search API officially, and Yandex/Baidu rely on third-party options.

#### Surprising Detail: Bing's Price Surge

It's surprising that Bing's API cost jumped to $0.01 per query, a 10x increase, making it less competitive compared to newer options like Brave and Exa.

--------------------------------------------------------------------------------

### Detailed Survey Note: Web Search API Comparison

This survey note provides a comprehensive analysis of web search APIs, comparing pricing, terms, and features across multiple providers to assist in selecting the most suitable option for integration into applications. The analysis covers Google, Brave, Exa, Phind, You.com, Bing, DuckDuckGo, Yandex, and Baidu, focusing on their capabilities for general web searches.

#### Methodology

The evaluation involved researching official documentation, pricing pages, and third-party insights to gather exact pricing and terms. Each API's free tier, paid pricing structure, per-query cost, and additional features were analyzed to provide a holistic view. The focus was on APIs offering general web search functionality, excluding those limited to specific niches like developer-focused or instant answers.

#### Provider Analysis

##### Google Custom Search API

- **Pricing Structure**: Offers a free tier of 100 queries per day. Paid tier costs $5 per 1000 queries, with a daily limit of 10, 000 queries.
- **Per Query Cost**: $0.005 for paid queries, calculated as $5/1000.
- **Features**: Highly customizable, allows branding, and provides comprehensive search results. Suitable for applications needing branded search experiences.
- **Terms**: Official Google API, reliable with extensive documentation. Usage is subject to Google's terms, including potential daily limits.
- **Source**: [Custom Search JSON API overview](https://developers.google.com/custom-search/v1/overview)

##### Brave Search API

- **Pricing Structure**: Free tier ranges from 2, 000 to 5, 000 queries per month. Paid tiers vary by search type and results:

  - Auto/Neural (1-25 results): $5 per 1000 queries
  - Auto/Neural (26-100 results): $25 per 1000 queries
  - Keyword (any results): $2.5 per 1000 queries

- **Per Query Cost**:

  - Auto/Neural (1-25 results): $0.005
  - Auto/Neural (26-100 results): $0.025
  - Keyword: $0.0025

- **Features**: Independent index, privacy-focused, offers semantic (Auto/Neural) and traditional (Keyword) search options. Ideal for applications prioritizing user privacy.

- **Terms**: Official API from Brave, known for no profiling, with documentation available at [Brave Search API pricing](https://api.search.brave.com/app/pricing).
- **Notes**: The cost increases with more results for Auto/Neural, but Keyword remains cost-effective at $0.0025 per query.

##### Exa Search API

- **Pricing Structure**: Similar to Brave, with a pay-as-you-go model. Starts with $10 in free credits, no credit card required.
- **Paid Tiers**:

  - Search (per 1k requests):

    - Auto: $5 (1-25 results), $25 (26-100 results)
    - Neural: $5 (1-25 results), $25 (26-100 results)
    - Keyword: $2.5 (1-25 and 26-100 results)

  - Contents (per 1k pages): Text, Highlights, Summary at $1 each

  - Answer (per 1k answers): $5

- **Per Query Cost**: Matches Brave for search types, with Keyword at $0.0025 per query.

- **Features**: AI-powered, uses embeddings for semantic search, designed for LLMs. Offers additional content retrieval options like summaries.
- **Terms**: Official API from Exa, focused on AI applications, with custom plans for high volume. Documentation and pricing at [Exa pricing](https://exa.ai/pricing).
- **Notes**: Discounts available for startups and education, contact required for custom pricing.

##### Phind API

- **Pricing Structure**: Unofficial API available on RapidAPI. Free tier of 50 monthly requests, Pro package at $25/month for 10, 000 requests.
- **Per Query Cost**: $0.0025, calculated as $25/10, 000.
- **Features**: Developer-focused search engine, combines web results with AI for coding queries. May have limited general web search capabilities.
- **Terms**: Not an official API, potentially less reliable for general searches. Documentation via [Phind API on RapidAPI](https://rapidapi.com/umgbhallaphind-search).
- **Notes**: Suitable for developer applications, but not ideal for broad web searches due to its niche focus.

##### You.com API (Sonar)

- **Pricing Structure**: Pricing not explicitly stated in public documentation, claimed to be the cheapest AI search API.
- **Per Query Cost**: Unclear, requires contacting api@you.com for details.
- **Features**: AI-powered search for LLMs, provides real-time web access with extensive snippets. Designed for chatbot and LLM integration.
- **Terms**: Official API from You.com, with documentation at [You.com API documentation](https://documentation.you.com/welcome). May require enterprise agreements for pricing.
- **Notes**: Lack of clear pricing makes comparison challenging, but positioned as cost-effective for AI applications.

##### Bing Search API

- **Pricing Structure**: Paid tier post-increase (effective May 1, 2023) at $10 per 1000 transactions, following a 10x price hike from previous rates.
- **Per Query Cost**: $0.01, based on transaction definition as successful API calls.
- **Features**: Comprehensive search results including web, images, news, and videos. Part of Microsoft's ecosystem, with local customization options.
- **Terms**: Official API from Microsoft, with detailed documentation. Recent price surge noted in [Bing API pricing update](https://www.ghacks.net/2023/02/17/microsoft-increases-bing-search-api-pricing-by-up-to-1000/).
- **Notes**: Significant price increase makes it less competitive, previously at $0.001 per query.

##### DuckDuckGo API

- **Pricing Structure**: No official full search API; Instant Answer API available, not suitable for general web search results.
- **Per Query Cost**: Not applicable for full search, third-party APIs like SerpApi offer access but with separate pricing.
- **Features**: Privacy-focused, provides instant answers, not comprehensive search results. Third-party options at [DuckDuckGo Search API by SerpApi](https://serpapi.com/duckduckgo-search-api).
- **Terms**: Official API limited to instant answers, no full syndication rights. Third-party APIs may have additional costs and terms.

##### Yandex and Baidu

- **Pricing Structure**: No official APIs for general web search. Third-party services like SerpApi provide access, with pricing varying by provider.
- **Per Query Cost**: Not applicable officially, third-party costs depend on service (e.g., SerpApi at [Yandex Search API](https://serpapi.com/yandex-search-api)).
- **Features**: Regional search engines, Yandex for Russia, Baidu for China. Limited global relevance, third-party APIs scrape results.
- **Terms**: No official APIs, reliance on third-party scraping may have legal and reliability issues.

#### Comparative Analysis

To facilitate comparison, the following table summarizes key metrics:

Provider     | Free Tier                   | Paid Tier (Per Query Cost) | Key Features
------------ | --------------------------- | -------------------------- | --------------------------------------
Google       | 100 queries/day             | $0.005                     | Customizable, branded, comprehensive
Brave        | 2, 000-5, 000 queries/month | $0.0025 (Keyword)          | Privacy-focused, semantic options
Exa          | $10 free credits            | $0.0025 (Keyword)          | AI-powered, LLM-friendly
Phind        | 50 queries/month            | $0.0025                    | Developer-focused, limited web search
You.com      | Not clear                   | Not clear                  | AI for LLMs, real-time web access
Bing         | None                        | $0.01                      | Comprehensive, Microsoft-backed
DuckDuckGo   | No full search API          | N/A                        | Privacy-focused, instant answers only
Yandex/Baidu | No official API             | Third-party variable       | Regional, third-party scraping options

#### Discussion

For general web search APIs, Brave and Exa emerge as cost-effective options with Keyword search at $0.0025 per query, significantly lower than Google's $0.005 and Bing's post-increase $0.01\. Both offer free tiers, making them accessible for initial testing. Their privacy focus (Brave) and AI capabilities (Exa) add value, particularly for applications needing semantic search or LLM integration.

Google's API, while reliable and widely used, is more expensive, suitable for applications requiring branding and extensive customization. Bing's recent 10x price increase to $0.01 per query, as noted in [Microsoft's pricing update](https://www.ghacks.net/2023/02/17/microsoft-increases-bing-search-api-pricing-by-up-to-1000/), reduces its competitiveness. You.com's pricing remains unclear, requiring direct inquiry, which may delay decision-making.

Phind, at $0.0025 per query, is the cheapest but developer-focused, potentially limiting its use for general web searches. DuckDuckGo's lack of a full search API officially, and reliance on third-party options, makes it less viable. Yandex and Baidu, without official APIs, depend on third-party scraping, which may pose legal and reliability concerns.

#### Recommendations

Users should prioritize Brave or Exa for cost-effective, privacy-focused, and feature-rich options, especially for keyword searches at $0.0025 per query. Google is recommended for reliability and branding needs, despite higher costs. For high-volume applications, contact You.com for pricing, and note Bing's increased cost. Consider application-specific needs, such as semantic search (Exa, Brave Auto/Neural) or developer focus (Phind).

#### Key Citations

- [Custom Search JSON API overview](https://developers.google.com/custom-search/v1/overview)
- [Brave Search API pricing](https://api.search.brave.com/app/pricing)
- [Exa pricing](https://exa.ai/pricing)
- [Phind API on RapidAPI](https://rapidapi.com/umgbhallaphind-search)
- [You.com API documentation](https://documentation.you.com/welcome)
- [Bing API pricing update](https://www.ghacks.net/2023/02/17/microsoft-increases-bing-search-api-pricing-by-up-to-1000/)
- [DuckDuckGo Search API by SerpApi](https://serpapi.com/duckduckgo-search-api)
- [Yandex Search API](https://serpapi.com/yandex-search-api)

================
File: plugins/repos/twat_llm/src/twat_llm/__version__.py
================


================
File: plugins/repos/twat_llm/src/twat_llm/twat_llm.py
================
logging.basicConfig(
logger = logging.getLogger(__name__)
class Config:
def process_data(
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        raise ValueError(msg)
def main() -> None:
        config = Config(name="default", value="test", options={"key": "value"})
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        logger.exception("An error occurred: %s", str(e))
    main()

================
File: plugins/repos/twat_llm/src/funchain.py
================
def tolower(s):
    return s.lower()
output = ask_chain(

================
File: plugins/repos/twat_llm/src/llm_plugins.py
================
class PackageResult:
def check_package(package: str) -> PackageResult:
    start_time = time.time()
        importlib.import_module(package)
        duration = time.time() - start_time
        return PackageResult(package, "SUCCESS", duration)
        return PackageResult(package, "FAILED", duration, str(e))
def create_results_table(results: Sequence[PackageResult]) -> Table:
    table = Table(show_header=True, header_style="bold magenta")
    table.add_column("Package")
    table.add_column("Status", justify="center")
    table.add_column("Time (s)", justify="right")
    table.add_column("Details")
    for result in sorted(results, key=lambda x: x.duration, reverse=True):
        table.add_row(
def check_llm_plugins(packages: Sequence[str] | None = None) -> None:
    results = [check_package(pkg) for pkg in packages]
    table = create_results_table(results)
    console = Console()
    console.print(table)
    Fire(check_llm_plugins)

================
File: plugins/repos/twat_llm/src/mallmo.py
================
class pathos_with:
    def __init__(self, pool_class=ProcessPool, nodes=None):
        self.nodes = nodes if nodes is not None else mp.cpu_count()
    def __enter__(self):
        self.pool = self.pool_class(nodes=self.nodes)
    def __exit__(self, exc_type, exc_value, traceback):
        self.pool.close()
        self.pool.join()
        self.pool.clear()
class LLMError(Exception):
def _resize_image(image: Image.Image, max_size: tuple = (512, 512)) -> bytes:
    image.thumbnail(max_size, Image.Resampling.LANCZOS)
    img_byte_arr = io.BytesIO()
    image.save(img_byte_arr, format="JPEG")
    return img_byte_arr.getvalue()
def _extract_middle_frame(video_path: str | Path) -> Image.Image:
    cap = cv2.VideoCapture(str(video_path))
    if not cap.isOpened():
        raise LLMError(msg)
    total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    cap.set(cv2.CAP_PROP_POS_FRAMES, middle_frame_index)
    ret, frame = cap.read()
    cap.release()
    return Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
def _prepare_media(path: str | Path) -> bytes:
    path = Path(path)
    file_ext = path.suffix.lower()
        _extract_middle_frame(path)
        else Image.open(path)
    return _resize_image(image)
@retry(
    retry=retry_if_exception_type(Exception),
    stop=stop_after_attempt(2),
    wait=wait_exponential(multiplier=1, min=4, max=10),
def _try_model(
    model = llm.get_model(model_id)
    response = model.prompt(prompt, attachments=attachments)
    return str(response)
def _process_step(step, current_data: str) -> str:
    if isinstance(step, str | callable):
    if not isinstance(step, tuple) or len(step) > 2:
        raise TypeError(msg)
    kwargs = step[1] if len(step) > 1 else {}
    if not isinstance(kwargs, dict):
    if callable(processor):
        result = processor(current_data, **kwargs)
    elif isinstance(processor, str):
        result = ask(prompt=processor, data=current_data, **kwargs)
            f"Step processor must be either a function or string, got {type(processor)}"
    return str(result)
def ask_chain(data: str, steps: Iterable) -> str:
    current_data = str(data)
        current_data = _process_step(step, current_data)
def ask(
            prompt.replace("$input", data)
                image_bytes = _prepare_media(path)
                attachments.append(llm.Attachment(content=image_bytes))
            return _try_model(prompt, model_id, attachments)
def _process_single_prompt(args: tuple) -> str:
    return ask(prompt, model_ids)
def ask_batch(
        with pathos_with(nodes=num_processes) as pool:
            results = pool.map(_process_single_prompt, args)
            return list(results)
def cli(
            responses = ask_batch(
            for _i, _response in enumerate(responses):
            ask(
                media_paths=[Path(path) for path in media] if media else None,
        sys.exit(1)
    Fire(cli)

================
File: plugins/repos/twat_llm/tests/test_twat_llm.py
================
def test_version():

================
File: plugins/repos/twat_llm/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_llm/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_llm/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_llm/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_llm/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v0.0.1] - 2025-02-15

### Added

- Initial release of the project
- Created `mallmo.py` with LLM interaction functionality:
  - Core `ask()` function for LLM prompting with media support
  - `ask_batch()` for parallel processing of multiple prompts
  - `ask_chain()` for chaining multiple prompts or functions
  - Support for multiple fallback models
  - Media file processing (images and video frames)
  - CLI interface
- Added `llm_plugins.py` for checking installed LLM plugins
- Created `funchain.py` as a simple example of chain functionality
- Basic project structure with Python package setup

### Changed

- Moved `twat_llm.py` to `src/twat_llm/` directory
- Enhanced code quality with type hints and modern Python features
- Improved error handling and logging

### Fixed

- Added missing newline at end of files
- Updated `.gitignore` to exclude `_private` directory

## [Unreleased]

### To Do

- Implement core data processing logic in `twat_llm.py`
- Add comprehensive test coverage
- Enhance documentation with usage examples
- Consider adding more LLM providers and models

[unreleased]: https://github.com/twardoch/twat-llm/compare/v0.0.1...HEAD
[v0.0.1]: https://github.com/twardoch/twat-llm/releases/tag/v0.0.1

================
File: plugins/repos/twat_llm/package.toml
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_llm/pyproject.toml
================
# this_file: twat_llm/pyproject.toml

# this_file: twat_llm/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_llm"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-llm"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-llm#readme"
Issues = "https://github.com/twardoch/twat-llm/issues"
Source = "https://github.com/twardoch/twat-llm"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_llm/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_llm --cov=tests {args:tests}"
type-check = "mypy src/twat_llm tests"
lint = ["ruff check src/twat_llm tests", "ruff format src/twat_llm tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_llm tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
fix = ["ruff check --fix --unsafe-fixes {args:.}", "ruff format {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_llm", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_llm/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_llm = ["src/twat_llm", "*/twat-llm/src/twat_llm"]
tests = ["tests", "*/twat-llm/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "twat>=1.8.1",           # Main twat package
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_llm --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_llm/README.md
================
# 



## Features

- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-llm
```

## Usage

```python
import twat_llm
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_llm/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_mp/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_mp/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs:
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_mp/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 896]  .
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 515]  cleanup.mdc
│       ├── [1.9K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [ 128]  .github
│   └── [ 128]  workflows
│       ├── [2.9K]  push.yml
│       └── [1.4K]  release.yml
├── [3.0K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [  96]  .specstory
│   └── [ 320]  history
│       ├── [2.7K]  .what-is-this.md
│       ├── [ 58K]  2025-03-04_03-11-implementing-todo-md-plan.md
│       ├── [ 69K]  2025-03-04_03-23-codebase-analysis-and-todo-list-creation.md
│       ├── [ 15K]  2025-03-04_06-17-fixing-hatch-configuration-error.md
│       ├── [103K]  2025-03-04_07-16-implementing-todo-md-phases-1-and-2.md
│       ├── [ 644]  2025-03-04_07-54-untitled.md
│       ├── [1.2K]  2025-03-04_07-59-project-maintenance-and-documentation-update.md
│       └── [ 93K]  2025-03-04_08-26-ruff-linting-errors-and-fixes.md
├── [ 288]  .venv
├── [6.0K]  API_REFERENCE.md
├── [3.8K]  CHANGELOG.md
├── [ 986]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [ 13K]  README.md
├── [ 77K]  REPO_CONTENT.txt
├── [2.9K]  TODO.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [  96]  docs
│   └── [6.4K]  architecture.md
├── [5.3K]  pyproject.toml
├── [ 128]  src
│   └── [ 256]  twat_mp
│       ├── [ 572]  __init__.py
│       ├── [ 17K]  async_mp.py
│       └── [ 25K]  mp.py
└── [ 224]  tests
    ├── [2.8K]  test_async_mp.py
    ├── [6.3K]  test_benchmark.py
    └── [ 13K]  test_twat_mp.py

13 directories, 33 files

================
File: plugins/repos/twat_mp/.cursor/rules/quality.mdc
================
---
description: Quality
globs:
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_mp/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

# Ensure that only one run per branch/commit is active at once.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  # === QUALITY JOB: Lint and format checks ===
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  # === TEST JOB: Run tests ===
  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_mp --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  # === BUILD JOB: Create distribution artifacts ===
  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_mp/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-mp
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_mp/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_mp/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_mp/dist/.gitignore
================
*

================
File: plugins/repos/twat_mp/docs/architecture.md
================
# twat-mp Architecture

This document explains the architecture of the `twat-mp` package using simple diagrams.

## Component Overview

```
+---------------------------+
|       twat-mp Package     |
+---------------------------+
           |
           |
           v
+---------------------------+
|      Core Components      |
+---------------------------+
           |
           +----------------+----------------+
           |                |                |
           v                v                v
+------------------+ +---------------+ +---------------+
|  Process-based   | |  Thread-based | |  Async-based  |
|  Parallelism     | |  Parallelism  | |  Parallelism  |
+------------------+ +---------------+ +---------------+
|                  | |               | |               |
| - MultiPool      | | - ThreadPool  | | - AsyncMultiPool |
| - ProcessPool    | |               | |               |
| - pmap, imap,    | |               | | - apmap       |
|   amap decorators| |               | |               |
+------------------+ +---------------+ +---------------+
           |                |                |
           |                |                |
           v                v                v
+---------------------------+---------------------------+
|                 Underlying Libraries                  |
+---------------------------+---------------------------+
|                                                       |
|  - pathos (ProcessPool, ThreadPool)                   |
|  - aiomultiprocess (AsyncMultiPool)                   |
|                                                       |
+-------------------------------------------------------+
```

## Execution Flow

### Process/Thread Pool Execution Flow

```
+-------------+     +-------------+     +----------------+
| User Code   |     | Pool        |     | Worker         |
| with        |---->| Creation    |---->| Processes      |
| ProcessPool |     | (Context)   |     | or Threads     |
+-------------+     +-------------+     +----------------+
      |                   |                    |
      |                   |                    |
      v                   v                    v
+-------------+     +-------------+     +----------------+
| Function    |     | Task        |     | Parallel       |
| to Execute  |---->| Distribution|---->| Execution      |
+-------------+     +-------------+     +----------------+
                          |                    |
                          |                    |
                          v                    v
                    +-------------+     +----------------+
                    | Result      |<----| Results        |
                    | Collection  |     | from Workers   |
                    +-------------+     +----------------+
                          |
                          |
                          v
                    +-------------+
                    | Return      |
                    | to User     |
                    +-------------+
```

### Async Pool Execution Flow

```
+-------------+     +-------------+     +----------------+
| Async       |     | AsyncMulti  |     | Worker         |
| User Code   |---->| Pool        |---->| Processes      |
+-------------+     | Creation    |     |                |
      |             +-------------+     +----------------+
      |                   |                    |
      v                   v                    v
+-------------+     +-------------+     +----------------+
| Async       |     | Task        |     | Parallel       |
| Function    |---->| Distribution|---->| Execution of   |
+-------------+     +-------------+     | Async Functions|
                          |             +----------------+
                          |                    |
                          v                    v
                    +-------------+     +----------------+
                    | Await       |<----| Async Results  |
                    | Results     |     | from Workers   |
                    +-------------+     +----------------+
                          |
                          |
                          v
                    +-------------+
                    | Return      |
                    | to User     |
                    +-------------+
```

## Decorator Pattern

```
+-------------+     +-------------+     +----------------+
| Function    |     | Decorator   |     | Wrapped        |
| Definition  |---->| Application |---->| Function       |
| @pmap       |     | (mmap)      |     |                |
+-------------+     +-------------+     +----------------+
                          |                    |
                          |                    |
                          v                    v
                    +-------------+     +----------------+
                    | Function    |     | MultiPool      |
                    | Call with   |---->| Creation &     |
                    | Iterable    |     | Management     |
                    +-------------+     +----------------+
                                               |
                                               |
                                               v
                                        +----------------+
                                        | Parallel       |
                                        | Execution &    |
                                        | Result Return  |
                                        +----------------+
```

## Class Hierarchy

```
                  +-------------+
                  |  MultiPool  |
                  +-------------+
                        ^
                        |
          +-------------+-------------+
          |                           |
+-----------------+         +-----------------+
|  ProcessPool    |         |   ThreadPool    |
+-----------------+         +-----------------+


+------------------+
| AsyncMultiPool   |
+------------------+
```

## Decorator Relationships

```
                  +-------------+
                  |    mmap     |
                  | (factory)   |
                  +-------------+
                        |
                        v
          +-------------+-------------+-------------+
          |             |             |             |
+-----------------+  +------+  +------+  +---------+
|      pmap       |  | imap |  | amap |  |  apmap  |
| (eager eval)    |  | (lazy)|  |(async)|  |(async) |
+-----------------+  +------+  +------+  +---------+
```

================
File: plugins/repos/twat_mp/src/twat_mp/__init__.py
================


================
File: plugins/repos/twat_mp/src/twat_mp/__version__.py
================


================
File: plugins/repos/twat_mp/src/twat_mp/async_mp.py
================
logger = logging.getLogger(__name__)
    class AioPool:  # type: ignore
T = TypeVar("T")
U = TypeVar("U")
V = TypeVar("V")
def _check_aiomultiprocess() -> None:
        raise ImportError(
class AsyncMultiPool:
    def __init__(
        _check_aiomultiprocess()
    async def __aenter__(self) -> "AsyncMultiPool":
                self.pool = aiomultiprocess.Pool(
                logger.debug(f"Created AsyncMultiPool with {self.processes} processes")
                logger.error(f"Failed to create AsyncMultiPool: {e}")
                raise RuntimeError(f"Failed to create AsyncMultiPool: {e}") from e
    async def __aexit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
            logger.debug("No pool to clean up in __aexit__")
            logger.warning("Cleanup already attempted, skipping")
            logger.warning(
            logger.debug("Attempting graceful pool shutdown with close()")
            self.pool.close()
            await self.pool.join()
            logger.debug("Pool gracefully closed and joined")
            logger.error(f"Error during graceful pool shutdown: {e}")
            logger.debug(f"Traceback: {traceback.format_exc()}")
                logger.debug("Attempting forceful pool termination")
                self.pool.terminate()
                logger.debug("Pool forcefully terminated and joined")
                logger.error(f"Error during forceful pool termination: {e2}")
            logger.debug("Clearing pool reference")
            logger.error(error_msg)
            raise RuntimeError(error_msg) from cleanup_error
    async def map(
            raise RuntimeError("Pool not initialized. Use 'async with' statement.")
            return await self.pool.map(func, iterable)
            logger.error(f"Error during parallel map operation: {e}")
            raise RuntimeError(f"Error during parallel map operation: {e}") from e
    async def starmap(
            return await self.pool.starmap(func, iterable)
            logger.error(f"Error during parallel starmap operation: {e}")
            raise RuntimeError(f"Error during parallel starmap operation: {e}") from e
    async def imap(
            async for result in self.pool.imap(func, iterable):
            logger.error(f"Error during parallel imap operation: {e}")
            raise RuntimeError(f"Error during parallel imap operation: {e}") from e
def apmap(
    @wraps(func)
    async def wrapper(iterable: Iterable[T]) -> list[U]:
            async with AsyncMultiPool() as pool:
                return await pool.map(func, iterable)
            logger.error(f"Error in apmap decorator: {e}")
            raise RuntimeError(f"Error in parallel processing: {e}") from e

================
File: plugins/repos/twat_mp/src/twat_mp/mp.py
================
logger = logging.getLogger(__name__)
T = TypeVar("T")
U = TypeVar("U")
DEBUG_MODE = os.environ.get("TWAT_MP_DEBUG", "0").lower() in ("1", "true", "yes", "on")
class WorkerError(Exception):
    def __init__(
                f": {type(original_exception).__name__}: {original_exception}"
        super().__init__(detailed_message)
def set_debug_mode(enabled: bool = True) -> None:
        logging.basicConfig(
            handlers=[logging.StreamHandler(sys.stdout)],
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled for twat_mp")
        logger.setLevel(logging.INFO)
        logger.info("Debug mode disabled for twat_mp")
def _worker_wrapper(func: Callable[[T], U], item: T, worker_id: int | None = None) -> U:
        return func(item)
        tb_str = traceback.format_exc()
        logger.error(
        logger.debug(f"Traceback: {tb_str}")
        raise WorkerError(
class MultiPool:
        self.nodes: int = nodes if nodes is not None else mp.cpu_count()  # type: ignore
            logger.debug(
    def __enter__(self) -> PathosPool:
            logger.debug(f"Creating pool with {self.nodes} nodes")
            self.pool = self.pool_class(nodes=self.nodes)
                logger.debug(f"Pool created successfully: {self.pool}")
                logger.error(msg)
                raise RuntimeError(msg)
            def enhanced_map(func: Callable[[T], U], iterable: Iterator[T]) -> Any:
                def wrapped_func(item, idx=None):
                    return _worker_wrapper(func, item, idx)
                    return original_map(wrapped_func, iterable)
                    if isinstance(e, WorkerError):
                    logger.error(f"Error during parallel map operation: {e}")
                        logger.debug(f"Traceback: {traceback.format_exc()}")
                    raise RuntimeError(
            logger.error(error_msg)
            raise RuntimeError(error_msg) from e
    def __exit__(self, exc_type: Any, exc_value: Any, traceback: Any) -> Literal[False]:
                    self.pool.terminate()
                    self.pool.join()
                        logger.debug("Closing pool gracefully")
                    self.pool.close()
                        logger.debug("Pool closed and joined successfully")
                        logger.debug("Pool terminated and joined after cleanup error")
                        logger.debug(f"Failed to terminate pool: {e2}")
                        logger.debug("Clearing pool resources")
                    self.pool.clear()
                        logger.debug("Pool resources cleared")
                        logger.debug(f"Error clearing pool resources: {e}")
                    logger.debug("Pool reference set to None")
            logger.debug("No pool to clean up")
class ProcessPool(MultiPool):
    def __init__(self, nodes: int | None = None, debug: bool | None = None) -> None:
        super().__init__(pool_class=PathosProcessPool, nodes=nodes, debug=debug)
class ThreadPool(MultiPool):
        super().__init__(pool_class=PathosThreadPool, nodes=nodes, debug=debug)
def mmap(
        raise ValueError(
    def decorator(func: Callable[[T], U]) -> Callable[[Iterator[T]], Iterator[U]]:
        @wraps(func)
        def wrapper(iterable: Iterator[T], *args: Any, **kwargs: Any) -> Any:
                    f"Executing {func.__name__} with {how} on {type(iterable).__name__}"
                with MultiPool(debug=use_debug) as pool:
                        mapping_method = getattr(pool, how)
                            logger.debug(f"Using pool method: {how}")
                        raise ValueError(error_msg) from err
                            logger.debug(f"Starting parallel execution with {how}")
                        result = mapping_method(func, iterable)
                                logger.debug("Getting results from async operation")
                            result = result.get()
                            logger.debug("Parallel execution completed successfully")
                                isinstance(e, WorkerError)
                            logger.debug("KeyboardInterrupt detected during execution")
def pmap(func: Callable[[T], U]) -> Callable[[Iterator[T]], Iterator[U]]:
    return mmap("map")(func)
def imap(func: Callable[[T], U]) -> Callable[[Iterator[T]], Iterator[U]]:
    return mmap("imap")(func)
def amap(func: Callable[[T], U]) -> Callable[[Iterator[T]], Any]:
    return mmap("amap", get_result=True)(func)

================
File: plugins/repos/twat_mp/tests/test_async_mp.py
================
async def async_double(x: int) -> int:
    await asyncio.sleep(0.01)  # Simulate some async work
async def async_raise_error(x: Any) -> Any:
    raise ValueError(f"Error processing {x}")
async def test_async_multi_pool_map() -> None:
    async with AsyncMultiPool() as pool:
        results = await pool.map(async_double, range(5))
async def test_async_multi_pool_empty() -> None:
        results = await pool.map(async_double, [])
async def test_async_multi_pool_error() -> None:
    with pytest.raises(ValueError):
            await pool.map(async_raise_error, range(5))
async def test_async_multi_pool_imap() -> None:
        async for result in pool.imap(async_double, range(5)):
            results.append(result)
async def test_async_multi_pool_starmap() -> None:
    async def async_sum(*args: int) -> int:
        await asyncio.sleep(0.01)
        return sum(args)
        results = await pool.starmap(async_sum, [(1, 2), (3, 4), (5, 6)])
async def test_apmap_decorator() -> None:
    async def double(x: int) -> int:
    results = await double(range(5))
async def test_pool_not_initialized() -> None:
    pool = AsyncMultiPool()
    with pytest.raises(RuntimeError, match="Pool not initialized"):
        await pool.map(async_double, range(5))
async def test_pool_cleanup() -> None:

================
File: plugins/repos/twat_mp/tests/test_benchmark.py
================
def _compute_intensive(x: int) -> int:
    for _ in range(1000):  # Simulate CPU-intensive work
def _io_intensive(x: int) -> int:
    time.sleep(0.001)  # Simulate I/O wait
def generate_data(size: int) -> list[int]:
    return list(range(size))
def small_data() -> list[int]:
    return generate_data(100)
def medium_data() -> list[int]:
    return generate_data(1000)
def large_data() -> list[int]:
    return generate_data(10000)
def run_parallel_operation(
    parallel_func = parallel_impl(func)
    return list(parallel_func(data))
class TestComputeIntensiveBenchmarks:
    def test_sequential_vs_process_pool(self, benchmark, medium_data):
        def sequential() -> list[int]:
            return list(map(_compute_intensive, medium_data))
        def parallel() -> list[int]:
            with ProcessPool() as pool:
                return list(pool.map(_compute_intensive, medium_data))
        result = sequential()  # Run once to get result
        assert parallel() == result  # Verify results match
        def run_both() -> tuple[list[int], list[int]]:
            return sequential(), parallel()
        benchmark(run_both)
    @pytest.mark.parametrize("data_size", [100, 1000, 10000])
    def test_parallel_implementations(self, benchmark, data_size):
        data = generate_data(data_size)
        def process_map(
            return mmap(how="map")(f)
        def thread_map(
            def wrapper(iterable: Any) -> Iterator[Any]:
                with ThreadPool() as pool:
                    return pool.map(f, iterable)
            "amap": lambda f: amap(f),
            "imap": lambda f: imap(f),
            "pmap": lambda f: pmap(f),
        reference_result = run_parallel_operation(
        for name, impl in implementations.items():
            result = run_parallel_operation(_compute_intensive, data, impl)
        def run_all() -> dict[str, list[int]]:
                name: run_parallel_operation(_compute_intensive, data, impl)
                for name, impl in implementations.items()
        benchmark(run_all)
class TestIOIntensiveBenchmarks:
    def test_thread_vs_process_pool(self, benchmark, medium_data):
        def process_pool() -> list[int]:
                return list(pool.map(_io_intensive, medium_data))
        def thread_pool() -> list[int]:
        result = process_pool()  # Run once to get result
        assert thread_pool() == result  # Verify results match
            return process_pool(), thread_pool()
class TestScalabilityBenchmarks:
    @pytest.mark.parametrize("nodes", [2, 4, 8, 16])
    def test_worker_scaling(self, benchmark, medium_data, nodes):
        def run_with_workers() -> list[int]:
            with ProcessPool(nodes=nodes) as pool:
        benchmark(run_with_workers)
class TestCompositionBenchmarks:
    def test_chained_operations(self, benchmark, medium_data):
        def sequential_chain() -> list[int]:
            return [_io_intensive(_compute_intensive(x)) for x in medium_data]
        def parallel_chain() -> list[int]:
            compute = amap(_compute_intensive)
            io_op = amap(_io_intensive)
            return list(io_op(compute(medium_data)))
        result = sequential_chain()  # Run once to get result
        assert parallel_chain() == result  # Verify results match
            return sequential_chain(), parallel_chain()

================
File: plugins/repos/twat_mp/tests/test_twat_mp.py
================
T = TypeVar("T")
U = TypeVar("U")
def test_version():
def _square(x: int) -> int:
def _subs(x: int) -> int:
isquare = amap(_square)
isubs = amap(_subs)
def test_process_pool_context():
    with ProcessPool() as pool:
        result = list(pool.map(_square, iter(range(5))))
def test_thread_pool_context():
    with ThreadPool() as pool:
def test_amap_decorator():
    result = isquare(iter(range(5)))
def test_pmap_decorator():
    def square(x: int) -> int:
    result = list(square(iter(range(5))))
def test_imap_decorator():
    iterator = square(iter(range(3)))
    assert next(iterator) == 0
    assert next(iterator) == 1
    assert next(iterator) == 4
def test_composed_operations():
    result = list(isubs(isquare(iter(range(5)))))
def test_pool_nodes_specification():
    with ProcessPool(nodes=TEST_PROCESS_POOL_SIZE) as pool:
    with ThreadPool(nodes=TEST_THREAD_POOL_SIZE) as pool:
def test_parallel_vs_sequential_performance():
    def slow_square(x: int) -> int:
        time.sleep(0.01)  # Simulate CPU work
    start_time = time.time()
    sequential_result = [slow_square(x) for x in range(20)]
    sequential_time = time.time() - start_time
        parallel_result = list(pool.map(slow_square, iter(range(20))))
    parallel_time = time.time() - start_time
    print(f"Sequential: {sequential_time:.4f}s, Parallel: {parallel_time:.4f}s")
def test_mmap_decorator_variants():
    map_decorator = mmap("map")
    def square1(x: int) -> int:
    assert list(square1(iter(range(5)))) == [0, 1, 4, 9, 16]
    imap_decorator = mmap("imap")
    def square2(x: int) -> int:
    assert list(square2(iter(range(5)))) == [0, 1, 4, 9, 16]
    amap_decorator = mmap("amap", get_result=True)
    def square3(x: int) -> int:
    assert square3(iter(range(5))) == [0, 1, 4, 9, 16]
def test_empty_iterable():
        result = list(pool.map(_square, iter([])))
    assert list(square(iter([]))) == []
    def square_imap(x: int) -> int:
    assert list(square_imap(iter([]))) == []
    def square_amap(x: int) -> int:
    assert square_amap(iter([])) == []
def test_error_propagation():
    def error_func(x: int) -> int:
            raise ValueError("Test error")
    with pytest.raises(WorkerError) as excinfo:
            list(pool.map(error_func, iter(range(5))))
    assert "ValueError" in str(excinfo.value)
    assert "Test error" in str(excinfo.value)
    assert "3" in str(excinfo.value)  # Check that the input item is mentioned
    def error_map(x: int) -> int:
            raise ValueError("Test error in pmap")
    with pytest.raises(RuntimeError, match="Failed to create or use pool"):
        list(error_map(iter(range(5))))
    def error_imap(x: int) -> int:
            raise ValueError("Test error in imap")
    with pytest.raises(ValueError):
        list(error_imap(iter(range(5))))
    def error_amap(x: int) -> int:
            raise ValueError("Test error in amap")
        error_amap(iter(range(5)))
def test_debug_mode():
    with patch("logging.Logger.debug") as mock_debug:
        set_debug_mode(True)
            list(pool.map(_square, iter(range(3))))
        set_debug_mode(False)
def test_invalid_mapping_method():
    with pytest.raises(ValueError, match="Invalid mapping method"):
        mmap("invalid_method")
    with patch("twat_mp.mp.MultiPool.__enter__") as mock_enter:
        mock_pool = MagicMock()
        delattr(mock_pool, valid_method)  # Remove the valid method from the mock
        decorator = mmap(valid_method)
        def test_func(x: int) -> int:
            test_func(iter(range(5)))
def test_pool_creation_failure():
    with patch(
        side_effect=RuntimeError("Test pool creation error"),
        with pytest.raises(RuntimeError, match="Test pool creation error"):
            with ProcessPool():
def test_resource_cleanup_after_exception():
    class TestError(Exception):
    with patch("twat_mp.mp.MultiPool.__exit__") as mock_exit:
                raise TestError("Test exception")
def test_keyboard_interrupt_handling():
    with patch("pathos.pools.ProcessPool.terminate") as mock_terminate:
        with patch("pathos.pools.ProcessPool.join") as mock_join:
                    raise KeyboardInterrupt()
def test_large_data_handling():
    large_data = list(range(1000))
        result = list(pool.map(_square, iter(large_data)))
        assert len(result) == 1000
def test_nested_pools():
    def nested_pool_func(x: int) -> List[int]:
        with ThreadPool(nodes=2) as inner_pool:
            return list(inner_pool.map(_square, iter(range(x + 1))))
    with ProcessPool(nodes=2) as outer_pool:
        results = list(outer_pool.map(nested_pool_func, iter(range(3))))
def test_pool_reuse_failure():
    with ProcessPool() as p:
        list(p.map(_square, iter(range(5))))
            result = list(pool.map(_square, iter(range(3))))
            print(f"Pool raised exception after context exit: {type(e).__name__}: {e}")
def test_custom_exception_handling():
    class CustomError(Exception):
    def raise_custom_error(x: int) -> int:
            raise CustomError(f"Value {x} is too large")
            list(pool.map(raise_custom_error, iter(range(5))))
    assert "CustomError" in str(excinfo.value)
    assert "Value 3 is too large" in str(excinfo.value)
    def decorated_error_func(x: int) -> int:
            raise CustomError(f"Decorated value {x} is too large")
        list(decorated_error_func(iter(range(5))))

================
File: plugins/repos/twat_mp/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python coverage
.coverage
.coverage.*
htmlcov/
coverage.xml
.pytest_cache/
.benchmarks/

_private

================
File: plugins/repos/twat_mp/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_mp/API_REFERENCE.md
================
# twat-mp API Reference

This document provides a comprehensive reference for the `twat-mp` package's API.

## Table of Contents

- [Core Classes](#core-classes)
  - [MultiPool](#multipool)
  - [ProcessPool](#processpool)
  - [ThreadPool](#threadpool)
  - [AsyncMultiPool](#asyncmultipool)
- [Decorators](#decorators)
  - [pmap](#pmap)
  - [imap](#imap)
  - [amap](#amap)
  - [apmap](#apmap)
- [Usage Patterns](#usage-patterns)
  - [Choosing the Right Pool](#choosing-the-right-pool)
  - [Error Handling](#error-handling)
  - [Resource Management](#resource-management)

## Core Classes

### MultiPool

```python
class MultiPool:
    def __init__(self, pool_class=PathosProcessPool, nodes=None):
        ...
```

Base class for managing Pathos parallel processing pools. This class abstracts the creation and cleanup of a parallel processing pool, automatically choosing the number of nodes (processes or threads) based on the CPU count if not provided.

**Parameters:**
- `pool_class`: The Pathos pool class to instantiate (default: `PathosProcessPool`)
- `nodes`: The number of processes/threads to use (default: CPU count)

**Methods:**
- `__enter__()`: Enter the runtime context and create the pool
- `__exit__(exc_type, exc_value, traceback)`: Exit the runtime context, ensuring the pool is properly closed and resources are freed

**Example:**
```python
with MultiPool(pool_class=PathosProcessPool) as pool:
    results = pool.map(lambda x: x * 2, range(5))
print(list(results))
```

### ProcessPool

```python
class ProcessPool(MultiPool):
    def __init__(self, nodes=None):
        ...
```

Context manager specifically for creating a process-based pool. This subclass of MultiPool defaults to using the ProcessPool from Pathos.

**Parameters:**
- `nodes`: Number of processes to use (default: CPU count)

**Example:**
```python
with ProcessPool() as pool:
    results = pool.map(lambda x: x * 2, range(10))
```

### ThreadPool

```python
class ThreadPool(MultiPool):
    def __init__(self, nodes=None):
        ...
```

Context manager specifically for creating a thread-based pool. This subclass of MultiPool defaults to using the ThreadPool from Pathos.

**Parameters:**
- `nodes`: Number of threads to use (default: CPU count)

**Example:**
```python
with ThreadPool() as pool:
    results = pool.map(lambda x: x * 2, range(10))
```

### AsyncMultiPool

```python
class AsyncMultiPool:
    def __init__(self, processes=None, initializer=None, initargs=None, **kwargs):
        ...
```

A context manager for managing an aiomultiprocess.Pool. Provides high-level interface for parallel processing with async/await support.

**Parameters:**
- `processes`: Number of processes to use (default: CPU count)
- `initializer`: Optional callable to initialize worker processes
- `initargs`: Arguments to pass to the initializer
- `**kwargs`: Additional keyword arguments passed to aiomultiprocess.Pool

**Methods:**
- `__aenter__()`: Enter the async context, creating and starting the pool
- `__aexit__(exc_type, exc_val, exc_tb)`: Exit the async context, closing and joining the pool
- `map(func, iterable)`: Apply the function to each item in the iterable in parallel
- `starmap(func, iterable)`: Like map() but unpacks arguments from the iterable
- `imap(func, iterable)`: Async iterator version of map()

**Example:**
```python
async def process_items():
    async with AsyncMultiPool() as pool:
        async def work(x):
            await asyncio.sleep(0.1)  # Some async work
            return x * 2

        results = await pool.map(work, range(10))
        return results
```

## Decorators

### pmap

```python
@pmap
def func(x):
    ...
```

Standard parallel map decorator (eager evaluation). Wraps a function so that when it is called with an iterable, the function is applied in parallel using a ProcessPool.

**Example:**
```python
@pmap
def square(x):
    return x * x

results = list(square(range(10)))
```

### imap

```python
@imap
def func(x):
    ...
```

Lazy parallel map decorator that returns an iterator. Results are yielded as they become available.

**Example:**
```python
@imap
def cube(x):
    return x * x * x

for result in cube(range(5)):
    print(result)  # Prints results as they become available
```

### amap

```python
@amap
def func(x):
    ...
```

Asynchronous parallel map decorator with automatic result retrieval. Uses the 'amap' method of Pathos pools.

**Example:**
```python
@amap
def double(x):
    return x * 2

results = list(double(range(10)))
```

### apmap

```python
@apmap
async def func(x):
    ...
```

Decorator for async functions to run in parallel using AsyncMultiPool. Requires the 'aio' extra to be installed.

**Example:**
```python
@apmap
async def double(x):
    await asyncio.sleep(0.1)  # Some async work
    return x * 2

async def main():
    results = await double(range(10))
    print(results)

asyncio.run(main())
```

## Usage Patterns

### Choosing the Right Pool

- **ProcessPool**: Best for CPU-intensive tasks that benefit from parallel execution across multiple cores
- **ThreadPool**: Best for I/O-bound tasks where threads can efficiently wait for I/O operations
- **AsyncMultiPool**: Best for combining async/await with multiprocessing, particularly useful for mixed workloads

### Error Handling

All pools provide proper error propagation. Exceptions raised in worker processes/threads are propagated to the main process:

```python
try:
    with ProcessPool() as pool:
        results = list(pool.map(potentially_failing_function, data))
except Exception as e:
    print(f"An error occurred: {e}")
```

### Resource Management

The context manager pattern ensures proper cleanup of resources:

```python
# Resources are automatically cleaned up when exiting the context
with ProcessPool() as pool:
    # Use the pool
    results = pool.map(func, data)

# Pool is now closed and resources are freed
```

For async pools:

```python
async def main():
    async with AsyncMultiPool() as pool:
        # Use the pool
        results = await pool.map(async_func, data)

    # Pool is now closed and resources are freed

asyncio.run(main())
```

================
File: plugins/repos/twat_mp/CHANGELOG.md
================
# Changelog

All notable changes to the `twat-mp` project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- Created comprehensive API reference documentation
- Added architecture diagrams explaining component relationships and workflows
- Added real-world examples for image processing, data processing, web scraping, and file operations
- Implemented better error handling with descriptive error messages and suggestions
- Added interactive examples in Jupyter notebooks
- Implemented debug mode with detailed logging

### Changed

- Improved resource cleanup in AsyncMultiPool.__aexit__ using close() instead of terminate()
- Enhanced error handling during pool cleanup to ensure proper resource management
- Updated docstrings with more examples and clearer explanations
- Improved compatibility with Python 3.12+ async features
- Enhanced exception propagation from worker processes
- Fixed build system configuration by adding proper `tool.hatch.version` settings
- Renamed `WorkerException` to `WorkerError` for consistency with Python naming conventions
- Updated tests to reflect the new error handling behavior
- Fixed `test_pool_reuse_failure` test to document current behavior
- Improved code quality by removing unused variables and lambda assignments

### Fixed

- Fixed AttributeError handling in exception handling code
- Improved graceful shutdown mechanism with better signal handling
- Fixed keyboard interrupt handling during parallel execution
- Addressed linter warnings and improved code quality
- Fixed package build error by adding missing version configuration in pyproject.toml
- Fixed inconsistent exception naming across the codebase
- Fixed test assertions to properly check for error messages in `WorkerError`

## [2.5.3] - 2025-03-04

### Added

- Added proper version configuration in pyproject.toml using hatch-vcs
- Improved build system configuration for better package distribution
- Created LOG.md file to track development progress (now merged into CHANGELOG.md)
- Updated TODO.md with prioritized tasks

### Fixed

- Fixed package build error related to missing version configuration
- Ensured proper version extraction from Git tags

### Changed

- Updated CHANGELOG.md with recent changes
- Updated README.md with information about recent updates

## [2.0.0] - 2025-02-20

### Added

- Added async support via `aiomultiprocess` integration
- New `AsyncMultiPool` class for combining async/await with multiprocessing
- New `apmap` decorator for easy async parallel processing
- Comprehensive test suite for async functionality
- Documentation for async features

### Changed

- Updated project dependencies to include optional `aiomultiprocess` support
- Enhanced type hints and error handling across the codebase
- Changed version handling to use static version instead of VCS-based versioning

## [1.7.5] - 2025-02-15

### Added

- Implemented AsyncMultiPool class for combining async/await with multiprocessing
- Added apmap decorator for easy async parallel processing
- Created comprehensive test suite for async functionality

### Changed

- Optimized CI/CD pipeline with improved GitHub Actions workflow
- Enhanced stability and performance optimizations in core multiprocessing functions
- Updated documentation and README with clearer usage examples
- Updated project dependencies to include optional aiomultiprocess support
- Enhanced type hints and error handling across the codebase

[Unreleased]: https://github.com/twardoch/twat-mp/compare/v2.5.3...HEAD
[2.5.3]: https://github.com/twardoch/twat-mp/compare/v2.0.0...v2.5.3
[2.0.0]: https://github.com/twardoch/twat-mp/compare/v1.7.5...v2.0.0
[1.7.5]: https://github.com/twardoch/twat-mp/compare/v1.7.3...v1.7.5
// ... existing references ...

================
File: plugins/repos/twat_mp/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_mp/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_mp/pyproject.toml
================
# this_file: twat_mp/pyproject.toml
# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0", # Core build backend for Hatch
    "hatch-vcs>=0.4.0", # Version Control System plugin for Hatch

]
build-backend = "hatchling.build" # Use Hatchling as the build backend

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-mp"
dynamic = ["version"]
description = "Parallel processing utilities using Pathos mpprocessing library"
readme = "README.md"
requires-python = ">=3.10" # Minimum Python version required
license = "MIT"
keywords = ["parallel", "mpprocessing", "pathos", "map", "pool"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]
# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "pathos>=0.3.3", # Parallel processing library
    "twat>=1.8.1",
    "aiomultiprocess>=0.9.1"
    # Main twat package
]

[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

[project.urls]
Documentation = "https://github.com/twardoch/twat-mp#readme"
Issues = "https://github.com/twardoch/twat-mp/issues"
Source = "https://github.com/twardoch/twat-mp"

[project.entry-points."twat.plugins"]
mp = "twat_mp"

[tool.hatch.build.targets.wheel]
packages = ["src/twat_mp"]

[tool.hatch.envs.default]
dependencies = ["mypy>=1.0.0", "ruff>=0.1.0"]

[project.optional-dependencies]
test = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "pytest-benchmark[histogram]>=4.0.0",
    "pytest-xdist>=3.5.0", # For parallel test execution
    "pandas>=2.0.0", # Required by some test dependencies
    "numpy>=1.24.0", # Required by pandas
    "matplotlib>=3.7.0", # For benchmark visualization

]
dev = ["pre-commit>=3.6.0"]
all = [
    "twat>=1.0.0",
    "pathos>=0.3.3",
    "aiomultiprocess>=0.9.1"
    # Include aio in all
]

[tool.hatch.envs.test]
dependencies = [".[test]"]

[tool.hatch.envs.test.scripts]
# Regular tests can run in parallel
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_mp --cov=tests {args:tests}"
# Benchmarks must run sequentially
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"
bench-hist = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-histogram=benchmark/hist"
bench-compare = "python -m pytest-benchmark compare benchmark/results.json --sort fullname --group-by func"

[tool.hatch.envs.lint]
detached = true
dependencies = ["black>=23.1.0", "mypy>=1.0.0", "ruff>=0.1.0"]

[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_mp tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

[tool.ruff]
target-version = "py310"
line-length = 88
lint.extend-select = [
    "I", # isort
    "N", # pep8-naming
    "B", # flake8-bugbear
    "RUF", # Ruff-specific rules

]
lint.ignore = [
    "ARG001", # Unused function argument
    "E501", # Line too long
    "I001",
]

[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "lf"

[tool.ruff.lint.per-file-ignores]
"tests/*" = ["S101"]

[tool.coverage.run]
source_pkgs = ["twat_mp", "tests"]
branch = true
parallel = true
omit = ["src/twat_mp/__about__.py"]

[tool.coverage.paths]
twat_mp = ["src/twat_mp", "*/twat-mp/src/twat_mp"]
tests = ["tests", "*/twat-mp/tests"]

[tool.coverage.report]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"

[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min", # Minimum time
    "max", # Maximum time
    "mean", # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr", # Inter-quartile range
    "ops", # Operations per second
    "rounds", # Number of rounds

]

[tool.hatch.version]
source = "vcs"

================
File: plugins/repos/twat_mp/README.md
================
# twat-mp

Parallel processing utilities using Pathos and aiomultiprocess libraries. This package provides convenient context managers and decorators for parallel processing, with process-based, thread-based, and async-based pools.

## Features

* Multiple parallel processing options:
  + `ProcessPool`: For CPU-intensive parallel processing using Pathos
  + `ThreadPool`: For I/O-bound parallel processing using Pathos
  + `AsyncMultiPool`: For combined async/await with multiprocessing using aiomultiprocess
* Decorators for common parallel mapping operations:
  + `pmap`: Standard parallel map (eager evaluation)
  + `imap`: Lazy parallel map returning an iterator
  + `amap`: Asynchronous map with automatic result retrieval
  + `apmap`: Async parallel map for use with async/await functions
* Automatic CPU core detection for optimal pool sizing
* Clean resource management with context managers
* Full type hints and modern Python features
* Flexible pool configuration with customizable worker count
* Graceful error handling and resource cleanup
* Enhanced exception propagation with detailed context
* Debug mode with comprehensive logging
* Optional dependencies to reduce installation footprint
* Version control system (VCS) based versioning using hatch-vcs

## Recent Updates

* Added debug mode with detailed logging via `set_debug_mode()`
* Enhanced error handling with `WorkerException` for better context
* Improved exception propagation from worker processes
* Added comprehensive docstrings to all public functions and classes
* Fixed build system configuration with proper version handling
* Enhanced error handling and resource cleanup
* Improved compatibility with Python 3.12+ async features
* Added comprehensive API reference documentation
* Added real-world examples for various use cases

## Installation

Basic installation:
```bash
pip install twat-mp
```

With async support:
```bash
pip install 'twat-mp[aio]'
```

With all extras and development tools:
```bash
pip install 'twat-mp[all,dev]'
```

## Usage

### Basic Usage

```python
from twat_mp import ProcessPool, pmap

# Using the pool directly
with ProcessPool() as pool:
    results = pool.map(lambda x: x * 2, range(10))

# Using the decorator
@pmap
def double(x):
    return x * 2

results = double(range(10))
```

### Async Support

The package provides async support through `aiomultiprocess`, allowing you to combine the benefits of async/await with multiprocessing:

```python
import asyncio
from twat_mp import AsyncMultiPool, apmap

# Using the pool directly
async def process_items():
    async with AsyncMultiPool() as pool:
        async def work(x):
            await asyncio.sleep(0.1)  # Some async work
            return x * 2

        results = await pool.map(work, range(10))
        return results

# Using the decorator
@apmap
async def double(x):
    await asyncio.sleep(0.1)  # Some async work
    return x * 2

async def main():
    results = await double(range(10))
    print(results)  # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

asyncio.run(main())
```

The async support is particularly useful when you need to:
- Perform CPU-intensive tasks in parallel
- Handle many concurrent I/O operations
- Combine async/await with true multiprocessing
- Process results from async APIs in parallel

### Advanced Async Features

The `AsyncMultiPool` provides additional methods for different mapping strategies:

```python
import asyncio
from twat_mp import AsyncMultiPool

async def main():
    # Using starmap for unpacking arguments
    async def sum_values(a, b):
        await asyncio.sleep(0.01)
        return a + b

    async with AsyncMultiPool() as pool:
        # Regular map
        double_results = await pool.map(
            lambda x: x * 2,
            range(5)
        )
        print(double_results)  # [0, 2, 4, 6, 8]

        # Starmap unpacks arguments
        sum_results = await pool.starmap(
            sum_values,
            [(1, 2), (3, 4), (5, 6)]
        )
        print(sum_results)  # [3, 7, 11]

        # imap returns an async iterator
        async for result in pool.imap(sum_values, [(1, 1), (2, 2), (3, 3)]):
            print(result)  # Prints 2, 4, 6 as they complete

asyncio.run(main())
```

### Using Process and Thread Pools

The package provides dedicated context managers for both process and thread pools:

```python
from twat_mp import ProcessPool, ThreadPool

# For CPU-intensive operations
with ProcessPool() as pool:
    results = pool.map(lambda x: x * x, range(10))
    print(list(results))  # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

# For I/O-bound operations
with ThreadPool() as pool:
    results = pool.map(lambda x: x * 2, range(10))
    print(list(results))  # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

# Custom number of workers
with ProcessPool(nodes=4) as pool:
    results = pool.map(lambda x: x * x, range(10))
```

### Using Map Decorators

The package provides three decorators for different mapping strategies:

```python
from twat_mp import amap, imap, pmap

# Standard parallel map (eager evaluation)
@pmap
def square(x: int) -> int:
    return x * x

results = list(square(range(10)))
print(results)  # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

# Lazy parallel map (returns iterator)
@imap
def cube(x: int) -> int:
    return x * x * x

for result in cube(range(5)):
    print(result)  # Prints results as they become available

# Asynchronous parallel map with automatic result retrieval
@amap
def double(x: int) -> int:
    return x * 2

results = list(double(range(10)))
print(results)  # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
```

### Function Composition

Decorators can be composed for complex parallel operations:

```python
from twat_mp import amap

@amap
def compute_intensive(x: int) -> int:
    result = x
    for _ in range(1000):  # Simulate CPU-intensive work
        result = (result * x + x) % 10000
    return result

@amap
def io_intensive(x: int) -> int:
    import time
    time.sleep(0.001)  # Simulate I/O wait
    return x * 2

# Chain parallel operations
results = list(io_intensive(compute_intensive(range(100))))
```

### Debug Mode and Error Handling

The package provides a debug mode for detailed logging and enhanced error handling:

```python
from twat_mp import ProcessPool, set_debug_mode
import time
import random

def process_item(x):
    """Process an item with random delay and potential errors."""
    # Simulate random processing time
    time.sleep(random.random() * 0.5)
    
    # Randomly fail for demonstration
    if random.random() < 0.2:  # 20% chance of failure
        raise ValueError(f"Random failure processing item {x}")
        
    return x * 10

# Enable debug mode for detailed logging
set_debug_mode(True)

try:
    with ProcessPool() as pool:
        results = list(pool.map(process_item, range(10)))
        print(f"Processed results: {results}")
except Exception as e:
    print(f"Caught exception: {e}")
    # The exception will include details about which worker and input item caused the error
finally:
    # Disable debug mode when done
    set_debug_mode(False)
```

The enhanced error handling provides detailed context about failures:

```python
from twat_mp import ProcessPool

def risky_function(x):
    if x == 5:
        raise ValueError("Cannot process item 5")
    return x * 2

try:
    with ProcessPool() as pool:
        results = list(pool.map(risky_function, range(10)))
except ValueError as e:
    # The exception will include the worker ID and input item that caused the error
    print(f"Caught error: {e}")
    # Handle the error appropriately
```

## Real-World Examples

### Image Processing

Processing images in parallel can significantly speed up operations like resizing, filtering, or format conversion:

```python
from twat_mp import ProcessPool
from PIL import Image
import os

def resize_image(file_path):
    """Resize an image to 50% of its original size."""
    try:
        with Image.open(file_path) as img:
            # Get the original size
            width, height = img.size
            # Resize to 50%
            resized = img.resize((width // 2, height // 2))
            # Save with '_resized' suffix
            output_path = os.path.splitext(file_path)[0] + '_resized' + os.path.splitext(file_path)[1]
            resized.save(output_path)
            return output_path
    except Exception as e:
        return f"Error processing {file_path}: {e}"

# Get all image files in a directory
image_files = [f for f in os.listdir('images') if f.lower().endswith(('.png', '.jpg', '.jpeg'))]
image_paths = [os.path.join('images', f) for f in image_files]

# Process images in parallel
with ProcessPool() as pool:
    results = list(pool.map(resize_image, image_paths))

print(f"Processed {len(results)} images")
```

### Web Scraping

Thread pools are ideal for I/O-bound operations like web scraping:

```python
import requests
from bs4 import BeautifulSoup
from twat_mp import ThreadPool

def fetch_page_title(url):
    """Fetch the title of a webpage."""
    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()
        soup = BeautifulSoup(response.text, 'html.parser')
        title = soup.title.string if soup.title else "No title found"
        return {"url": url, "title": title, "status": response.status_code}
    except Exception as e:
        return {"url": url, "error": str(e), "status": None}

# List of URLs to scrape
urls = [
    "https://www.python.org",
    "https://www.github.com",
    "https://www.stackoverflow.com",
    "https://www.wikipedia.org",
    "https://www.reddit.com"
]

# Use ThreadPool for I/O-bound operations
with ThreadPool() as pool:
    results = list(pool.map(fetch_page_title, urls))

# Print results
for result in results:
    if "error" in result:
        print(f"Error fetching {result['url']}: {result['error']}")
    else:
        print(f"{result['url']} - {result['title']} (Status: {result['status']})")
```

### Data Processing with Pandas

Process large datasets in parallel chunks:

```python
import pandas as pd
import numpy as np
from twat_mp import ProcessPool

def process_chunk(chunk_data):
    """Process a chunk of data."""
    # Simulate some data processing
    chunk_data['processed'] = chunk_data['value'] * 2 + np.random.randn(len(chunk_data))
    chunk_data['category'] = pd.cut(chunk_data['processed'], 
                                    bins=[-np.inf, 0, 10, np.inf], 
                                    labels=['low', 'medium', 'high'])
    # Calculate some statistics
    result = {
        'chunk_id': chunk_data['chunk_id'].iloc[0],
        'mean': chunk_data['processed'].mean(),
        'median': chunk_data['processed'].median(),
        'std': chunk_data['processed'].std(),
        'count': len(chunk_data),
        'categories': chunk_data['category'].value_counts().to_dict()
    }
    return result

# Create a large DataFrame
n_rows = 1_000_000
df = pd.DataFrame({
    'value': np.random.randn(n_rows),
    'group': np.random.choice(['A', 'B', 'C', 'D'], n_rows)
})

# Split into chunks for parallel processing
chunk_size = 100_000
chunks = []
for i, chunk_start in enumerate(range(0, n_rows, chunk_size)):
    chunk_end = min(chunk_start + chunk_size, n_rows)
    chunk = df.iloc[chunk_start:chunk_end].copy()
    chunk['chunk_id'] = i
    chunks.append(chunk)

# Process chunks in parallel
with ProcessPool() as pool:
    results = list(pool.map(process_chunk, chunks))

# Combine results
summary = pd.DataFrame(results)
print(summary)
```

### Async File Processing

Combine async I/O with parallel processing:

```python
import asyncio
import aiofiles
import os
from twat_mp import AsyncMultiPool

async def count_words(filename):
    """Count words in a file asynchronously."""
    try:
        async with aiofiles.open(filename, 'r') as f:
            content = await f.read()
            word_count = len(content.split())
            return {"filename": filename, "word_count": word_count}
    except Exception as e:
        return {"filename": filename, "error": str(e)}

async def main():
    # Get all text files in a directory
    files = [os.path.join("documents", f) for f in os.listdir("documents") 
             if f.endswith(".txt")]
    
    # Process files in parallel
    async with AsyncMultiPool() as pool:
        results = await pool.map(count_words, files)
    
    # Calculate total word count
    total_words = sum(r.get("word_count", 0) for r in results)
    
    # Print results
    for result in results:
        if "error" in result:
            print(f"Error processing {result['filename']}: {result['error']}")
        else:
            print(f"{result['filename']}: {result['word_count']} words")
    
    print(f"Total word count: {total_words}")

# Run the async main function
asyncio.run(main())
```

## API Reference

For detailed API documentation, see the [API Reference](API_REFERENCE.md).

## License

MIT

================
File: plugins/repos/twat_mp/TODO.md
================
---
this_file: TODO.md
---

# TODO

Tip: Periodically run `python ./cleanup.py status` to see results of lints and tests. Use `uv pip ...` not `pip ...`


## Phase 1

- [ ] Add a CLI interface for common operations
- [ ] Add support for cancellation of running tasks
- [ ] Implement progress tracking for long-running parallel operations
- [ ] Implement backpressure mechanisms for memory-intensive operations
- [ ] Add support for process affinity and priority settings
- [ ] Implement a unified interface for all pool types
- [ ] Add support for custom serialization/deserialization
- [ ] Add context manager for automatic resource cleanup across all parallel operations
- [ ] Implement adaptive pool sizing based on workload and system resources
- [ ] Add support for task prioritization and scheduling

## Phase 2

- [ ] Increase test coverage to at least 90%
- [ ] Implement stress tests for high concurrency scenarios
- [ ] Add tests for resource leaks and memory usage
- [ ] Add tests for different Python versions and platforms
- [ ] Implement property-based testing for robustness

## Phase 3

- [ ] Optimize serialization/deserialization for better performance
- [ ] Optimize memory usage for large datasets
- [ ] Implement caching mechanisms for repeated operations
- [ ] Profile and optimize critical code paths
- [ ] Add performance comparison with native multiprocessing
- [ ] Implement workload-specific optimization strategies

## Phase 4

- [ ] Implement compatibility with other multiprocessing libraries
- [ ] Add support for distributed processing across multiple machines
- [ ] Add support for GPU acceleration where applicable
- [ ] Implement integration with dask and other distributed computing frameworks
- [ ] Add support for cloud-based parallel processing

## User Experience

- [ ] Implement automatic pool selection based on workload characteristics
- [ ] Add progress bars and status reporting for long-running tasks
- [ ] Create wizard-like interfaces for common operations

## Package Management

- [ ] Create a conda package for easier installation
- [ ] Implement automated release process with GitHub Actions
- [ ] Add support for Windows-specific optimizations
- [ ] Create development documentation for contributors

================
File: plugins/repos/twat_mp/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_os/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_os/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_os/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 736]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 976]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [1.0K]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [ 742]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 192]  dist
│   └── [   1]  .gitkeep
├── [5.6K]  pyproject.toml
├── [  96]  src
│   └── [ 224]  twat_os
│       ├── [  95]  __init__.py
│       ├── [7.4K]  paths.py
│       └── [1.0K]  paths.toml
└── [ 128]  tests
    └── [ 145]  test_package.py

10 directories, 19 files

================
File: plugins/repos/twat_os/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_os/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_os --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_os/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-os
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_os/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_os/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_os/dist/.gitignore
================
*

================
File: plugins/repos/twat_os/dist/.gitkeep
================


================
File: plugins/repos/twat_os/src/twat_os/__init__.py
================
__version__ = metadata.version(__name__)

================
File: plugins/repos/twat_os/src/twat_os/__version__.py
================


================
File: plugins/repos/twat_os/src/twat_os/paths.py
================
PATHS_TOML = Path(__file__).parent / "paths.toml"
DEFAULT_PATHS = tomli.loads(PATHS_TOML.read_text())
class PathConfig(BaseModel):
    @field_validator("base_dir", "package_dir")
    def expand_path(cls, v: str | Path | None) -> Path | None:
        if isinstance(v, str):
            expanded = os.path.expandvars(os.path.expanduser(v))
            return Path(expanded)
    @model_validator(mode="after")
    def validate_and_create_dirs(self) -> Self:
            self.base_dir.mkdir(parents=True, exist_ok=True)
                self.package_dir.mkdir(parents=True, exist_ok=True)
class CacheConfig(PathConfig):
    base_dir: Path = Field(
        default_factory=lambda: Path(platformdirs.user_cache_dir()) / "twat"
class ConfigDirConfig(PathConfig):
        default_factory=lambda: Path(platformdirs.user_config_dir()) / "twat"
class DataDirConfig(PathConfig):
        default_factory=lambda: Path(platformdirs.user_data_dir()) / "twat"
class TempDirConfig(PathConfig):
        default_factory=lambda: Path(platformdirs.user_runtime_dir()) / "twat"
class GenAIConfig(PathConfig):
    lora_dir: Path = Field(
        default_factory=lambda: Path(platformdirs.user_data_dir()) / "twat/genai/loras"
    model_dir: Path = Field(
        default_factory=lambda: Path(platformdirs.user_data_dir()) / "twat/genai/models"
    output_dir: Path = Field(
        default_factory=lambda: Path.home() / "Pictures/twat_genai"
    def validate_and_create_all_dirs(self) -> Self:
            self.lora_dir.mkdir(parents=True, exist_ok=True)
            self.model_dir.mkdir(parents=True, exist_ok=True)
            self.output_dir.mkdir(parents=True, exist_ok=True)
class LogConfig(PathConfig):
        default_factory=lambda: Path(platformdirs.user_state_dir()) / "twat/logs"
class PathManager:
    def __init__(
            config_path = Path(config_file)
            if not config_path.exists():
                raise FileNotFoundError(msg)
            self.config = tomli.loads(config_path.read_text())
        self._init_paths()
    def _init_paths(self) -> None:
        def format_path(path_str: str) -> Path | None:
            expanded = os.path.expandvars(os.path.expanduser(path_str))
            return Path(expanded.format(package_name=self.package_name))
        self.cache = CacheConfig(
            package_dir=format_path(self.config["cache"]["package_dir"]),
        self.config_dir = ConfigDirConfig(
            package_dir=format_path(self.config["config"]["package_dir"]),
        self.data = DataDirConfig(
            package_dir=format_path(self.config["data"]["package_dir"]),
        self.temp = TempDirConfig(
            package_dir=format_path(self.config["temp"]["package_dir"]),
        self.genai = GenAIConfig(
        self.logs = LogConfig(
            package_dir=format_path(self.config["logs"]["package_dir"]),
    def get_path(self, category: str, key: str = "base_dir") -> Path:
        config = getattr(self, category)
        return getattr(config, key)
    def for_package(
        return cls(package_name=package_name, config_file=config_file)
    def __repr__(self) -> str:

================
File: plugins/repos/twat_os/src/twat_os/paths.toml
================
[cache]
# Base directory for all cache operations
base_dir = "~/.cache/twat"
# Directory for storing package-specific cache data
package_dir = "~/.cache/twat/{package_name}"

[config]
# Base directory for configuration files
base_dir = "~/.config/twat"
# Directory for package-specific configuration
package_dir = "~/.config/twat/{package_name}"

[data]
# Base directory for user data
base_dir = "~/.local/share/twat"
# Directory for package-specific data
package_dir = "~/.local/share/twat/{package_name}"

[temp]
# Base directory for temporary files
base_dir = "~/tmp/twat"
# Directory for package-specific temporary files
package_dir = "~/tmp/twat/{package_name}"

[genai]
# Directory for storing LoRA files
lora_dir = "~/.local/share/twat/genai/loras"
# Directory for model weights
model_dir = "~/.local/share/twat/genai/models"
# Directory for generated images
output_dir = "~/Pictures/twat_genai"

[logs]
# Base directory for log files
base_dir = "~/.local/state/twat/logs"
# Directory for package-specific logs
package_dir = "~/.local/state/twat/logs/{package_name}"

================
File: plugins/repos/twat_os/tests/test_package.py
================
def test_version():

================
File: plugins/repos/twat_os/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_os/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_os/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(["tree", "-a", "-I", ".git", "--gitignore", "-n", "-h", "-I", "*_cache"])
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_os/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_os/pyproject.toml
================
# this_file: pyproject.toml 
# this_project: twat_os
[project]
name = "twat-os"
dynamic = ["version"]
description = "Operating system utilities for twat"
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
keywords = [
    "os",
    "system",
    "twat"
]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

dependencies = [
    "fire>=0.6.0",
    "loguru>=0.7.2",
    "twat>=1.8.1"
]

[project.optional-dependencies]
dev = [
    "hatch>=1.14.0",
    "hatchling>=1.27.0",
    "hatch-vcs>=0.4.0",
    "mypy>=1.15.0",
    "pre-commit>=4.1.0",
    "pyupgrade>=3.19.1",
    "ruff>=0.9.6"
]

test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-benchmark>=5.1.0",
    "pytest-mock>=3.14.0",
    "pytest-asyncio>=0.25.3",
    "pytest-timeout>=2.3.1"
]

all = [
    "fire>=0.6.0",
    "loguru>=0.7.2",
    "twat>=1.8.1"
]

[project.scripts]
twat-os = "twat_os.__main__:main"

[project.entry-points."twat.plugins"]
os = "twat_os"

[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

[project.urls]
Documentation = "https://github.com/twardoch/twat-os#readme"
Issues = "https://github.com/twardoch/twat-os/issues"
Source = "https://github.com/twardoch/twat-os"

[build-system]
build-backend = "hatchling.build"
requires = [
    "hatchling>=1.27.0",
    "hatch-vcs>=0.4.0"
]

[tool.coverage.paths]
twat_os = ["src/twat_os", "*/twat-os/src/twat_os"]
tests = ["tests", "*/twat-os/tests"]

[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:"
]

[tool.coverage.run]
source_pkgs = ["twat_os", "tests"]
branch = true
parallel = true
omit = ["src/twat_os/__about__.py"]

[tool.hatch.build.hooks.vcs]
version-file = "src/twat_os/__version__.py"

[tool.hatch.build.targets.wheel]
packages = ["src/twat_os"]
include = [
    "src/twat_os/**/*.py",
    "src/twat_os/py.typed"
]

[tool.hatch.envs.default]
dependencies = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "ruff>=0.9.6",
    "mypy>=1.15.0"
]

[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_os --cov=tests {args:tests}"
type-check = "mypy src/twat_os tests"
lint = ["ruff check src/twat_os tests", "ruff format --respect-gitignore src/twat_os tests"]
fix = ["ruff check --fix --unsafe-fixes src/twat_os tests", "ruff format --respect-gitignore src/twat_os tests"]

[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

[tool.hatch.envs.lint]
detached = true
dependencies = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "ruff>=0.9.6",
    "mypy>=1.15.0"
]

[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_os tests}"
style = ["ruff check {args:.}", "ruff format --respect-gitignore {args:.}"]
fmt = ["ruff format --respect-gitignore {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

[tool.hatch.envs.test]
dependencies = []

[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto -p no:briefcase {args:tests}"
test-cov = "python -m pytest -n auto -p no:briefcase --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_os --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

[tool.mypy]
python_version = "3.10"
check_untyped_defs = true
disallow_incomplete_defs = true
disallow_untyped_decorators = true
disallow_untyped_defs = true
no_implicit_optional = true
warn_no_return = true
warn_redundant_casts = true
warn_return_any = true
warn_unreachable = true
warn_unused_configs = true
warn_unused_ignores = true

[tool.ruff]
line-length = 88
target-version = "py310"

[tool.ruff.lint]
extend-select = [
    "A",
    "ARG",
    "B",
    "C",
    "DTZ",
    "E",
    "EM",
    "F",
    "FBT",
    "I",
    "ICN",
    "ISC",
    "N",
    "PLC",
    "PLE",
    "PLR",
    "PLW",
    "Q",
    "RUF",
    "S",
    "T",
    "TID",
    "UP",
    "W",
    "YTT"
]
ignore = [
    "ARG001",
    "E501",
    "I001",
    "RUF001",
    "PLR2004",
    "EXE003",
    "ISC001"
]

[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]

[tool.pytest.ini_options]
addopts = "-v --durations=10 -p no:briefcase"
asyncio_mode = "auto"
console_output_style = "progress"
filterwarnings = [
    "ignore::DeprecationWarning",
    "ignore::UserWarning"
]
log_cli = true
log_cli_level = "INFO"
markers = [
    "benchmark: marks tests as benchmarks (select with '-m benchmark')",
    "unit: mark a test as a unit test",
    "integration: mark a test as an integration test",
    "permutation: tests for permutation functionality",
    "parameter: tests for parameter parsing",
    "prompt: tests for prompt parsing"
]
norecursedirs = [
    ".*",
    "build",
    "dist",
    "venv",
    "__pycache__",
    "*.egg-info",
    "_private"
]
python_classes = ["Test*"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
testpaths = ["tests"]

[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",
    "max",
    "mean",
    "stddev",
    "median",
    "iqr",
    "ops",
    "rounds"
]

================
File: plugins/repos/twat_os/README.md
================
# 



## Features

- Plugin for twat
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-os
```

## Usage

```python
import twat_os
plugin = twat_os.plugin
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License

================
File: plugins/repos/twat_os/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_search/.cursor/rules/0project.mdc
================
---
description: About this project
globs: 
alwaysApply: false
---
# About this project

`twat-search` is a multi-provider search 

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new search backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_search/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
alwaysApply: false
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in [PROGRESS.md](mdc:PROGRESS.md) and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_search/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[1.3K]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 192]  rules
│       ├── [ 334]  0project.mdc
│       ├── [ 559]  cleanup.mdc
│       └── [ 10K]  filetree.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [1.2K]  .pre-commit-config.yaml
├── [ 128]  .specstory
│   ├── [1.6K]  history
│   │   ├── [2.7K]  .what-is-this.md
│   │   ├── [ 52K]  2025-02-25_01-58-creating-and-tracking-project-tasks.md
│   │   ├── [7.4K]  2025-02-25_02-17-project-task-continuation-and-progress-update.md
│   │   ├── [ 11K]  2025-02-25_02-24-planning-tests-for-twat-search-web-package.md
│   │   ├── [196K]  2025-02-25_02-27-implementing-tests-for-twat-search-package.md
│   │   ├── [ 46K]  2025-02-25_02-58-transforming-python-script-into-cli-tool.md
│   │   ├── [ 93K]  2025-02-25_03-09-generating-a-name-for-the-chat.md
│   │   ├── [5.5K]  2025-02-25_03-33-untitled.md
│   │   ├── [ 57K]  2025-02-25_03-54-integrating-search-engines-into-twat-search.md
│   │   ├── [ 72K]  2025-02-25_04-05-consolidating-you-py-and-youcom-py.md
│   │   ├── [6.1K]  2025-02-25_04-13-missing-env-api-key-names-in-pplx-py.md
│   │   ├── [118K]  2025-02-25_04-16-implementing-functions-for-brave-search-engines.md
│   │   ├── [286K]  2025-02-25_04-48-unifying-search-engine-parameters-in-twat-search.md
│   │   ├── [ 83K]  2025-02-25_05-36-implementing-duckduckgo-search-engine.md
│   │   ├── [194K]  2025-02-25_05-43-implementing-the-webscout-search-engine.md
│   │   ├── [ 23K]  2025-02-25_06-07-implementing-bing-scraper-engine.md
│   │   ├── [ 15K]  2025-02-25_06-12-continuing-bing-scraper-engine-implementation.md
│   │   ├── [121K]  2025-02-25_06-34-implementing-safe-import-patterns-in-modules.md
│   │   ├── [9.9K]  2025-02-25_07-09-refactoring-plan-and-progress-update.md
│   │   ├── [ 40K]  2025-02-25_07-17-implementing-phase-1-from-todo-md.md
│   │   ├── [292K]  2025-02-25_07-34-integrating-hasdata-google-serp-apis.md
│   │   ├── [142K]  2025-02-25_08-19-implementing-search-engines-from-nextengines-md.md
│   │   ├── [175K]  2025-02-26_09-54-implementing-plain-option-for-search-commands.md
│   │   ├── [264K]  2025-02-26_10-55-standardizing-engine-naming-conventions.md
│   │   ├── [4.3K]  2025-02-26_11-30-untitled.md
│   │   ├── [102K]  2025-02-26_12-11-update-config-py-to-use-engine-constants.md
│   │   ├── [278K]  2025-02-26_12-18-update-engine-imports-and-exports-in-init-py.md
│   │   ├── [268K]  2025-02-26_13-40-search-engine-initialization-errors.md
│   │   ├── [ 61K]  2025-02-26_14-15-codebase-issue-analysis-and-fix-plan.md
│   │   ├── [ 43K]  2025-02-26_14-52-resolving-critical-issues-in-todo-md.md
│   │   ├── [247K]  2025-02-26_17-48-progress-update-for-twat-search-project.md
│   │   ├── [ 28K]  2025-02-26_18-28-fixing-num-results-parameter-in-twat-search.md
│   │   ├── [ 63K]  2025-02-26_18-44-fixing-search-engine-issues-in-codebase.md
│   │   ├── [ 72K]  2025-02-26_19-37-removing-anywebsearch-references-from-codebase.md
│   │   ├── [ 98K]  2025-02-26_19-49-fixing-linting-errors-in-code.md
│   │   ├── [761K]  2025-02-26_20-02-analyzing-todo-md-and-updating-progress-md.md
│   │   ├── [ 69K]  2025-02-26_22-55-bing-scraper-result-inconsistency-investigation.md
│   │   ├── [591K]  2025-02-26_23-13-google-scraper-result-validation-issues.md
│   │   ├── [ 23K]  2025-02-27_12-02-plan-for-adding-support-for-falla-engines.md
│   │   ├── [615K]  2025-02-27_13-08-implementing-falla-and-tracking-progress.md
│   │   ├── [ 43K]  2025-02-27_15-08-project-update-and-task-management.md
│   │   ├── [317K]  2025-02-27_15-45-updating-falla-library-selenium-to-playwright.md
│   │   ├── [133K]  2025-02-27_17-02-fixing-falla-py-async-issues.md
│   │   ├── [ 17K]  2025-02-27_17-34-improving-search-engine-selectors-in-falla.md
│   │   ├── [ 76K]  2025-03-04_05-12-fixing-ruff-linting-errors-in-code.md
│   │   ├── [488K]  2025-03-04_05-38-implementing-phases-1-and-2-from-todo-md.md
│   │   ├── [2.8K]  2025-03-04_07-32-project-documentation-and-cleanup-tasks.md
│   │   └── [ 644]  2025-03-04_07-53-untitled.md
│   └── [2.2M]  history.txt
├── [2.8K]  CHANGELOG.md
├── [ 499]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [ 14K]  README.md
├── [ 12K]  TODO.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [6.3K]  debug_fetch.py
├── [ 256]  debug_output
│   ├── [ 445]  qwant_analysis.txt
│   ├── [100K]  qwant_content.html
│   ├── [153K]  qwant_screenshot.png
│   ├── [ 476]  yahoo_analysis.txt
│   ├── [ 88K]  yahoo_content.html
│   └── [402K]  yahoo_screenshot.png
├── [ 192]  dist
├── [7.4K]  falla_search.py
├── [4.0K]  google_debug_Python_programming_language.html
├── [3.9K]  google_debug_test_query.html
├── [5.4K]  pyproject.toml
├── [  43]  requirements.txt
├── [ 224]  resources
│   ├── [ 224]  brave
│   │   ├── [ 65K]  brave.md
│   │   ├── [ 29K]  brave_image.md
│   │   ├── [ 22K]  brave_news.md
│   │   └── [ 22K]  brave_video.md
│   ├── [ 128]  pplx
│   │   ├── [ 32K]  pplx.md
│   │   └── [ 335]  pplx_urls.txt
│   ├── [ 15K]  pricing.md
│   └── [ 192]  you
│       ├── [ 54K]  you.md
│       ├── [ 251]  you.txt
│       ├── [ 58K]  you_news.md
│       └── [  98]  you_news.txt
├── [ 128]  src
│   └── [ 256]  twat_search
│       ├── [ 613]  __init__.py
│       ├── [2.5K]  __main__.py
│       └── [ 416]  web
│           ├── [1.8K]  __init__.py
│           ├── [ 12K]  api.py
│           ├── [ 48K]  cli.py
│           ├── [ 19K]  config.py
│           ├── [2.8K]  engine_constants.py
│           ├── [ 576]  engines
│           │   ├── [8.7K]  __init__.py
│           │   ├── [ 17K]  base.py
│           │   ├── [ 11K]  bing_scraper.py
│           │   ├── [ 15K]  brave.py
│           │   ├── [9.2K]  critique.py
│           │   ├── [7.8K]  duckduckgo.py
│           │   ├── [ 12K]  falla.py
│           │   ├── [ 12K]  google_scraper.py
│           │   ├── [7.9K]  hasdata.py
│           │   ├── [ 288]  lib_falla
│           │   │   ├── [ 822]  __init__.py
│           │   │   ├── [ 608]  core
│           │   │   │   ├── [1.4K]  __init__.py
│           │   │   │   ├── [ 763]  aol.py
│           │   │   │   ├── [ 892]  ask.py
│           │   │   │   ├── [2.4K]  bing.py
│           │   │   │   ├── [ 855]  dogpile.py
│           │   │   │   ├── [6.7K]  duckduckgo.py
│           │   │   │   ├── [ 18K]  falla.py
│           │   │   │   ├── [2.4K]  fetch_page.py
│           │   │   │   ├── [ 860]  gibiru.py
│           │   │   │   ├── [ 12K]  google.py
│           │   │   │   ├── [ 762]  mojeek.py
│           │   │   │   ├── [5.9K]  qwant.py
│           │   │   │   ├── [ 923]  searchencrypt.py
│           │   │   │   ├── [ 900]  startpage.py
│           │   │   │   ├── [5.4K]  yahoo.py
│           │   │   │   └── [2.1K]  yandex.py
│           │   │   ├── [2.9K]  main.py
│           │   │   ├── [ 365]  requirements.txt
│           │   │   ├── [ 378]  settings.py
│           │   │   └── [4.8K]  utils.py
│           │   ├── [7.7K]  pplx.py
│           │   ├── [7.3K]  serpapi.py
│           │   ├── [8.2K]  tavily.py
│           │   └── [8.6K]  you.py
│           ├── [1.0K]  exceptions.py
│           ├── [1.6K]  models.py
│           └── [4.1K]  utils.py
├── [ 453]  test_async_falla.py
├── [ 443]  test_falla.py
├── [3.3K]  test_google_falla_debug.py
├── [1.7K]  test_simple.py
├── [ 341]  test_sync_falla.py
├── [ 256]  tests
│   ├── [  64]  .benchmarks
│   ├── [2.0K]  conftest.py
│   ├── [ 193]  test_twat_search.py
│   ├── [ 192]  unit
│   │   ├── [  78]  __init__.py
│   │   ├── [1.6K]  mock_engine.py
│   │   └── [ 320]  web
│   │       ├── [  82]  __init__.py
│   │       ├── [ 160]  engines
│   │       │   ├── [  73]  __init__.py
│   │       │   └── [4.4K]  test_base.py
│   │       ├── [5.2K]  test_api.py
│   │       ├── [2.7K]  test_config.py
│   │       ├── [2.0K]  test_exceptions.py
│   │       ├── [4.1K]  test_models.py
│   │       └── [3.5K]  test_utils.py
│   └── [ 160]  web
│       └── [ 10K]  test_bing_scraper.py
├── [664K]  twat_search.txt
└── [195K]  uv.lock

26 directories, 150 files

================
File: plugins/repos/twat_search/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_search --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_search/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-search
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_search/.idea/inspectionProfiles/profiles_settings.xml
================
<component name="InspectionProjectProfileManager">
  <settings>
    <option name="USE_PROJECT_PROFILE" value="false" />
    <version value="1.0" />
  </settings>
</component>

================
File: plugins/repos/twat_search/.idea/.gitignore
================
# Default ignored files
/shelf/
/workspace.xml

================
File: plugins/repos/twat_search/.idea/misc.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="Black">
    <option name="sdkName" value="Python 3.12 (twat_search)" />
  </component>
  <component name="ProjectRootManager" version="2" project-jdk-name="Python 3.12 (twat_search)" project-jdk-type="Python SDK" />
</project>

================
File: plugins/repos/twat_search/.idea/modules.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="ProjectModuleManager">
    <modules>
      <module fileurl="file://$PROJECT_DIR$/.idea/twat_search.iml" filepath="$PROJECT_DIR$/.idea/twat_search.iml" />
    </modules>
  </component>
</project>

================
File: plugins/repos/twat_search/.idea/ruff.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="RuffConfigService">
    <option name="globalRuffExecutablePath" value="/Library/Frameworks/Python.framework/Versions/3.12/bin/ruff" />
  </component>
</project>

================
File: plugins/repos/twat_search/.idea/twat_search.iml
================
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
  <component name="NewModuleRootManager">
    <content url="file://$MODULE_DIR$">
      <sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
      <excludeFolder url="file://$MODULE_DIR$/.venv" />
    </content>
    <orderEntry type="jdk" jdkName="Python 3.12 (twat_search)" jdkType="Python SDK" />
    <orderEntry type="sourceFolder" forTests="false" />
  </component>
  <component name="PyDocumentationSettings">
    <option name="format" value="PLAIN" />
    <option name="myDocStringFormat" value="Plain" />
  </component>
  <component name="TestRunnerService">
    <option name="PROJECT_TEST_RUNNER" value="py.test" />
  </component>
</module>

================
File: plugins/repos/twat_search/.idea/vcs.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="VcsDirectoryMappings">
    <mapping directory="" vcs="Git" />
  </component>
</project>

================
File: plugins/repos/twat_search/.idea/workspace.xml
================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="AnalysisUIOptions">
    <option name="SPLITTER_PROPORTION" value="0.32892907" />
    <option name="ANALYZE_INJECTED_CODE" value="false" />
  </component>
  <component name="AutoImportSettings">
    <option name="autoReloadType" value="SELECTIVE" />
  </component>
  <component name="ChangeListManager">
    <list default="true" id="7e893f62-f4ac-4be0-a4f1-7ef663c5b4bf" name="Changes" comment="">
      <change afterPath="$PROJECT_DIR$/.specstory/history/2025-02-27_15-45-updating-falla-library-selenium-to-playwright.md" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/falla_search.py" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/fetch_page.py" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/requirements.txt" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/test_async_falla.py" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/test_falla.py" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/test_simple.py" afterDir="false" />
      <change afterPath="$PROJECT_DIR$/test_sync_falla.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/.cursor/rules/filetree.mdc" beforeDir="false" afterPath="$PROJECT_DIR$/.cursor/rules/filetree.mdc" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/.pre-commit-config.yaml" beforeDir="false" afterPath="$PROJECT_DIR$/.pre-commit-config.yaml" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/CLEANUP.txt" beforeDir="false" afterPath="$PROJECT_DIR$/CLEANUP.txt" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/PROGRESS.md" beforeDir="false" afterPath="$PROJECT_DIR$/PROGRESS.md" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/TODO.md" beforeDir="false" afterPath="$PROJECT_DIR$/TODO.md" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/pyproject.toml" beforeDir="false" afterPath="$PROJECT_DIR$/pyproject.toml" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/falla.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/falla.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/bing.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/bing.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/duckduckgo.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/duckduckgo.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/falla.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/falla.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/google.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/google.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/qwant.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/qwant.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/yahoo.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/yahoo.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/yandex.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/core/yandex.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/main.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/main.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/utils.py" beforeDir="false" afterPath="$PROJECT_DIR$/src/twat_search/web/engines/lib_falla/utils.py" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/twat_search.txt" beforeDir="false" afterPath="$PROJECT_DIR$/twat_search.txt" afterDir="false" />
      <change beforePath="$PROJECT_DIR$/uv.lock" beforeDir="false" afterPath="$PROJECT_DIR$/uv.lock" afterDir="false" />
    </list>
    <option name="SHOW_DIALOG" value="false" />
    <option name="HIGHLIGHT_CONFLICTS" value="true" />
    <option name="HIGHLIGHT_NON_ACTIVE_CHANGELIST" value="false" />
    <option name="LAST_RESOLUTION" value="IGNORE" />
  </component>
  <component name="Git.Settings">
    <option name="RECENT_GIT_ROOT_PATH" value="$PROJECT_DIR$" />
  </component>
  <component name="GitHubPullRequestSearchHistory">{
  &quot;lastFilter&quot;: {
    &quot;state&quot;: &quot;OPEN&quot;,
    &quot;assignee&quot;: &quot;twardoch&quot;
  }
}</component>
  <component name="GithubPullRequestsUISettings">{
  &quot;selectedUrlAndAccountId&quot;: {
    &quot;url&quot;: &quot;https://github.com/twardoch/twat-search.git&quot;,
    &quot;accountId&quot;: &quot;e13b4c44-74d4-444d-8d41-a0014a4f5e46&quot;
  }
}</component>
  <component name="ProblemsViewState">
    <option name="selectedTabId" value="DEPENDENCY_CHECKER_PROBLEMS_TAB" />
  </component>
  <component name="ProjectColorInfo">{
  &quot;associatedIndex&quot;: 6
}</component>
  <component name="ProjectId" id="2tVu0eV0ttkimuCVbQl164d1fNA" />
  <component name="ProjectViewState">
    <option name="hideEmptyMiddlePackages" value="true" />
    <option name="showLibraryContents" value="true" />
  </component>
  <component name="PropertiesComponent">{
  &quot;keyToString&quot;: {
    &quot;RunOnceActivity.ShowReadmeOnStart&quot;: &quot;true&quot;,
    &quot;RunOnceActivity.git.unshallow&quot;: &quot;true&quot;,
    &quot;git-widget-placeholder&quot;: &quot;main&quot;,
    &quot;last_opened_file_path&quot;: &quot;/Users/adam/Developer/vcs/github.twardoch/pub/twat-packages/_good/twat/plugins/repos/twat_search&quot;,
    &quot;run.code.analysis.last.selected.profile&quot;: &quot;aDefault&quot;,
    &quot;settings.editor.selected.configurable&quot;: &quot;preferences.pluginManager&quot;
  }
}</component>
  <component name="SharedIndexes">
    <attachedChunks>
      <set>
        <option value="bundled-python-sdk-fc1d50d2d915-aa17d162503b-com.jetbrains.pycharm.community.sharedIndexes.bundled-PC-243.24978.54" />
      </set>
    </attachedChunks>
  </component>
  <component name="SpellCheckerSettings" RuntimeDictionaries="0" Folders="0" CustomDictionaries="0" DefaultDictionary="application-level" UseSingleDictionary="true" transferred="true" />
  <component name="TaskManager">
    <task active="true" id="Default" summary="Default task">
      <changelist id="7e893f62-f4ac-4be0-a4f1-7ef663c5b4bf" name="Changes" comment="" />
      <created>1740449340899</created>
      <option name="number" value="Default" />
      <option name="presentableId" value="Default" />
      <updated>1740449340899</updated>
    </task>
    <servers />
  </component>
</project>

================
File: plugins/repos/twat_search/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_search/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_search/debug_output/qwant_analysis.txt
================
Analysis for qwant
================================================================================

Potential containers found:
div.dd.algo: 0 elements found
div.algo-sr: 0 elements found
div.dd.algo.algo-sr: 0 elements found
article.webResult: 0 elements found
div.result: 0 elements found
div.web-result: 0 elements found
div.result: 0 elements found
div.search-result: 0 elements found
li.result: 0 elements found

Other potential elements:

================
File: plugins/repos/twat_search/debug_output/qwant_content.html
================
<!DOCTYPE html><html lang="en"><head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta http-equiv="Accept-CH" content="Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA, Sec-CH-Prefers-Color-Scheme">
    <base href="https://www.qwant.com">
    <meta name="theme-color" content="#FFFFFF">
    <meta name="twitter:card" content="summary_large_image">
    <meta name="twitter:site" content="@QwantCom">
    <meta name="twitter:creator" content="@QwantCom">
    <meta name="twitter:domain" content="https://www.qwant.com">
    <meta name="twitter:widgets:csp" content="on">
    
        <meta property="og:image" content="https://www.qwant.com/public/qwant-logo-seo.0ea4e62ad0a6047e242cdb57128d7995.png">
        <meta property="og:image:width" content="1000">
        <meta property="og:image:height" content="536">
        <meta name="twitter:image" content="https://www.qwant.com/public/qwant-logo-seo.0ea4e62ad0a6047e242cdb57128d7995.png">
      
    <meta property="og:type" content="website">
    <meta property="og:url" content="https://www.qwant.com" id="meta_og_url">
    <meta name="application-name" content="Qwant">
    <meta name="viewport" content="width=device-width,initial-scale=1.0,minimum-scale=1.0">
    <meta name="description" content="Fast, reliable answers and still in trust: Qwant does not store your search data, does not sell your personal data and is hosted in Europe.">
    <meta name="keywords" content="search engine">
    <meta property="og:description" content="Fast, reliable answers and still in trust: Qwant does not store your search data, does not sell your personal data and is hosted in Europe.">
    <meta property="og:site_name" content="Qwant">
    <title>Qwant - The search engine that values you as a user, not as a product</title>
    <meta property="og:title" content="FontLab – Qwant Search">
    <meta property="twitter:title" content="FontLab – Qwant Search">
    <link rel="manifest" href="/manifest.json">
    <link data-chunk="app" rel="preload" as="style" href="/1f7c4e111261b3ad8a04_app.chunks.css">
<link data-chunk="app" rel="preload" as="script" href="/a61ec0921f734a0d5dee_runtime.Phoenix.js">
<link data-chunk="app" rel="preload" as="script" href="/eccc33be0ced324f778a_vendors~app.chunks.js">
<link data-chunk="app" rel="preload" as="script" href="/46a6c1fdc290b3b0cbd5_app.chunks.js">
    <link data-chunk="app" rel="stylesheet" href="/1f7c4e111261b3ad8a04_app.chunks.css">
    <meta name="robots" content="index">
    <link rel="shortcut icon" href="/public/favicon.066f5ee2ab77b590bb5846c32c57cb84.ico" type="image/x-icon">
    <link rel="icon" href="/public/favicon.97e9bd6cd3d07b5f096430c1b09ea705.png" type="image/png">
    <link rel="icon" sizes="32x32" type="image/png" href="/public/favicon-32.9d2af599f2a0364630ddb8a2a32026ec.png">
    <link rel="icon" sizes="64x64" type="image/png" href="/public/favicon-64.ddb1092939bad1aaca378c29744e7be3.png">
    <link rel="icon" sizes="96x96" type="image/png" href="/public/favicon-96.3a477836f89ad0ed8e0420c7c7ef4679.png">
    <link rel="icon" sizes="144x144" type="image/png" href="/public/favicon-144.6fdb01169d8537aeadd4f95dce2982e4.png">
    <link rel="icon" sizes="152x152" href="/public/favicon-152.8c87cc8fd7862d72eee478c8f503d3e8.png" type="image/png">
    <link rel="icon" sizes="196x196" href="/public/favicon-196.b2a1214ee3c261334512a377f1d678f7.png" type="image/png">
    <link rel="apple-touch-icon" sizes="152x152" href="/public/apple-touch-icon.e9d3092ec9fb9e1a28c1ea1a3b10777a.png">
    <link rel="apple-touch-icon" sizes="60x60" href="/public/apple-touch-icon-60x60.2f06b254c6f7bc88c95cb88780b2548d.png">
    <link rel="apple-touch-icon" sizes="76x76" href="/public/apple-touch-icon-76x76.64dd5e4f44dc904646e7b304031cf763.png">
    <link rel="apple-touch-icon" sizes="114x114" href="/public/apple-touch-icon-114x114.442d3c1be69b232f3eaae7d19ca653c0.png">
    <link rel="apple-touch-icon" sizes="120x120" href="/public/apple-touch-icon-120x120.97e9bd6cd3d07b5f096430c1b09ea705.png">
    <link rel="apple-touch-icon" sizes="144x144" href="/public/apple-touch-icon-144x144.6fdb01169d8537aeadd4f95dce2982e4.png">
    <link rel="apple-touch-icon-precomposed" href="/public/favicon-152.8c87cc8fd7862d72eee478c8f503d3e8.png">
    <meta name="msapplication-config" content="https://www.qwant.com/browserconfig.xml"> 
    <meta name="referrer" content="origin">
    <link rel="canonical" href="https://www.qwant.com">
    <link rel="search" type="application/opensearchdescription+xml" title="Qwant" href="https://www.qwant.com/opensearch.xml">
    <link rel="chrome-webstore-item" href="https://chrome.google.com/webstore/detail/hnlkiofnhhoahaiimdicppgemmmomijo">
    <link rel="alternate" hreflang="fr" href="https://www.qwant.com/?l=fr">
	<link rel="alternate" hreflang="en" href="https://www.qwant.com/?l=en">
	<link rel="alternate" hreflang="de" href="https://www.qwant.com/?l=de">
	<link rel="alternate" hreflang="it" href="https://www.qwant.com/?l=it">
	<link rel="alternate" hreflang="br" href="https://www.qwant.com/?l=br">
	<link rel="alternate" hreflang="ca" href="https://www.qwant.com/?l=ca">
	<link rel="alternate" hreflang="co" href="https://www.qwant.com/?l=co">
	<link rel="alternate" hreflang="es" href="https://www.qwant.com/?l=es">
	<link rel="alternate" hreflang="eu" href="https://www.qwant.com/?l=eu">
	<link rel="alternate" hreflang="nl" href="https://www.qwant.com/?l=nl">
	<link rel="alternate" hreflang="pl" href="https://www.qwant.com/?l=pl">
	<link rel="alternate" hreflang="pt" href="https://www.qwant.com/?l=pt">
    <link rel="alternate" href="https://www.qwant.com" hreflang="x-default">
    <link rel="preconnect" href="https://mn.qwant.com/v2">
    <link rel="preconnect" href="https://api.qwant.com/v3">
  <link rel="stylesheet" type="text/css" href="/c1288907ef4c62505104_apps-errors-DatadomeCaptcha.chunks.css"><script charset="utf-8" src="/9c0275a1ca1698783ba2_apps-errors-DatadomeCaptcha.chunks.js"></script><script id="ppas_container_configuration" data-appid="98cbd59f-731c-4874-a402-04f51ed9069a" data-host="k.qwant.com"></script></head>

  <body class="" data-device="desktop" data-product="qwant" data-theme="light">
  
  <!-- render markup phoenix -->
  <div id="root"><div style="display: block;"><iframe class="_m56B" data-testid="captchaDatadomeContainer" src="https://geo.captcha-delivery.com/captcha/?initialCid=AHrlqAAAAAMA_uUIbTzAlFwAWUAcMA==&amp;cid=Yi476GtTtdBnxk2U_0z0GWu~JUehKSVKrRM8oOfEy_IcQLbhLuWRlquGdPMnbzh9uTqMIqypBfMuuBs5P8VExp~iFBF73Atlzexr~xQIsLS3aMC8cUsRZqRcwoe~ZN2U&amp;referer=http%3A%2F%2Ffdn.qwant.com%2Fv3%2Fsearch%2Fweb%3Fq%3De236108c4c1e3f45885edc3f14e36afa80de4c2f8e1f5397262455c8b1d8b72d4cb7078aeb61790c3a3125164edcf12d15544bfc9b511f3d23a0c4e9d17fc8da%26count%3D10%26locale%3Dpl_PL%26offset%3D0%26device%3Ddesktop%26tgp%3D1%26safesearch%3D1%26displayed%3Dtrue%26llm%3Dtrue&amp;hash=78B13B7513D180B7AB6D6FF9EB0A51&amp;t=fe&amp;s=44484&amp;e=920c0bb4642e95d5bd768bfa18dad6705c3137aad0fb0271e0735cdb175b0446" title="Antiscrap Captcha"></iframe></div></div>
  
  <!-- inject i18n and phoenix scripts -->
  <script type="text/javascript" async="" defer="" src="https://k.qwant.com/2aecbbf86c74c1c2fb798e0a39f0678e.js"></script><script async="" src="https://k.qwant.com/containers/98cbd59f-731c-4874-a402-04f51ed9069a.js"></script><script>INITIAL_PROPS = {"testGroupPer":1,"llm":{"apiUrl":"https:\u002F\u002Fapi.qwant.com"},"piwik":{"url":"https:\u002F\u002Fk.qwant.com","id":"98cbd59f-731c-4874-a402-04f51ed9069a","permittedDomains":["shdw.me","www.qwantjunior.com"],"crossDomains":["shdw.me","www.qwantjunior.com","about.qwant.com","help.qwant.com","betterweb.qwant.com","auth.qwant.com"]},"didomi":{"apiKey":"0dbf2631-6bba-4497-9048-745301a3f637","noticeFr":"NrdXrK89","noticeRow":"j4NDeL4k","noticeEs":"a7gixD3Q","clarityId":"km57m6oo9k","datadomeId":"78B13B7513D180B7AB6D6FF9EB0A51"},"device":{"browserName":"Chrome","completeUa":"Mozilla\u002F5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\u002F537.36 (KHTML, like Gecko) Chrome\u002F91.0.4472.124 Safari\u002F537.36","isWindowsDesktop":true,"isMacDesktop":false,"isAndroid":false,"isIos":false,"isMobile":false,"isNativeApp":false,"isPhone":false,"isTablet":false},"urls":{"domain":"qwant.com","apiUrlClient":"https:\u002F\u002Fapi.qwant.com\u002Fapi","apiUrlClientV2":"https:\u002F\u002Fapi.qwant.com\u002Fv3","minervaUrlClient":"https:\u002F\u002Fmn.qwant.com\u002Fv2","apmServerExternal":"https:\u002F\u002Fapm.qwant.com","reportUrl":"https:\u002F\u002Freport.qwant.com","martinUrl":"https:\u002F\u002Ftiles.qwant.com","maps":"https:\u002F\u002Fmap.qwant.com\u002F","lite":"https:\u002F\u002Flite.qwant.com","junior":"https:\u002F\u002Fwww.qwantjunior.com\u002F","blog":"\u002F\u002Fbetterweb.qwant.com\u002F","shop":"\u002F\u002Fshop.spreadshirt.fr\u002Fqwantstore","help":"\u002F\u002Fhelp.qwant.com\u002F","jobs":"\u002F\u002Fwww.welcometothejungle.com\u002Ffr\u002Fcompanies\u002Fqwant\u002Fjobs","downloads":"\u002F\u002Fdownloads.qwant.com"},"isMailEnabled":false,"features":{"drawerSettingLogin":false,"drawerSettingSync":false,"enableAccount":true,"enableApmMonitoring":true,"enableCssLint":false,"enableDatadomeCaptcha":true,"enableDebugRenderReact":false,"enableDegradedMode":false,"enableDidomiConsentEs":true,"enableDidomiConsentFr":true,"enableDidomiConsentRow":true,"enableGoogleModule":true,"enableHomeTiles":true,"enableHomeIaGen":true,"enableLlm":true,"enableOutage":false,"enableLandingGame":false,"enableProductsAdsMicrosoft":true,"enableQwantAlpha":false,"enableQwantJunior":false,"enableQwantReward":true,"enableQwantRewardPopover":false,"enableTextToKids":true,"piwik":true,"productsAds":false,"settingSearchLanguage":true},"configuration":{"baseUrl":"https:\u002F\u002Fwww.qwant.com","appName":"phoenix","appVersion":"ad433566e0d622b5837cd198b58cc0e489397c5b","clientRequestTimeoutInMS":12000,"clientRequestTimeoutWithRetryInMS":12000,"apmTransactionSampleRateForRUM":0.5,"pageLoadTraceId":"f9e3327f9e6017f9c792248e66bb6f62","pageLoadSpanId":"06d71cad59c3bdfb","pageLoadSampled":true},"regionLocale":"pl_PL","urlParams":{"q":"FontLab"},"query":"FontLab","languages":{"fr":{"name":"Français"},"en":{"name":"English"},"de":{"name":"Deutsch"},"it":{"name":"Italiano"},"br":{"name":"Brezhoneg"},"ca":{"name":"Català"},"co":{"name":"Corsu"},"es":{"name":"Español"},"eu":{"name":"Euskara"},"nl":{"name":"Nederlands"},"pl":{"name":"Polski"},"pt":{"name":"Português"}},"locales":{"FR":{"langs":["fr"]},"GB":{"langs":["en"]},"DE":{"langs":["de"]},"IT":{"langs":["it"]},"AR":{"langs":["es"]},"AU":{"langs":["en"]},"US":{"langs":["en"]},"ES":{"langs":["es","ca"]},"CZ":{"langs":["cs"]},"RO":{"langs":["ro"]},"GR":{"langs":["el"]},"CN":{"langs":["zh"]},"HK":{"langs":["zh"]},"NZ":{"langs":["en"]},"TH":{"langs":["th"]},"KR":{"langs":["ko"]},"SE":{"langs":["sv"]},"NO":{"langs":["nb"]},"DK":{"langs":["da"]},"HU":{"langs":["hu"]},"EE":{"langs":["et"]},"MX":{"langs":["es"]},"CL":{"langs":["es"]},"CA":{"langs":["en","fr"]},"MY":{"langs":["en"]},"BG":{"langs":["bg"]},"FI":{"langs":["fi"]},"PL":{"langs":["pl"]},"NL":{"langs":["nl"]},"PT":{"langs":["pt"]},"CH":{"langs":["de","fr","it"]},"AT":{"langs":["de"]},"BE":{"langs":["fr","nl"]},"IE":{"langs":["en"]},"IL":{"langs":["he"]}},"blacklistedMarkets":["JP","RU","TR","PH","SG","ID","TW","BR","IN","VN"],"market":"pl","geoCountryCode":"PL","serverState":{"theme":"-1","extInstalled":false,"randomAutoPromo":0},"dataPage":{"data":null},"accountConfiguration":{"iamUrl":"https:\u002F\u002Fauth.qwant.com\u002Fkratos","accountApiUrl":"https:\u002F\u002Faccount.qwant.com","authenticatorKratosUrl":"https:\u002F\u002Fauth.qwant.com\u002Fui\u002Fsettings\u002Fsecurity\u002Fauthenticator","loginKratosUrl":"https:\u002F\u002Fauth.qwant.com\u002Fkratos\u002Fself-service\u002Flogin\u002Fbrowser","registrationKratosUrl":"https:\u002F\u002Fauth.qwant.com\u002Fkratos\u002Fself-service\u002Fregistration\u002Fbrowser","verificationKratosUrl":"https:\u002F\u002Fauth.qwant.com\u002Fkratos\u002Fself-service\u002Fverification\u002Fbrowser","profileKratosUrl":"https:\u002F\u002Fauth.qwant.com\u002Fui\u002Fsettings\u002Fprofile","socialsKratosUrl":"https:\u002F\u002Fauth.qwant.com\u002Fui\u002Fsettings\u002Fsocials"},"isCookieAccountPresent":false}</script>
  <script>
                  initialI18nStore = {"en": {"account":{"misc":{"loggedOutByError":"An error occurred while retrieving your account. Please try logging in again."},"menu":{"profile":"My profile","security":"Login and security","reward":"Rewards","notifications":"Notifications"},"pages":{"notifications":{"title":"Notifications","newsletter":{"title":"Newsletter","description":"Receive occasional information about our technological advances or product releases"},"qwant_community":{"title":"Communauté Qwant","description":"Be notified to preview our new products and give your opinion"},"qwant_rewards":{"title":"Qwant loyalty program","description":"View progress notifications on Qwant.com","second_description":"Receive emails (required)"}},"reward":{"title":"Rewards","program":{"title":"Qwant loyalty program","inactive":"Not active","requiredConsent":"Consent required","active":"Active","inactiveCta":"Start","requiredConsentCta":"Change","activeCta":"Opt out"},"bank_account":{"title":"Account holder","description":"No information","cta":"Add"},"iban":{"title":"IBAN","description":"No information","cta":"Add"},"terms":{"title":"Terms of use"},"privacy_policy":{"title":"Privacy policy"}},"profile":{"title":"My profile","avatar":{"cta":"Change","secondCta":"Reset"},"email":{"title":"Email","meta":"Verified","cta":"Edit"},"username":{"title":"Username","cta":"Change","modal":{"title":"Change your username ","label":"Your username","cancel":"Cancel","confirm":"Edit"}},"language":{"title":"Language","cta":"Change"}},"security":{"title":"Login and security","password":{"title":"Password","cta":"Modify"},"2fa":{"title":"Double authentication","description":"Two-factor authentication protects your account by adding an extra step of security when you log in.","cta":"Manage"},"external_account":{"title":"Alternative login methods","description":"Link your Qwant account to a social account to log in.","cta":"Manage"},"my_privacy_preferences":{"title":"My privacy preferences","description":"Adjust your privacy settings","cta":"Modify"},"personal_data":{"title":"My personal data","description":"Receive a copy of my personal data"},"delete":{"title":"Delete my account","description":"Irreversible action"}}},"modals":{"delete":{"title":"Delete my account","body":"All associated data will be deleted and you will no longer be able to log in to Qwant","confirmCta":"Confirm account deletion","cancelCta":"Cancel and return to my account"},"optOutReward":{"title":"Opt out of the Qwant loyalty program","body":"By opting out, you agree to lose all your points, your ranking and the value of your Qwant wallet.","confirmCta":"Confirm my opting out","cancelCta":"Cancel and return to my account"},"deleted":{"title":"Account deleted !","body":"We're sorry to see you go.\u003Cbr\u002F\u003EWe hope you will continue to use Qwant products in the future.\u003Cbr\u002F\u003ESee you soon!","cta":"Back to Qwant.com"}}},"alpha":{},"common":{"brandName":"Qwant","close":"Close","share":"Share","send":"Send","reportContent":"Report content","alreadyReported":"Already reported","fullScreen":"Full screen","edit":"Edit","ad":"Ad","partnership":"Partnership","event":"Event","new":"New","resultsNotRelevant":"The following results are probably not relevant, please rephrase your query.","comCenter":{"privacy":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fconfidentialite\u002F","hrefHuawei":"https:\u002F\u002Fabout.qwant.com\u002Fprivacy-happ\u002F"}},"seo":{"searchTitle":"Qwant Search","description":"Fast, reliable answers and still in trust: Qwant does not store your search data, does not sell your personal data and is hosted in Europe.","tagline":"The search engine that values you as a user, not as a product","keywords":"search engine"},"clipboard":{"copied":"Copied","success":"The URL was copied to your clipboard.","failure":"The URL could not be copied to your clipboard. Your browser might not allow it.","textSuccess":"Text copied!","label":"Copy"},"validationErrors":{"required":"This field is required","minLength":"This field requires at least {{min}} character","maxLength":"This field cannot exceed {{max}} characters","alphaDashPoint":"This field must only contain alpha-numeric characters, dashes, underscores and dots"},"next":"Next","no_results":"No results found.","showAllResults":"Show all the results","search":"Search","language":"Language","aria_search":"Enter your search term","aria_search_button":"Search the web","suggestions":"Suggestions","country":"Country","confirm":"Confirm","cancel":"Cancel","errorInPage":{"firstPart":"Qwant is temporarily unavailable","secondPart":"Please check your connection or \u003C0\u003Erefresh page\u003C\u002F0\u003E.","refreshButton":"Relaunch your search","strictModeTitle":"The content you are looking for may be inappropriate","strictModeDescription":"Your current SafeSearch setting filters out results that might return adult content. To view those results as well, \u003C0\u003Echange your SafeSearch setting\u003C\u002F0\u003E"},"accessibility":{"share":"Share","fullScreen":"Full screen","report":"Report"},"account":{"login":"Login","logout":"Logout"},"secondAgo":"{{count}} second ago","secondAgo_plural":"{{count}} seconds ago","minuteAgo":"{{count}} minute ago","minuteAgo_plural":"{{count}} minutes ago","hourAgo":"{{count}} hour ago","hourAgo_plural":"{{count}} hours ago","yesterday":"yesterday","dayAgo":"{{count}} day ago","dayAgo_plural":"{{count}} days ago","now":"just now","weekAgo":"{{count}} week ago","weekAgo_plural":"{{count}} weeks ago","monthAgo":"{{count}} month ago","monthAgo_plural":"{{count}} months ago","countries":{"AR":"Argentina","AT":"Austria","AU":"Australia","BE":"Belgium","BR":"Brazil","BG":"Bulgaria","CA":"Canada","CH":"Switzerland","CL":"Chile","CZ":"Czech Republic","DK":"Denmark","DE":"Germany","FI":"Finland","GR":"Greece","HK":"Hong Kong","HU":"Hungary","KR":"South Korea","EE":"Estonia","ES":"Spain","FR":"France","GB":"Great Britain","IE":"Ireland","IN":"India","IT":"Italy","MY":"Malaysia","MX":"Mexico","NO":"Norway","PL":"Poland","PT":"Portugal","NL":"Netherlands","NZ":"New Zealand","RO":"Romania","SE":"Sweden","TH":"Thailand","US":"United States","CN":"China","IL":"Israel"},"addon":{"firefoxAddon":"https:\u002F\u002Faddons.mozilla.org\u002Fen-US\u002Ffirefox\u002Faddon\u002Fqwant-the-search-engine\u002F","edgeAddon":"https:\u002F\u002Fmicrosoftedge.microsoft.com\u002Faddons\u002Fdetail\u002Fqwant-the-search-engine\u002Fadmmanannmnegodaboehkdhangccabio","chromeAddon":"https:\u002F\u002Fchromewebstore.google.com\u002Fdetail\u002Fqwant-the-search-engine\u002Fkplfenefaakjhjkklghidleljeocgdap","safariAddon":"https:\u002F\u002Fapps.apple.com\u002Fapp\u002Fqwant-for-safari\u002Fid1645050159"},"userFeedback":{"option_useful":"It's useful","option_useless":"It's not useful","option_bug":"There's a mistake","option_other":"Other comments","inputPlaceholder":"Tell us more!","modalTitle":"What do you think?","successSubtitle":"Thank you for your comment","successTitle":"Well received!","moreConfetti":"More confetti!","errorRequired":"Could you please tell us more?","labelFloating":"Feedback","label":"Share your feedback","title":"What do you think of your Qwant experience?","selectElementNegative":"What can we improve?","selectElementPositive":"What did you like?","selectElementLabel":"Select an item...","commentPlaceholder":"Comment","selectElementOptionHome":"Home page","selectElementOptionResults":"Results page","selectElementOptionAds":"Advertisements","selectElementOptionNews":"News","selectElementOptionImages":"Images","selectElementOptionVideos":"Videos","selectElementOptionShopping":"Shopping products","selectElementOptionBrand":"The Qwant brand","selectElementOptionOther":"Other"},"contactUs":"Contact us","degradedMode":{"ribbonDescription":"MAINTENANCE in progress. Some features, as well as links at the bottom of the page, may be temporarily unavailable. Thank you for your understanding."},"outage":{"ribbonDescription":"We are currently experiencing issues with search service being temporarily unavailable. Thank you for your understanding."},"ribbons":{"firefox":{"title":"New","description":"Make your choice","linkText":"Use Qwant on Firefox","linkUrl":"https:\u002F\u002Fbetterweb.qwant.com\u002Fen\u002F2024\u002F03\u002F19\u002Ftake-your-pick-use-qwant-on-firefox\u002F?utm_source=ap_Ribbon_desk_EN&utm_medium=clicks&utm_campaign=blog_camp_mozilla&utm_id=art.blog"},"newQwantRibbon":{"new":"New!","text":"Qwant artificial intelligence for your searches : ","linkText":"Learn more","link":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002F"},"retrib":{"new":"New!","text":"Qwant launches its fidelity program: win up to 10€ per week with your search engine. ","linkText":"Learn More","link":"https:\u002F\u002Fabout.qwant.com\u002Fprogramme-de-fidelite\u002F"}}},"drawers":{"drawerSettings":{"titleDrawerMobileMenu":"Settings","titleSafeSearchFilter":"Filter adult content","titleCountryFilter":"Country\u002FRegion","titleInterfaceLanguage":"Interface Language","cookieInfo":"Any changes you make to these options must be saved \u003C0\u003Ein a cookie\u003C\u002F0\u003E created exclusively for this purpose. You can delete this cookie in your browser options.","cookieHide":"Do not show anymore"},"bottomMenu":{"home":"Home","products":"Our Products","about":"About","settings":"Settings"},"drawerProducts":{"cardNewModelLink":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002F","partner":{"label":"Partner","link":"https:\u002F\u002Fshdw.me\u002Fqwant-tool","name":"Shadow Drive","title":"Get 20GB of free storage!","wording":"Keep your private files safe and secure with Shadow Drive, the cloud storage solution from Qwant’s partner, designed and hosted in Europe.","cta":"Discover"}},"drawerCookieManagement":{"title":"Managing your cookie preferences","wording1":"Certain features of this site rely on the use of cookies.\nThe cookies banner is not displayed on the home page during your navigation because only cookies necessary for the operation of the site are deposited (you can, however, object to this by clicking on the “Refuse” button below).","wording2":"Statistical audience measurement services are necessary for the operation of the site by enabling its proper administration. However, you have the possibility to object to their use.","actionTitle":"Statistical audience measurement","authorize":"Authorize","refuse":"Refuse","TestsAB":"Performance Measurement Tests (AB Test)"},"pushExtension":{"description":"Qwant becomes your default search engine, blocks trackers and protects your personal data","name":"Install {{name}} extension"}},"header":{"appMenu":{"llmMode":"Flash Response","llmBanner":{"title":"Flash Response saves you time","description":"Qwant finds the right information from the most relevant web pages. You get your answer immediately at the top of the page!","linkText":"Find out more","link":"https:\u002F\u002Fabout.qwant.com\u002F#ia-gen"},"Web":"All","News":"News","Images":"Images","Videos":"Videos","Shopping":"Shopping","Maps":"Maps","about":"About us","mail":"Mail","new":"new","products":"Products","settings":"Settings","burgerMenu":"Show menu"},"backToQwantCom":"Back to Qwant.com","login":"Login","search":{"placeholder":"Search","altLogo":"Qwant Logo"}},"errorpage":{"errorNotFound":{"title":"Oops 404 lunar page","description":"You have been sent too far mea culpa!","link":"Back to the planet Qwant"},"errorPage":{"title":"Maintenance","description":"Our service will be back in a few moments, we apologize for the inconvenience.","link":"Refresh page"},"errorUnavailable":{"title":"Service unavailable","thanks":"Thanks for your visit","sadly":"Unfortunately we are not yet available in your country.","learnMore":"Would you like to know more about our actions?","linkToBlog":"Visit our blog"}},"filters":{"Brand_filters":"Brand","Category_filters":"Category","Any color_filters":"Any color","Color_filters":"Color","Format_filters":"Format","Any freshness_filters":"Any freshness","Freshness_filters":"Freshness","Language_filters":"Language","Any license_filters":"Any license","License_filters":"License","Price_filters":"Price","min_filters":"min","max_filters":"max","Region_filters":"Region","Shop_filters":"Shop","Any size_filters":"Any size","Size_filters":"Size","order_by_filters":"Order by","Any source_filters":"Any source","Source_filters":"Any source","Any type_filters":"Any type","Type_filters":"Type","Hide_filters":"Hide","OK_filters":"OK","all_filters":"all","Relevance_filters":"Relevance","Views_filters":"Views","Most recent_filters":"Most recent","Price : Low to High_filters":"Price : Low to High","Price : High to Low_filters":"Price : High to Low","All sizes_filters":"All sizes","Small_filters":"Small","Medium_filters":"Medium","Large_filters":"Large","Wallpaper_filters":"Wallpaper","Wide wallpaper_filters":"Wide wallpaper","All_filters":"All","Public domain_filters":"Public domain","Non-commercial reproduction and sharing_filters":"Non-commercial reproduction and sharing","Reproduction and sharing_filters":"Reproduction and sharing","Non-commercial reproduction, sharing and modification_filters":"Non-commercial reproduction, sharing and modification","Reproduction, sharing and modification_filters":"Reproduction, sharing and modification","Color only_filters":"Color only","Monochrome_filters":"Monochrome","black_filters":"Black","blue_filters":"Blue","brown_filters":"Brown","gray_filters":"Gray","green_filters":"Green","orange_filters":"Orange","pink_filters":"Pink","purple_filters":"Purple","red_filters":"Red","teal_filters":"Teal","white_filters":"White","yellow_filters":"Yellow","Animated Gif_filters":"Animated Gif","Photograph_filters":"Photograph","Transparent_filters":"Transparent","All time_filters":"All time","Less than 1 hour ago_filters":"Less than 1 hour ago","Past 24 hours_filters":"Past 24 hours","Past week_filters":"Past week","Past month_filters":"Past month","Low price_filters":"Low price","High price_filters":"High price","Newest first_filters":"Newest first","Oldest first_filters":"Oldest first","Show the next {number} filter_brand_filter":"Show the next {{number}}","Show the next {number} filter_brand_filter_plural":"Show the next {{number}}","Show less brands_filter":"Show less brands","All_filter":"All","Show the next {number} filter_shop_filter":"Show the next {{number}}","Show the next {number} filter_shop_filter_plural":"Show the next {{number}}","Show less shops_filter":"Show less shops","Min price {min}€_filter":"Min price {{min}}€","Max price {max}€_filter":"Max price {{max}}€","An error occurred in removing filters_filter":"An error occurred in removing filters","Filters_filter":"Filters","Back_filter":"Back","Validate_filter":"Validate","Filter_filter":"Filter","Reinitialize filters_filter":"Reinitialize filters","Reinitialize_filter":"Reinitialize","Show the products_filter":"Show the products","Filters_button":"Filters"},"footer":{"label":"Footer","tagline":"The search engine that values you as a user, not as a product","results":{"title":"Results","links":{"ranking":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fclassement\u002F","label":"Ranking"},"report":{"label":"Report"}},"copyright":"© 2025 Qwant. All rights reserved."},"privacy":{"label":"Privacy"},"legal":{"title":"Legal","links":{"termsOfService":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fqwant-search\u002F","label":"Terms of Service"},"legalNotices":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fmentions-legales\u002F","label":"Legal notices"}}},"company":{"title":"Companies","links":{"qwantatwork":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fqwantatwork\u002F","label":"Qwant @ Work"},"blog":{"href":"https:\u002F\u002Fbetterweb.qwant.com\u002Fen\u002F","label":"Better Web"},"jobs":{"href":"https:\u002F\u002Fwww.welcometothejungle.com\u002Fen\u002Fcompanies\u002Fqwant","label":"Join Us"},"press":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Fpresse\u002F","label":"Press"},"contact":{"href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Fcontact\u002F","label":"Contact Us"}}},"helpCenter":{"href":"https:\u002F\u002Fhelp.qwant.com\u002Fen","label":"Help Center"},"apps":{"android":"Android App","ios":"iOS App"},"whyQwant":{"label":"Why Qwant?","href":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002F"},"products":{"qwantExtension":"Qwant Extension","maps":"Maps","search":"Search","junior":"Junior"},"aboutQwant":"About Qwant","followUs":"Follow us","services":"Services","makeDefaultSearchEngine":"Make Qwant my default search engine","settings":"Settings","cookieManagement":"Cookie management","discoverQwant":{"label":"Discover the service","href":{"multi":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002F"}},"socials":{"instagram":"Instagram","facebook":"Facebook","twitter":"X (ex-Twitter)","tiktok":"TikTok","linkedin":"LinkedIn","discord":"Discord"}},"images":{"websitePreview":"View website","openFullscreen":"Open image in fullscreen","tooltip":"Qwant is careful to keep your searches private (\u003Ca href=\"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fconfidentialite\u002F\"\u003ELearn more\u003C\u002Fa\u003E). These ads from the Microsoft Advertising network match the keywords entered. Clicks are subject to the \u003Ca href=\"http:\u002F\u002Fgo.microsoft.com\u002Ffwlink\u002F?LinkId=248681\"\u003EMicrosoft Privacy Statement\u003C\u002Fa\u003E"},"junior":{},"modules":{"pushDesktop":{"title":"Qwant extension","content":"Install Qwant as your default search engine.","button":"Add to {{name}}"}},"robots":{"title":"Information","introduction":"A high amount of connections have been detected from your location and you have been blocked.","introduction2":"Please, validate the anti-robot below to be allowed access to the website.","instruction":"Click on the only unique symbol.","description":"The use of a Virtual Private Network (VPN), a browser plugin or another program might be related to the current blocking. You might try uninstalling the VPN from your computer or network and see if that makes a difference.\u003Cbr\u003E Some VPNs send traffic that violates the law or websites' terms of service. When the abuse to Qwant's network stops, we automatically stop blocking the IP that were sending the bad traffic.\u003Cbr\u003E If you are unsure about a malicious usage of your network, \u003Ca href=\"http:\u002F\u002Fadios-hola.org\u002F\"\u003Ethis website\u003C\u002Fa\u003E (english only) may be a good reference to start with.","error":"You haven't clicked on the right symbol.","serverError":"Server error, cannot display the captcha.","refresh":"Please check your connection or \u003C0\u003E refresh page \u003C\u002F0\u003E."},"ia":{"showMore":"Show more","allocineWebsite":"allocine.fr","readMore":"Read more","readMoreOn":"Read more on {{source}}","seeAll":"See all","contributeWiki":"How to contribute?","wikipediaContributeUrl":"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWikipedia:Contributing_to_Wikipedia","wikidataContributeUrl":"https:\u002F\u002Fwww.wikidata.org\u002Fwiki\u002FWikidata:Contribute\u002Fen","newsOn":"News","productsBuy":"Buy \u003C0\u003E{{name}}\u003C\u002F0\u003E","ads":"Ads","fromTo":"{{start}} to {{end}}","birthInformation":"Date and place of birth","nationality":"Nationality","filmography":"Filmography","from":"From","minutesShort":"min","duration":"Duration","watchMovie":"Watch this movie","durationText":"{{hours}}h {{min}}min","seasons":"Seasons","otherSeasons":"Other seasons","episodes":"Episodes","actors":"Actors","actorsOfSeason":"Actors of season {{n}}","episodesOfSeason":"Episodes of season {{n}}","seeAllEpisodes":"See all the episodes of season {{n}}","seasonNumber":"Season {{n}}","seasonNumberOf":"Season {{n}} of {{name}}","videoOfSeason":"Video of season {{n}}","press":"Press","audience":"Audience","ratings":"notes","video":"Video","startingPrice":"Starting at {{price}}","watch":"Watch","answerBy":"Answer by","status":{"progress":"In progress","cancelled":"Cancelled","toCome":"To come"},"movie":{"genres":{"ACTION":"Action","ADVENTURE":"Adventure","ANIMATION":"Animation","BIOPIC":"Biopic","BOLLYWOOD":"Bollywood","CARTOON":"Cartoon","CLASSIC":"Classic","COMEDY":"Comedy","COMEDY_DRAMA":"Comedy\u002FDrama","CONCERT":"Concert","DETECTIVE":"Detective","DIVERS":"Misc","DOCUMENTARY":"Documentary","DRAMA":"Drama","EROTIC":"Erotic","EXPERIMENTAL":"Experimental","FAMILY":"Family","FANTASY":"Fantasy","HISTORICAL":"Historical","HISTORICAL_EPIC":"Historical Epic","HORROR":"Horror","JUDICIAL":"Judicial","KOREAN_DRAMA":"Korean drama","MARTIAL_ARTS":"Martial arts","MEDICAL":"Medical","MOVIE_NIGHT":"Movie night","MUSIC":"Music","MUSICAL":"Musical","OPERA":"Opera","ROMANCE":"Romance","SCIENCE_FICTION":"Science fiction","PERFORMANCE":"Performance","SOAP":"Soap opera","SPORT_EVENT":"Sport event","SPY":"Spy","THRILLER":"Thriller","WARMOVIE":"Warmovie","WEB_SERIES":"Web series","WESTERN":"Western"}},"shopping":{"allColors":"All colors"},"daysShort":{"monday":"Mon.","tuesday":"Tue.","wednesday":"Wed.","thursday":"Thu.","friday":"Fri.","saturday":"Sat.","sunday":"Sun."},"daysLong":{"monday":"Monday","tuesday":"Tuesday","wednesday":"Wednesday","thursday":"Thursday","friday":"Friday","saturday":"Saturday","sunday":"Sunday"},"weather":{"precipitation":"Precipitation","humidity":"Humidity","wind":"Wind","temperatures":"Temperatures","precipitations":"Precipitations","type":{"CLOUDY":"Cloudy","FOGGY":"Foggy","HAIL":"Hail","NIGHT":"Night","NIGHT_CLOUDY":"Cloudy night","RAINBOW":"Rainbow","RAINY":"Rainy","SNOW":"Snowy","SUNNY":"Sunny","PARTLY_CLOUDY":"Partly cloudy","THUNDER":"Thunder","WINDY":"Windy"}},"maps":{"directions":"Directions","website":"Website","open":"Open","closed":"Closed","inPartnershipWith":"in partnership with","revealPhoneNumber":"Show the number","review":"{{count}} review","review_plural":"{{count}} reviews","seeMoreResults":"See more results"},"flight":{"titleWithLocations":"Flights {{from}} - {{to}}","allAirports":"All","cheapFlight":"Find a cheap flight","roundTrip":"Round trip","oneWay":"One way","departure":"Depart","return":"Return","searchFlight":"Search for a flight","moreFlight":"Display more results","from":"From","to":"To","type":"Type","direct":"Direct","stop":"Stop(s)","priceFrom":"Starting from {{price}}","multiCarriers":"Multiple airlines","anywhere":"Anywhere","searchAnywhere":"Search everywhere!","searchAnywhereDescription":"In case you don't know where to go","currency":"Prices shown in {{currency}}","reset":"Reset","submit":"Ok","flags":{"cheapest":"Low price","top-travel":"Top destination"},"where":"Where are we going?","tooltip":"Qwant takes care to preserve the confidentiality of your searches  (\u003Ca href=\"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fconfidentialite\u002F\"\u003ELearn more\u003C\u002Fa\u003E). \nThese results from the Skyscanner network match the keywords entered. Some prices are cached. The clicks are subject to \u003Ca href=\"https:\u002F\u002Fwww.skyscanner.fr\u002Fmedia\u002Fpolitique-confidentialite\"\u003ESkyscanner's privacy policy\u003C\u002Fa\u003E."},"technicalHelp":{"copy":"The code was copied to your clipboard.","example":"Example"},"productsAds":{"refurbished":"Refurbished"},"music":{"discography":"Discography","officialWebsite":"Official website","discoverArtist":"Discover {{artist}}"},"recipes":{"recipes":"Recipes of \u003C0\u003E{{value}}\u003C\u002F0\u003E","quantityInterval":"{{a}} to {{b}}"},"overline":{"new":"New"},"translate":{"placeholderSource":"Enter text","placeholderTarget":"Translation","warning":"The first {{limit}} characters were translated.","translatedBy":"Translated by","sourceSelectTitle":"Source language","targetSelectTitle":"Target language","detected":"{{lang}} (detected language)","automaticLabel":"Automatic detection","swapLabel":"Swap languages","clearSourceLabel":"Clear source text","thanks":"Thanks","deeplPro":"DeepL Pro","translateMoreOnDeepL":"Translate more on DeepL","deeplProSubtext":"Fast and secure translations","tryDeeplPro":"Try DeepL Pro for free","language":{"CS":"Czech","DA":"Danish","DE":"German","EL":"Greek","EN":"English","ES":"Spanish","ET":"Estonian","FI":"Finnish","FR":"French","HU":"Hungarian","IT":"Italian","JA":"Japanese","LT":"Lithuanian","LV":"Latvian","NL":"Dutch","PL":"Polish","PT":"Portuguese","RO":"Romanian","RU":"Russian","SK":"Slovak","SL":"Slovenian","SV":"Swedish","ZH":"Chinese"}},"olympics":{"games_winter":"Winter Games 2022","location_beijing":"Beijing, China","title":"Medals ranking","countries":"Countries","athletes":"Athletes","seemore":"See more on","date":"February 4 - February 20","cc3":"Content subject to CC-BY-SA 3.0 license. Source:","cc3_link":"https:\u002F\u002Fcreativecommons.org\u002Flicenses\u002Fby-sa\u002F3.0\u002F","para_games_winter":"Winter Paralympics Games 2022","para_date":"March 4 - March 13"},"knowledge":{"website":"Website","related":{"french_president":"French Presidents","explore":"Explore"}},"snapearth":{"longitude":"Longitude","latitude":"Latitude","min":"Min","max":"Max","seeMore":"See more"},"howTo":{"seemoreSteps":"Discover the {{count}} steps","seemoreMethods":"Discover the {{count}} methods","seemoreFallback":"Discover the steps","videos":"Tutorials on the same subject"},"worldcup":{"title":"FIFA World Cup","subtitle":"2022 Season","matches":"Matches","rankings":"Groups","footerMention":"All times are Romance Standard Time","ongoing":"Ongoing","halftime":"Half-time","location":"Qatar","worldcup":"World cup","share":"Share","upcomingGame":"TBD","lastUpdate":"Last updated:","rankingMj":"MP","rankingG":"W","rankingN":"D","rankingP":"L","rankingBp":"GF","rankingBc":"GA","rankingDb":"GD","rankingPts":"PTS","group":"Group","eight":"Round of 16","quarter":"Quarter-final","semi":"Semi-final","third":"Third place match","final":"Final","AU":"Australia","CA":"Canada","CH":"Switzerland","CR":"Costa Rica","DE":"Germany","DK":"Denmark","EC":"Ecuador","GH":"Ghana","IR":"Iran","MA":"Morocco","MX":"Mexico","RS":"Serbia","SA":"Saudi Arabia","SN":"Senegal","US":"USA","UY":"Uruguay","AR":"Argentina","BE":"Belgium","BR":"Brazil","CM":"Cameroon","ES":"Spain","FR":"France","GB-ENG":"England","GB-WLS":"Wales","HR":"Croatia","JP":"Japan","KR":"Republic of Korea","NL":"Netherlands","PL":"Poland","PT":"Portugal","QA":"Qatar","TN":"Tunisia","1c-ou-2d":"1C or 2D","1d-ou-2c":"1D or 2C","1g-ou-2h":"1G or 2H","1h-ou-2g":"1H or 2G","2e-groupe-a":"2nd Group A","2e-groupe-b":"2nd Group B","2e-groupe-c":"2nd Group C","2e-groupe-d":"2nd Group D","2e-groupe-e":"2nd Group E","2e-groupe-f":"2nd Group F","2e-groupe-g":"2nd Group G","2e-groupe-h":"2nd Group H","vainqueur-dm2":"Winner SF2","vainqueur-q2":"Winner Q2","vainqueur-q4":"Winner Q4","1a-ou-2b":"1A or 2B","1b-ou-2a":"1B or 2A","1e-ou-2f":"1E or 2F","1er-groupe-a":"1st Group A","1er-groupe-b":"1st Group B","1er-groupe-c":"1st Group C","1er-groupe-d":"1st Group D","1er-groupe-e":"1st Group E","1er-groupe-f":"1st Group F","1er-groupe-g":"1st Group G","1er-groupe-h":"1st Group H","1f-ou-2e":"1F or 2E","vainqueur-dm1":"Winner SF1","vainqueur-q1":"Winner Q1","vainqueur-q3":"Winner Q3","perdant-dm1":"Looser SM1","perdant-dm2":"Looser SM2","sourceUrl":"https:\u002F\u002Fwww.lequipe.fr\u002F","sourceText":"lequipe.fr"},"autopromo":{"pushMobileApps":{"adoptMobileApp":"Download Qwant app!"},"extension":{"title":"Surf safely on all websites with Qwant.com","titleMiniDesktop":"Surf safely on all websites with Qwant.com","titleMobile":"Switch to Qwant on Safari","addQwantSafari":"Add Qwant to Safari","addQwant":"Add Qwant to {{browser}}","addQwantItsFree":"Add Qwant to {{browser}} - It's free!","defaultSearchEngine":"Qwant as default search engine.","lessCookies":"Fewer cookies, no more intrusive banners!","trackerBlocking":"Fewer trackers, finally free!"},"firefox":{"try":"Make your choice","contribute":"Use Qwant on Firefox"},"PrivacyDay":{"title":"Protégez vos données personnelles !\u003Cbr\u002F\u003EOui mais comment ?","button":"Téléchargez notre guide"}},"llm":{"summary":"Summary by Qwant's artificial intelligence","placeholderFeedback":"Share your opinion","disableAnswers":"Disable Artificial Intelligence Answers","titleFeedback":"Give your feedback","dangerous":"Inappropriate\u002Fdangerous","inexact":"This is incorrect","other":"Other","sendButton":"Send","alertSuccess":"Thank you, your feedback has been received","errorSummary":"For technical reasons, we are unable to summarize the content of this page. Thank you for your understanding","llmPromoTag":"Instant answer","llmPromoText":"The new service to know everything about","llmPromoTry":"Try it","llmPromoClose":"No thanks","tooltipSummarizer":"Summarize with Qwant's\u003Cbr\u002F\u003E artificial intelligence","ctaSummarizer":"See the site","viewDetailedAnswer":"View detailed answer","readingSources":"Reading sources","signUpBanner":{"title":"Right now: Enjoy instant answers with Qwant's artificial intelligence!"},"sources":"Sources","dday":{"promo":{"labelCta":"Activate unlimited AI","labelCta2":"Discover","description":"Save time with unlimited access to Qwant's artificial intelligence summaries"},"cta":{"labelCta":"Trigger AI","description":"Save time with an artificial intelligence summary from Qwant"}}},"google":{"title":"Are you leaving us already?","subtitle":"On Qwant:","argument1":"You save time with fast, relevant answers generated by artificial intelligence.","argument2":"Your searches are not stored, your personal data is not sold.","cta1":"I search on Qwant","cta2":"I prefer to go to Google"}},"settings":{"settings":"Settings","interface":"Interface","homeTrends":"News on homepage","homeTiles":"Sponsored shortcuts on Homepage","smartNews":"Display news next to web results","outgoingTab":"Open outgoing links in a new tab","displaySiteIcons":"Display site icons","link":"Every settings available in a link","linkDescription":"Do you sometimes delete your cookies? Use this link as your browser's start page, or drag it to your favorites:","searchResults":"Search results","searchLanguage":"Language","saveSuccessfully":"Settings updated successfully","localSave":"Local backup","drawers":{"interface":"Interface language","locale":"Region"},"helloUsername":"\u003C0\u003E{{username}}\u003C\u002F0\u003E","personalizeYourSearches":"Personalize your searches","manageAccount":"Manage my account","llm":{"placeholder":"Flash Responses","label":"Generated by artificial intelligence","title":{"0":"Deactivated","1":"Activated by Qwant","2":"Always activated"},"description":{"0":"The flash response will not be displayed","1":"The flash response will be displayed only when relevant","2":"The flash response will always be displayed"}},"theme":{"placeholder":"Mode","label":"Appearance","default":"Automatic","light":"Light","dark":"Dark"},"safeSearch":{"placeholder":"Filter","label":"Filter adult content","title":{"0":"None","1":"Moderate","2":"Strict"},"description":{"0":"Do not filter adult content","1":"No adult images or videos","2":"No adult texts, images or videos"}},"privacy":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fconfidentialite\u002Fcookie-policy\u002F"},"report":{"title":"Report content","desc_de":"In Germany, Qwant displays the search results as provided in real-time by our partner Microsoft Bing. \nIn case of illicit content, please notify the result to Microsoft so they can review and delete it.","button_label_de":"Report to Microsoft"},"homepage":{"hero":{"switch":"Go to the homepage","no_search_tracking":"Zero tracking of your searches","no_ad_tracking":"Zero advertising tracking","no_data_sell":"Zero sale of your personal data","title":"The search engine that values you as a user, not as a product","title_branding":"The search engine that doesn't know anything about you, and that changes everything!","aria_scrolling_button":"Next section","title_carousel":"\u003C1\u003E90\u003C\u002F1\u003E \u003C0\u003Eof anonymous users satisfied\u003C\u002F0\u003E\u003C3\u003E*\u003C\u002F3\u003E","title_credits":"*regular anonymous surveys on Qwant.com"},"toggle":{"qwant":"What is Qwant?","news":"News of the day"},"skin":{"partnership":"Partnership","ad":"Ad"},"sections":{"section_clickmachine_description":"Turning our users into click machines goes against our ethics. That's why, at Qwant, we don't track advertising or analyze your personal data. Qwant puts aside the algorithms that pre-format the web and turn you into a commodity.","section_tracker_description":"Have you noticed that there is no advertising cookie banner on Qwant? Does that surprise you? We think that's normal. We do not sell your data, it belongs to you!","section_ads_description":"Simply to finance our model, as a European search engine that does not track or store your data and intends to remain free. In short, contextual ads are the same for everyone, we don't track you while you search.","section_clickmachine_title":"\u003C0\u003EBeing informed should not turn you into a\u003C\u002F0\u003E \u003C1\u003EClick machine\u003C\u002F1\u003E","section_clickmachine_image_alt":"Being informed should not turn you into a \"Click machine\"","section_tracker_title":"\u003C0\u003EHere,\u003C\u002F0\u003E \u003C1\u003Eno trackers\u003C\u002F1\u003E \u003C0\u003Eor advertising\u003C\u002F0\u003E \u003C2\u003Ecookies\u003C\u002F2\u003E","section_tracker_image_alt":"Here, no trackers or advertising cookies","section_ads_title":"\u003C0\u003EBut why is there\u003C\u002F0\u003E \u003C1\u003Eadvertising\u003C\u002F1\u003E \u003C0\u003Eon\u003C\u002F0\u003E \u003C2\u003EQwant\u003C\u002F2\u003E\u003C0\u003E?\u003C\u002F0\u003E","section_ads_image_alt":"But why is there advertising on Qwant?","section_searchWithQwant_link":"Search with Qwant","section_searchWithQwant_link_url":"https:\u002F\u002Fqwant.com\u002F","section_knowMore_link":"Know more","section_knowMore_link_url":"https:\u002F\u002Fabout.qwant.com\u002F","section_extension_title_app":"\u003C0\u003EDownload\u003C\u002F0\u003E \u003C1\u003EQwant\u003C\u002F1\u003E","section_extension_title_extension":"\u003C0\u003EDownload\u003C\u002F0\u003E \u003C1\u003EQwant\u003C\u002F1\u003E","brands_checklist":{"title":"\u003C0\u003EDiscover our check-list to keep you private life\u003C\u002F0\u003E \u003C1\u003Ereally private\u003C\u002F1\u003E","discover":"Discover","qwant_search":{"text":"We do not care about your personal data, nor your searches.","name":"Qwant Search","link":"https:\u002F\u002Fwww.qwant.com"},"qwant_maps":{"text":"Get around anywhere untracked.","name":"Qwant Maps","link":"https:\u002F\u002Fmap.qwant.com"},"qwant_junior":{"text":"Let your children discover the Internet safely.","name":"Qwant Junior","link":"https:\u002F\u002Fwww.qwantjunior.com"},"qwant_VIPrivacy":{"text":"Browse the web without trackers on any websites.","name":"Qwant VIPrivacy","link":"https:\u002F\u002Fabout.qwant.com\u002Fextension\u002F"}},"brands_userReviews":{"0":{"text":"« Finally, some digital peace of mind thanks to this application. And it's French ! Thank you very much. »","date":"User of September 24, 2022"},"1":{"text":"« After years of wanting to get rid of Google, I finally did and I must say well done!  The results are relevant, there are plenty of options and the search engine interface is well-designed across all devices. »","date":"User of February 4, 2022"},"2":{"text":"« I have been using Qwant for many year and it is great. »","date":"User of October 7, 2022"},"3":{"text":"« Great search engine, I recommend it to all who want to get away from censored results and who care about protecting their personal data. »","date":"User of July 30, 2022"},"4":{"text":"« Excellent French search engine. »","date":"User of October 7, 2022"},"5":{"text":"« I have used Qwant for many years to protect my personal data. »","date":"User of October 13, 2022"},"title":"\u003C1\u003E90\u003C\u002F1\u003E \u003C0\u003Esatisfied users\u003C\u002F0\u003E\u003C3\u003E*\u003C\u002F3\u003E","credits":"*regular anonymous surveys on Qwant.com"},"branding":{"title":"Always \u003C0\u003Econfident and secure with\u003C\u002F0\u003E Qwant","knowMore":"Know more","joinFree":"Sign up for free","enjoyNewQwant":"Enjoy the new Qwant","noConservation":"Qwant does not retain your search data","noSell":"No selling of your personal data","madeInFrance":"French search engine, hosted in Europe"}},"tiles":{"sponsored":"Sponsored","sponsoredTooltip":"This sponsored content is published without any sending of personal data to the advertiser. Hyperlinks are provided by our \u003Cstrong\u003Eadvertising partner\u003C\u002Fstrong\u003E whose \u003Cstrong\u003Eprivacy policy\u003C\u002Fstrong\u003E is applied when you click on them.","sponsoredShortcuts":"Sponsored shortcuts"},"trends":{"moreNews":"Discover more news","sponsored":"Sponsored","altAutoPromo":"A person with a panda mask looking at his phone, under which the values of Qwant.com are presented. A search engine that does not track your searches, does not sell your personal data, does not target your ads.","altAutoPromoOpera":"Reverse, A person with a panda mask looking at his phone, under which the values of Qwant.com are presented. A search engine that does not track your searches, does not sell your personal data, does not target your ads."},"addon":{"banner":"\u003Cstrong\u003EQwant VIP\u003C\u002Fstrong\u003Erivacy protects you from trackers and makes Qwant your default search engine","learn_more":"Learn more","add_qwant":"Add Qwant to {{browser}}","add_qwant_for_safari":"Add Qwant for Safari","make_default":"Make Qwant my default search engine","available_soon":"Available soon on {{browser}}","already_available_on_store":"Already available on \u003Ca href='https:\u002F\u002Faddons.mozilla.org\u002Fen-US\u002Ffirefox\u002Faddon\u002Fqwantcom-for-firefox' alt='Qwant VIPrivacy for Firefox' target='_blank' rel='noopener'\u003EFirefox\u003C\u002Fa\u003E and \u003Ca href='https:\u002F\u002Fchrome.google.com\u002Fwebstore\u002Fdetail\u002Fqwant-for-chrome\u002Fhnlkiofnhhoahaiimdicppgemmmomijo\u002F' alt='Qwant VIPrivacy for Chrome' target='_blank' rel='noopener'\u003EChrome\u003C\u002Fa\u003E"},"mobile_app":{"title":"Switch to Qwant app","download":"Use the application","reviews":"+ {{total}} reviews"},"products":{"mobile_apps":"Our mobile apps","search":"Search","search_text":"The search engine that doesn't know anything about you, and that changes everything: zero tracking of your searches, zero personal data use, zero targeted advertising.","maps":"Maps","maps_text":"The map service that finds the right addresses and guides you around without tracking you.","junior":"Junior","junior_text":"The only secure search engine designed for children's learning and their parents' peace of mind."},"flag":{"branding":{"brand_phrase":"The search engine that respects your privacy"}},"welcome":{"alt_test":"Arrow pointing towards searchbar saying \"Test me\"","reasons":"\u003C1\u003E3 reasons\u003C\u002F1\u003E \u003C0\u003Eto use Qwant\u003C\u002F0\u003E","suggest":{"title":"Examples of searches","fake1":"TikTok digital majority","fake2":"The shortcomings of ChatGPT","fake3":"Pension reform strikes","fake3_highlighted":"Pension reform"},"alt_try":"Arrow pointing towards the above list and et says try these searches"},"rtb":{"fr":{"frenchMade":"Developed in \u003C0\u003EFrance\u003C\u002F0\u003E by our teams","hostedInFrance":"Hosted in France in \u003C0\u003Eour data centers\u003C0\u003E","privacy":"\u003C0\u003ENo. 1 in Europe\u003C\u002F0\u003E for privacy"},"yellow":{"data":"\u003C0\u003EHundreds\u003C\u002F0\u003E of data collected per day on you","profits":"\u003C0\u003EBillions\u003C\u002F0\u003E in profits made by you for others","cookies":"\u003C0\u003EHundreds\u003C\u002F0\u003E of cookie banners to refuse","tooltipCookies":"With Qwant VIPrivacy navigator extension"}},"homepageCards":{"reward":{"title":"Earn up to €10 per week by using Qwant","description":"Do research, earn points and move up the leaderboard to receive rewards every week!","buttonLabel":"Join"},"ia":{"title":"Flash Response saves you time.","description":"Qwant finds the right information among the most relevant web pages. You get your answer instantly!","tagLabel":"Free","buttonLabel":"Let's try"}},"dday":{"title":"Enjoy instant answers with Qwant's artificial intelligence"}},"reward":{"drawers":{"promo":{"title":"Loyalty program","title2":"Earn up to €10 per week by using Qwant!","claim1":"Do your searches on Qwant","claim2":"Earn points and climb the rankings","claim3":"Over 1,300 winners receive up to €10 each week!","cta":"Sign up for free","info":"How does it work?"},"ranking":{"title":"Loyalty program","walletLinkTitle":"Wallet","walletLinkSubtitle":"According to your ranking, receive your reward at the end of the week.","walletLinkSubtitleAttributionPending":"This week's new earnings are being allocated, so please be patient","countdownTitle":"End in","countdownUnitDays":"d","countdownUnitHours":"h","countdownUnitMinutes":"min","rankingTitle":"Weekly ranking","rankingDescription":"Use Qwant every day to move up the rankings. Over 1,300 winners each week.","rankingWinningZone":"Winner's zone"},"activity":{"title":"Today's Activities","bonusMobile":"Search on the Qwant mobile app","bonusDesktop":"Search on the web","days":{"title":"7 consecutive days bonus","text":"Use Qwant regularly to boost your ranking!","day":{"large":{"monday":"Mon.","tuesday":"Tue.","wednesday":"Wed.","thursday":"Thu.","friday":"Fri.","saturday":"Sat.","sunday":"Sun."},"small":{"monday":"M","tuesday":"T","wednesday":"W","thursday":"T","friday":"F","saturday":"S","sunday":"S"}}}},"wallet":{"title":"Wallet","cta":"Transfer","text":"The minimum amount for making a transfer is €5","textWithExpiry":"As soon as your wallet reaches €5, each earning expires after 3 months.","attributionPending":"New earnings in the process of being awarded","emptyOperation":"No transactions conducted at the moment.","history":{"title":"List of operations","credit":"Rewards earned","debit":"Debit","expiry":"Expiry of more than 3 months' earnings"}}},"alerts":{"welcome":"✅ Loyalty program activated\u003Cbr\u002F\u003E💰🏆 It’s your turn! Use Qwant to move up the rankings","welcomeTitle":"Congratulations on your registration","firstDay":"First activity day of the week... and we're off!","newGame":"A new week begins, we reset the counters and here we go again!","rankingRise":"Congratulation, you're moving up in the ranking! You're on the right track!","maxActivity":"Congratulations, you've used Qwant for 7 consecutive days. Your daily points are now doubled!","activityStreak":"{{count}}\u003Cspan\u003Eth\u003C\u002Fspan\u003E consecutive day on Qwant. Congrats, keep it up!","activityStreakEnd":"Ouch, you didn't come yesterday. You've lost your bonus days in a row.","winningZone":"Wow, you've just entered the winner's zone. Don't let up until the end of the week."},"popovers":{"unlogged":{"title":"Loyalty program","desc":"Earn up to €10 per week by using Qwant !","cta":"Start"}},"onboarding":{"form":{"title":"Join the Qwant loyalty program","text":"Your username","checkboxProgram":"I accept the \u003Ca href='https:\u002F\u002Fabout.qwant.com\u002Flegal\u002Fqwant-search\u002Fconditions-particulieres-programme-de-fidelite' target='_blank'\u003Eterms of use\u003C\u002Fa\u003E and the \u003Ca href='https:\u002F\u002Fabout.qwant.com\u002Flegal\u002Fpolitique-de-confidentialite-programme-de-fidelite\u002F' target='_blank'\u003Eprivacy policy\u003C\u002Fa\u003E","checkboxEmails":"I agree to receive emails from the loyalty program to keep me informed of my rankings and earnings","checkboxAdult":"I certify that I am over 18","cta1":"Validate","cta2":"I do not wish to participate","avatarTitle":"Choose you avatar","avatarbgColorTitle":"Choose a background color","avatarCancel":"Cancel","avatarConfirm":"Confirm"},"privacyAgreement":{"title":"Just one more step to go!","content":"To participate in the loyalty program, you must accept all Qwant's privacy settings.","adblocker":"If you use an AdBlocker, you won’t be able to see the privacy settings. Turn it off to proceed.","openCMP":"Access privacy settings","modalTitle":"Refusal of privacy settings","modalBody":"Are you sure you want to refuse all Qwant privacy settings? If you confirm, you won't be able to join the loyalty program.","modalAccept":"I've changed my mind, and I accept","modalRefuse":"I confirm my refusal","needHelp":"Need help?"},"privacyAgreementFinal":{"title":"Sorry, you cannot join the loyalty program","content":"Without accepting Qwant's privacy settings, you cannot participate in the loyalty program.","openCMP":"Return to privacy settings","backToQwant":"Back to Qwant.com"},"info":{"title":"\u003Cspan\u003EEarn up to €10\u003C\u002Fspan\u003E per week by using Qwant !","text":"It's simple:","paragraph1":"Do your searches on Qwant","paragraph2":"Earn points and climb the rankings","paragraph3":"Over 1,300 winners receive up to €10 each week!"}},"transfer":{"title":"Transfer to your bank account","description":"The transfer to your bank account may take 1 to 2 working days.","tax":"Bank charges of €0.70 apply to this transaction.","start":"Start"},"transferUserInfo":{"title":"Fill in your bank information","holder":"Account holder","iban":"IBAN","submit":"Submit"},"transferSuccess":{"title":"Transfer completed","alertTitle":"The money transfer has been successfully completed.","alertMessage":"The usual transfer time is between 1 and 2 working days.","backToQwantCom":"Back to Qwant.com"},"transferError":{"title":"Transfer failed","alertDefaultTitle":"The money transfer failed.","alertDefaultMessage":"Please try again in a few minutes","alertIbanAlreadyUsedMessage":"IBAN already used by another user","alertAmountTooLowMessage":"Amount too low","alertWalletLockedMessage":"An error has occurred. Please try again later.","alertIbanInvalidMessage":"Invalid IBAN","alertBeneficiaryInvalidMessage":"Invalid beneficiary","alertBeneficiaryEmptyMessage":"Invalid beneficiary","retry":"Retry"},"disableAdblockerModal":{"title":"You do not manage to display the privacy settings?","step1":"Disable your AdBlocker if you have one.","step2":"For Firefox users, check that the browser's privacy settings are set to \u003Cb\u003EStandard\u003C\u002Fb\u003E to display the consent window.","closeButton":"Close"},"landing":{"hero":{"title":"Win up to 10€ per week!","description":"Join the Qwant loyalty program and earn money thanks to your searches on the internet.","statistic":{"participants":{"number":"20,000+","description":"Participants"},"winners":{"number":"2,000+","description":"Winners"}}},"reasons":{"1":{"title":"An \u003C0\u003Einnovative\u003C\u002F0\u003E search engine","description":"Qwant, this is the innovative search engine that offers you a more pleasant and fluid search experience. By creating an account, you benefit from Qwant at 100% thanks to our AI and our \"Summary\" feature, which offers you a summary of a web page in one click!","subdescription":"Our AI answers all your questions and all your searches to accompany you every day."},"2":{"title":"No search \u003C0\u003Ehistory\u003C\u002F0\u003E","description":"With the Qwant search engine, there is no search history or sale of your personal data! Your searches belong to you and we don't know your name, gender, or what you're interested in on the web.","subdescription":"Make the choice of a more discreet and respectful navigation of your online privacy."}},"steps":{"1":{"description":"Make your daily searches on Qwant.com"},"2":{"description":"Accumulate points and climb the ranking"},"3":{"description":"Win up to 10 € per week"}},"message":{"title":"Are you ready to win money thanks to your searches on the internet?","button":"Register"},"faq":{"1":{"title":"What is Qwant?","description":"Qwant is a 100% free French search engine that allows you to earn money through your internet searches. It also incorporates artificial intelligence modules for a more relevant search experience, all without keeping your search history!"},"2":{"title":"How do I participate in the loyalty program?","description":"To participate in the loyalty program, it is enough simply to create a Qwant account and register for the program. Once your pseudo and your avatar are chosen, you will just have to make your searches."},"3":{"title":"Earning money through my searches, how does it work?","description":"Once your account is created, you can perform your searches on Qwant to earn points. These points allow you to climb the ranking and at the end of each week, the first in the ranking win money. The higher you finish in the ranking, the greater your reward. And that, every week!"},"4":{"title":"How do I withdraw my money?","description":"To withdraw your money, you must have reached a minimum of 5 euros in your Qwant wallet. Then you will be able to transfer this money from your Qwant wallet to your bank account in a totally secure way. Note, once the sum of 5 euros is reached, you have 3 months to withdraw your money before it expires."},"5":{"title":"Is it secure?","description":"The respect of personal data is a deep and strong commitment at Qwant. We work with our French banking partner MemoBank to ensure the highest security of all bank transactions."},"title":"Have a question? We've got answers!"}}},"signup":{"form":{"title":"Sign up","googleCta":"Sign up with Google","appleCta":"Sign up with Apple","or":"or","emailLabel":"Email","emailPlaceholder":"mary@example.com","passwordLabel":"Password","passwordHelperMessage":"Minimum 10 characters with at least one uppercase letter, one lowercase letter, one number and one special character","checkboxTerms":"I have read and accept the \u003Ca href='https:\u002F\u002Fabout.qwant.com\u002Flegal\u002Fqwant-search' target='_blank'\u003Eterms and conditions\u003C\u002Fa\u003E and the \u003Ca href='https:\u002F\u002Fabout.qwant.com\u002Flegal\u002Fconfidentialite' target='_blank'\u003Eprivacy policy\u003C\u002Fa\u003E","checkboxNewsletter":"I agree to receive the newsletter","submit":"Create an account","submitLanding":"Create an account with an email","connectionMessage":"Already have an account?","connectionLink":"Log in"}},"videos":{"copyright":"This video may be subject to copyright.","personal_data_title":"Information relating to your personal data","personal_data_description":"This video is not hosted by Qwant, which is not responsible for it's content or the personal data that may be used. The host of the video {{brand}} may, when it is displayed, access certain data such as your IP address and cookies, depending on its own \u003C0\u003Eprivacy policies\u003C\u002F0\u003E.","personal_data_remember":"Remember my choice and no longer display this message on this browser.","personal_data_accept":"watch the video","personal_data_cancel":"Cancel","playOnQwant":"Always play videos on Qwant.com","playOnQwantConfirm":"Are you sure ?","playOnQwantExplanations":"By disabling playback on Qwant.com, your videos will open in a new tab on the publisher's site (Dailymotion, Youtube…)","playOnQwantCancel":"Keep playing videos on Qwant.com","playOnQwantAccept":"Yes I'm sure"},"web":{"suggestTooltip":"This search suggestion is an advertisement for {{brand}}. If you click on this advertisement, you will be redirected to the advertiser's site: {{domain}}","ecoSuggest":{"suggestion":"Suggestion","ecoSuggestTooltip":"This suggestion is displayed when Qwant finds a more sustainable alternative that is likely to match your search."},"carousel":{"buttonAll":"Show all"},"shopping":{"landing":{"titleShoppingAds":"Shopping ads","popularCategories":"Most popular categories","legalSalesWarning":"The ads on this page may correspond to products that are not on sale. Check the merchant's website.","tooltipPriceDrop":"The products correspond to a selection of products whose price has recently fallen. For more offers, we invite you to specify your search in our dedicated shopping engine.","tooltipPopularity":"The products correspond to a selection of the most popular products. For more offers, we invite you to specify your search in our dedicated shopping engine.","tooltipCategory":"The products correspond to a selection of the most popular products in the category. For more offers, we invite you to specify your search in our dedicated shopping engine."},"buyProductNow":"Buy product now","useCode":"Use this discount code in the shopping cart","copyCode":"Press and hold to copy the code.","specialOffer":"Special offer","showProducts":"Show the products","mainCategory":"Main category","secondaryCategory":"Secondary category","seeMore":"Show the next {{number}}","seeLess":"Show less","allCategories":"All categories","min":"Min","max":"Max","ok":"OK","filter":"Filter","result":"Result","results":"Results \u003C0\u003E{{value}}\u003C\u002F0\u003E","noResult":"No product matches your request.","noResultSuggest":"We suggest removing some filters:","noResultQuery":"No product matches your query «{{query}}».","noResultTypo":"Check the spelling of your search","noResultBroader":"Use a less specific keyword","webResult":"Show web results","lessThan":"Less than","moreThan":"More than","between":"Between \u003C0\u002F\u003E and \u003C1\u002F\u003E","resetAll":"Reset all","allCategory":"All categories","tooltip":"The results presented on this page do not necessarily represent all products and services available on the Web. These are results for which Qwant is paid by Microsoft Bing on a CPC (\"cost per click\") basis. Qwant does not rank the Shopping offers by default; they are provided automatically by Bing Shopping based on the keywords entered. As with all of Qwant's services, we are committed to respecting your privacy when you search for products and services on our platform. The merchants from which you finalise your purchases have their own general terms and conditions of sale (applicable taxes, shipping costs, delivery options, etc.) which you should read before purchasing. The crossed-out prices are indicative information provided by the merchant when indexing its product data \u003C0 href=\"https:\u002F\u002Fabout.qwant.com\u002Fen\u002Flegal\u002Fclassement\u002F\"\u003ELearn more\u003C\u002F0\u003E","errors":{"priceRequired":"You need to fill one or both input above","priceNotANumber":"You need to use numbers","priceNegative":"The price can't be a negative number","priceMinMax":"The minimum price must be below the maximum price"},"filters":{"brands":"Brands","category":"Category","merchant":"Merchant","order":"Order","price":"Price"}},"news":{"title":"News on \u003C0\u003E{{value}}\u003C\u002F0\u003E","moreNews":"See more news","moreInUniverse":"See the other {{n}} articles","hideInUniverse":"Hide the other {{n}} articles"},"images":{"title":"Images of \u003C0\u003E{{value}}\u003C\u002F0\u003E","moreImages":"See more images"},"videos":{"title":"Videos of \u003C0\u003E{{value}}\u003C\u002F0\u003E","moreVideos":"See more videos"},"ads":{"microsoftMention":"Ads by Microsoft","qwantMention":"Ads by Qwant","microsoftToolTip":"Qwant takes care to preserve the confidentiality of your searches (\u003C0\u003ELearn more\u003C\u002F0\u003E). These ads from the Microsoft advertising network match the search terms you entered. The clicks are subject to \u003C1\u003EMicrosoft's privacy policy\u003C\u002F1\u003E.","ad":"Ad","from":"From {{ s }}","minOrder":"starting from {{s}} worth of purchases","morePrice":"Show {{ n }} prices from {{ price }}","code":"Code","clipboardCode":"The code was copied to your clipboard.","validity":"Validity","confidentiality":"Confidentiality","tooltip":"This sponsored content is published without transmission of personal data to the advertiser. Hyperlinks are provided by our partner whose Privacy policy applies when you click them.","seeMoreLinks":"See more links"},"productAds":{"title":"Products associated with {{value}}","titlePhone":"Products {{value}}","moreProducts":"See more products from {{value}}","moreProductsMobile":"See more products","adsMention":"Sponsored ads","adsMentionPhone":"Ad","promo":"Promotion"},"buttonShowMore":"Display more results","brandingBing":"Results by Microsoft Bing","relatedSearch":{"title":"To go further","titleSidebar":"Other searches"},"didYouMean":{"alteredQuery":"Includes results for \u003C0\u003E{{alteredQuery}}\u003C\u002F0\u003E","originalQuery":"Do you only want results for \u003C0\u003E{{originalQuery}}\u003C\u002F0\u003E?"},"report":"Click on a result to report it","adblockerModal":{"title":"Advertising keeps us free,\u003Cbr\u002F\u003E so play the game!","content":"Qwant offers a free search engine that respects your privacy. By blocking ads, you're blocking our only source of funding.","highlight":"To access your services, please disable your ad blocker!","howToCta":"How to disable my ad blocker?","deactivateCta":"I have disabled my ad blocker","helpTitle":"How to disable my ad blocker?","helpStep1":"Click on the ad blocker icon located at the top of your screen.","helpStep1Ex":"Examples of ad blocker icons:","helpStep1Sub1":"It's usually located in the top right-hand corner of your screen","helpStep1Sub2":"It's possible that a number partially covers the blocker icon","helpStep1Sub3":"If you use several blockers, make sure to disable all of them to access Qwant","helpStep2":"Follow the instructions provided by your blocker to disable it","helpStep3":"Click on the button below to take your changes into account:","helpMobileStep1":"Blocker disabling varies according to your browser (Firefox, Chrome, Safari...):","helpMobileStep2":"Open your phone or application settings","helpMobileStep2Or":"or","helpMobileStep2Bis":"Click on the icon (lock, shield, aA) to the left of the URL","helpMobileStep3":"Then click on the button below to take your changes into account: ","helpNeedHelpCta":"Need help ? Contact us","helpNeedHelpCtaUrl":"https:\u002F\u002Fabout.qwant.com\u002Fen\u002F"}},"upselling":{"download_tablet":"Download Qwant app for tablet and enjoy a european search engine that respects privacy.","download_mobile":"Download Qwant app for mobile and enjoy a european search engine that respects privacy.","ios_link":"https:\u002F\u002Fitunes.apple.com\u002Fus\u002Fapp\u002Fqwant\u002Fid924470452","android_link":"https:\u002F\u002Fplay.google.com\u002Fstore\u002Fapps\u002Fdetails?id=com.qwant.liberty","junior_ios_link":"https:\u002F\u002Fapps.apple.com\u002Fus\u002Fapp\u002Fqwant-junior\u002Fid1318660239","ios_alt":"Download on the App Store","android_alt":"Get it on Google Play","download":"Download the app","pluginThanks":{"title":"The extension has been added to {{browser}} !","titleSecondary":"In detail what does this mean?","defaultSearchEngine":"Qwant is now your default search engine in the address bar","startPage":"Qwant will be displayed on the start page when you launch your browser","search":"Start a new search","mobile":"Continue on mobile"},"pluginSurvey":"https:\u002F\u002Ffr.surveymonkey.com\u002Fr\u002FKS9WPBC "}}};
                  initialLanguage = "en";
                  initialWhiteListLanguages = ["br","ca","co","de","en","es","eu","fr","it","nl","pl","pt","cimode"];
                  i18nNamespaces = ["account","alpha","common","drawers","errorpage","filters","footer","header","homepage","ia","images","junior","modules","report","reward","robots","settings","signup","upselling","videos","web"]
                  i18nNamespacesMapping = {"locales\u002Fbr\u002Faccount.json":"\u002Flocales\u002Fbr\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fbr\u002Falpha.json":"\u002Flocales\u002Fbr\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fbr\u002Fcommon.json":"\u002Flocales\u002Fbr\u002Fcommon.6c33f161dcb279662d6ca87d4ff9b24e.json","locales\u002Fbr\u002Fdrawers.json":"\u002Flocales\u002Fbr\u002Fdrawers.0d8ca2a729365e734aec0043bdb9ebc7.json","locales\u002Fbr\u002Ferrorpage.json":"\u002Flocales\u002Fbr\u002Ferrorpage.c0864410ebc0d5491aeab8c59f0718ce.json","locales\u002Fbr\u002Ffilters.json":"\u002Flocales\u002Fbr\u002Ffilters.4469fcf249f0bbbf6955d9300f2a54c9.json","locales\u002Fbr\u002Ffooter.json":"\u002Flocales\u002Fbr\u002Ffooter.da6fb7cf687a85ea1931954e01864df7.json","locales\u002Fbr\u002Fheader.json":"\u002Flocales\u002Fbr\u002Fheader.43a8dd5c1156632854e724143191214e.json","locales\u002Fbr\u002Fhomepage.json":"\u002Flocales\u002Fbr\u002Fhomepage.548fd5b8f363354dfc99fb31df8ac363.json","locales\u002Fbr\u002Fia.json":"\u002Flocales\u002Fbr\u002Fia.07eed66846e1a0f6bcaaf61e320f0c06.json","locales\u002Fbr\u002Fimages.json":"\u002Flocales\u002Fbr\u002Fimages.3c69e336cd11ac97c92305ed5e4e5096.json","locales\u002Fbr\u002Fjunior.json":"\u002Flocales\u002Fbr\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fbr\u002Fmodules.json":"\u002Flocales\u002Fbr\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fbr\u002Freport.json":"\u002Flocales\u002Fbr\u002Freport.ce2a24c6e48de34786133d46d9c523b6.json","locales\u002Fbr\u002Freward.json":"\u002Flocales\u002Fbr\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fbr\u002Frobots.json":"\u002Flocales\u002Fbr\u002Frobots.a35be9ac5086bfe9dda35a38109d8345.json","locales\u002Fbr\u002Fsettings.json":"\u002Flocales\u002Fbr\u002Fsettings.f7830d0e83068e0876f791c2c1f68fe7.json","locales\u002Fbr\u002Fsignup.json":"\u002Flocales\u002Fbr\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fbr\u002Fupselling.json":"\u002Flocales\u002Fbr\u002Fupselling.f23a938c9aad8060a07a8b10e481c8ae.json","locales\u002Fbr\u002Fvideos.json":"\u002Flocales\u002Fbr\u002Fvideos.e4a091c0595bf89959fd7e65403d9463.json","locales\u002Fbr\u002Fweb.json":"\u002Flocales\u002Fbr\u002Fweb.427c688236552cc397f2fb74e83b4e28.json","locales\u002Fca\u002Faccount.json":"\u002Flocales\u002Fca\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Falpha.json":"\u002Flocales\u002Fca\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Fcommon.json":"\u002Flocales\u002Fca\u002Fcommon.07600d3dd688d512f4687ae517a17229.json","locales\u002Fca\u002Fdrawers.json":"\u002Flocales\u002Fca\u002Fdrawers.e1f3534792603f768df90172b664c7a9.json","locales\u002Fca\u002Ferrorpage.json":"\u002Flocales\u002Fca\u002Ferrorpage.c0864410ebc0d5491aeab8c59f0718ce.json","locales\u002Fca\u002Ffilters.json":"\u002Flocales\u002Fca\u002Ffilters.2821d56083595f32bb142673fe16848f.json","locales\u002Fca\u002Ffooter.json":"\u002Flocales\u002Fca\u002Ffooter.772857597902c15b5da976176ad8c76f.json","locales\u002Fca\u002Fheader.json":"\u002Flocales\u002Fca\u002Fheader.2f4b61628da98888d0590eb8d943eed9.json","locales\u002Fca\u002Fhomepage.json":"\u002Flocales\u002Fca\u002Fhomepage.29aa26e09ced252f8a2c7aa358192281.json","locales\u002Fca\u002Fia.json":"\u002Flocales\u002Fca\u002Fia.f145032675715d328c724d9db4563515.json","locales\u002Fca\u002Fimages.json":"\u002Flocales\u002Fca\u002Fimages.706779ddaccc870460850e2494dca786.json","locales\u002Fca\u002Fjunior.json":"\u002Flocales\u002Fca\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Fmodules.json":"\u002Flocales\u002Fca\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Freport.json":"\u002Flocales\u002Fca\u002Freport.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Freward.json":"\u002Flocales\u002Fca\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Frobots.json":"\u002Flocales\u002Fca\u002Frobots.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Fsettings.json":"\u002Flocales\u002Fca\u002Fsettings.15203e84be1798846a6ffbd86a6ed619.json","locales\u002Fca\u002Fsignup.json":"\u002Flocales\u002Fca\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fca\u002Fupselling.json":"\u002Flocales\u002Fca\u002Fupselling.f23a938c9aad8060a07a8b10e481c8ae.json","locales\u002Fca\u002Fvideos.json":"\u002Flocales\u002Fca\u002Fvideos.a786faac46fea020656ddd003e11a4df.json","locales\u002Fca\u002Fweb.json":"\u002Flocales\u002Fca\u002Fweb.75df64eefb907d7a27675f1cb7faf462.json","locales\u002Fco\u002Faccount.json":"\u002Flocales\u002Fco\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Falpha.json":"\u002Flocales\u002Fco\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Fcommon.json":"\u002Flocales\u002Fco\u002Fcommon.a4ffafad8d8aad5343169f2948eb65e4.json","locales\u002Fco\u002Fdrawers.json":"\u002Flocales\u002Fco\u002Fdrawers.9a65da9e90f24d9896047971efe0f0e3.json","locales\u002Fco\u002Ferrorpage.json":"\u002Flocales\u002Fco\u002Ferrorpage.c0864410ebc0d5491aeab8c59f0718ce.json","locales\u002Fco\u002Ffilters.json":"\u002Flocales\u002Fco\u002Ffilters.9c3522024cf93abb6625e267972ec524.json","locales\u002Fco\u002Ffooter.json":"\u002Flocales\u002Fco\u002Ffooter.ca3a4e874f88ff764c5da84d895dc6d7.json","locales\u002Fco\u002Fheader.json":"\u002Flocales\u002Fco\u002Fheader.17377a577a1fba907444c85c7893b286.json","locales\u002Fco\u002Fhomepage.json":"\u002Flocales\u002Fco\u002Fhomepage.7fb448492b208a737831ac9d21325209.json","locales\u002Fco\u002Fia.json":"\u002Flocales\u002Fco\u002Fia.8bf7127319d9dd55dbd6486d7fcdcdc3.json","locales\u002Fco\u002Fimages.json":"\u002Flocales\u002Fco\u002Fimages.795870d2dfb94a8d5a84998c18a9ca76.json","locales\u002Fco\u002Fjunior.json":"\u002Flocales\u002Fco\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Fmodules.json":"\u002Flocales\u002Fco\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Freport.json":"\u002Flocales\u002Fco\u002Freport.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Freward.json":"\u002Flocales\u002Fco\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Frobots.json":"\u002Flocales\u002Fco\u002Frobots.917a885e115fb6aefc352d9ea291e0de.json","locales\u002Fco\u002Fsettings.json":"\u002Flocales\u002Fco\u002Fsettings.bc1455e57d9dde35981680e7bf741992.json","locales\u002Fco\u002Fsignup.json":"\u002Flocales\u002Fco\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fco\u002Fupselling.json":"\u002Flocales\u002Fco\u002Fupselling.f23a938c9aad8060a07a8b10e481c8ae.json","locales\u002Fco\u002Fvideos.json":"\u002Flocales\u002Fco\u002Fvideos.4552e2eaad67e61108c0905d43861530.json","locales\u002Fco\u002Fweb.json":"\u002Flocales\u002Fco\u002Fweb.df9fe8f311f02142dca457b8ac3b1789.json","locales\u002Fde\u002Faccount.json":"\u002Flocales\u002Fde\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fde\u002Falpha.json":"\u002Flocales\u002Fde\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fde\u002Fcommon.json":"\u002Flocales\u002Fde\u002Fcommon.652b35b2e2cc7a079d5e0bbcd9b15a3e.json","locales\u002Fde\u002Fdrawers.json":"\u002Flocales\u002Fde\u002Fdrawers.46f5af0ff82bcd60fc9a7590b006da1d.json","locales\u002Fde\u002Ferrorpage.json":"\u002Flocales\u002Fde\u002Ferrorpage.1d97e08d76852d2f30e51f1724c9ae2f.json","locales\u002Fde\u002Ffilters.json":"\u002Flocales\u002Fde\u002Ffilters.06e53912c17cfb669f81b2328aed609b.json","locales\u002Fde\u002Ffooter.json":"\u002Flocales\u002Fde\u002Ffooter.46a66f55715d00503dba9f6f686df290.json","locales\u002Fde\u002Fheader.json":"\u002Flocales\u002Fde\u002Fheader.34d294b510d55097173653607726c0a6.json","locales\u002Fde\u002Fhomepage.json":"\u002Flocales\u002Fde\u002Fhomepage.3235dcba4209d6b123be1c7ebbb58b55.json","locales\u002Fde\u002Fia.json":"\u002Flocales\u002Fde\u002Fia.002cdb68f22c27360f751fea9fea7d1d.json","locales\u002Fde\u002Fimages.json":"\u002Flocales\u002Fde\u002Fimages.f1904767176897c4cbfca551b6c535a8.json","locales\u002Fde\u002Fjunior.json":"\u002Flocales\u002Fde\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fde\u002Fmodules.json":"\u002Flocales\u002Fde\u002Fmodules.63ba6495e81ea7987ebcf20cdbd59173.json","locales\u002Fde\u002Freport.json":"\u002Flocales\u002Fde\u002Freport.d7815847a14bc6e1e78d71d599b8a278.json","locales\u002Fde\u002Freward.json":"\u002Flocales\u002Fde\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fde\u002Frobots.json":"\u002Flocales\u002Fde\u002Frobots.3cf49368e3329f2ca497efc3e09f8cb5.json","locales\u002Fde\u002Fsettings.json":"\u002Flocales\u002Fde\u002Fsettings.967d8ef298bf3f888f4bfa0c738c96ef.json","locales\u002Fde\u002Fsignup.json":"\u002Flocales\u002Fde\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fde\u002Fupselling.json":"\u002Flocales\u002Fde\u002Fupselling.e8cc7e33ed56d1bbe6866b58368161af.json","locales\u002Fde\u002Fvideos.json":"\u002Flocales\u002Fde\u002Fvideos.6509e644b434f7c0c99e7b7418de9e4b.json","locales\u002Fde\u002Fweb.json":"\u002Flocales\u002Fde\u002Fweb.a2b13d4ab18b33a1c36f08d4a7468ba4.json","locales\u002Fen\u002Faccount.json":"\u002Flocales\u002Fen\u002Faccount.358febc37c0e34e87ed17322e00a65b4.json","locales\u002Fen\u002Falpha.json":"\u002Flocales\u002Fen\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fen\u002Fcommon.json":"\u002Flocales\u002Fen\u002Fcommon.95817679658d7f30ef75c241098b2ce3.json","locales\u002Fen\u002Fdrawers.json":"\u002Flocales\u002Fen\u002Fdrawers.2b06a816d1fc451568d5c4b1ef180790.json","locales\u002Fen\u002Ferrorpage.json":"\u002Flocales\u002Fen\u002Ferrorpage.36ca021ca16e84a8d0a5e344ac937345.json","locales\u002Fen\u002Ffilters.json":"\u002Flocales\u002Fen\u002Ffilters.3b50eb5f7f009c8a6f9a12fcfdc84523.json","locales\u002Fen\u002Ffooter.json":"\u002Flocales\u002Fen\u002Ffooter.9703391ad5ff880503e0667e5e9673a7.json","locales\u002Fen\u002Fheader.json":"\u002Flocales\u002Fen\u002Fheader.76585ad62c4cf74c14ed1a0bbe6aabdf.json","locales\u002Fen\u002Fhomepage.json":"\u002Flocales\u002Fen\u002Fhomepage.6ddfc82b591758e6cd209e852b44aa1c.json","locales\u002Fen\u002Fia.json":"\u002Flocales\u002Fen\u002Fia.74cf6996b53b2b1bb01896e756496a3c.json","locales\u002Fen\u002Fimages.json":"\u002Flocales\u002Fen\u002Fimages.bda1382b2e5b9e93defa841c412811fc.json","locales\u002Fen\u002Fjunior.json":"\u002Flocales\u002Fen\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fen\u002Fmodules.json":"\u002Flocales\u002Fen\u002Fmodules.8a97433b342fa5c192c847aa7df6a58b.json","locales\u002Fen\u002Freport.json":"\u002Flocales\u002Fen\u002Freport.b81b19b650a7676f929829a54ffb6547.json","locales\u002Fen\u002Freward.json":"\u002Flocales\u002Fen\u002Freward.96f6abd410c9d199e13a3191eb97f8b7.json","locales\u002Fen\u002Frobots.json":"\u002Flocales\u002Fen\u002Frobots.54e0c6af5f73d4cc8b38a36c36414c7d.json","locales\u002Fen\u002Fsettings.json":"\u002Flocales\u002Fen\u002Fsettings.3fa1e80f6685e4d7c1406a0ff9bec723.json","locales\u002Fen\u002Fsignup.json":"\u002Flocales\u002Fen\u002Fsignup.fe9cc651a7a3939b466117718aa66679.json","locales\u002Fen\u002Fupselling.json":"\u002Flocales\u002Fen\u002Fupselling.83553fb7da247bb426bc148116b3fa5f.json","locales\u002Fen\u002Fvideos.json":"\u002Flocales\u002Fen\u002Fvideos.74c026c708aac14c3c34bb97972fd029.json","locales\u002Fen\u002Fweb.json":"\u002Flocales\u002Fen\u002Fweb.a7f0bf29c355e8f7ba51544d65bd9eee.json","locales\u002Fes\u002Faccount.json":"\u002Flocales\u002Fes\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fes\u002Falpha.json":"\u002Flocales\u002Fes\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fes\u002Fcommon.json":"\u002Flocales\u002Fes\u002Fcommon.93861c7a4488f92ee79b307c04bb6bc5.json","locales\u002Fes\u002Fdrawers.json":"\u002Flocales\u002Fes\u002Fdrawers.77d5dd88eb2014730cb22034ca5ce046.json","locales\u002Fes\u002Ferrorpage.json":"\u002Flocales\u002Fes\u002Ferrorpage.d0f2c506a5c31ecbcd364f187b70b4c7.json","locales\u002Fes\u002Ffilters.json":"\u002Flocales\u002Fes\u002Ffilters.2968569b890787437a636792b8879fdf.json","locales\u002Fes\u002Ffooter.json":"\u002Flocales\u002Fes\u002Ffooter.3c4e116bc31cfd4deb6b85e0325851cb.json","locales\u002Fes\u002Fheader.json":"\u002Flocales\u002Fes\u002Fheader.4c69145a698aba2f2a1df20110fabc30.json","locales\u002Fes\u002Fhomepage.json":"\u002Flocales\u002Fes\u002Fhomepage.fe7ae2352142f177a6db2c9482702cd4.json","locales\u002Fes\u002Fia.json":"\u002Flocales\u002Fes\u002Fia.0bb4b001e2f9caf83de9c62cc3135dee.json","locales\u002Fes\u002Fimages.json":"\u002Flocales\u002Fes\u002Fimages.fa9e6250dc7babe07ae0c3297446c0a2.json","locales\u002Fes\u002Fjunior.json":"\u002Flocales\u002Fes\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fes\u002Fmodules.json":"\u002Flocales\u002Fes\u002Fmodules.83a800232ca23d476cfd9015b09d7c2e.json","locales\u002Fes\u002Freport.json":"\u002Flocales\u002Fes\u002Freport.9ebf8bd17bcb2b310c960d3d349633ae.json","locales\u002Fes\u002Freward.json":"\u002Flocales\u002Fes\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fes\u002Frobots.json":"\u002Flocales\u002Fes\u002Frobots.c5171ae2a720fe195f41c13b810f048f.json","locales\u002Fes\u002Fsettings.json":"\u002Flocales\u002Fes\u002Fsettings.718da31837a5b9509c6b64c842a42ef8.json","locales\u002Fes\u002Fsignup.json":"\u002Flocales\u002Fes\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fes\u002Fupselling.json":"\u002Flocales\u002Fes\u002Fupselling.2f5e8b301c5875857027d5d31c174d80.json","locales\u002Fes\u002Fvideos.json":"\u002Flocales\u002Fes\u002Fvideos.ba762c47f77d82b835ac318bba8e400c.json","locales\u002Fes\u002Fweb.json":"\u002Flocales\u002Fes\u002Fweb.8ef462fae87d5bf75855bdb2a64ceda4.json","locales\u002Feu\u002Faccount.json":"\u002Flocales\u002Feu\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Falpha.json":"\u002Flocales\u002Feu\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Fcommon.json":"\u002Flocales\u002Feu\u002Fcommon.3a9efe604c86ff0f37f697ab50f9c57c.json","locales\u002Feu\u002Fdrawers.json":"\u002Flocales\u002Feu\u002Fdrawers.2efce8da92feb311d7fe05259c1b4d2d.json","locales\u002Feu\u002Ferrorpage.json":"\u002Flocales\u002Feu\u002Ferrorpage.c0864410ebc0d5491aeab8c59f0718ce.json","locales\u002Feu\u002Ffilters.json":"\u002Flocales\u002Feu\u002Ffilters.a680b92d1c3fafe374ba5b4245d859bc.json","locales\u002Feu\u002Ffooter.json":"\u002Flocales\u002Feu\u002Ffooter.bfb41b5790f10190ea53a59c873a0383.json","locales\u002Feu\u002Fheader.json":"\u002Flocales\u002Feu\u002Fheader.59ecc4ed6035a8330c564c7bb7e8d67b.json","locales\u002Feu\u002Fhomepage.json":"\u002Flocales\u002Feu\u002Fhomepage.9887b04e8f861d966e2cd7895cd0cb5c.json","locales\u002Feu\u002Fia.json":"\u002Flocales\u002Feu\u002Fia.c6294da27c8a8012b91dc98137b93a08.json","locales\u002Feu\u002Fimages.json":"\u002Flocales\u002Feu\u002Fimages.20c81074513e1cb723787f5fbf8b3ab1.json","locales\u002Feu\u002Fjunior.json":"\u002Flocales\u002Feu\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Fmodules.json":"\u002Flocales\u002Feu\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Freport.json":"\u002Flocales\u002Feu\u002Freport.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Freward.json":"\u002Flocales\u002Feu\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Frobots.json":"\u002Flocales\u002Feu\u002Frobots.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Fsettings.json":"\u002Flocales\u002Feu\u002Fsettings.57ed81984b72d31d0c3ae58ccf91bfed.json","locales\u002Feu\u002Fsignup.json":"\u002Flocales\u002Feu\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Feu\u002Fupselling.json":"\u002Flocales\u002Feu\u002Fupselling.f23a938c9aad8060a07a8b10e481c8ae.json","locales\u002Feu\u002Fvideos.json":"\u002Flocales\u002Feu\u002Fvideos.b5bb016593ef9da2048d69481e321693.json","locales\u002Feu\u002Fweb.json":"\u002Flocales\u002Feu\u002Fweb.8dd9228ebc79ff97410706b68cee7ab8.json","locales\u002Ffr\u002Faccount.json":"\u002Flocales\u002Ffr\u002Faccount.55a19b3151742a037cd2e23a4b2cf0ae.json","locales\u002Ffr\u002Falpha.json":"\u002Flocales\u002Ffr\u002Falpha.61aee2460c3ea3419127100c54749876.json","locales\u002Ffr\u002Fcommon.json":"\u002Flocales\u002Ffr\u002Fcommon.2c27f1c5e945da1680bb3d8c0f993bea.json","locales\u002Ffr\u002Fdrawers.json":"\u002Flocales\u002Ffr\u002Fdrawers.993c41454180ea284a915c9c6bf2884b.json","locales\u002Ffr\u002Ferrorpage.json":"\u002Flocales\u002Ffr\u002Ferrorpage.bed62535d3cf6ad5dd76f4acea9c0aa8.json","locales\u002Ffr\u002Ffilters.json":"\u002Flocales\u002Ffr\u002Ffilters.179f8df8c0278217982a41756d733465.json","locales\u002Ffr\u002Ffooter.json":"\u002Flocales\u002Ffr\u002Ffooter.a1c1a320b71c38c8a41eebe034c5609e.json","locales\u002Ffr\u002Fheader.json":"\u002Flocales\u002Ffr\u002Fheader.28a0776aef58a44b5072ad281f3b310b.json","locales\u002Ffr\u002Fhomepage.json":"\u002Flocales\u002Ffr\u002Fhomepage.fec59e287c043ac1bf71490123fa96d6.json","locales\u002Ffr\u002Fia.json":"\u002Flocales\u002Ffr\u002Fia.36563b564d6fca3024079fc2b8cc06ee.json","locales\u002Ffr\u002Fimages.json":"\u002Flocales\u002Ffr\u002Fimages.b6b4177a09b19388f1998d6d3c955c2f.json","locales\u002Ffr\u002Fjunior.json":"\u002Flocales\u002Ffr\u002Fjunior.3a106e4f68b08441cde18952f9ecdb80.json","locales\u002Ffr\u002Fmodules.json":"\u002Flocales\u002Ffr\u002Fmodules.e0aaa73e99c02b791cf71684c2b593ae.json","locales\u002Ffr\u002Freport.json":"\u002Flocales\u002Ffr\u002Freport.7d8ddb4a7b6a7d51921d7782771c880e.json","locales\u002Ffr\u002Freward.json":"\u002Flocales\u002Ffr\u002Freward.8b53c3d4ba478f8cd2954ebc504c3b0e.json","locales\u002Ffr\u002Frobots.json":"\u002Flocales\u002Ffr\u002Frobots.c594365c2bbfba9ae39ba09dc90d1b38.json","locales\u002Ffr\u002Fsettings.json":"\u002Flocales\u002Ffr\u002Fsettings.46aaf75cbf6d3b8dcac1a2251b40b7ad.json","locales\u002Ffr\u002Fsignup.json":"\u002Flocales\u002Ffr\u002Fsignup.4ab11feb6b9138fd3ff21fcde8f0d84e.json","locales\u002Ffr\u002Fupselling.json":"\u002Flocales\u002Ffr\u002Fupselling.7c3880c29b680887a7893175318850e5.json","locales\u002Ffr\u002Fvideos.json":"\u002Flocales\u002Ffr\u002Fvideos.a2a7fad56ac85341769fcd54cee9fd7e.json","locales\u002Ffr\u002Fweb.json":"\u002Flocales\u002Ffr\u002Fweb.23652a522b42e3c0c1e77db7b420ba1e.json","locales\u002Fit\u002Faccount.json":"\u002Flocales\u002Fit\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fit\u002Falpha.json":"\u002Flocales\u002Fit\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fit\u002Fcommon.json":"\u002Flocales\u002Fit\u002Fcommon.43bd6ed8f12156c29ef20d30d329677c.json","locales\u002Fit\u002Fdrawers.json":"\u002Flocales\u002Fit\u002Fdrawers.48b1bca8d3c34a6c0d3016ce91e1f5d3.json","locales\u002Fit\u002Ferrorpage.json":"\u002Flocales\u002Fit\u002Ferrorpage.f81d0015359f4cff93d3aa75799e9bf9.json","locales\u002Fit\u002Ffilters.json":"\u002Flocales\u002Fit\u002Ffilters.471a1dbf9415fe03105ae0c5f905f525.json","locales\u002Fit\u002Ffooter.json":"\u002Flocales\u002Fit\u002Ffooter.eba4abc4aa2d8c222afacb5a033c537d.json","locales\u002Fit\u002Fheader.json":"\u002Flocales\u002Fit\u002Fheader.ec8b3324114970e6f42ab9ade90f9074.json","locales\u002Fit\u002Fhomepage.json":"\u002Flocales\u002Fit\u002Fhomepage.4dc0c7e72aeb73b720faa33220b02b31.json","locales\u002Fit\u002Fia.json":"\u002Flocales\u002Fit\u002Fia.e76a1914cfaf482c0e6b1e3f2490e321.json","locales\u002Fit\u002Fimages.json":"\u002Flocales\u002Fit\u002Fimages.313f9a2ba1ecdd355b22eeedeaf7b20c.json","locales\u002Fit\u002Fjunior.json":"\u002Flocales\u002Fit\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fit\u002Fmodules.json":"\u002Flocales\u002Fit\u002Fmodules.1d71eb1f4457d80e785aad4b91079d8d.json","locales\u002Fit\u002Freport.json":"\u002Flocales\u002Fit\u002Freport.85ac2e078dc399cd4d0517414c4b4948.json","locales\u002Fit\u002Freward.json":"\u002Flocales\u002Fit\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fit\u002Frobots.json":"\u002Flocales\u002Fit\u002Frobots.7ae7a2bb9101636b3daea89ba62b3a38.json","locales\u002Fit\u002Fsettings.json":"\u002Flocales\u002Fit\u002Fsettings.3fac7654a397173f21b8b4bf6d1f46d1.json","locales\u002Fit\u002Fsignup.json":"\u002Flocales\u002Fit\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fit\u002Fupselling.json":"\u002Flocales\u002Fit\u002Fupselling.62a7a0a84d003f4132425688754765b7.json","locales\u002Fit\u002Fvideos.json":"\u002Flocales\u002Fit\u002Fvideos.f2ac79be2c8cb231df3c0c1d5f074749.json","locales\u002Fit\u002Fweb.json":"\u002Flocales\u002Fit\u002Fweb.a169282ed7de5f9d5e42ebb98e1850bc.json","locales\u002Fnl\u002Faccount.json":"\u002Flocales\u002Fnl\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fnl\u002Falpha.json":"\u002Flocales\u002Fnl\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fnl\u002Fcommon.json":"\u002Flocales\u002Fnl\u002Fcommon.b264532d5e4571c69e0e07ce0083fee8.json","locales\u002Fnl\u002Fdrawers.json":"\u002Flocales\u002Fnl\u002Fdrawers.cbe29e61187b8950da22e601ef035bc6.json","locales\u002Fnl\u002Ferrorpage.json":"\u002Flocales\u002Fnl\u002Ferrorpage.04a7201f98a33b735e1328a9c6dcf054.json","locales\u002Fnl\u002Ffilters.json":"\u002Flocales\u002Fnl\u002Ffilters.a31cd1c18cfdaa1499a0982e12bc428b.json","locales\u002Fnl\u002Ffooter.json":"\u002Flocales\u002Fnl\u002Ffooter.98840b68f8c8e8c04cfa0b58c8936d91.json","locales\u002Fnl\u002Fheader.json":"\u002Flocales\u002Fnl\u002Fheader.2c2cea073389ce4a7dc22fcaa0e21fe4.json","locales\u002Fnl\u002Fhomepage.json":"\u002Flocales\u002Fnl\u002Fhomepage.11794695e5a91e433b640cff748493a6.json","locales\u002Fnl\u002Fia.json":"\u002Flocales\u002Fnl\u002Fia.b65c4f7441a588c6c9550b37767a0e0b.json","locales\u002Fnl\u002Fimages.json":"\u002Flocales\u002Fnl\u002Fimages.56446d53181567fdbb47a8f162c5b767.json","locales\u002Fnl\u002Fjunior.json":"\u002Flocales\u002Fnl\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fnl\u002Fmodules.json":"\u002Flocales\u002Fnl\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fnl\u002Freport.json":"\u002Flocales\u002Fnl\u002Freport.5166bdaadc5904e7d42735c30984ba83.json","locales\u002Fnl\u002Freward.json":"\u002Flocales\u002Fnl\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fnl\u002Frobots.json":"\u002Flocales\u002Fnl\u002Frobots.843cbdf018cd6eb16f957bb3b0453b4f.json","locales\u002Fnl\u002Fsettings.json":"\u002Flocales\u002Fnl\u002Fsettings.477f6e2eab992b8c4316f52a11c0488d.json","locales\u002Fnl\u002Fsignup.json":"\u002Flocales\u002Fnl\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fnl\u002Fupselling.json":"\u002Flocales\u002Fnl\u002Fupselling.3dd02a372ce0e4092c7a238e513c0272.json","locales\u002Fnl\u002Fvideos.json":"\u002Flocales\u002Fnl\u002Fvideos.594d0afe0a2dbb24b23b98bb5d1d603c.json","locales\u002Fnl\u002Fweb.json":"\u002Flocales\u002Fnl\u002Fweb.f885e4a165b49c9bba30654b81c0eeac.json","locales\u002Fpl\u002Faccount.json":"\u002Flocales\u002Fpl\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpl\u002Falpha.json":"\u002Flocales\u002Fpl\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpl\u002Fcommon.json":"\u002Flocales\u002Fpl\u002Fcommon.59b8b79ae7a0933e6086315bd4954ac5.json","locales\u002Fpl\u002Fdrawers.json":"\u002Flocales\u002Fpl\u002Fdrawers.092f784d4914c3befd5928ab9472dcd2.json","locales\u002Fpl\u002Ferrorpage.json":"\u002Flocales\u002Fpl\u002Ferrorpage.b54759e1886b998375830ef8f0b51be2.json","locales\u002Fpl\u002Ffilters.json":"\u002Flocales\u002Fpl\u002Ffilters.ca0ce709b95fca716f0c4801e51a25de.json","locales\u002Fpl\u002Ffooter.json":"\u002Flocales\u002Fpl\u002Ffooter.9cd8a417fd57959a8d488c4e0f14b49a.json","locales\u002Fpl\u002Fheader.json":"\u002Flocales\u002Fpl\u002Fheader.1676ed16d6c6d646098414b6f7b81297.json","locales\u002Fpl\u002Fhomepage.json":"\u002Flocales\u002Fpl\u002Fhomepage.9b99ab99c890853c8db423b52f7981bd.json","locales\u002Fpl\u002Fia.json":"\u002Flocales\u002Fpl\u002Fia.67ec246200caca73d0c99ef57fbd66b7.json","locales\u002Fpl\u002Fimages.json":"\u002Flocales\u002Fpl\u002Fimages.ab4ccf038fdaec4f3b1b0de12303ccb4.json","locales\u002Fpl\u002Fjunior.json":"\u002Flocales\u002Fpl\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpl\u002Fmodules.json":"\u002Flocales\u002Fpl\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpl\u002Freport.json":"\u002Flocales\u002Fpl\u002Freport.4635a57cbcf59de28ec1e44dd90ad136.json","locales\u002Fpl\u002Freward.json":"\u002Flocales\u002Fpl\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpl\u002Frobots.json":"\u002Flocales\u002Fpl\u002Frobots.955a4716904658e36acf0dd656446ca4.json","locales\u002Fpl\u002Fsettings.json":"\u002Flocales\u002Fpl\u002Fsettings.bdde3b891fec1e0825c1207ada56ec0f.json","locales\u002Fpl\u002Fsignup.json":"\u002Flocales\u002Fpl\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpl\u002Fupselling.json":"\u002Flocales\u002Fpl\u002Fupselling.d43346a35b7d262ab42aecb62796bf02.json","locales\u002Fpl\u002Fvideos.json":"\u002Flocales\u002Fpl\u002Fvideos.33ca5650aecb308f3bc552b39c9a2dfd.json","locales\u002Fpl\u002Fweb.json":"\u002Flocales\u002Fpl\u002Fweb.073f92f2fa27c7f5c58b8105a307671a.json","locales\u002Fpt\u002Faccount.json":"\u002Flocales\u002Fpt\u002Faccount.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpt\u002Falpha.json":"\u002Flocales\u002Fpt\u002Falpha.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpt\u002Fcommon.json":"\u002Flocales\u002Fpt\u002Fcommon.dee568940dec2b31771ab93d0c88c589.json","locales\u002Fpt\u002Fdrawers.json":"\u002Flocales\u002Fpt\u002Fdrawers.b0aef0d5a71c0945ae057d2962d2d0fc.json","locales\u002Fpt\u002Ferrorpage.json":"\u002Flocales\u002Fpt\u002Ferrorpage.3779611bfe87b4a2a4d8705edab8c925.json","locales\u002Fpt\u002Ffilters.json":"\u002Flocales\u002Fpt\u002Ffilters.391333197dfb63b30135095554087bb7.json","locales\u002Fpt\u002Ffooter.json":"\u002Flocales\u002Fpt\u002Ffooter.833a6c178b7d2ae255e00bdcaa78ab6d.json","locales\u002Fpt\u002Fheader.json":"\u002Flocales\u002Fpt\u002Fheader.21fd56dfd0a81410597729a02d5d8260.json","locales\u002Fpt\u002Fhomepage.json":"\u002Flocales\u002Fpt\u002Fhomepage.55acee7811abe82e2e64037dc53f0b2e.json","locales\u002Fpt\u002Fia.json":"\u002Flocales\u002Fpt\u002Fia.20cdd39f97b815753632550424686b12.json","locales\u002Fpt\u002Fimages.json":"\u002Flocales\u002Fpt\u002Fimages.9e35dbbca5021d98e2710aa33aa80c34.json","locales\u002Fpt\u002Fjunior.json":"\u002Flocales\u002Fpt\u002Fjunior.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpt\u002Fmodules.json":"\u002Flocales\u002Fpt\u002Fmodules.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpt\u002Freport.json":"\u002Flocales\u002Fpt\u002Freport.a9326ba3d0d11eb0efacf7123b04d3dd.json","locales\u002Fpt\u002Freward.json":"\u002Flocales\u002Fpt\u002Freward.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpt\u002Frobots.json":"\u002Flocales\u002Fpt\u002Frobots.477af6e6f0334b53ba9b1941e4321625.json","locales\u002Fpt\u002Fsettings.json":"\u002Flocales\u002Fpt\u002Fsettings.77349e65131463fdd753e9e5c3204b01.json","locales\u002Fpt\u002Fsignup.json":"\u002Flocales\u002Fpt\u002Fsignup.8a80554c91d9fca8acb82f023de02f11.json","locales\u002Fpt\u002Fupselling.json":"\u002Flocales\u002Fpt\u002Fupselling.2ec100079420ed38eb1dce690318bc30.json","locales\u002Fpt\u002Fvideos.json":"\u002Flocales\u002Fpt\u002Fvideos.2f05062993dec435b8a26edb8a1f12ce.json","locales\u002Fpt\u002Fweb.json":"\u002Flocales\u002Fpt\u002Fweb.270fbd3802963f2c25a28611b5bfd461.json"}
                </script>
  <script id="__LOADABLE_REQUIRED_CHUNKS__" type="application/json">[]</script><script id="__LOADABLE_REQUIRED_CHUNKS___ext" type="application/json">{"namedChunks":[]}</script>
<script async="" data-chunk="app" src="/a61ec0921f734a0d5dee_runtime.Phoenix.js"></script>
<script async="" data-chunk="app" src="/eccc33be0ced324f778a_vendors~app.chunks.js"></script>
<script async="" data-chunk="app" src="/46a6c1fdc290b3b0cbd5_app.chunks.js"></script>


<script>(function(window, document, dataLayerName, id) {
window[dataLayerName]=window[dataLayerName]||[],window[dataLayerName].push({start:(new Date).getTime(),event:"stg.start"});var scripts=document.getElementsByTagName('script')[0],tags=document.createElement('script');
function stgCreateCookie(a,b,c){var d="";if(c){var e=new Date;e.setTime(e.getTime()+24*c*60*60*1e3),d="; expires="+e.toUTCString()}document.cookie=a+"="+b+d+"; path=/"}
var isStgDebug=(window.location.href.match("stg_debug")||document.cookie.match("stg_debug"))&&!window.location.href.match("stg_disable_debug");stgCreateCookie("stg_debug",isStgDebug?1:"",isStgDebug?14:-1);
var qP=[];dataLayerName!=="dataLayer"&&qP.push("data_layer_name="+dataLayerName),isStgDebug&&qP.push("stg_debug");var qPString=qP.length>0?("?"+qP.join("&")):"";
tags.async=!0,tags.src="https://k.qwant.com/containers/"+id+".js"+qPString,scripts.parentNode.insertBefore(tags,scripts);
!function(a,n,i){a[n]=a[n]||{};for(var c=0;c<i.length;c++)!function(i){a[n][i]=a[n][i]||{},a[n][i].api=a[n][i].api||function(){var a=[].slice.call(arguments,0);"string"==typeof a[0]&&window[dataLayerName].push({event:n+"."+i+":"+a[0],parameters:[].slice.call(arguments,1)})}}(i[c])}(window,"ppms",["tm","cm"]);
})(window, document, 'dataLayer', '98cbd59f-731c-4874-a402-04f51ed9069a')</script><script type="text/javascript">
    var _paq = _paq || [];
    _paq.push(['setTrackingSource', 'jstc_tm']);
    _paq.push(['setSecureCookie', 1]);
    _paq.push(['setCookieDomain', '.qwant.com']);
    _paq.push(['enableCrossDomainLinking']);
    _paq.push(['setDomains', ['.qwant.com', '.qwantjunior.com']]);
    (function(p,i,w,ik) {
        var g=ik.createElement('script'),s=ik.getElementsByTagName('script')[0];
        _paq.push(['setTrackerUrl', p]);
        _paq.push(['setSiteId', w]);
        g.type='text/javascript';g.async=true;g.defer=true;g.src=i;s.parentNode.insertBefore(g,s);
    })('https://k.qwant.com/2aecbbf86c74c1c2fb798e0a39f0678e','https://k.qwant.com/2aecbbf86c74c1c2fb798e0a39f0678e.js','98cbd59f-731c-4874-a402-04f51ed9069a',document)
</script></body></html>

================
File: plugins/repos/twat_search/debug_output/yahoo_analysis.txt
================
Analysis for yahoo
================================================================================

Potential containers found:
div.dd.algo: 0 elements found
div.algo-sr: 0 elements found
div.dd.algo.algo-sr: 0 elements found
article.webResult: 0 elements found
div.result: 0 elements found
div.web-result: 0 elements found
div.result: 0 elements found
div.search-result: 0 elements found
li.result: 0 elements found

Other potential elements:
h1: 1 elements found
  1. guce

================
File: plugins/repos/twat_search/debug_output/yahoo_content.html
================
<!DOCTYPE html><html dir="ltr" class="ltr  yahoo-page height100"><head>
    <title>Yahoo is part of the Yahoo family of brands</title>

    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, minimum-scale=1, user-scalable=no, viewport-fit=cover">
    <link rel="stylesheet" href="https://s.yimg.com/oa/build/css/site-ltr-1e589816.css">
    <link rel="icon" type="image/png" href="https://s.yimg.com/oa/build/images/favicons/yahoo.png">
    <style nonce="">:root,html[data-color-scheme=dark] [data-maintain-color],html[data-color-theme-enabled] [data-color-scheme=light],html[data-color-theme-enabled][data-color-scheme=light]{--yb-default-font-family:"Helvetica Neue", Helvetica, Tahoma, Geneva, Arial, sans-serif;--yb-fuji2-font-family:"YahooSans VF", YahooSans, "Yahoo Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;--yb-uh3-cond-font-family:"Yahoo Sans Cond", YahooSansCond, "Yahoo Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;--yb-font-regular:400;--yb-font-medium:500;--yb-font-bold:600;--yb-color-att-dark-blue:#0057b8;--yb-color-frontier-red:#c41230;--yb-color-rogers-red:#c41522;--yb-color-a11y-gray:#787679;--yb-barbie:#f80e5d;--yb-barney:#cc008c;--yb-batcave:#232a31;--yb-battleship:#5b636a;--yb-black:#000;--yb-blurple:#5d5eff;--yb-bob:#b0b9c1;--yb-bonsai:#008751;--yb-charcoal:#464e56;--yb-cobalt:#003abc;--yb-denim:#1a0dab;--yb-dirty-seagull:#e0e4e9;--yb-dolphin:#6e7780;--yb-dory:#0f69ff;--yb-gandalf:#979ea8;--yb-grape-jelly:#6001d2;--yb-gray-hair:#f0f3f5;--yb-hulk-pants:#7e1fff;--yb-inkwell:#1d2228;--yb-malbec:#39007d;--yb-marshmallow:#f5f8fa;--yb-midnight:#101518;--yb-mulberry:#5015b0;--yb-pebble:#c7cdd2;--yb-peeps:#7dcbff;--yb-ramones:#2c363f;--yb-sea-foam:#11d3cd;--yb-scooter:#0063eb;--yb-shark:#828a93;--yb-sky:#12a9ff;--yb-smurfette:#188fff;--yb-solo-cup:#eb0f29;--yb-starfish:#7759ff;--yb-swedish-fish:#ff333a;--yb-thanos:#907cff;--yb-turmeric:#ffa700;--yb-white:#fff;--yb-rgb-battleship:91 99 106;--yb-rgb-white:255 255 255;--yb-rgb-blurple:93 94 255;--yb-rgb-starfish:119 89 255;--yb-rgb-hulk-pants:126 31 255;--yb-rgb-grape-jelly:96 1 210;--yb-rgb-peeps:125 203 255;--yb-rgb-sky:18 169 255;--yb-rgb-dory:15 105 255;--yb-rgb-scooter:0 99 235;--yb-rgb-cobalt:0 58 188;--yb-rgb-sea-foam:17 211 205;--yb-battleship-fog:rgb(var(--yb-rgb-battleship) / 10%);--yb-white-fog:rgb(var(--yb-rgb-white) / 10%);--yb-blurple-fog:rgb(var(--yb-rgb-blurple) / 10%);--yb-starfish-fog:rgb(var(--yb-rgb-starfish) / 10%);--yb-hulk-pants-fog:rgb(var(--yb-rgb-hulk-pants) / 10%);--yb-grape-jelly-fog:rgb(var(--yb-rgb-grape-jelly) / 10%);--yb-peeps-fog:rgb(var(--yb-rgb-peeps) / 10%);--yb-sky-fog:rgb(var(--yb-rgb-sky) / 10%);--yb-dory-fog:rgb(var(--yb-rgb-dory) / 10%);--yb-scooter-fog:rgb(var(--yb-rgb-scooter) / 10%);--yb-cobalt-fog:rgb(var(--yb-rgb-cobalt) / 10%);--yb-sea-foam-fog:rgb(var(--yb-rgb-sea-foam) / 10%);--yb-hulk-pants-accent:rgb(var(--yb-rgb-hulk-pants) / 50%);--yb-grape-jelly-accent:rgb(var(--yb-rgb-grape-jelly) / 50%);--yb-sky-accent:rgb(var(--yb-rgb-sky) / 50%);--yb-logo-brand:var(--yb-grape-jelly);--yb-logo-property:var(--yb-batcave);--yb-finance-logo-brand:var(--yb-grape-jelly);--yb-text-primary:var(--yb-batcave);--yb-text-secondary:var(--yb-battleship);--yb-text-mail-primary:var(--yb-grape-jelly);--yb-text-mail-hover:var(--yb-mulberry);--yb-background-color:var(--yb-white);--yb-l2-background-color:var(--yb-marshmallow);--yb-popover-background:var(--uh-popover-background, var(--yb-white));--yb-tab-focus-accent:var(--yb-grape-jelly-accent);--yb-selected-item-background:var(--yb-marshmallow);--yb-sidenav-close-btn-background:var(--yb-marshmallow);--yb-sidenav-btn-color-hover:var(--yb-grape-jelly);--yb-sidenav-btn-color-active:var(--yb-dirty-seagull);--yb-sidenav-btn-text-active:var(--yb-batcave);--yb-sidenav-finance-plus-icon:var(--yb-svg-icon-color-non-l2);--yb-sidenav-bg:var(--yb-background-color);--yb-moremenu-finance-plus-icon:var(--yb-svg-icon-color-non-l2);--yb-profile-avatar-bg:var(--uh-profile-avatar-bg, var(--yb-gray-hair));--yb-profile-panel-hover:var(--uh-profile-panel-hover-bg, var(--yb-dirty-seagull));--yb-profile-alphatar-background:var(--yb-gray-hair);--yb-profile-alphatar-background-pressed:var(--yb-white);--yb-signin-border-color:var(--yb-dirty-seagull);--yb-mail-border-color:var(--yb-grape-jelly);--yb-unread-container-bg:rgb(var(--yb-rgb-grape-jelly) / 5%);--yb-profile-character-hover:var(--yb-white);--yb-profile-pressed-state:var(--yb-bob);--yb-profile-character-pressed-state-bg:var(--yb-dirty-seagull);--yb-profile-character-pressed-state-text:var(--yb-batcave);--yb-profile-account-status-container-bg:var(--yb-battleship-fog);--yb-searchbox-bg:var(--yb-gray-hair);--yb-search-btn-hover:#5409b2;--yb-search-faux-icon:var(--yb-batcave);--yb-search-input-box-shadow:0 0 1px 0 rgb(0 0 0 / 0%), 0 1px 0 0 var(--yb-dirty-seagull);--yb-search-assist-primary:var(--yb-batcave);--yb-search-assist-secondary:var(--yb-dolphin);--yb-search-assist-history:#6e329d;--yb-search-assist-history-pc:var(--yb-grape-jelly);--yb-search-assist-back-btn:var(--yb-dory);--yb-search-assist-item-hover-background:var(--yb-marshmallow);--yb-elevation3-box-shadow:0 4px 8px 0 rgb(0 0 0 / 10%), 0 0 1px 0 rgb(0 0 0 / 10%);--yb-elevation4-box-shadow:0 4px 16px 0 rgb(0 0 0 / 20%), 0 0 2px 0 rgb(0 0 0 / 5%);--yb-l2-news-background:linear-gradient(0deg, rgb(255 255 255 / 100%) 0%, rgb(248 250 251 / 100%) 100%);--yb-current-account-fog:var(--yb-dory-fog);--yb-editions-modal-outline:var(--yb-dirty-seagull);--yb-editions-text:var(--yb-dolphin);--yb-cobrand-logos-divider:var(--yb-dirty-seagull);--yb-l2-background:var(--uh-l2-background, var(--yb-l2-background-color));--yb-search-carat:var(--uh-search-carat-color, var(--yb-grape-jelly));--yb-l2-divider-color:var(--uh-l2-divider-color, var(--yb-gray-hair));--yb-l2-text-color:var(--uh-l2-text-color, var(--yb-batcave));--yb-default-font:var(--uh-font-family, var(--yb-default-font-family));--yb-fuji2-font:var(--uh-font-family, var(--yb-fuji2-font-family));--yb-uh3-cond-font:var(--uh-font-family, var(--yb-uh3-cond-font-family));--yb-outline:4px solid var(--uh-tab-focus-color, var(--yb-tab-focus-accent));--yb-search-btn-color:var(--uh-search-btn-color, var(--yb-search-carat));--yb-search-btn-hover-color:var(--uh-search-btn-hover-color, var(--yb-search-btn-hover));--yb-text-hover-color:var(--uh-text-hover-color, var(--yb-sidenav-btn-color-hover));--yb-profile-panel-hover-text:var(--uh-text-hover-color, var(--yb-grape-jelly));--yb-searchbox-transition:border-radius 0s 0.2s ease-out, background-color 0s 0.2s ease-out, box-shadow 0.2s ease-out, padding-left 50ms 150ms ease-out;--yb-property-ruler-color:var(--uh-property-ruler-color, var(--yb-grape-jelly));--yb-svg-icon-color:var(--uh-icon-color, var(--yb-grape-jelly));--yb-svg-icon-color-non-l2:var(--uh-icon-color-non-l2, var(--yb-grape-jelly) );--yb-svg-icon-color-selected:var(--uh-icon-color-selected, var(--yb-white));--yb-mail-bubble-color:var(--uh-mail-bubble-color, var(--yb-grape-jelly));--yb-mail-indicator:var(--yb-profile-panel-hover-text);--yb-mail-indicator-border:1px solid var(--yb-background-color);--yb-mail-count:var(--yb-text-primary);--yb-tap-color:var(--uh-text-hover-color, var(--yb-text-primary));--yb-text-pressed-color:var(--uh-text-pressed-color, var(--yb-mulberry));--yb-l2-margin:var(--uh-l2-margin);--yb-text-shadow:none;--yb-nav-color:var(--yb-text-secondary);--yb-sports-l2-dropdown-hover-bg:#e6e6e6;--yb-sports-l2-dropdown-hover-color:var(--yb-black);--yb-sports-l2-dropdown-focus-bg:#ecf8ff;--yb-nav-selected-color:var(--yb-text-primary);--yb-topnav-bg:var(--yb-batcave);--yb-back-button-fill:var(--yb-batcave);--yb-sa-stock-up:var(--yb-bonsai);--yb-sa-stock-down:var(--yb-solo-cup);--yb-sa-flight-label-border:1px solid var(--yb-dirty-seagull);--yb-sa-flight-label-positive:var(--yb-bonsai);--yb-sa-flight-label-negative:var(--yb-solo-cup);--yb-popover-border:1px solid #d8dade;--yb-popover-border-radius:4px;--yb-popover-shadow:0 2px 8px 0 rgb(0 0 0 / 36%);--yb-yns-title:var(--batcave);--yb-l2-nav-selected:var(--yb-grape-jelly);--yb-l2-nav-dropdown-focus:var(--yb-text-hover-color);--yb-mail-icon-text:var(--yb-batcave);--yb-mail-signin-link:#0078ff;--yb-light-img-logo-display:block;--yb-dark-img-logo-display:none;--yb-account-item-border:#d8dade;--yb-account-item-hover-bg:rgb(227 236 248);--yb-account-info-bg:#ededf3;--yb-account-menu-item:var(--yb-white);--yb-account-link:var(--yb-smurfette);--yb-account-text:#26282a;--yb-editions-button:var(--yb-marshmallow);--yb-editions-modal-background:var(--yb-white);--yb-mobile-search-carat:var(--uh-search-carat-color, var(--yb-dory));--yb-dialpad:var(--yb-black);--yb-dialpad-label-text:#1b1b1b;--yb-dialpad-regular:#1d1d1d;--yb-dialpad-focus-background:var(--uh-dialpad-focus-background, #F1F2FF);--yb-dialpad-hover:rgba(43, 53, 218, 1);--yb-dialpad-outline:rgba(43, 53, 217, 0.45);--yb-dialpad-icon:#626262;--yb-dialpad-icon-hover:var(--uh-dialpad-icon-hover, #1d1d1d);--yb-dialpad-icon-backplate:var(--uh-dialpad-icon-backplate, rgb(19 19 19 / 10%));--yb-poweredby-container-bg:var(--yb-marshmallow);--yb-poweredby-text:var(--yb-black);--yb-l1-selected-color:var(--uh-l1-selected-color, var(--yb-text-primary));--yb-l1-selected-font-weight:var(--uh-l1-selected-font-weight, var(--yb-font-bold));--yb-l1-selected-hover-color:var(--uh-l1-selected-hover-color, var(--yb-text-primary));--yb-visual-viewport-height:100vh;--yb-sa-height-max:auto;--yb-sa-height-min:auto}html[data-color-theme-enabled][data-color-scheme=dark],html[data-color-theme-enabled] [data-color-scheme=dark]{--yb-sea-foam-fog:rgb(var(--yb-rgb-sea-foam) / 20%);--yb-cobalt-fog:rgb(var(--yb-rgb-cobalt) / 30%);--yb-scooter-fog:rgb(var(--yb-rgb-scooter) / 30%);--yb-dory-fog:rgb(var(--yb-rgb-dory) / 30%);--yb-sky-fog:rgb(var(--yb-rgb-sky) / 30%);--yb-peeps-fog:rgb(var(--yb-rgb-peeps) / 30%);--yb-grape-jelly-fog:rgb(var(--yb-rgb-grape-jelly) / 30%);--yb-hulk-pants-fog:rgb(var(--yb-rgb-hulk-pants) / 30%);--yb-starfish-fog:rgb(var(--yb-rgb-starfish) / 30%);--yb-blurple-fog:rgb(var(--yb-rgb-blurple) / 30%);--yb-logo-brand:var(--yb-white);--yb-logo-property:rgb(255 255 255 / 70%);--yb-finance-logo-brand:#B896E1;--yb-text-primary:var(--yb-gray-hair);--yb-text-secondary:var(--yb-bob);--yb-text-mail-primary:var(--yb-sky);--yb-text-mail-hover:var(--yb-sky);--yb-background-color:var(--yb-midnight);--yb-l2-background-color:transparent;--yb-popover-background:var(--uh-popover-background, var(--yb-batcave));--yb-tab-focus-accent:var(--yb-sky-accent);--yb-selected-item-background:var(--yb-inkwell);--yb-sidenav-close-btn-background:var(--yb-batcave);--yb-sidenav-btn-color-hover:var(--yb-sky);--yb-sidenav-btn-color-active:var(--yb-inkwell);--yb-sidenav-btn-text-active:var(--yb-bob);--yb-sidenav-finance-plus-icon:var(--yb-white);--yb-moremenu-finance-plus-icon:var(--yb-white);--yb-profile-avatar-bg:var(--uh-profile-avatar-bg, var(--yb-ramones));--yb-profile-panel-hover:var(--uh-profile-panel-hover-bg, var(--yb-ramones));--yb-profile-alphatar-background:var(--yb-ramones);--yb-profile-alphatar-background-pressed:var(--yb-batcave);--yb-signin-border-color:var(--yb-gandalf);--yb-mail-border-color:var(--yb-sky);--yb-profile-character-hover:var(--yb-midnight);--yb-profile-pressed-state:var(--yb-battleship);--yb-profile-character-pressed-state-bg:var(--yb-inkwell);--yb-profile-character-pressed-state-text:var(--yb-bob);--yb-profile-account-status-container-bg:var(--yb-white-fog);--yb-searchbox-bg:var(--yb-batcave);--yb-search-btn-hover:var(--yb-dory);--yb-search-faux-icon:var(--yb-marshmallow);--yb-search-input-box-shadow:0 0 1px 0 rgb(0 0 0 / 0%), 0 1px 0 0 var(--yb-midnight);--yb-search-assist-primary:var(--yb-marshmallow);--yb-search-assist-secondary:var(--yb-bob);--yb-search-assist-history:var(--yb-thanos);--yb-search-assist-history-pc:#be8fff;--yb-search-assist-back-btn:var(--yb-sky);--yb-search-assist-item-hover-background:var(--yb-ramones);--yb-elevation3-box-shadow:0 4px 8px 0 rgb(0 0 0 / 90%), 0 0 1px 0 rgb(0 0 0 / 92%);--yb-elevation4-box-shadow:0 4px 16px 0 rgb(0 0 0 / 80%), 0 0 2px 0 rgb(0 0 0 / 95%);--yb-l2-news-background:var(--yb-midnight);--yb-current-account-fog:var(--yb-white-fog);--yb-mail-count:var(--yb-sky);--yb-sports-l2-dropdown-hover-bg:var(--yb-selected-item-background);--yb-sports-l2-dropdown-hover-color:var(--yb-text-hover-color);--yb-sports-l2-dropdown-focus-bg:var(--yb-selected-item-background);--yb-topnav-bg:var(--yb-starfish);--yb-back-button-fill:var(--yb-white);--yb-sa-stock-up:#21d87d;--yb-sa-stock-down:#fc7a6e;--yb-sa-flight-label-border:1px solid var(--yb-marshmallow);--yb-sa-flight-label-positive:#1ac567;--yb-sa-flight-label-negative:#ff5257;--yb-popover-border:none;--yb-popover-border-radius:8px;--yb-popover-shadow:0 4px 8px 0 rgb(0 0 0 / 9%), 0 0 1px 0 rgb(0 0 0 / 9%);--yb-yns-title:var(--yb-marshmallow);--yb-l2-nav-selected:var(--yb-white);--yb-l2-nav-dropdown-focus:var(--yb-white);--yb-mail-icon-text:var(--yb-thanos);--yb-mail-signin-link:var(--yb-gray-hair);--yb-light-img-logo-display:none;--yb-dark-img-logo-display:block;--yb-account-item-border:var(--yb-inkwell);--yb-account-item-hover-bg:var(--yb-gandalf);--yb-account-info-bg:transparent;--yb-account-menu-item:var(--yb-inkwell);--yb-account-link:var(--yb-account-text);--yb-account-text:var(--yb-gray-hair);--yb-editions-button:var(--yb-selected-item-background);--yb-editions-modal-background:var(--yb-inkwell);--yb-editions-modal-outline:var(--yb-battleship);--yb-editions-text:var(--yb-gandalf);--yb-cobrand-logos-divider:var(--yb-gandalf);--yb-unread-container-bg:#06334d;--yb-l2-background:var(--uh-l2-background, var(--yb-l2-background-color));--yb-search-carat:var(--uh-search-carat-color, var(--yb-sky));--yb-l2-divider-color:var(--uh-l2-divider-color, var(--yb-sky));--yb-l2-text-color:var(--uh-l2-text-color, var(--yb-text-primary));--yb-default-font:var(--uh-font-family, var(--yb-default-font-family));--yb-fuji2-font:var(--uh-font-family, var(--yb-fuji2-font-family));--yb-uh3-cond-font:var(--uh-font-family, var(--yb-uh3-cond-font-family));--yb-outline:4px solid var(--uh-tab-focus-color, var(--yb-tab-focus-accent));--yb-search-btn-color:var(--uh-search-btn-color, var(--yb-search-carat));--yb-search-btn-hover-color:var(--uh-search-btn-hover-color, var(--yb-search-btn-hover));--yb-text-hover-color:var(--uh-text-hover-color, var(--yb-sidenav-btn-color-hover));--yb-profile-panel-hover-text:var(--uh-text-hover-color, var(--yb-sky));--yb-property-ruler-color:var(--uh-property-ruler-color, var(--yb-sky));--yb-svg-icon-color:var(--uh-icon-color, var(--yb-white));--yb-svg-icon-color-non-l2:var(--uh-icon-color-non-l2, var(--yb-white));--yb-svg-icon-color-selected:var(--uh-icon-color-selected, var(--yb-batcave));--yb-mail-bubble-color:var(--uh-mail-bubble-color, var(--yb-sky));--yb-tap-color:var(--uh-text-hover-color, var(--yb-text-primary));--yb-text-pressed-color:var(--uh-text-pressed-color, #1289ff);--yb-l2-margin:var(--uh-l2-margin);--yb-mobile-search-carat:var(--uh-search-carat-color, var(--yb-sky));--yb-dialpad:var(--yb-white);--yb-dialpad-label-text:#F0F3F5;--yb-dialpad-regular:var(--yb-bob);--yb-dialpad-focus-background:var(--uh-dialpad-focus-background, #414364);--yb-dialpad-hover:#96A1FF;--yb-dialpad-icon:#ffffff;--yb-dialpad-icon-hover:var(--uh-dialpad-icon-hover, #ffffff);--yb-dialpad-icon-backplate:var(--uh-dialpad-icon-backplate, rgb(255 255 255 / 10%));--yb-poweredby-container-bg:var(--yb-batcave);--yb-poweredby-text:var(--yb-white);--yb-l1-selected-color:var(--uh-l1-selected-color, var(--yb-text-primary));--yb-l1-selected-font-weight:var(--uh-l1-selected-font-weight, var(--yb-font-bold));--yb-l1-selected-hover-color:var(--uh-l1-selected-hover-color, var(--yb-text-primary))}html[data-color-theme-enabled][data-color-scheme=transparent],html[data-color-theme-enabled] [data-color-scheme=transparent]{--yb-sea-foam-fog:rgb(var(--yb-rgb-sea-foam) / 20%);--yb-cobalt-fog:rgb(var(--yb-rgb-cobalt) / 30%);--yb-scooter-fog:rgb(var(--yb-rgb-scooter) / 30%);--yb-dory-fog:rgb(var(--yb-rgb-dory) / 30%);--yb-sky-fog:rgb(var(--yb-rgb-sky) / 30%);--yb-peeps-fog:rgb(var(--yb-rgb-peeps) / 30%);--yb-grape-jelly-fog:rgb(var(--yb-rgb-grape-jelly) / 30%);--yb-hulk-pants-fog:rgb(var(--yb-rgb-hulk-pants) / 30%);--yb-starfish-fog:rgb(var(--yb-rgb-starfish) / 30%);--yb-blurple-fog:rgb(var(--yb-rgb-blurple) / 30%);--yb-logo-brand:var(--yb-white);--yb-logo-property:rgb(255 255 255 / 70%);--yb-finance-logo-brand:#B896E1;--yb-text-primary:var(--yb-gray-hair);--yb-text-secondary:var(--yb-bob);--yb-text-mail-primary:var(--yb-sky);--yb-text-mail-hover:var(--yb-sky);--yb-background-color:rgba(16, 21, 24, 0.5);--yb-l2-background-color:transparent;--yb-popover-background:var(--uh-popover-background, var(--yb-batcave));--yb-tab-focus-accent:var(--yb-sky-accent);--yb-selected-item-background:var(--yb-inkwell);--yb-sidenav-close-btn-background:var(--yb-batcave);--yb-sidenav-btn-color-hover:var(--yb-sky);--yb-sidenav-btn-color-active:var(--yb-inkwell);--yb-sidenav-btn-text-active:var(--yb-bob);--yb-sidenav-finance-plus-icon:var(--yb-white);--yb-sidenav-bg:rgba(16, 21, 24, 0.6);--yb-moremenu-finance-plus-icon:var(--yb-white);--yb-profile-avatar-bg:var(--uh-profile-avatar-bg, var(--yb-ramones));--yb-profile-panel-hover:var(--uh-profile-panel-hover-bg, var(--yb-ramones));--yb-profile-alphatar-background:var(--yb-ramones);--yb-profile-alphatar-background-pressed:var(--yb-batcave);--yb-signin-border-color:var(--yb-gandalf);--yb-mail-border-color:var(--yb-sky);--yb-profile-character-hover:var(--yb-midnight);--yb-profile-pressed-state:var(--yb-battleship);--yb-profile-character-pressed-state-bg:var(--yb-inkwell);--yb-profile-character-pressed-state-text:var(--yb-bob);--yb-profile-account-status-container-bg:var(--yb-white-fog);--yb-searchbox-bg:rgba(35, 42, 49, 0.50);--yb-search-btn-hover:var(--yb-dory);--yb-search-faux-icon:var(--yb-marshmallow);--yb-search-input-box-shadow:0 0 1px 0 rgb(0 0 0 / 0%), 0 1px 0 0 var(--yb-midnight);--yb-search-assist-primary:var(--yb-marshmallow);--yb-search-assist-secondary:var(--yb-bob);--yb-search-assist-history:var(--yb-thanos);--yb-search-assist-history-pc:#be8fff;--yb-search-assist-back-btn:var(--yb-sky);--yb-search-assist-item-hover-background:var(--yb-ramones);--yb-elevation3-box-shadow:none;--yb-elevation4-box-shadow:0 4px 16px 0 rgb(0 0 0 / 80%), 0 0 2px 0 rgb(0 0 0 / 95%);--yb-l2-news-background:initial;--yb-current-account-fog:var(--yb-white-fog);--yb-mail-indicator:var(--yb-batcave);--yb-mail-indicator-border:1px solid var(--yb-white);--yb-mail-count:var(--yb-white);--yb-text-shadow:0px 0px 1px rgba(0, 0, 0, 0.50), 0px 2px 4px rgba(0, 0, 0, 0.40);--yb-nav-color:var(--yb-text-primary);--yb-sports-l2-dropdown-hover-bg:var(--yb-selected-item-background);--yb-sports-l2-dropdown-hover-color:var(--yb-text-hover-color);--yb-nav-selected-color:var(--yb-white);--yb-sports-l2-dropdown-focus-bg:var(--yb-selected-item-background);--yb-topnav-bg:var(--yb-starfish);--yb-back-button-fill:var(--yb-white);--yb-sa-stock-up:#21d87d;--yb-sa-stock-down:#fc7a6e;--yb-sa-flight-label-border:1px solid var(--yb-marshmallow);--yb-sa-flight-label-positive:#1ac567;--yb-sa-flight-label-negative:#ff5257;--yb-popover-border:none;--yb-popover-border-radius:8px;--yb-popover-shadow:0 4px 8px 0 rgb(0 0 0 / 9%), 0 0 1px 0 rgb(0 0 0 / 9%);--yb-yns-title:var(--yb-marshmallow);--yb-l2-nav-selected:var(--yb-white);--yb-l2-nav-dropdown-focus:var(--yb-white);--yb-mail-icon-text:var(--yb-thanos);--yb-mail-signin-link:var(--yb-gray-hair);--yb-light-img-logo-display:none;--yb-dark-img-logo-display:block;--yb-account-item-border:var(--yb-inkwell);--yb-account-item-hover-bg:var(--yb-gandalf);--yb-account-info-bg:transparent;--yb-account-menu-item:var(--yb-inkwell);--yb-account-link:var(--yb-account-text);--yb-account-text:var(--yb-gray-hair);--yb-editions-button:var(--yb-selected-item-background);--yb-editions-modal-background:var(--yb-inkwell);--yb-cobrand-logos-divider:var(--yb-gandalf);--yb-icon-filter:drop-shadow(0 1px 2px rgb(0 0 0 / 40%));--yb-transparent-popover-background:rgba(16, 21, 24, 0.8);--yb-transparent-popover-backdrop-filter:blur(4px);--yb-transparent-popover-box-shadow:0 4px 8px 0 rgb(0 0 0 / 10%), 0 0 1px 0 rgb(0 0 0 / 10%);--yb-unread-container-bg:#06334d;--yb-l2-background:var(--uh-l2-background, var(--yb-l2-background-color));--yb-search-carat:var(--uh-search-carat-color, var(--yb-sky));--yb-l2-divider-color:var(--uh-l2-divider-color, var(--yb-sky));--yb-l2-text-color:var(--uh-l2-text-color, var(--yb-text-primary));--yb-default-font:var(--uh-font-family, var(--yb-default-font-family));--yb-fuji2-font:var(--uh-font-family, var(--yb-fuji2-font-family));--yb-uh3-cond-font:var(--uh-font-family, var(--yb-uh3-cond-font-family));--yb-outline:4px solid var(--uh-tab-focus-color, var(--yb-tab-focus-accent));--yb-search-btn-color:var(--uh-search-btn-color, var(--yb-search-carat));--yb-search-btn-hover-color:var(--uh-search-btn-hover-color, var(--yb-search-btn-hover));--yb-text-hover-color:var(--uh-text-hover-color, var(--yb-sidenav-btn-color-hover));--yb-profile-panel-hover-text:var(--uh-text-hover-color, var(--yb-sky));--yb-property-ruler-color:var(--uh-property-ruler-color, var(--yb-sky));--yb-svg-icon-color:var(--uh-icon-color, var(--yb-white));--yb-svg-icon-color-non-l2:var(--uh-icon-color-non-l2, var(--yb-white));--yb-svg-icon-color-selected:var(--uh-icon-color-selected, var(--yb-batcave));--yb-mail-bubble-color:var(--uh-mail-bubble-color, var(--yb-sky));--yb-tap-color:var(--uh-text-hover-color, var(--yb-text-primary));--yb-text-pressed-color:var(--uh-text-pressed-color, #1289ff);--yb-l2-margin:var(--uh-l2-margin);--yb-mobile-search-carat:var(--uh-search-carat-color, var(--yb-sky));--yb-l1-selected-color:var(--uh-l1-selected-color, var(--yb-text-primary));--yb-l1-selected-font-weight:var(--uh-l1-selected-font-weight, var(--yb-font-bold));--yb-l1-selected-hover-color:var(--uh-l1-selected-hover-color, var(--yb-text-primary))}@media (prefers-color-scheme:dark){html[data-color-theme-enabled]{--yb-sea-foam-fog:rgb(var(--yb-rgb-sea-foam) / 20%);--yb-cobalt-fog:rgb(var(--yb-rgb-cobalt) / 30%);--yb-scooter-fog:rgb(var(--yb-rgb-scooter) / 30%);--yb-dory-fog:rgb(var(--yb-rgb-dory) / 30%);--yb-sky-fog:rgb(var(--yb-rgb-sky) / 30%);--yb-peeps-fog:rgb(var(--yb-rgb-peeps) / 30%);--yb-grape-jelly-fog:rgb(var(--yb-rgb-grape-jelly) / 30%);--yb-hulk-pants-fog:rgb(var(--yb-rgb-hulk-pants) / 30%);--yb-starfish-fog:rgb(var(--yb-rgb-starfish) / 30%);--yb-blurple-fog:rgb(var(--yb-rgb-blurple) / 30%);--yb-logo-brand:var(--yb-white);--yb-logo-property:rgb(255 255 255 / 70%);--yb-finance-logo-brand:#B896E1;--yb-text-primary:var(--yb-gray-hair);--yb-text-secondary:var(--yb-bob);--yb-text-mail-primary:var(--yb-sky);--yb-text-mail-hover:var(--yb-sky);--yb-background-color:var(--yb-midnight);--yb-l2-background-color:transparent;--yb-popover-background:var(--uh-popover-background, var(--yb-batcave));--yb-tab-focus-accent:var(--yb-sky-accent);--yb-selected-item-background:var(--yb-inkwell);--yb-sidenav-close-btn-background:var(--yb-batcave);--yb-sidenav-btn-color-hover:var(--yb-sky);--yb-sidenav-btn-color-active:var(--yb-inkwell);--yb-sidenav-btn-text-active:var(--yb-bob);--yb-sidenav-finance-plus-icon:var(--yb-white);--yb-moremenu-finance-plus-icon:var(--yb-white);--yb-profile-avatar-bg:var(--uh-profile-avatar-bg, var(--yb-ramones));--yb-profile-panel-hover:var(--uh-profile-panel-hover-bg, var(--yb-ramones));--yb-profile-alphatar-background:var(--yb-ramones);--yb-profile-alphatar-background-pressed:var(--yb-batcave);--yb-signin-border-color:var(--yb-gandalf);--yb-mail-border-color:var(--yb-sky);--yb-profile-character-hover:var(--yb-midnight);--yb-profile-pressed-state:var(--yb-battleship);--yb-profile-character-pressed-state-bg:var(--yb-inkwell);--yb-profile-character-pressed-state-text:var(--yb-bob);--yb-profile-account-status-container-bg:var(--yb-white-fog);--yb-searchbox-bg:var(--yb-batcave);--yb-search-btn-hover:var(--yb-dory);--yb-search-faux-icon:var(--yb-marshmallow);--yb-search-input-box-shadow:0 0 1px 0 rgb(0 0 0 / 0%), 0 1px 0 0 var(--yb-midnight);--yb-search-assist-primary:var(--yb-marshmallow);--yb-search-assist-secondary:var(--yb-bob);--yb-search-assist-history:var(--yb-thanos);--yb-search-assist-history-pc:#be8fff;--yb-search-assist-back-btn:var(--yb-sky);--yb-search-assist-item-hover-background:var(--yb-ramones);--yb-elevation3-box-shadow:0 4px 8px 0 rgb(0 0 0 / 90%), 0 0 1px 0 rgb(0 0 0 / 92%);--yb-elevation4-box-shadow:0 4px 16px 0 rgb(0 0 0 / 80%), 0 0 2px 0 rgb(0 0 0 / 95%);--yb-l2-news-background:var(--yb-midnight);--yb-current-account-fog:var(--yb-white-fog);--yb-mail-count:var(--yb-sky);--yb-sports-l2-dropdown-hover-bg:var(--yb-selected-item-background);--yb-sports-l2-dropdown-hover-color:var(--yb-text-hover-color);--yb-sports-l2-dropdown-focus-bg:var(--yb-selected-item-background);--yb-topnav-bg:var(--yb-starfish);--yb-back-button-fill:var(--yb-white);--yb-sa-stock-up:#21d87d;--yb-sa-stock-down:#fc7a6e;--yb-sa-flight-label-border:1px solid var(--yb-marshmallow);--yb-sa-flight-label-positive:#1ac567;--yb-sa-flight-label-negative:#ff5257;--yb-popover-border:none;--yb-popover-border-radius:8px;--yb-popover-shadow:0 4px 8px 0 rgb(0 0 0 / 9%), 0 0 1px 0 rgb(0 0 0 / 9%);--yb-yns-title:var(--yb-marshmallow);--yb-l2-nav-selected:var(--yb-white);--yb-l2-nav-dropdown-focus:var(--yb-white);--yb-mail-icon-text:var(--yb-thanos);--yb-mail-signin-link:var(--yb-gray-hair);--yb-light-img-logo-display:none;--yb-dark-img-logo-display:block;--yb-account-item-border:var(--yb-inkwell);--yb-account-item-hover-bg:var(--yb-gandalf);--yb-account-info-bg:transparent;--yb-account-menu-item:var(--yb-inkwell);--yb-account-link:var(--yb-account-text);--yb-account-text:var(--yb-gray-hair);--yb-editions-button:var(--yb-selected-item-background);--yb-editions-modal-background:var(--yb-inkwell);--yb-editions-modal-outline:var(--yb-battleship);--yb-editions-text:var(--yb-gandalf);--yb-cobrand-logos-divider:var(--yb-gandalf);--yb-unread-container-bg:#06334d;--yb-l2-background:var(--uh-l2-background, var(--yb-l2-background-color));--yb-search-carat:var(--uh-search-carat-color, var(--yb-sky));--yb-l2-divider-color:var(--uh-l2-divider-color, var(--yb-sky));--yb-l2-text-color:var(--uh-l2-text-color, var(--yb-text-primary));--yb-default-font:var(--uh-font-family, var(--yb-default-font-family));--yb-fuji2-font:var(--uh-font-family, var(--yb-fuji2-font-family));--yb-uh3-cond-font:var(--uh-font-family, var(--yb-uh3-cond-font-family));--yb-outline:4px solid var(--uh-tab-focus-color, var(--yb-tab-focus-accent));--yb-search-btn-color:var(--uh-search-btn-color, var(--yb-search-carat));--yb-search-btn-hover-color:var(--uh-search-btn-hover-color, var(--yb-search-btn-hover));--yb-text-hover-color:var(--uh-text-hover-color, var(--yb-sidenav-btn-color-hover));--yb-profile-panel-hover-text:var(--uh-text-hover-color, var(--yb-sky));--yb-property-ruler-color:var(--uh-property-ruler-color, var(--yb-sky));--yb-svg-icon-color:var(--uh-icon-color, var(--yb-white));--yb-svg-icon-color-non-l2:var(--uh-icon-color-non-l2, var(--yb-white));--yb-svg-icon-color-selected:var(--uh-icon-color-selected, var(--yb-batcave));--yb-mail-bubble-color:var(--uh-mail-bubble-color, var(--yb-sky));--yb-tap-color:var(--uh-text-hover-color, var(--yb-text-primary));--yb-text-pressed-color:var(--uh-text-pressed-color, #1289ff);--yb-l2-margin:var(--uh-l2-margin);--yb-mobile-search-carat:var(--uh-search-carat-color, var(--yb-sky));--yb-dialpad:var(--yb-white);--yb-dialpad-label-text:#F0F3F5;--yb-dialpad-regular:var(--yb-bob);--yb-dialpad-focus-background:var(--uh-dialpad-focus-background, #414364);--yb-dialpad-hover:#96A1FF;--yb-dialpad-icon:#ffffff;--yb-dialpad-icon-hover:var(--uh-dialpad-icon-hover, #ffffff);--yb-dialpad-icon-backplate:var(--uh-dialpad-icon-backplate, rgb(255 255 255 / 10%));--yb-poweredby-container-bg:var(--yb-batcave);--yb-poweredby-text:var(--yb-white);--yb-l1-selected-color:var(--uh-l1-selected-color, var(--yb-text-primary));--yb-l1-selected-font-weight:var(--uh-l1-selected-font-weight, var(--yb-font-bold));--yb-l1-selected-hover-color:var(--uh-l1-selected-hover-color, var(--yb-text-primary))}html[data-color-theme-enabled] [data-maintain-color]{--yb-barbie:#f80e5d;--yb-barney:#cc008c;--yb-batcave:#232a31;--yb-battleship:#5b636a;--yb-black:#000;--yb-blurple:#5d5eff;--yb-bob:#b0b9c1;--yb-bonsai:#008751;--yb-charcoal:#464e56;--yb-cobalt:#003abc;--yb-denim:#1a0dab;--yb-dirty-seagull:#e0e4e9;--yb-dolphin:#6e7780;--yb-dory:#0f69ff;--yb-gandalf:#979ea8;--yb-grape-jelly:#6001d2;--yb-gray-hair:#f0f3f5;--yb-hulk-pants:#7e1fff;--yb-inkwell:#1d2228;--yb-malbec:#39007d;--yb-marshmallow:#f5f8fa;--yb-midnight:#101518;--yb-mulberry:#5015b0;--yb-pebble:#c7cdd2;--yb-peeps:#7dcbff;--yb-ramones:#2c363f;--yb-sea-foam:#11d3cd;--yb-scooter:#0063eb;--yb-shark:#828a93;--yb-sky:#12a9ff;--yb-smurfette:#188fff;--yb-solo-cup:#eb0f29;--yb-starfish:#7759ff;--yb-swedish-fish:#ff333a;--yb-thanos:#907cff;--yb-turmeric:#ffa700;--yb-white:#fff;--yb-rgb-battleship:91 99 106;--yb-rgb-white:255 255 255;--yb-rgb-blurple:93 94 255;--yb-rgb-starfish:119 89 255;--yb-rgb-hulk-pants:126 31 255;--yb-rgb-grape-jelly:96 1 210;--yb-rgb-peeps:125 203 255;--yb-rgb-sky:18 169 255;--yb-rgb-dory:15 105 255;--yb-rgb-scooter:0 99 235;--yb-rgb-cobalt:0 58 188;--yb-rgb-sea-foam:17 211 205;--yb-battleship-fog:rgb(var(--yb-rgb-battleship) / 10%);--yb-white-fog:rgb(var(--yb-rgb-white) / 10%);--yb-blurple-fog:rgb(var(--yb-rgb-blurple) / 10%);--yb-starfish-fog:rgb(var(--yb-rgb-starfish) / 10%);--yb-hulk-pants-fog:rgb(var(--yb-rgb-hulk-pants) / 10%);--yb-grape-jelly-fog:rgb(var(--yb-rgb-grape-jelly) / 10%);--yb-peeps-fog:rgb(var(--yb-rgb-peeps) / 10%);--yb-sky-fog:rgb(var(--yb-rgb-sky) / 10%);--yb-dory-fog:rgb(var(--yb-rgb-dory) / 10%);--yb-scooter-fog:rgb(var(--yb-rgb-scooter) / 10%);--yb-cobalt-fog:rgb(var(--yb-rgb-cobalt) / 10%);--yb-sea-foam-fog:rgb(var(--yb-rgb-sea-foam) / 10%);--yb-hulk-pants-accent:rgb(var(--yb-rgb-hulk-pants) / 50%);--yb-grape-jelly-accent:rgb(var(--yb-rgb-grape-jelly) / 50%);--yb-sky-accent:rgb(var(--yb-rgb-sky) / 50%);--yb-logo-brand:var(--yb-grape-jelly);--yb-logo-property:var(--yb-batcave);--yb-finance-logo-brand:var(--yb-grape-jelly);--yb-text-primary:var(--yb-batcave);--yb-text-secondary:var(--yb-battleship);--yb-text-mail-primary:var(--yb-grape-jelly);--yb-text-mail-hover:var(--yb-mulberry);--yb-background-color:var(--yb-white);--yb-l2-background-color:var(--yb-marshmallow);--yb-popover-background:var(--uh-popover-background, var(--yb-white));--yb-tab-focus-accent:var(--yb-grape-jelly-accent);--yb-selected-item-background:var(--yb-marshmallow);--yb-sidenav-close-btn-background:var(--yb-marshmallow);--yb-sidenav-btn-color-hover:var(--yb-grape-jelly);--yb-sidenav-btn-color-active:var(--yb-dirty-seagull);--yb-sidenav-btn-text-active:var(--yb-batcave);--yb-sidenav-finance-plus-icon:var(--yb-svg-icon-color-non-l2);--yb-sidenav-bg:var(--yb-background-color);--yb-moremenu-finance-plus-icon:var(--yb-svg-icon-color-non-l2);--yb-profile-avatar-bg:var(--uh-profile-avatar-bg, var(--yb-gray-hair));--yb-profile-panel-hover:var(--uh-profile-panel-hover-bg, var(--yb-dirty-seagull));--yb-profile-alphatar-background:var(--yb-gray-hair);--yb-profile-alphatar-background-pressed:var(--yb-white);--yb-signin-border-color:var(--yb-dirty-seagull);--yb-mail-border-color:var(--yb-grape-jelly);--yb-unread-container-bg:rgb(var(--yb-rgb-grape-jelly) / 5%);--yb-profile-character-hover:var(--yb-white);--yb-profile-pressed-state:var(--yb-bob);--yb-profile-character-pressed-state-bg:var(--yb-dirty-seagull);--yb-profile-character-pressed-state-text:var(--yb-batcave);--yb-profile-account-status-container-bg:var(--yb-battleship-fog);--yb-searchbox-bg:var(--yb-gray-hair);--yb-search-btn-hover:#5409b2;--yb-search-faux-icon:var(--yb-batcave);--yb-search-input-box-shadow:0 0 1px 0 rgb(0 0 0 / 0%), 0 1px 0 0 var(--yb-dirty-seagull);--yb-search-assist-primary:var(--yb-batcave);--yb-search-assist-secondary:var(--yb-dolphin);--yb-search-assist-history:#6e329d;--yb-search-assist-history-pc:var(--yb-grape-jelly);--yb-search-assist-back-btn:var(--yb-dory);--yb-search-assist-item-hover-background:var(--yb-marshmallow);--yb-elevation3-box-shadow:0 4px 8px 0 rgb(0 0 0 / 10%), 0 0 1px 0 rgb(0 0 0 / 10%);--yb-elevation4-box-shadow:0 4px 16px 0 rgb(0 0 0 / 20%), 0 0 2px 0 rgb(0 0 0 / 5%);--yb-l2-news-background:linear-gradient(0deg, rgb(255 255 255 / 100%) 0%, rgb(248 250 251 / 100%) 100%);--yb-current-account-fog:var(--yb-dory-fog);--yb-editions-modal-outline:var(--yb-dirty-seagull);--yb-editions-text:var(--yb-dolphin);--yb-cobrand-logos-divider:var(--yb-dirty-seagull);--yb-l2-background:var(--uh-l2-background, var(--yb-l2-background-color));--yb-search-carat:var(--uh-search-carat-color, var(--yb-grape-jelly));--yb-l2-divider-color:var(--uh-l2-divider-color, var(--yb-gray-hair));--yb-l2-text-color:var(--uh-l2-text-color, var(--yb-batcave));--yb-default-font:var(--uh-font-family, var(--yb-default-font-family));--yb-fuji2-font:var(--uh-font-family, var(--yb-fuji2-font-family));--yb-uh3-cond-font:var(--uh-font-family, var(--yb-uh3-cond-font-family));--yb-outline:4px solid var(--uh-tab-focus-color, var(--yb-tab-focus-accent));--yb-search-btn-color:var(--uh-search-btn-color, var(--yb-search-carat));--yb-search-btn-hover-color:var(--uh-search-btn-hover-color, var(--yb-search-btn-hover));--yb-text-hover-color:var(--uh-text-hover-color, var(--yb-sidenav-btn-color-hover));--yb-profile-panel-hover-text:var(--uh-text-hover-color, var(--yb-grape-jelly));--yb-searchbox-transition:border-radius 0s 0.2s ease-out, background-color 0s 0.2s ease-out, box-shadow 0.2s ease-out, padding-left 50ms 150ms ease-out;--yb-property-ruler-color:var(--uh-property-ruler-color, var(--yb-grape-jelly));--yb-svg-icon-color:var(--uh-icon-color, var(--yb-grape-jelly));--yb-svg-icon-color-non-l2:var(--uh-icon-color-non-l2, var(--yb-grape-jelly) );--yb-svg-icon-color-selected:var(--uh-icon-color-selected, var(--yb-white));--yb-mail-bubble-color:var(--uh-mail-bubble-color, var(--yb-grape-jelly));--yb-mail-indicator:var(--yb-profile-panel-hover-text);--yb-mail-indicator-border:1px solid var(--yb-background-color);--yb-mail-count:var(--yb-text-primary);--yb-tap-color:var(--uh-text-hover-color, var(--yb-text-primary));--yb-text-pressed-color:var(--uh-text-pressed-color, var(--yb-mulberry));--yb-l2-margin:var(--uh-l2-margin);--yb-text-shadow:none;--yb-nav-color:var(--yb-text-secondary);--yb-sports-l2-dropdown-hover-bg:#e6e6e6;--yb-sports-l2-dropdown-hover-color:var(--yb-black);--yb-sports-l2-dropdown-focus-bg:#ecf8ff;--yb-nav-selected-color:var(--yb-text-primary);--yb-topnav-bg:var(--yb-batcave);--yb-back-button-fill:var(--yb-batcave);--yb-sa-stock-up:var(--yb-bonsai);--yb-sa-stock-down:var(--yb-solo-cup);--yb-sa-flight-label-border:1px solid var(--yb-dirty-seagull);--yb-sa-flight-label-positive:var(--yb-bonsai);--yb-sa-flight-label-negative:var(--yb-solo-cup);--yb-popover-border:1px solid #d8dade;--yb-popover-border-radius:4px;--yb-popover-shadow:0 2px 8px 0 rgb(0 0 0 / 36%);--yb-yns-title:var(--batcave);--yb-l2-nav-selected:var(--yb-grape-jelly);--yb-l2-nav-dropdown-focus:var(--yb-text-hover-color);--yb-mail-icon-text:var(--yb-batcave);--yb-mail-signin-link:#0078ff;--yb-light-img-logo-display:block;--yb-dark-img-logo-display:none;--yb-account-item-border:#d8dade;--yb-account-item-hover-bg:rgb(227 236 248);--yb-account-info-bg:#ededf3;--yb-account-menu-item:var(--yb-white);--yb-account-link:var(--yb-smurfette);--yb-account-text:#26282a;--yb-editions-button:var(--yb-marshmallow);--yb-editions-modal-background:var(--yb-white);--yb-mobile-search-carat:var(--uh-search-carat-color, var(--yb-dory));--yb-dialpad:var(--yb-black);--yb-dialpad-label-text:#1b1b1b;--yb-dialpad-regular:#1d1d1d;--yb-dialpad-focus-background:var(--uh-dialpad-focus-background, #F1F2FF);--yb-dialpad-hover:rgba(43, 53, 218, 1);--yb-dialpad-outline:rgba(43, 53, 217, 0.45);--yb-dialpad-icon:#626262;--yb-dialpad-icon-hover:var(--uh-dialpad-icon-hover, #1d1d1d);--yb-dialpad-icon-backplate:var(--uh-dialpad-icon-backplate, rgb(19 19 19 / 10%));--yb-poweredby-container-bg:var(--yb-marshmallow);--yb-poweredby-text:var(--yb-black);--yb-l1-selected-color:var(--uh-l1-selected-color, var(--yb-text-primary));--yb-l1-selected-font-weight:var(--uh-l1-selected-font-weight, var(--yb-font-bold));--yb-l1-selected-hover-color:var(--uh-l1-selected-hover-color, var(--yb-text-primary));--yb-visual-viewport-height:100vh;--yb-sa-height-max:auto;--yb-sa-height-min:auto}}.use-homepage-blue{--yb-search-btn-color:var(--yb-dory);--yb-outline:4px solid var(--yb-dory);--yb-search-btn-hover-color:var(--yb-scooter);--yb-text-hover-color:var(--yb-dory);--yb-mail-count:var(--yb-dory);--yb-mail-indicator:var(--yb-dory);--yb-profile-panel-hover-text:var(--yb-dory);--yb-logo-brand:var(--yb-batcave);--yb-logo-property:rgb(35 42 49 / 70%);--yb-search-carat:var(--yb-dory);--yb-text-pressed-color:var(--yb-scooter)}.use-homepage-blue-v2{--yb-profile-avatar-bg:#ecf7ff;--yb-searchbox-bg:#ecf7ff}.ybar-property-finance{--yb-mail-border-color:var(--uh-mail-bubble-color);--yb-text-mail-primary:var(--uh-mail-bubble-color);--yb-text-mail-hover:var(--uh-mail-bubble-color)}._yb_5t6xy._yb_1u9bsx1,._yb_3zjksb._yb_1u9bsx1{--yb-text-hover-color:var(--yb-color-att-dark-blue);--yb-search-btn-color:var(--yb-color-att-dark-blue);--yb-search-btn-hover-color:var(--yb-color-att-dark-blue);--yb-profile-panel-hover-text:var(--yb-color-att-dark-blue);--yb-mail-border-color:var(--yb-color-att-dark-blue);--yb-text-mail-primary:var(--yb-color-att-dark-blue);--yb-text-mail-hover:var(--yb-color-att-dark-blue)}._yb_1bwhbi8._yb_iswvcq,._yb_1bqr3cv._yb_iswvcq{--yb-text-hover-color:var(--yb-color-rogers-red);--yb-search-btn-color:var(--yb-color-rogers-red);--yb-search-btn-hover-color:var(--yb-color-rogers-red);--yb-profile-panel-hover-text:var(--yb-color-rogers-red);--yb-mail-border-color:var(--yb-color-rogers-red);--yb-text-mail-primary:var(--yb-color-rogers-red);--yb-text-mail-hover:var(--yb-color-rogers-red)}
/*! Copyright 2017 Yahoo Holdings, Inc. All rights reserved. */
template{display:none}._yb_bbmou4{font-family:"Helvetica Neue",Helvetica,Tahoma,Geneva,Arial,sans-serif;font-family:var(--yb-default-font);font-weight:400;font-weight:var(--yb-font-regular);font-stretch:normal;direction:ltr;display:block;box-sizing:border-box;text-align:start;-webkit-font-smoothing:antialiased;z-index:1000;overflow-anchor:none}.ybar-ytheme-crunch._yb_bbmou4,.ybar-ytheme-fuji2._yb_bbmou4{font-family:"YahooSans VF",YahooSans,"Yahoo Sans","Helvetica Neue",Helvetica,Arial,sans-serif;font-family:var(--yb-fuji2-font)}html[data-color-theme-enabled][data-color-scheme=transparent] .ybar-ytheme-crunch{text-shadow:none;text-shadow:var(--yb-text-shadow)}html[data-color-theme-enabled][data-color-scheme=transparent] .ybar-ytheme-crunch button,html[data-color-theme-enabled][data-color-scheme=transparent] .ybar-ytheme-crunch ._yb_1muppqd,html[data-color-theme-enabled][data-color-scheme=transparent] #ybar-nav-placement{text-shadow:none}html[data-color-theme-enabled][data-color-scheme=transparent] .ybar-ytheme-crunch svg{filter:var(--yb-icon-filter)}html[data-color-theme-enabled][data-color-scheme=transparent] .ybar-ytheme-crunch button svg,html[data-color-theme-enabled][data-color-scheme=transparent] .ybar-ytheme-crunch ._yb_1muppqd svg,html[data-color-theme-enabled][data-color-scheme=transparent] #ybar-nav-placement svg{filter:none}#ybar._yb_v1r5ph{margin:0 auto}._yb_bbmou4 ._yb_16kwvy4{display:flex;flex-direction:column}.ybar-ytheme-crunch #ybar-inner-wrap{background:#fff;background:var(--yb-background-color)}._yb_1u6ljkh{display:flex;justify-content:center}._yb_dzr3yz{display:flex;justify-content:flex-start;align-items:center;width:100%;min-width:0;max-width:1920px;box-sizing:border-box;transition:margin .2s ease-out,opacity .15s linear .2s;opacity:1}.ybar-ytheme-crunch ._yb_dzr3yz{max-width:calc(100% - 32px);max-width:var(--uh-max-width, calc(100% - 32px))}.typing .ybar-ytheme-crunch ._yb_16kwvy4{transition:margin .2s ease-out}.ybar-ytheme-fuji2 ._yb_dzr3yz{max-width:1340px;padding:0 20px;transition:margin .2s ease-out,opacity .15s linear .2s,padding .4s}.ybar-property-sports._yb_115wtas #ybar-navigation{max-width:100%;padding:0}.ybar-sticky #ybar._yb_115wtas #ybar-inner-wrap{left:0}.ybar-ytheme-fuji2._yb_2erz1k ._yb_dzr3yz,.ybar-ytheme-fuji2._yb_2erz1k ._yb_1smugqe ._yb_dzr3yz{padding:0;margin:0}.ybar-ytheme-fuji2._yb_2erz1k ._yb_1u6ljkh._yb_1emv3g7,.ybar-ytheme-fuji2._yb_2erz1k ._yb_1u6ljkh._yb_1smugqe{padding:0 20px;transition:padding .4s}@media screen and (min-width:1020px){.ybar-ytheme-fuji2 ._yb_dzr3yz{padding:0 40px}.ybar-ytheme-fuji2._yb_2erz1k ._yb_1u6ljkh._yb_1emv3g7,.ybar-ytheme-fuji2._yb_2erz1k ._yb_1u6ljkh._yb_1smugqe{padding:0 40px}}.ybar-ytheme-fuji2._yb_1sik2rh ._yb_dzr3yz{padding:0;max-width:100%}.ybar-ytheme-fuji2._yb_jrobl8._yb_2erz1k ._yb_dzr3yz{max-width:1264px;min-width:0}._yb_1jtlmql ._yb_dzr3yz,.ybar-ytheme-fuji2._yb_ist0jp ._yb_1jtlmql ._yb_dzr3yz,.ybar-ytheme-fuji2._yb_2erz1k ._yb_1jtlmql ._yb_dzr3yz{padding:0;position:relative;display:none}.ybar-ytheme-fuji2 ._yb_1jtlmql ._yb_dzr3yz{display:flex}.ybar-ytheme-fuji2.ybar-property-mail.fuji2-dialpad ._yb_1jtlmql{display:none}._yb_1s3151i{background:#232a31;background:var(--yb-topnav-bg);flex:1}.ybar-dark ._yb_1s3151i{background:#464e56;background:var(--yb-charcoal)}.ybar-dark ._yb_vp1vt8 ._yb_1s3151i{background-color:#7e1fff;background-color:var(--yb-hulk-pants)}.ybar-ytheme-fuji2 ._yb_1jtlmql{overflow:hidden}@media screen and (max-width:1340px){.ybar-ytheme-fuji2 ._yb_1jtlmql{width:100%;min-width:1032px}.ybar-ytheme-fuji2 ._yb_1s3151i{display:none}}._yb_wutdic{min-width:0;padding:0;height:84px}._yb_v5t4r9 ._yb_wutdic,._yb_1os1r4w ._yb_wutdic,._yb_v1r5ph ._yb_wutdic,._yb_1q0vcon ._yb_wutdic,._yb_1fg69p4 ._yb_wutdic,._yb_69p0bm ._yb_wutdic,._yb_zfzvr2 ._yb_wutdic{padding:0 64px 0 50px}._yb_v9nhk7._yb_1u6ljkh._yb_1muppqd{padding:0 16px;margin-bottom:8px;justify-content:center;width:auto}._yb_ist0jp ._yb_v9nhk7._yb_1u6ljkh._yb_1muppqd,.ybar-property-generic ._yb_v9nhk7._yb_1u6ljkh._yb_1muppqd,.ybar-property-homepage ._yb_v9nhk7._yb_1u6ljkh._yb_1muppqd{margin-bottom:16px}._yb_1smugqe ._yb_dzr3yz{height:34px;padding-left:24px}.ybar-ytheme-crunch ._yb_1smugqe._yb_l2-news{background:linear-gradient(0deg,rgb(255 255 255/100%) 0,rgb(248 250 251/100%) 100%);background:var(--yb-l2-news-background)}.ybar-ytheme-crunch ._yb_1smugqe._yb_mjszbe{background:#f5f8fa;background:var(--yb-l2-background)}.ybar-ytheme-crunch ._yb_1smugqe._yb_cg7fqk{box-shadow:0 -1px 0 #f0f3f5;box-shadow:0 -1px 0 var(--yb-l2-divider-color)}.ybar-ytheme-crunch ._yb_1smugqe._yb_15sv7cz{background-color:#6001d2;background-color:var(--yb-grape-jelly)}.ybar-ytheme-crunch.ybar-property-finance ._yb_1smugqe{background-color:#f5f8fa;background-color:var(--yb-l2-background)}.ybar-ytheme-fuji2 ._yb_wutdic{height:72px}.ybar-ytheme-fuji2._yb_1sik2rh._yb_189cmx2 ._yb_wutdic{height:84px}._yb_ist0jp ._yb_wutdic,._yb_2erz1k ._yb_wutdic{height:80px;padding:5px 0 7px;box-sizing:content-box}#ybar.ybar-ytheme-crunch.ybar-property-homepage._yb_phn8ti._yb_1u9bsx1,#ybar.ybar-ytheme-crunch._yb_phn8ti{--uh-visible-height:112px;height:112px}#ybar.ybar-ytheme-crunch.ybar-property-finance._yb_phn8ti{--uh-visible-height:117px;height:117px}#ybar.ybar-ytheme-crunch.ybar-property-finance._yb_phn8ti._yb_kx18b1{--uh-visible-height:113px;height:113px}#ybar.ybar-ytheme-crunch.ybar-property-homepage._yb_phn8ti,#ybar.ybar-ytheme-crunch,.modal-open #ybar.ybar-ytheme-crunch,.modal-open #ybar.ybar-ytheme-crunch.ybar-property-homepage._yb_phn8ti._yb_1u9bsx1,#ybar.ybar-ytheme-crunch ._yb_wutdic{--uh-visible-height:64px;height:64px;padding:0}.ybar-ytheme-fuji2 ._yb_1smugqe ._yb_dzr3yz,.ybar-ytheme-fuji2 ._yb_wutdic{padding:0 20px;box-sizing:border-box}.ybar-ytheme-crunch ._yb_1smugqe ._yb_dzr3yz{height:48px;padding:0}.ybar-ytheme-crunch.ybar-property-finance:not(._yb_kx18b1) ._yb_1smugqe ._yb_dzr3yz{height:50px}@media screen and (min-width:1020px){.ybar-ytheme-fuji2 ._yb_1smugqe ._yb_dzr3yz,.ybar-ytheme-fuji2 ._yb_wutdic{padding:0 40px}}.ybar-ytheme-fuji2._yb_1sik2rh ._yb_wutdic{padding:0}.modal-open ._yb_1smugqe,.ybar-hide-navigation ._yb_1smugqe{overflow:hidden}.ybar-hide-navigation .ybar-ytheme-crunch._yb_rq7mm4 ._yb_1smugqe{overflow:unset}.ybar-hide-navigation #ybar ._yb_1smugqe._yb_hwx3um>._yb_dzr3yz{margin-top:-82px}.modal-open #ybar ._yb_1smugqe>._yb_dzr3yz,.ybar-hide-navigation #ybar ._yb_1smugqe>._yb_dzr3yz{opacity:0;margin-top:-60px}.ybar-ytheme-crunch._yb_kx18b1 ._yb_1wmqq67{margin:0;height:1px;background-color:#f0f3f5;background-color:var(--yb-gray-hair)}.ybar-ytheme-crunch:not(._yb_kx18b1)._yb_rq7mm4 ._yb_16kwvy4{border-bottom:#f0f3f5 3px solid;border-bottom:var(--yb-l2-divider-color) 3px solid}.ybar-ytheme-crunch ._yb_15sv7cz #ybar-navigation{height:49px}.ybar-hide-navigation #ybar.ybar-ytheme-crunch._yb_rq7mm4 ._yb_1smugqe>._yb_dzr3yz{opacity:1;margin-top:0}.ybar-hide-navigation #ybar ._yb_19s5ae1{opacity:0;margin-top:-42px}.modal-open ._yb_1jtlmql>._yb_dzr3yz,.ybar-hide-topnavigation ._yb_1jtlmql>._yb_dzr3yz{margin-top:-39px}@media screen and (max-width:1340px){.ybar-ytheme-fuji2 ._yb_1jtlmql{width:100%;min-width:1032px}.ybar-ytheme-fuji2._yb_rq7mm4._yb_19dmi0p ._yb_1jtlmql{min-width:initial}.ybar-ytheme-fuji2 ._yb_1s3151i{display:none}}@media screen and (min-width:1020px){.ybar-ytheme-fuji2 ._yb_ist0jp ._yb_wutdic ._yb_2qgpj,.ybar-ytheme-fuji2.ybar-property-homepage ._yb_wutdic ._yb_2qgpj{width:320px;justify-content:flex-end}}@supports (display:grid){@media screen and (max-width:1019px){#ybar.ybar-ytheme-fuji2 ._yb_wutdic{grid-column-gap:20px}}.ybar-ytheme-fuji2 ._yb_wutdic{display:grid;grid-template-columns:[main-start] repeat(9,minmax(0,1fr)) [main-end] repeat(3,minmax(0,1fr));grid-template-rows:auto;grid-column-gap:32px}#ybar.ybar-ytheme-fuji2.ybar-property-mail ._yb_wutdic{display:flex;-moz-column-gap:normal;column-gap:normal}.ybar-ytheme-fuji2 ._yb_wutdic>div{grid-row-start:1}.ybar-ytheme-fuji2 ._yb_1muppqd{grid-column:main;justify-self:end;width:calc(100% - 240px)}.ybar-ytheme-fuji2._yb_2erz1k ._yb_1muppqd{width:calc(100% - 220px)}.ybar-ytheme-fuji2._yb_2erz1k._yb_5t6xy._yb_1u9bsx1 ._yb_1muppqd,.ybar-ytheme-fuji2._yb_2erz1k._yb_3zjksb._yb_1u9bsx1 ._yb_1muppqd,.ybar-ytheme-fuji2._yb_2erz1k._yb_3zjksb._yb_z83zjv ._yb_1muppqd,.ybar-ytheme-fuji2._yb_2erz1k._yb_1bwhbi8._yb_iswvcq ._yb_1muppqd,.ybar-ytheme-fuji2._yb_2erz1k._yb_1bqr3cv._yb_iswvcq ._yb_1muppqd{width:calc(100% - 240px)}.ybar-ytheme-crunch ._yb_wutdic>._yb_26wpdh,.ybar-ytheme-fuji2 ._yb_wutdic>._yb_26wpdh{grid-column-start:main-start;z-index:1}.ybar-ytheme-fuji2 ._yb_wutdic ._yb_2qgpj,.ybar-ytheme-fuji2.ybar-property-generic ._yb_wutdic ._yb_2qgpj,.ybar-ytheme-fuji2.ybar-property-homepage ._yb_wutdic ._yb_2qgpj{grid-column:main-end/-1;justify-self:end;width:auto}.ybar-ytheme-fuji2 ._yb_wutdic ._yb_y280k9{grid-column-start:main-start}}@supports (display:grid){@media screen and (max-width:1023px){#ybar.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_wutdic{grid-template-columns:[main-start] repeat(5,minmax(0,1fr)) [main-end toolbar-start]repeat(1,minmax(0,1fr)) [toolbar-end]}#ybar.ybar-ytheme-crunch:not(._yb_rq7mm4) :not(._yb_1qomz0q)>._yb_wutdic{grid-column-gap:24px}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_1w5vwjh{display:none}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_11pef5i{align-items:center;display:flex;height:36px;margin-right:12px;margin-left:-8px;padding:0 8px}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_1qomz0q ._yb_11pef5i{margin-right:8px}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_1qomz0q ._yb_11pef5i:has(._yb_19mvr2h:focus-visible),.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_11pef5i:hover{background-color:#f5f8fa;background-color:var(--yb-sidenav-close-btn-background);border-radius:100px}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_11pef5i:active{background-color:#e0e4e9;background-color:var(--yb-sidenav-btn-color-active);border-radius:100px}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_1qomz0q ._yb_19mvr2h:focus-visible svg path,.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_11pef5i:hover ._yb_19mvr2h svg path{fill:#6001d2;fill:var(--yb-text-hover-color)}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_11pef5i:active ._yb_19mvr2h svg path{fill:#232a31;fill:var(--yb-sidenav-btn-text-active)}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_1muppqd{max-width:unset;min-width:193px;width:100%;padding-right:139px}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_19mvr2h:focus-visible{outline:4px solid rgb(96 1 210/50%);outline:var(--yb-outline);border-radius:1px;outline-offset:6px}.ybar-ytheme-crunch:not(._yb_rq7mm4) ._yb_1qomz0q ._yb_19mvr2h:focus-visible{border-radius:100px}}@media screen and (max-width:1049px){#ybar.ybar-ytheme-crunch._yb_kx18b1 ._yb_wutdic{grid-template-columns:[main-start] repeat(3,minmax(0,1fr)) [main-end space-start] repeat(1,minmax(0,1fr)) [space-end nav-start]repeat(1,minmax(0,1fr)) [nav-end toolbar-start]repeat(1,minmax(250px,1fr)) [toolbar-end];grid-column-gap:1rem}.ybar-ytheme-crunch._yb_kx18b1 ._yb_1w5vwjh{grid-column-start:6}.ybar-ytheme-crunch._yb_kx18b1 ._yb_11pef5i,.ybar-ytheme-crunch.ybar-property-finance ._yb_1qomz0q ._yb_11pef5i{align-items:center;display:flex;height:36px;margin-right:10px;padding:0 9px 2px}#ybar.ybar-ytheme-crunch._yb_rq7mm4._yb_phn8ti{height:67px}#ybar.ybar-ytheme-crunch._yb_rq7mm4._yb_phn8ti._yb_kx18b1{height:64px}.ybar-hide-navigation #ybar.ybar-ytheme-crunch._yb_rq7mm4 ._yb_1smugqe>._yb_dzr3yz,#ybar.ybar-ytheme-crunch._yb_rq7mm4 ._yb_1smugqe>._yb_dzr3yz{opacity:0;margin-top:-60px}.ybar-hide-navigation .ybar-ytheme-crunch._yb_rq7mm4 ._yb_1smugqe,.ybar-ytheme-crunch._yb_rq7mm4 ._yb_1smugqe{overflow:hidden}.ybar-ytheme-crunch._yb_kx18b1 ._yb_uy5fkg{grid-column:1/span 5}.ybar-ytheme-crunch._yb_kx18b1 ._yb_11pef5i:hover{background-color:#f5f8fa;background-color:var(--yb-marshmallow);border-radius:100px}.ybar-ytheme-crunch._yb_rq7mm4 ._yb_1muppqd{max-width:632px;min-width:193px;width:calc(100% - 174px);margin-right:19px}}@media screen and (max-width:767px){#ybar.ybar-ytheme-crunch._yb_kx18b1,#ybar.ybar-ytheme-crunch._yb_kx18b1._yb_phn8ti{height:92px}#ybar.ybar-ytheme-crunch :not(._yb_1qomz0q)>._yb_wutdic{grid-column-gap:20px}#ybar.ybar-ytheme-crunch._yb_kx18b1 ._yb_wutdic{height:32px;margin:8px 16px 0;padding-bottom:84px}.ybar-ytheme-crunch._yb_kx18b1 ._yb_1muppqd{position:absolute;top:48px;max-width:100vw;width:calc(100vw - 88px)}.ybar-ytheme-crunch ._yb_uy5fkg{grid-column:1/span 5}.ybar-ytheme-crunch._yb_rq7mm4:not(._yb_kx18b1) ._yb_1muppqd{min-width:100px}}.ybar-ytheme-crunch :not(._yb_1qomz0q)>._yb_wutdic{display:grid;grid-template-columns:[main-start] repeat(11,minmax(0,1fr)) [main-end toolbar-start]repeat(1,minmax(0,1fr)) [toolbar-end];grid-template-rows:auto;grid-column-gap:32px}.ybar-ytheme-crunch._yb_kx18b1 ._yb_wutdic{grid-template-columns:[main-start] repeat(7,minmax(0,1fr)) [main-end space-start] repeat(1,minmax(0,1fr)) [space-end nav-start]repeat(4,minmax(0,1fr)) [nav-end toolbar-start]repeat(1,minmax(250px,1fr)) [toolbar-end]}._yb_uy5fkg{align-items:center;display:flex;flex-wrap:nowrap;grid-column:1/span 6}.ybar-ytheme-crunch ._yb_uy5fkg{transition:width .15s;width:inherit}._yb_1w5vwjh{display:flex;justify-content:flex-end}.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_1w5vwjh{grid-column:10}._yb_kx18b1 ._yb_1w5vwjh{grid-column-start:12}.ybar-ytheme-crunch ._yb_wutdic>div{grid-row-start:1}.ybar-ytheme-crunch ._yb_1muppqd{width:auto;min-width:193px}.ybar-ytheme-crunch ._yb_wutdic>._yb_26wpdh{grid-column-start:main-start;z-index:1}.ybar-ytheme-crunch ._yb_wutdic ._yb_2qgpj{grid-column:toolbar-start;width:100%;justify-content:flex-end}}.ybar-ytheme-crunch._yb_1jtdxyl ._yb_1emv3g7:not(._yb_1qomz0q) ._yb_11pef5i,._yb_11pef5i{display:none}._yb_19mvr2h{background:0 0;display:inline-block;border:0;margin:0;padding:0;height:20px;width:20px}._yb_19mvr2h:hover svg path{fill:#6001d2;fill:var(--yb-text-hover-color)}._yb_19mvr2h svg{display:inline-block;vertical-align:baseline}._yb_19mvr2h svg path{fill:#232a31;fill:var(--yb-text-primary)}._yb_v5t4r9 ._yb_wutdic{padding-left:54px}._yb_1ahofbt ._yb_wutdic{width:1020px}._yb_1os1r4w ._yb_wutdic{background:#6302de}._yb_1os1r4w._yb_2y1n8n ._yb_wutdic{background:0 0}._yb_1os1r4w._yb_189cmx2 ._yb_wutdic,._yb_1os1r4w._yb_1knq4zo ._yb_wutdic,._yb_1os1r4w._yb_5xpw6f ._yb_wutdic,._yb_1os1r4w._yb_td6sjk ._yb_wutdic{background:#000;background:var(--yb-black)}._yb_1os1r4w._yb_3w2bsd ._yb_wutdic{background:#2b2c2f}._yb_1os1r4w._yb_okuc2 ._yb_wutdic{background:#333}._yb_1os1r4w._yb_116dmfv ._yb_wutdic{background:#2b2d32}._yb_1o9isxs._yb_1996gte ._yb_wutdic{background:#6302de}._yb_1os1r4w._yb_g7a2wu ._yb_wutdic{background:#222}._yb_1os1r4w._yb_1nnw83b ._yb_wutdic{background:#0a4ea3}._yb_1os1r4w._yb_m0uu5w ._yb_wutdic{background:#0a0a0a}._yb_1os1r4w._yb_13gzied ._yb_wutdic{background:#fff;background:var(--yb-white)}._yb_1os1r4w._yb_pi28zo ._yb_wutdic{background:#1e4e9d}._yb_1os1r4w._yb_elmbj ._yb_wutdic{background:linear-gradient(303deg,#00d301,#36c275 50%,#00a562)}._yb_1os1r4w._yb_c5xir8 ._yb_wutdic{background:#36465d}._yb_26wpdh{display:flex;width:142px;min-width:142px}._yb_ist0jp ._yb_26wpdh,._yb_2erz1k ._yb_26wpdh{height:58px;width:205px;min-width:205px;margin-top:10px;margin-left:15px}.ybar-ytheme-crunch ._yb_wutdic ._yb_26wpdh{height:auto;width:auto;min-width:-moz-fit-content;min-width:fit-content;margin:0}._yb_1sik2rh ._yb_26wpdh{justify-content:center;min-width:192px}@media screen and (max-width:1024px){._yb_ist0jp ._yb_26wpdh,._yb_2erz1k ._yb_26wpdh{width:90px;min-width:90px;height:27px}}.ybar-ytheme-fuji2 ._yb_wutdic ._yb_26wpdh{height:auto;width:166px;min-width:166px;margin-right:20px;margin-top:0;margin-left:0;transition:min-width .4s,width .4s}.ybar-ytheme-fuji2._yb_1sik2rh ._yb_wutdic ._yb_26wpdh{margin-right:0}@media screen and (min-width:1020px){.ybar-ytheme-fuji2 ._yb_wutdic ._yb_26wpdh{width:220px;min-width:220px}}.ybar-ytheme-fuji2.ybar-property-mail ._yb_26wpdh{width:192px;min-width:192px}@media screen and (min-width:1440px){._yb_1sik2rh ._yb_26wpdh,.ybar-ytheme-fuji2._yb_1sik2rh ._yb_26wpdh{max-width:224px;width:14%}}._yb_1muppqd{display:flex;flex:1}._yb_1sik2rh ._yb_1muppqd{flex:initial}._yb_1muppqd #mail-search{flex:1}._yb_69p0bm:not(.ybar-ytheme-crunch) ._yb_1muppqd{display:none}#ybar.ybar-searchbox-assist-fullscreen #ybar-inner-wrap{position:initial}._yb_19s5ae1{margin:0 auto;padding:0 40px;width:1340px;max-width:100%;box-sizing:border-box;height:42px;display:flex;opacity:1;transition:margin .2s ease-out,opacity .15s linear .2s}._yb_1v4azts{overflow:hidden}._yb_15yb1e4{width:220px;margin-right:20px;flex-shrink:0}@media screen and (max-width:1019px){._yb_19s5ae1{padding:0 20px;transition:padding .4s}._yb_1hrqc53{width:143px;margin-left:105px;flex-shrink:0}}._yb_2qgpj{margin-left:auto;display:flex;align-items:center}.ybar-ytheme-crunch ._yb_2qgpj{margin:0}.ybar-ytheme-crunch ._yb_1qomz0q ._yb_2qgpj._yb_1mkycac{gap:10px}._yb_1sik2rh ._yb_2qgpj{padding-right:32px}.ybar-ytheme-crunch._yb_rq7mm4 ._yb_2qgpj{gap:12px}.ybar-ytheme-fuji2._yb_ist0jp ._yb_2qgpj,.ybar-ytheme-fuji2._yb_2erz1k ._yb_2qgpj{margin-right:0}._yb_2qgpj>div{margin-left:32px;white-space:nowrap}.ybar-ytheme-crunch ._yb_1qomz0q ._yb_1bud7df,.ybar-ytheme-crunch ._yb_1qomz0q ._yb_26w2jb{margin-left:0}.ybar-ytheme-crunch:not(.ybar-property-finance)._yb_19dqzcm ._yb_1qomz0q ._yb_26w2jb{width:60px;display:flex;justify-content:flex-end}.ybar-ytheme-crunch ._yb_1qomz0q ._yb_az9x9b{margin-left:16px}.ybar-ytheme-crunch ._yb_az9x9b,.ybar-ytheme-crunch._yb_19dqzcm ._yb_1qomz0q ._yb_az9x9b,.ybar-ytheme-crunch.ybar-mail-v3 ._yb_1qomz0q ._yb_2qgpj ._yb_az9x9b,.ybar-ytheme-crunch.ybar-mail-v3._yb_19dqzcm ._yb_1qomz0q ._yb_2qgpj ._yb_az9x9b{margin-left:12px}.ybar-ytheme-crunch.ybar-property-finance.ybar-mail-v3 ._yb_1qomz0q ._yb_2qgpj ._yb_az9x9b,.ybar-ytheme-crunch.ybar-property-finance._yb_19dqzcm ._yb_1qomz0q ._yb_az9x9b{margin-left:0}.ybar-ytheme-crunch ._yb_2qgpj>div{display:inline-block}@media screen and (max-width:600px){.ybar-ytheme-fuji2 ._yb_2qgpj{display:none}}@media screen and (min-width:1344px){.ybar-ytheme-crunch ._yb_wutdic ._yb_26wpdh{margin-left:0}}._yb_1bwhbi8._yb_iswvcq ._yb_2qgpj ._yb_1bud7df{display:none}._yb_2qgpj ._yb_zndw9d{max-width:250px}@media screen and (max-width:768px){:not(._yb_1qomz0q) ._yb_2qgpj>div{margin-left:24px}._yb_v5t4r9 ._yb_wutdic{height:54px;padding:0 24px 0 20px}._yb_1os1r4w ._yb_wutdic{height:50px;padding:0}._yb_v5t4r9 ._yb_wutdic,._yb_1os1r4w ._yb_wutdic{justify-content:center;position:relative}._yb_v5t4r9._yb_okuc2 ._yb_2qgpj,._yb_v5t4r9._yb_1996gte ._yb_2qgpj,._yb_v5t4r9._yb_m0uu5w ._yb_2qgpj,._yb_1os1r4w ._yb_2qgpj{position:absolute;right:24px}._yb_v5t4r9 ._yb_26wpdh,._yb_1os1r4w ._yb_26wpdh{width:auto;min-width:auto}._yb_189cmx2 ._yb_26wpdh{height:18px}._yb_1996gte ._yb_26wpdh,._yb_116dmfv ._yb_26wpdh{height:24px}._yb_1nnw83b ._yb_26wpdh{height:15px}._yb_2y1n8n ._yb_26wpdh{height:25px;width:64px}._yb_1knq4zo ._yb_26wpdh{height:22px}._yb_1os1r4w._yb_pi28zo ._yb_26wpdh,._yb_elmbj ._yb_26wpdh{height:20px}.ybar-property-finance ._yb_1qomz0q ._yb_uy5fkg ._yb_1muppqd{padding-right:0}}._yb_14tgfka{display:block}._yb_1sik2rh ._yb_1u6ljkh{max-width:none}._yb_1sik2rh ._yb_wutdic{padding:0;max-width:none}._yb_1sik2rh ._yb_wutdic ._yb_1muppqd{height:48px;margin:0;min-width:496px;padding:0;width:44%}._yb_1sik2rh._yb_19dmi0p ._yb_wutdic>._yb_26wpdh,._yb_1sik2rh._yb_17dx6fk ._yb_wutdic>._yb_26wpdh,._yb_1sik2rh._yb_156l1ls ._yb_wutdic>._yb_26wpdh,._yb_1sik2rh._yb_ozcu3o ._yb_wutdic>._yb_26wpdh{width:224px;min-width:224px}._yb_1sik2rh ._yb_wutdic ._yb_2qgpj{padding-right:32px}._yb_rq7mm4._yb_19dmi0p ._yb_16kwvy4{background:#fff;background:var(--yb-white)}._yb_1sik2rh ._yb_1s3151i{display:none}.ybar-ytheme-fuji2.ybar-property-mail ._yb_2t4frf{display:none}.ybar-ytheme-fuji2.ybar-property-mail.fuji2-dialpad ._yb_2t4frf{display:block}.ybar-amp,.ybar-amp .ybar-row{min-width:initial;max-width:initial;padding-right:0}#ybar._yb_50ho25{height:84px}#ybar.ybar-ytheme-fuji2.ybar-property-mail.fuji2-dialpad._yb_50ho25{height:72px}#ybar.ybar-ytheme-fuji2._yb_50ho25{height:111px}#ybar.ybar-ytheme-fuji2.ybar-property-generic._yb_50ho25{height:123px}#ybar.ybar-ytheme-fuji2._yb_phn8ti{height:104px}#ybar._yb_phn8ti,#ybar._yb_50ho25._yb_phn8ti,#ybar.ybar-ytheme-fuji2.ybar-property-homepage._yb_phn8ti,#ybar.ybar-ytheme-fuji2.ybar-property-generic._yb_phn8ti,#ybar.ybar-ytheme-fuji2.ybar-property-mail.fuji2-dialpad._yb_50ho25._yb_phn8ti,#ybar.ybar-ytheme-fuji2.ybar-property-generic._yb_50ho25._yb_phn8ti,#ybar.ybar-ytheme-fuji2.ybar-property-homepage._yb_50ho25._yb_phn8ti{height:114px}.ybar-ytheme-crunch.ybar-property-homepage._yb_phn8ti:not(._yb_1u9bsx1) ._yb_1smugqe{display:none}#ybar.ybar-ytheme-fuji2._yb_50ho25._yb_phn8ti{height:143px}.modal-open #ybar.ybar-ytheme-fuji2._yb_50ho25._yb_phn8ti{height:72px}#ybar.ybar-ytheme-fuji2.ybar-property-generic.ybar-show-homepage-topnavigation._yb_50ho25._yb_phn8ti,#ybar.ybar-ytheme-fuji2.ybar-property-homepage.ybar-show-homepage-topnavigation._yb_50ho25._yb_phn8ti{height:155px}#ybar.ybar-property-generic._yb_phn8ti,#ybar.ybar-property-homepage._yb_phn8ti{height:126px}#ybar.ybar-ytheme-fuji2.ybar-property-generic._yb_50ho25._yb_qezay3{height:165px}#ybar.ybar-ytheme-fuji2._yb_50ho25._yb_phn8ti._yb_qezay3{height:185px}#ybar.ybar-ytheme-fuji2._yb_50ho25._yb_phn8ti._yb_qezay3._yb_rq7mm4._yb_19dmi0p{height:203px}.modal-open .ybar-sticky #ybar.ybar-ytheme-fuji2._yb_2erz1k{height:92px!important}.modal-open .ybar-sticky #ybar.ybar-ytheme-fuji2._yb_2erz1k{height:80px!important}@media screen and (orientation:portrait){html.Reader-open ._yb_1muppqd{display:none}html.Reader-open ._yb_b7aa7v{display:inline-block}._yb_26qcwt{display:inline-block}._yb_2718zi{display:none}}#ybar input[type=hidden]{visibility:hidden}.ybar-show-outline{outline-offset:2px;outline:3px solid #0f69ff;outline:3px solid var(--yb-dory)}.ybar-hide-outline{outline:0!important}.ybar-ytheme-crunch .ybar-show-outline:focus{outline-offset:5px;outline:4px solid rgb(96 1 210/50%);outline:var(--yb-outline);border-radius:1px}._yb_bbmou4 :focus{outline:solid #0f69ff 2px;outline:solid var(--yb-dory) 2px}._yb_bbmou4 :focus:not(:focus-visible){outline:0}.ybar-theme-dark ._yb_bbmou4 :focus{outline-color:#12a9ff;outline-color:var(--yb-sky)}.ybar-theme-dark ._yb_bbmou4 :focus:not(:focus-visible){outline:0}body.typing .ybar-ytheme-fuji2[data-ovrly-bkt=darker] ._yb_1u6ljkh._yb_1smugqe{background-color:rgb(16 24 21/70%);pointer-events:none}body.typing .ybar-ytheme-fuji2[data-ovrly-bkt=light] ._yb_1u6ljkh._yb_1smugqe{background-color:#fff;background-color:var(--yb-white);opacity:.2;pointer-events:none}.ybar-page-is-scrolled:not(.ybar-hide-navigation) body.typing .ybar-ytheme-crunch[data-ovrly-bkt=midnight60] ._yb_1vxyy3,:not(.ybar-page-is-scrolled) body.typing .ybar-ytheme-crunch[data-ovrly-bkt=midnight60] ._yb_1vxyy3{background-color:#101518;background-color:var(--yb-midnight);height:64px;height:var(--uh-visible-height, 64px);opacity:.6;position:fixed;width:100%;z-index:1}.ybar-page-is-scrolled body.typing .ybar-ytheme-crunch[data-ovrly-bkt=midnight60] ._yb_1smugqe,:not(.ybar-page-is-scrolled) body.typing .ybar-ytheme-crunch[data-ovrly-bkt=midnight60] #ybar-inner-wrap{pointer-events:none}:not(.ybar-page-is-scrolled) body.typing .ybar-ytheme-crunch[data-ovrly-bkt=midnight60] #ybar-inner-wrap #ybar-search-box-container{pointer-events:auto}.ybar-ytheme-crunch ._yb_1u6ljkh{margin:0}@media screen and (min-width:768px){.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_dzr3yz{max-width:calc(100% - 39px);max-width:var(--uh-max-width, calc(100% - 39px))}}@media screen and (min-width:1024px){.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_dzr3yz{max-width:calc(100% - 40px);max-width:var(--uh-max-width, calc(100% - 40px))}.ybar-ytheme-crunch.ybar-property-finance:not(._yb_kx18b1) ._yb_dzr3yz{margin-left:12px;margin-left:var(--uh-row-margin-left, 12px);margin-right:0;margin-right:var(--uh-row-margin-right, 0)}}@media screen and (min-width:1050px){.ybar-ytheme-crunch.property-finance:not(._yb_kx18b1) ._yb_dzr3yz{max-width:calc(100% - 36px);max-width:var(--uh-max-width, calc(100% - 36px))}}@media screen and (min-width:1050px){.ybar-ytheme-crunch._yb_kx18b1 ._yb_dzr3yz{margin-left:calc((100vw - 1440px)/2);margin-right:calc((100vw - 1440px)/2)}}@media screen and (max-width:1049px){.ybar-ytheme-crunch.ybar-property-finance ._yb_1qomz0q ._yb_dzr3yz{max-width:calc(100% - 28px);max-width:var(--uh-max-width, calc(100% - 28px));margin-left:3px;margin-left:var(--uh-row-margin-left, 3px);margin-right:0;margin-right:var(--uh-row-margin-right, 0)}}@media screen and (min-width:1280px){.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_dzr3yz{max-width:min(1264px,calc(100% - 40px));max-width:var(--uh-max-width, min(1264px, calc(100% - 40px)))}.ybar-ytheme-crunch.ybar-property-finance ._yb_dzr3yz{max-width:calc(100% - 36px);max-width:var(--uh-max-width, calc(100% - 36px))}}@media screen and (min-width:1440px){.ybar-ytheme-crunch:not(._yb_kx18b1) ._yb_dzr3yz{max-width:1264px;max-width:var(--uh-max-width, 1264px)}.ybar-ytheme-crunch.ybar-property-finance ._yb_dzr3yz{max-width:1404px;max-width:var(--uh-max-width, 1404px)}}._yb_1qomz0q ._yb_wutdic ._yb_2qgpj{width:auto}:not(.ybar-page-is-scrolled) body.typing .ybar-ytheme-crunch[data-ovrly-bkt=midnight60] #ybar-inner-wrap ._yb_1qomz0q ._yb_uy5fkg{width:inherit}._yb_19dqzcm ._yb_1qomz0q ._yb_1w5vwjh{margin-left:0}@media screen and (max-width:1023px){._yb_1qomz0q :not(._yb_rq7mm4) ._yb_1muppqd{padding-right:0}.ybar-property-finance ._yb_1qomz0q ._yb_1muppqd{padding-right:40px}._yb_1qomz0q ._yb_1w5vwjh,._yb_1qomz0q ._yb_1muppqd,._yb_19dqzcm ._yb_1qomz0q ._yb_1muppqd{margin-right:24px}.ybar-mail-v5 ._yb_1qomz0q ._yb_1muppqd{margin-right:46px}._yb_kx18b1 ._yb_1qomz0q ._yb_dzr3yz{max-width:calc(100% - 23px)}._yb_kx18b1 ._yb_1qomz0q ._yb_11pef5i{margin-right:10px}._yb_kx18b1 ._yb_1qomz0q ._yb_26wpdh{margin-right:10px}}@media screen and (min-width:1024px){._yb_1qomz0q ._yb_1w5vwjh{margin-right:40px}._yb_1qomz0q ._yb_1muppqd{margin-right:40px}._yb_19dqzcm ._yb_1qomz0q ._yb_1muppqd{margin-right:40px}._yb_kx18b1 ._yb_1qomz0q ._yb_uy5fkg{margin-left:12px}._yb_kx18b1 ._yb_1qomz0q ._yb_26wpdh{margin-right:10px}._yb_kx18b1 ._yb_1qomz0q ._yb_dzr3yz{max-width:calc(100% - 25px)}._yb_kx18b1 ._yb_1qomz0q ._yb_1muppqd{margin-right:48px}}@media screen and (min-width:1280px){._yb_1qomz0q ._yb_1w5vwjh{margin-right:56px}.ybar-property-finance ._yb_1qomz0q ._yb_1w5vwjh{margin-right:56px}._yb_1qomz0q ._yb_1muppqd{margin-right:56px}.ybar-property-finance ._yb_1qomz0q ._yb_1muppqd{margin-right:60px}._yb_19dqzcm ._yb_1qomz0q ._yb_1muppqd{margin-right:56px}._yb_kx18b1 ._yb_1qomz0q ._yb_1w5vwjh{margin-right:0}._yb_kx18b1 ._yb_1qomz0q ._yb_1muppqd{margin-right:70px}._yb_kx18b1 ._yb_1qomz0q ._yb_dzr3yz{max-width:calc(100% - 24px)}}@media screen and (min-width:1440px){.ybar-property-finance ._yb_1qomz0q ._yb_1w5vwjh,._yb_1qomz0q ._yb_1w5vwjh{margin-right:64px}.ybar-property-finance ._yb_1qomz0q ._yb_1muppqd,._yb_19dqzcm ._yb_1qomz0q ._yb_1muppqd,._yb_1qomz0q ._yb_1muppqd{margin-right:64px}.ybar-property-finance:not(._yb_g6ro8m,._yb_13pabky,._yb_1e2sz6e) ._yb_1qomz0q ._yb_1muppqd{max-width:740px}._yb_kx18b1 ._yb_1qomz0q ._yb_1muppqd{margin-right:78px}._yb_kx18b1 ._yb_1qomz0q ._yb_1w5vwjh{margin-right:46px}}._yb_1qomz0q #ybarAccountMenu{padding:0}._yb_18dd21{color:red}.ybar-sticky #ybar-inner-wrap{position:fixed;z-index:20;width:100%;transition:box-shadow .2s ease-out}.ybar-page-is-scrolled body:not(.uh-disable-bottom-box-shadow) .ybar-sticky #ybar-inner-wrap{box-shadow:rgb(0 0 0/40%) 0 0 10px 0}.ybar-sticky .ybar-ytheme-crunch.ybar-property-finance #ybar-inner-wrap{transition:box-shadow .5s ease-in-out;top:0}._yb_1s8ibak,._yb_1b9x91s ._yb_1jc4gg0,._yb_fajdfd,._yb_t3vv1p,._yb_1jc4gg0,._yb_mkdxrx,.ybar-ytheme-fuji2 ._yb_mkdxrx,._yb_tyjfd1,._yb_1xyb1po,._yb_wvwmz5,._yb_12p4kq1,._yb_1utz9cx,._yb_1xs54pa,._yb_1qxcmf5,._yb_1856sns,._yb_19gmgo9,._yb_7zky42,._yb_66i12g,._yb_1jc3ac5,._yb_wklvr4,.ybar-ytheme-fuji2 ._yb_fajdfd,.ybar-ytheme-fuji2.ybar-property-homepage ._yb_fajdfd,._yb_laz8r0,._yb_7dxmux,._yb_l06g4f,._yb_p92pzg,._yb_19xr1yw,._yb_14v0pqj,._yb_1jugtd5,._yb_iva9fe,._yb_ycmex2,._yb_18m475z,._yb_106tojv,._yb_v10t20,._yb_13ix2cd,._yb_1ipstdz,._yb_1mre8d0,._yb_1g07l37,._yb_1ogiuyw,._yb_1knjkik,._yb_alkw6r,._yb_uuu0ny,._yb_1yhgbi0,._yb_lkxfo3,._yb_dcvoaw,._yb_1vifz6d,._yb_1qkoi2y,._yb_745sfo,._yb_nrk6cq,._yb_alfv4i,._yb_sstvpn,._yb_702g33,._yb_prtnec,._yb_1c7ti4i,._yb_1qlxi9,._yb_2w8yu1,._yb_1phjup6,._yb_137fl4z,._yb_1ig6q1v,._yb_1bxd6y2,._yb_1fg4gb1,._yb_g65rpo,._yb_1jpak4i,._yb_1ipqnqk,._yb_1gkesjf,._yb_1dfeie,._yb_128o2yp{background-image:url(https://s.yimg.com/kr/assets/spritify-sprite-dark-fd484ded-615432bc.png);background-size:32px 1713px;background-repeat:no-repeat}._yb_1s8ibak,._yb_1b9x91s ._yb_1jc4gg0{background-position:0 -1582px;width:24px;height:24px}._yb_fajdfd,._yb_t3vv1p,._yb_1jc4gg0{background-position:0 -1611px;width:24px;height:24px}._yb_mkdxrx{background-position:0 -1524px;width:24px;height:24px}.ybar-ytheme-fuji2 ._yb_mkdxrx{background-position:0 -1640px;width:32px;height:32px}._yb_tyjfd1{background-position:0 -1553px;width:24px;height:24px}._yb_1xyb1po{background-position:0 -1677px;width:32px;height:36px}.ybar-light ._yb_1s8ibak,.ybar-light ._yb_1b9x91s ._yb_1jc4gg0,.ybar-light ._yb_fajdfd,.ybar-light ._yb_t3vv1p,.ybar-light ._yb_1jc4gg0,.ybar-light .ybar-property-homepage ._yb_fajdfd,.ybar-light ._yb_mkdxrx,.ybar-light .ybar-ytheme-fuji2 ._yb_mkdxrx,.ybar-light ._yb_tyjfd1,.ybar-light ._yb_1xyb1po{background-image:url(https://s.yimg.com/kr/assets/spritify-sprite-light-fd484ded-e3dce7a4.png);background-size:32px 1740px;background-repeat:no-repeat}.ybar-light ._yb_1s8ibak,.ybar-light ._yb_1b9x91s ._yb_1jc4gg0{background-position:0 -1609px;width:24px;height:24px}.ybar-light ._yb_fajdfd,.ybar-light ._yb_t3vv1p,.ybar-light ._yb_1jc4gg0{background-position:0 -1638px;width:24px;height:24px}.ybar-light .ybar-property-homepage ._yb_fajdfd{background-position:0 -1524px;width:27px;height:20px}.ybar-light ._yb_mkdxrx{background-position:0 -1549px;width:24px;height:26px}.ybar-light .ybar-ytheme-fuji2 ._yb_mkdxrx{background-position:0 -1667px;width:32px;height:32px}.ybar-light ._yb_tyjfd1{background-position:0 -1580px;width:24px;height:24px}.ybar-light ._yb_1xyb1po{background-position:0 -1704px;width:32px;height:36px}._yb_wvwmz5{background-position:0 -1188px;width:24px;height:24px}._yb_12p4kq1{background-position:0 -1245px;width:13px;height:12px}._yb_1utz9cx{background-position:0 -1291px;width:18px;height:18px}._yb_1xs54pa{background-position:0 -1314px;width:18px;height:18px}._yb_1qxcmf5{background-position:0 -1337px;width:18px;height:16px}._yb_1856sns{background-position:0 -1262px;width:24px;height:24px}._yb_19gmgo9{background-position:0 -1358px;width:24px;height:24px}._yb_7zky42{background-position:0 -1387px;width:12px;height:12px}._yb_66i12g{background-position:0 -1434px;width:16px;height:16px}._yb_1jc3ac5{background-position:0 -1455px;width:18px;height:18px}._yb_wklvr4{background-position:0 -1478px;width:18px;height:18px}.ybar-ytheme-fuji2 ._yb_fajdfd,.ybar-ytheme-fuji2.ybar-property-homepage ._yb_fajdfd{background-position:0 -1404px;width:23px;height:25px}._yb_laz8r0{background-position:0 -99px;width:28px;height:28px}._yb_7dxmux{background-position:0 -132px;width:28px;height:28px}._yb_l06g4f,._yb_p92pzg{background-position:0 -165px;width:28px;height:28px}._yb_19xr1yw,._yb_14v0pqj{background-position:0 -66px;width:28px;height:28px}._yb_1jugtd5,._yb_iva9fe{background-position:0 -231px;width:28px;height:28px}._yb_ycmex2{background-position:0 -33px;width:28px;height:28px}._yb_18m475z{background-position:0 -198px;width:28px;height:28px}._yb_106tojv,._yb_v10t20{background-position:0 0;width:28px;height:28px}._yb_13ix2cd{background-position:0 -297px;width:28px;height:28px}._yb_1ipstdz,._yb_1mre8d0{background-position:0 -264px;width:28px;height:28px}._yb_1g07l37{background-position:0 -429px;width:28px;height:28px}._yb_1ogiuyw{background-position:0 -495px;width:28px;height:28px}._yb_1knjkik{background-position:0 -462px;width:28px;height:28px}._yb_alkw6r{background-position:0 -396px;width:28px;height:28px}._yb_uuu0ny{background-position:0 -594px;width:28px;height:28px}._yb_1yhgbi0{background-position:0 -528px;width:28px;height:28px}._yb_lkxfo3{background-position:0 -561px;width:28px;height:28px}._yb_dcvoaw{background-position:0 -363px;width:28px;height:28px}._yb_1vifz6d{background-position:0 -330px;width:28px;height:28px}._yb_1qkoi2y{background-position:0 -627px;width:28px;height:28px}._yb_745sfo{background-position:0 -660px;width:28px;height:28px}._yb_nrk6cq{background-position:0 -693px;width:28px;height:28px}._yb_alfv4i{background-position:0 -726px;width:28px;height:28px}._yb_sstvpn{background-position:0 -759px;width:28px;height:28px}._yb_702g33{background-position:0 -792px;width:28px;height:28px}._yb_prtnec{background-position:0 -825px;width:28px;height:28px}._yb_1c7ti4i{background-position:0 -858px;width:28px;height:28px}._yb_1qlxi9{background-position:0 -891px;width:28px;height:28px}._yb_2w8yu1{background-position:0 -924px;width:28px;height:28px}._yb_1phjup6{background-position:0 -957px;width:28px;height:28px}._yb_137fl4z{background-position:0 -990px;width:28px;height:28px}._yb_1ig6q1v{background-position:0 -1155px;width:28px;height:28px}._yb_1bxd6y2{background-position:0 -1023px;width:28px;height:28px}._yb_1fg4gb1,._yb_g65rpo{background-position:0 -1056px;width:28px;height:28px}._yb_1jpak4i{background-position:0 -1089px;width:28px;height:28px}._yb_1ipqnqk{background-position:0 -1122px;width:28px;height:28px}._yb_1gkesjf{background-position:0 -1501px;width:18px;height:18px}._yb_1dfeie{background-position:0 -1217px;width:14px;height:9px}._yb_128o2yp{background-position:0 -1231px;width:14px;height:9px}@media only screen and (-webkit-min-device-pixel-ratio:2),only screen and (min-device-pixel-ratio:2),only screen and (min-resolution:192dpi),only screen and (min-resolution:2dppx){._yb_1s8ibak,._yb_1b9x91s ._yb_1jc4gg0,._yb_fajdfd,._yb_t3vv1p,._yb_1jc4gg0,._yb_mkdxrx,.ybar-ytheme-fuji2 ._yb_mkdxrx,._yb_tyjfd1,._yb_wvwmz5,._yb_12p4kq1,._yb_1utz9cx,._yb_1xs54pa,._yb_1qxcmf5,._yb_19gmgo9,._yb_7zky42,._yb_66i12g,._yb_1jc3ac5,._yb_wklvr4,.ybar-ytheme-fuji2 ._yb_fajdfd,.ybar-ytheme-fuji2.ybar-property-homepage ._yb_fajdfd,._yb_laz8r0,._yb_7dxmux,._yb_l06g4f,._yb_19xr1yw,._yb_1jugtd5,._yb_ycmex2,._yb_18m475z,._yb_106tojv,._yb_13ix2cd,._yb_1ipstdz,._yb_1g07l37,._yb_1ogiuyw,._yb_1knjkik,._yb_alkw6r,._yb_uuu0ny,._yb_1yhgbi0,._yb_lkxfo3,._yb_dcvoaw,._yb_1vifz6d,._yb_1qkoi2y,._yb_745sfo,._yb_nrk6cq,._yb_alfv4i,._yb_sstvpn,._yb_702g33,._yb_prtnec,._yb_1c7ti4i,._yb_1qlxi9,._yb_2w8yu1,._yb_1phjup6,._yb_137fl4z,._yb_1ig6q1v,._yb_1bxd6y2,._yb_1fg4gb1,._yb_g65rpo,._yb_1jpak4i,._yb_1ipqnqk,._yb_1gkesjf,._yb_1dfeie,._yb_128o2yp{background-image:url(https://s.yimg.com/kr/assets/spritify-sprite-dark-2x-fd484ded-89f6b234.png);background-size:32px 1672px;background-repeat:no-repeat}._yb_1s8ibak,._yb_1b9x91s ._yb_1jc4gg0{background-position:0 -1541px;width:24px;height:24px}._yb_fajdfd,._yb_t3vv1p,._yb_1jc4gg0{background-position:0 -1570px;width:24px;height:24px}._yb_mkdxrx{background-position:0 -1483px;width:24px;height:24px}.ybar-ytheme-fuji2 ._yb_mkdxrx{background-position:0 -1599px;width:32px;height:32px}._yb_tyjfd1{background-position:0 -1512px;width:24px;height:24px}.ybar-light ._yb_1s8ibak,.ybar-light ._yb_1b9x91s ._yb_1jc4gg0,.ybar-light ._yb_fajdfd,.ybar-light ._yb_t3vv1p,.ybar-light ._yb_1jc4gg0,.ybar-light .ybar-property-homepage ._yb_fajdfd,.ybar-light ._yb_mkdxrx,.ybar-light .ybar-ytheme-fuji2 ._yb_mkdxrx,.ybar-light ._yb_tyjfd1{background-image:url(https://s.yimg.com/kr/assets/spritify-sprite-light-2x-fd484ded-374cd76d.png);background-size:32px 1699px;background-repeat:no-repeat}.ybar-light ._yb_1s8ibak,.ybar-light ._yb_1b9x91s ._yb_1jc4gg0{background-position:0 -1568px;width:24px;height:24px}.ybar-light ._yb_fajdfd,.ybar-light ._yb_t3vv1p,.ybar-light ._yb_1jc4gg0{background-position:0 -1597px;width:24px;height:24px}.ybar-light .ybar-property-homepage ._yb_fajdfd{background-position:0 -1483px;width:27px;height:20px}.ybar-light ._yb_mkdxrx{background-position:0 -1508px;width:24px;height:26px}.ybar-light .ybar-ytheme-fuji2 ._yb_mkdxrx{background-position:0 -1626px;width:32px;height:32px}.ybar-light ._yb_tyjfd1{background-position:0 -1539px;width:24px;height:24px}._yb_wvwmz5{background-position:0 -1188px;width:24px;height:24px}._yb_12p4kq1{background-position:0 -1245px;width:13px;height:12px}._yb_1utz9cx{background-position:0 -1262px;width:18px;height:18px}._yb_1xs54pa{background-position:0 -1285px;width:18px;height:18px}._yb_1qxcmf5{background-position:0 -1308px;width:18px;height:16px}._yb_19gmgo9{background-position:0 -1329px;width:12px;height:12px}._yb_7zky42{background-position:0 -1346px;width:12px;height:12px}._yb_66i12g{background-position:0 -1393px;width:16px;height:16px}._yb_1jc3ac5{background-position:0 -1414px;width:18px;height:18px}._yb_wklvr4{background-position:0 -1437px;width:18px;height:18px}.ybar-ytheme-fuji2 ._yb_fajdfd,.ybar-ytheme-fuji2.ybar-property-homepage ._yb_fajdfd{background-position:0 -1363px;width:23px;height:25px}._yb_laz8r0{background-position:0 -99px;width:28px;height:28px}._yb_7dxmux{background-position:0 -132px;width:28px;height:28px}._yb_l06g4f{background-position:0 -165px;width:28px;height:28px}._yb_19xr1yw{background-position:0 -66px;width:28px;height:28px}._yb_1jugtd5{background-position:0 -231px;width:28px;height:28px}._yb_ycmex2{background-position:0 -33px;width:28px;height:28px}._yb_18m475z{background-position:0 -198px;width:28px;height:28px}._yb_106tojv{background-position:0 0;width:28px;height:28px}._yb_13ix2cd{background-position:0 -297px;width:28px;height:28px}._yb_1ipstdz{background-position:0 -264px;width:28px;height:28px}._yb_1g07l37{background-position:0 -429px;width:28px;height:28px}._yb_1ogiuyw{background-position:0 -495px;width:28px;height:28px}._yb_1knjkik{background-position:0 -462px;width:28px;height:28px}._yb_alkw6r{background-position:0 -396px;width:28px;height:28px}._yb_uuu0ny{background-position:0 -594px;width:28px;height:28px}._yb_1yhgbi0{background-position:0 -528px;width:28px;height:28px}._yb_lkxfo3{background-position:0 -561px;width:28px;height:28px}._yb_dcvoaw{background-position:0 -363px;width:28px;height:28px}._yb_1vifz6d{background-position:0 -330px;width:28px;height:28px}._yb_1qkoi2y{background-position:0 -627px;width:28px;height:28px}._yb_745sfo{background-position:0 -660px;width:28px;height:28px}._yb_nrk6cq{background-position:0 -693px;width:28px;height:28px}._yb_alfv4i{background-position:0 -726px;width:28px;height:28px}._yb_sstvpn{background-position:0 -759px;width:28px;height:28px}._yb_702g33{background-position:0 -792px;width:28px;height:28px}._yb_prtnec{background-position:0 -825px;width:28px;height:28px}._yb_1c7ti4i{background-position:0 -858px;width:28px;height:28px}._yb_1qlxi9{background-position:0 -891px;width:28px;height:28px}._yb_2w8yu1{background-position:0 -924px;width:28px;height:28px}._yb_1phjup6{background-position:0 -957px;width:28px;height:28px}._yb_137fl4z{background-position:0 -990px;width:28px;height:28px}._yb_1ig6q1v{background-position:0 -1155px;width:28px;height:28px}._yb_1bxd6y2{background-position:0 -1023px;width:28px;height:28px}._yb_1fg4gb1,._yb_g65rpo{background-position:0 -1056px;width:28px;height:28px}._yb_1jpak4i{background-position:0 -1089px;width:28px;height:28px}._yb_1ipqnqk{background-position:0 -1122px;width:28px;height:28px}._yb_1gkesjf{background-position:0 -1460px;width:18px;height:18px}._yb_1dfeie{background-position:0 -1217px;width:14px;height:9px}._yb_128o2yp{background-position:0 -1231px;width:14px;height:9px}}._yb_1s7oh70{display:flex;align-items:0;font-size:0;height:100%}._yb_1ekgad8._yb_1s7oh70,._yb_nqixwx._yb_1s7oh70{width:100%}.ybar-ytheme-crunch ._yb_1s7oh70{width:auto}.ybar-ytheme-fuji2 ._yb_1s7oh70{max-height:24px;transition:height .4s}.ybar-ytheme-fuji2 ._yb_1ekgad8._yb_1s7oh70,.ybar-ytheme-fuji2 ._yb_nqixwx._yb_1s7oh70{margin-top:8px;max-height:45px}@media screen and (min-width:1020px){.ybar-ytheme-fuji2 ._yb_1s7oh70{max-height:32px}.ybar-ytheme-fuji2 ._yb_1ekgad8._yb_1s7oh70,.ybar-ytheme-fuji2 ._yb_nqixwx._yb_1s7oh70{margin-top:6px;max-height:56px}}.ybar-ytheme-fuji2 ._yb_1lf3kkk._yb_1s7oh70{width:auto;max-height:40px}._yb_1s7oh70:focus{outline-offset:2px}._yb_ftk0fa{align-self:flex-start;max-height:100%;max-width:100%}@media screen and (min-width:768px){._yb_ftk0fa{max-height:40px}}._yb_nqixwx ._yb_ftk0fa{height:100%;max-height:100%}.ybar-ytheme-fuji2 ._yb_ftk0fa{height:auto;width:auto;max-height:100%;max-width:100%;flex-shrink:0;-o-object-fit:contain;object-fit:contain;-o-object-position:left;object-position:left}.ybar-ytheme-fuji2._yb_1lf3kkk ._yb_ftk0fa{-o-object-position:center;object-position:center}.ybar-ytheme-crunch ._yb_9ow7t3 ._yb_ftk0fa{max-height:24px}._yb_1near09{display:block;display:var(--yb-light-img-logo-display)}._yb_1s7rouz{display:none;display:var(--yb-dark-img-logo-display)}.ybar-dark ._yb_1s7rouz{display:block}.ybar-dark ._yb_1near09,.ybar-ytheme-crunch ._yb_1s7rouz,.ybar-light ._yb_1s7rouz{display:none}.ybar-amp ._yb_1s7oh70{display:block;margin:auto;padding:10px 0;text-align:center}@media screen and (max-width:768px){._yb_miwkng ._yb_ftk0fa,._yb_19jntx5 ._yb_ftk0fa{height:100%;max-height:32px}}._yb_ovs6in{position:absolute;left:-9999px;top:auto;width:1px;height:1px;overflow:hidden}._yb_1mrxhn3{display:flex;align-items:center}._yb_aqhy15{width:1px;height:20px;background-color:#e0e4e9;background-color:var(--yb-cobrand-logos-divider);margin:0 8px}._yb_1mrxhn3 svg{width:auto;height:100%}._yb_1mrxhn3 a._yb_4wfnek,._yb_1mrxhn3 ._yb_6nnjs4{height:24px}._yb_1mrxhn3 ._yb_2y654h{height:20px}._yb_1mrxhn3 a._yb_w9fh6r{height:18px}</style>
    <style nonce="">#consent-page #ybar div{background: #fff;padding-left: 0}.btn {padding: 15px;font-size:20px;min-width: 30%;}.btn.primary{background-color: #0f69ff;color: #fff;cursor: pointer;}</style>
</head>
<body id="tcf2-layer1" class="no-touch blur-preview-tpl wizard tcf-2 yahoo eu-localized en-GB js">
<script nonce="">
    if ('classList' in document.body) {
        document.body.classList.add('js');
        document.body.classList.remove('no-js');
    }

</script>

<div id="consent-page" class="theme-2 v4 brandtype-yahoo ">
    <div class="consent-overlay">
        <div class="container con-container is-reject-all-enabled" aria-labelledby="consent-title" aria-describedby="consent-text">
            <div class="con-wizard">
                <form method="post" class="consent-form" action="">
                <div class="wizard-header"><div class="consent-brand-logo"><div id="ybar" role="banner" data-spaceid="" data-testid="" data-version="3.10.48" data-issinglerowheader="false" data-issinglerowsearchbox="" class="ybar-ytheme-classic  ybar-property-guce  ybar-variant-yahoo _yb_bbmou4 _yb_jrobl8 _yb_1os1r4w    ybar-light       ybar-track-link-views          "> <script id="ybarConfig" type="text/x-template">
        {}
    </script>  <div id="ybar-inner-wrap" class="_yb_16kwvy4 "><div class="_yb_1vxyy3"></div>          <div class="_yb_1u6ljkh _yb_1emv3g7"><div class="_yb_dzr3yz _yb_wutdic">   <div class="_yb_26wpdh"><h1 class="_yb_ovs6in">guce</h1>     <a href="https://uk.yahoo.com/" target="_self" id="ybar-logo" class="_yb_1s7oh70  _yb_19jntx5    " data-ylk="slk:guce;elm:logo;sec:ybar;subsec:logo;itc:0;">             <img class="_yb_ftk0fa _yb_1near09" src="https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_p_bestfit_frontpage.png" srcset="https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_p_bestfit_frontpage.png 1x, https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_p_bestfit_frontpage_2x.png 2x" title="" aria-hidden="true" alt=""><img class="_yb_ftk0fa _yb_1s7rouz" src="https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_w_bestfit_frontpage.png" srcset="https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_w_bestfit_frontpage.png 1x, https://s.yimg.com/rz/p/yahoo_frontpage_en-US_s_f_w_bestfit_frontpage_2x.png 2x" title="" aria-hidden="true" alt="">          guce    </a>  </div>    <div role="toolbar" class="_yb_2qgpj ybar-menu-hover-open"> </div></div></div>  </div></div></div></div>
                <div class="wizard-body">
                    <div class="consent-text" id="consent-text">
                        <div class="content-section first">
                            <p>    We, Yahoo, are part of the <span tabindex="0" aria-label="Yahoo family of brands" aria-describedby="consent-yahoo-tooltip" class="info hover touched-once" data-tooltip="The sites and apps that we own and operate, including Yahoo and AOL, and our digital advertising service, Yahoo Advertising."><span class="info-popup"><span class="info-popup-content"><span class="info-popup-heading">Yahoo family of brands</span><img src="../static/images/close.svg" alt=""><span id="consent-yahoo-tooltip" class="info-popup-description">The sites and apps that we own and operate, including Yahoo and AOL, and our digital advertising service, Yahoo Advertising.</span></span></span>Yahoo</span> family of brands.
</p>
                            <div>    When you use our sites and apps, we use <span tabindex="0" aria-label="Cookies" aria-describedby="consent-cookie-tooltip" class="info hover" data-tooltip="Cookies (including similar technologies such as web storage) allow the operators of websites and apps to store and read information from your device. Learn more in our <a href='/redirect?to=https%3A%2F%2Flegal.yahoo.com%2Fie%2Fen%2Fyahoo%2Fprivacy%2Fcookies%2Findex.html&amp;brandDomain=&amp;brandId=yahoo&amp;tos=eu&amp;step=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6&amp;userType=NON_REG' target='_blank'>cookie policy</a>."><span class="info-popup"><span class="info-popup-content"><span class="info-popup-heading">Cookies</span><img src="../static/images/close.svg" alt=""><span id="consent-cookie-tooltip" class="info-popup-description">Cookies (including similar technologies such as web storage) allow the operators of websites and apps to store and read information from your device. Learn more in our <a href="/redirect?to=https%3A%2F%2Flegal.yahoo.com%2Fie%2Fen%2Fyahoo%2Fprivacy%2Fcookies%2Findex.html&amp;brandDomain=&amp;brandId=yahoo&amp;tos=eu&amp;step=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6&amp;userType=NON_REG" target="_blank">cookie policy</a>.</span></span></span>cookies</span> to:
</div>
                            <ul class="content-list">
                                <li class="list-item">provide our sites and apps to you</li>
                                <li class="list-item">authenticate users, apply security measures, and prevent spam and abuse, and</li>
                                <li class="list-item">measure your use of our sites and apps</li>
                            </ul>
                        </div>
                        <div class="content-section">
                            <div>    If you click '<strong>Accept all</strong>', we and <a href="/v2/partners-list?sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6" target="_blank">our partners</a>, including 238 who are part of the IAB Transparency &amp; Consent Framework, will also store and/or access information on a device (in other words, use cookies) and use precise geolocation data and other personal data such as IP address and browsing and search data, for personalised advertising and content, advertising and content measurement and audience research and services development.
</div>
                                <p>    If you do not want us and our partners to use cookies and personal data for these additional purposes, click '<strong>Reject all</strong>'.
</p>
                            <p>    If you would like to customise your choices, click '<strong>Manage privacy settings</strong>'.
</p>
                            <p>    You can withdraw your consent or change your choices at any time by clicking on the 'Privacy &amp; cookie settings' or 'Privacy dashboard' links on our sites and apps. Find out more about how we use your personal data in our <a href="/redirect?to=https%3A%2F%2Flegal.yahoo.com%2Fie%2Fen%2Fyahoo%2Fprivacy%2Findex.html&amp;brandDomain=&amp;brandId=yahoo&amp;tos=eu&amp;step=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6&amp;userType=NON_REG" target="_blank">privacy policy</a> and <a href="/redirect?to=https%3A%2F%2Flegal.yahoo.com%2Fie%2Fen%2Fyahoo%2Fprivacy%2Fcookies%2Findex.html&amp;brandDomain=&amp;brandId=yahoo&amp;tos=eu&amp;step=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6&amp;userType=NON_REG" target="_blank">cookie policy</a>.
</p>
                        </div>
                    </div>
                    <div class="actions couple">
    <input type="hidden" name="csrfToken" value="LmxhE88" style="">
    <input type="hidden" name="sessionId" value="3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6" style="">
    <input type="hidden" name="originalDoneUrl" value="https://search.yahoo.com/search?p=FontLab&amp;guccounter=1" style="">
    <input type="hidden" name="namespace" value="yahoo" style="">
    <button type="submit" data-beacon="/beacon?tag=TCF2&amp;step=Layer1-AcceptAll&amp;brandDomain=search.yahoo.com&amp;brandBid=&amp;userType=nonreg&amp;sdk=false&amp;tos=en-GB&amp;country=PL&amp;x=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6" class="btn secondary accept-all " name="agree" value="agree" data-ylk="slk:accept-all;elm:consent;sec:general_consent;subsec:guce;itc:1;outcm:consent-layer1_click">Accept all</button>
        <button type="submit" data-beacon="/beacon?tag=TCF2&amp;step=Layer1-RejectAll&amp;brandDomain=search.yahoo.com&amp;brandBid=&amp;userType=nonreg&amp;sdk=false&amp;tos=en-GB&amp;country=PL&amp;x=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6" class="btn secondary reject-all" name="reject" value="reject" data-ylk="slk:reject-all;elm:consent;sec:general_consent;subsec:guce;itc:1;outcm:consent-layer1_click">Reject all</button>
    <a href="/v2/partners?sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6" class="btn secondary mng-btn manage-settings" role="button" data-ylk="slk:manage-privacy-settings;elm:setting;sec:general_consent;subsec:guce;itc:1;outcm:consent-layer1_click">Manage privacy settings</a>
                    </div>
                </div>
                </form>
                <div class="scroll-down-wrapper">
                    <button id="scroll-down-btn" class="btn">
                        <span class="scroll-down-arrow"></span>Go to end
                    </button>
                </div>
            </div>
        </div>
    </div>
</div>

<script src="https://s.yimg.com/oa/build/js/site-820ed71c.js" nonce=""></script>
    <img class="beacon hidden" src="/beacon?tag=TCF2&amp;step=Layer1-View&amp;brandDomain=search.yahoo.com&amp;brandBid=&amp;userType=nonreg&amp;sdk=false&amp;tos=en-GB&amp;country=PL&amp;x=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6">
    <input type="hidden" data-impressionbeaconjs="/beacon?tag=TCF2&amp;step=Layer1-View-Js&amp;brandDomain=search.yahoo.com&amp;brandBid=&amp;userType=nonreg&amp;sdk=false&amp;tos=en-GB&amp;country=PL&amp;x=&amp;sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6" style="">
<noscript>
    <img class="beacon hidden" src="/beacon?tag=code72&step=Layer1-View&sessionId=3_cc-session_7f1945f4-bcb0-4d8f-b35c-290f1b848dd6">
</noscript>
    <script nonce="" src="https://s.yimg.com/ss/rapid-3.53.30.js"> </script>
    <script nonce="">
              var custKeys = {
                      outcm : "consent-layer1_imp",
                      mrkt : "gb",
                      lang : "en-GB",
                      pt : "utility",
                      pct : "story",
                      ver : "java",
                      dmi_consent: "false",
                      site : "www.yahoo.com"
              };
              var rapidConfigs = {
                      spaceid : "1197812781",
                      keys : custKeys,
                      anonymized : "true",
              };
    
              var rapidInstance = new YAHOO.i13n.Rapid(rapidConfigs);
    </script>


</body></html>

================
File: plugins/repos/twat_search/dist/.gitignore
================
*

================
File: plugins/repos/twat_search/resources/brave/brave_image.md
================
[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave Search Image Search API

Brave Search API is a REST API to query Brave Search and get back search
results from the web. The following sections describe how to curate requests,
including parameters and headers, to Brave Search API and get a JSON response
back.

> To try the API on a Free plan, you’ll still need to subscribe — you simply
> won’t be charged. Once subscribed, you can get an API key in the [API Keys
> section](/app/keys).

## Endpoints

Brave Search API exposes multiple endpoints for specific types of data, based
on the level of your subscription. If you don’t see the endpoint you’re
interested in, you may need to change your subscription.

Brave Image Search API is currently available at the following endpoint and
exposes an API to get images from the web relevant to the query.

    
    
    https://api.search.brave.com/res/v1/images/search
    
    

## Example

Get started immediately with CURL. An example request will look something like
this:

    
    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/images/search?q=munich&safesearch=strict&count=20&search_lang=en&country=us&spellcheck=1" \
      -H "Accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "X-Subscription-Token: <YOUR_API_KEY>"
    

## Next Steps

To learn what parameters are available and what responses can be expected
while querying Brave Search, please review the following pages:

  * [Query Parameters](/app/documentation/image-search/query)
  * [Request Headers](/app/documentation/image-search/request-headers)
  * [Response Headers](/app/documentation/image-search/response-headers)
  * [Response Objects](/app/documentation/image-search/responses)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Image Search API

## Query Parameters

#### # Image Search API

This table lists the query parameters supported by the Image Search API. Some
are required, but most are optional.

Parameter| Required| Type| Default| Description  
---|---|---|---|---  
q| true| string| |  The user’s search query term. Query can not be empty. Maximum of 400 characters and 50 words in the query.  
country| false| string| US|  The search query country, where the results come
from. The country string is limited to 2 character country codes of supported
countries. For a list of supported values, see [Country
Codes](/app/documentation/image-search/codes#country-codes).  
search_lang| false| string| en|  The search language preference. The 2 or more
character language code for which the search results are provided. For a list
of possible values, see [Language Codes](/app/documentation/image-
search/codes#language-codes).  
count| false| number| 50|  The number of search results returned in response.
The maximum is `100`. The actual number delivered may be less than requested.  
safesearch| false| string| strict|  Filters search results for adult content.
The following values are supported:

  * `off`: No filtering is done.
  * `strict`: Drops all adult content from search results.

  
spellcheck| false| bool| 1|  Whether to spellcheck provided query. If the
spellchecker is enabled, the modified query is always used for search. The
modified query can be found in `altered` key from the
[query](/app/documentation/image-search/responses#Query) response model.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Image Search API

## Request Headers

#### Image Search API Request Headers

This table lists the request headers supported by the Image Search API, most
of which are optional.

Header| Required| Name| Description  
---|---|---|---  
Accept| false| Accept|  The default supported media type is `application/json`  
Accept-Encoding| false| Accept Encoding|  The supported compression type is
`gzip`.  
Api-Version| false| Web Search API Version|  The Brave Web Search API version
to use. This is denoted by the format `YYYY-MM-DD`. The latest version is used
by default, and the previous ones can be found in the [API Changelog](./api-
changelog).  
Cache-Control| false| Cache Control|  Search will return cached web search
results by default. To prevent caching set the Cache-Control header to `no-
cache`. This is currently done as best effort.  
User-Agent| false| User Agent|  The user agent of the client sending the
request. Search can utilize the user agent to provide a different experience
depending on the client sending the request. The user agent should follow the
commonly used browser agent strings on each platform. For more information on
curating user agents, see [RFC 9110](https://www.rfc-
editor.org/rfc/rfc9110.html#name-user-agent). User agent string examples by
platform:

  * **Android** : Mozilla/5.0 (Linux; Android 13; Pixel 7 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36
  * **iOS** : Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Mobile/15E148 Safari/604.1
  * **macOS** : Mozilla/5.0 (Macintosh; Intel Mac OS X 12_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
  * **Windows** : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/

  
X-Subscription-Token| true| Authentication token|  The secret token for the
subscribed plan to authenticate the request. Can be obtained from [API
Keys](/app/keys).

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Image Search API

## Response Headers

#### Global

This table lists the response headers supported by the Image Search API.

Header| Name| Description  
---|---|---  
X-RateLimit-Limit| Rate Limit|  Rate limits associated with the requested
plan. An example rate limit `X-RateLimit-Limit: 1, 15000` means 1 request per
second and 15000 requests per month.  
X-RateLimit-Policy| Rate Limit Policy|  Rate limit policies currently
associated with the requested plan. An example policy `X-RateLimit-Policy:
1;w=1, 15000;w=2592000` means a limit of 1 request over a 1 second window and
15000 requests over a month window. The windows are always given in seconds.  
X-RateLimit-Remaining| Rate Limit Remaining|  Remaining quota units associated
with the expiring limits. An example remaining limit `X-RateLimit-Remaining:
1, 1000` indicates the API is able to be accessed once during the current
second, and 1000 times over the current month. **Note** : Only successful
requests are counted and billed.  
X-RateLimit-Reset| Rate Limit Reset|  The number of seconds until the quota
associated with the expiring limits resets. An example reset limit
`X-RateLimit-Reset: 1, 1419704` means a single request can be done again in a
second and in 1419704 seconds the full monthly quota associated with the plan
will be available again.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Image Search API

## Response Objects

#### # ImageSearchApiResponse

Top level response model for successful Image Search API requests. The API can
also respond back with an error response based on invalid subscription keys
and rate limit events.

Field| Type| Description  
---|---|---  
type| "images"| The type of search API result. The value is always images.  
query| Query| Image search query string.  
results| list [ ImageResult ]| The list of image results for the given query.  
  
#### # Query

A model representing information gathered around the requested query.

Field| Type| Description  
---|---|---  
original| string| The original query that was requested.  
altered| string| The altered query by the spellchecker. This is the query that
is used to search.  
spellcheck_off| bool| Whether the spell checker is enabled or disabled.  
show_strict_warning| string| The value is True if the lack of results is due
to a 'strict' safesearch setting. Adult content relevant to the query was
found, but was blocked by safesearch.  
  
#### # ImageResult

A model representing an image result for the requested query.

Field| Type| Description  
---|---|---  
type| image_result| The type of image search API result. The value is always
image_result.  
title| string| The title of the image.  
url| string| The original page url where the image was found.  
source| string| The source domain where the image was found.  
page_fetched| string| The iso date time when the page was last fetched. The
format is YYYY-MM-DDTHH:MM:SSZ  
thumbnail| Thumbnail| The thumbnail for the image.  
properties| Properties| Metadata for the image.  
meta_url| MetaUrl| Aggregated information on the url associated with the image
search result.  
  
#### # Thumbnail

Aggregated details representing the image thumbnail

Field| Type| Description  
---|---|---  
src| string| The served url of the image.  
  
#### # Properties

Metadata on an image.

Field| Type| Description  
---|---|---  
url| string| The image URL.  
placeholder| string| The lower resolution placeholder image url.  
  
#### # MetaUrl

Aggregated information about a url.

Field| Type| Description  
---|---|---  
scheme| string| The protocol scheme extracted from the url.  
netloc| string| The network location part extracted from the url.  
hostname| string| The lowercased domain name extracted from the url.  
favicon| string| The favicon used for the url.  
path| string| The hierarchical path of the url useful as a display string.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

## Codes

#### [#](/app/documentation/image-search/query#country) Country Codes

This table lists the country codes supported by the `country` parameter.

Country| Code  
---|---  
All Regions| ALL  
Argentina| AR  
Australia| AU  
Austria| AT  
Belgium| BE  
Brazil| BR  
Canada| CA  
Chile| CL  
Denmark| DK  
Finland| FI  
France| FR  
Germany| DE  
Hong Kong| HK  
India| IN  
Indonesia| ID  
Italy| IT  
Japan| JP  
Korea| KR  
Malaysia| MY  
Mexico| MX  
Netherlands| NL  
New Zealand| NZ  
Norway| NO  
Peoples Republic of China| CN  
Poland| PL  
Portugal| PT  
Republic of the Philippines| PH  
Russia| RU  
Saudi Arabia| SA  
South Africa| ZA  
Spain| ES  
Sweden| SE  
Switzerland| CH  
Taiwan| TW  
Turkey| TR  
United Kingdom| GB  
United States| US  
  
#### [#](/app/documentation/image-search/query#language) Language Codes

This table lists the language codes supported by the `search_lang` parameter.

Language| Code  
---|---  
Arabic| ar  
Basque| eu  
Bengali| bn  
Bulgarian| bg  
Catalan| ca  
Chinese Simplified| zh-hans  
Chinese Traditional| zh-hant  
Croatian| hr  
Czech| cs  
Danish| da  
Dutch| nl  
English| en  
English United Kingdom| en-gb  
Estonian| et  
Finnish| fi  
French| fr  
Galician| gl  
German| de  
Gujarati| gu  
Hebrew| he  
Hindi| hi  
Hungarian| hu  
Icelandic| is  
Italian| it  
Japanese| jp  
Kannada| kn  
Korean| ko  
Latvian| lv  
Lithuanian| lt  
Malay| ms  
Malayalam| ml  
Marathi| mr  
Norwegian Bokmål| nb  
Polish| pl  
Portuguese Brazil| pt-br  
Portuguese Portugal| pt-pt  
Punjabi| pa  
Romanian| ro  
Russian| ru  
Serbian Cyrylic| sr  
Slovak| sk  
Slovenian| sl  
Spanish| es  
Swedish| sv  
Tamil| ta  
Telugu| te  
Thai| th  
Turkish| tr  
Ukrainian| uk  
Vietnamese| vi

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave Search API Changelog - Images

This changelog lists all updates to the Brave Image Search API in
chronological order.

### 2023-08-09

  * Initial release of the Image Search API.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search)

[ Get Started](/app/documentation/image-search/get-started)[ Query
Parameters](/app/documentation/image-search/query)[ Request
Headers](/app/documentation/image-search/request-headers)[ Response
Headers](/app/documentation/image-search/response-headers)[ Response
Objects](/app/documentation/image-search/responses)[
Codes](/app/documentation/image-search/codes)[ API
Changelog](/app/documentation/image-search/api-changelog)[ How to
Guides](/app/documentation/image-search/guides)

[ Video Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# A simple guide to handle missing and small image results

##### Brave Search Image API

There are two limitations of the Brave Search Image API:

  * the image results could be pointing to urls that cannot be resolved (404)
  * the image dimensions `width` and `height` are not provided ahead of time.

We’ve worked around the very same limitations to build our image search
vertical at [search.brave.com/images](https://search.brave.com/images?q=cats).
This guide is a simpler version of the same approach.

Our project structure will be this:

    
    
    index.js
    public/
        images.html
        styles.css
    

To get started, let’s create a folder for our image search page as well as
create the files we will need later on:

    
    
    mkdir img-search && cd img-search
    mkdir public
    touch public/images.html
    touch public/styles.css
    

We’ll use a simple [node.js express server](https://expressjs.com/) to call
the API. So let’s go ahead and install it as a dependency (our only one).

    
    
    npm install express --save
    

The following `index.js` is our server, and will be handling the API calls. It
has to be done through a server environment so that there are no Cross-Origin
Resource Sharing ([CORS](https://developer.mozilla.org/en-
US/docs/Web/HTTP/CORS)) issues.

#### index.js

    
    
    const express = require('express');
    const app = express();
    const port = 4000;
    
    app.use(express.static('public'));
    
    const API_KEY = '<YOUR_API_KEY>';
    const API_PATH = 'https://api.search.brave.com/res/v1/images/search';
    
    app.get('/api/images', async (req, res) => {
      try {
        const params = new URLSearchParams({
          q: req.query.q,
          count: 20,
          search_lang: 'en',
          country: 'us',
          spellcheck: 1,
        });
        const response = await fetch(`${API_PATH}?${params}`, {
          headers: {
            'x-subscription-token': API_KEY,
            accept: 'application/json',
          },
        });
        const data = await response.json();
        res.json(data);
        return;
      } catch (err) {
        console.log(err);
      }
      res.status(500).send('Internal Server Error');
    });
    
    app.listen(port, () => {
      console.log(`Example app listening on port ${port}`);
    });
    

Now that we have our server, let’s focus on the client side scripts needed to
load image outside of the DOM, to learn if they exist, and what the `width`
and `height` is.

#### public/images.html

    
    
    <html>
      <head>
        <title>Image Search</title>
        <link rel="stylesheet" href="/styles.css" />
      </head>
      <body>
        <script lang="javascript">
          async function fetchImages(query) {
            const params = new URLSearchParams({ q: query });
            const response = await fetch(`/api/images?${params}`);
            return await response.json();
            return data;
          }
    
          function renderImages(images) {
            const imagesContainer = document.getElementById('images');
            imagesContainer.innerHTML = '';
            images.forEach(({ image }) => {
              const figElement = document.createElement('figure');
    
              const imgElement = document.createElement('img');
              imgElement.src = image.thumbnail.src;
              imgElement.alt = image.title;
              imgElement.width = image.thumbnail.width;
              imgElement.height = image.thumbnail.height;
    
              const figCaptionElement = document.createElement('figcaption');
              figCaptionElement.innerHTML =
                `<div class="dimensions">${image.thumbnail.width} x ${image.thumbnail.height}</div>` +
                image.title;
    
              figElement.appendChild(imgElement);
              figElement.appendChild(figCaptionElement);
    
              imagesContainer.appendChild(figElement);
            });
          }
    
          function loadImage(result) {
            return new Promise((resolve, reject) => {
              const img = new Image();
              img.crossOrigin = 'anonymous';
              img.onload = (e) => {
                if (e.target) {
                  const image = e.target;
                  const width = image.naturalWidth;
                  const height = image.naturalHeight;
    
                  // Filter images that are too small
                  if (width < 275 || height < 275) {
                    console.error('[Img] Image too small', result);
                    reject(result);
                    return;
                  }
    
                  // Fill missing info; use the size the image will be scaled into instead of actual size
                  result.thumbnail.width = width;
                  result.thumbnail.height = height;
    
                  resolve(result);
                } else {
                  console.error('[Img] onLoad returned no image', result);
                  reject(result);
                }
              };
              img.onerror = (e) => {
                console.error('[Img] onError loading img', e);
                reject(result);
              };
              img.onabort = (e) => {
                console.error('[Img] onAbort loading img', e);
                reject(result);
              };
              img.src = result.thumbnail.src;
            });
          }
    
          async function load(query) {
            // Load images from API
            const response = await fetchImages(query);
            const loadedImages = [];
            let i = 0;
            const count = 5;
            const initialPromises = [];
    
            // Load the first 5 images in parallel
            for (let i = 0; i < count; i++) {
              initialPromises.push(async () => {
                const result = response.results[i];
                try {
                  const image = await loadImage(result);
                  loadedImages.push({
                    image,
                  });
                } catch (err) {
                  // pass
                }
              });
            }
    
            // Wait for the first 5 images to resolve
            await Promise.all(initialPromises);
    
            // Load images sequentially until there is 5 results, then show them.
            while (loadedImages.length < 5 || i >= response.results.length) {
              const result = response.results[i];
              try {
                const image = await loadImage(result);
                loadedImages.push({
                  image,
                });
              } catch (err) {
                // pass
              }
              i++;
            }
    
            renderImages(loadedImages);
          }
        </script>
        <div id="search">
          <input type="text" id="query" value="" onchan />
          <button onclick="load(document.getElementById('query').value)">Search</button>
        </div>
        <div id="images"></div>
      </body>
    </html>
    

The approach here, is to load images outside of the DOM, 5 at a time (which
you can configure), and images that we manage successfully, and are larger
than a given size (`width > 275 || height > 275` in the example), we show in
the DOM.

Lastly, we need some styles to make sure what we see does not look too bad:

### styles.css

    
    
    #images {
      display: flex;
      flex-wrap: wrap;
      gap: 20px;
      max-width: 1000px;
      margin: 2rem auto;
      justify-content: center;
    }
    #search {
      display: flex;
      gap: 1rem;
      align-items: center;
      width: 600px;
      margin: auto;
    }
    #search input {
      width: 100%;
      padding: 10px 20px;
      border-radius: 12px;
      outline: none;
      border: solid 1px #d3d3d3;
      background: #f3f3f3;
    }
    button {
      cursor: pointer;
      padding: 10px 20px;
      border-radius: 12px;
      outline: none;
      border: solid 1px #d3d3d3;
      background: #f3f3f3;
    }
    button:hover {
      background: #111;
      color: #fff;
    }
    figure {
      border: solid 1px #d3d3d3;
      display: flex;
      flex-flow: column;
      padding: 5px 5px 10px 5px;
      max-width: 220px;
      gap: 1rem;
      border-radius: 12px;
      margin: 0;
    }
    
    img {
      max-width: 220px;
      max-height: 150px;
      object-fit: contain;
      border-radius: 12px;
    }
    
    figcaption {
      display: flex;
      text-align: center;
      font-family: sans-serif;
      font-size: 14px;
      flex-direction: column;
      gap: 5px;
      align-items: center;
    }
    figcaption .dimensions {
      font-size: 12px;
      color: blue;
      background: #e7e7fc;
      padding: 5px 10px;
      border-radius: 12px;
      width: fit-content;
    }
    

And we’re ready to go. You can now run the project:

    
    
    node index.js
    

Open `localhost:4000/images.html` in the browser, and start searching.

![Image Search Client](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/img-search-client.Yjc8cU7X.png)

================
File: plugins/repos/twat_search/resources/brave/brave_news.md
================
[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave News Search API

Brave Search API is a REST API to query Brave Search and get back search
results from the web. The following sections describe how to curate requests,
including parameters and headers, to Brave Search API and get a JSON response
back.

> To try the API on a Free plan, you’ll still need to subscribe — you simply
> won’t be charged. Once subscribed, you can get an API key in the [API Keys
> section](/app/keys).

## Endpoints

Brave Search API exposes multiple endpoints for specific types of data, based
on the level of your subscription. If you don’t see the endpoint you’re
interested in, you may need to change your subscription.

Brave News Search API is currently available at the following endpoint and
exposes an API to get news from the web relevant to the query.

    
    
    https://api.search.brave.com/res/v1/news/search
    

## Example

Get started immediately with CURL. An example request will look something like
this:

    
    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/news/search?q=munich&count=10&country=us&search_lang=en&spellcheck=1" \
      -H "Accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "X-Subscription-Token: <YOUR_API_KEY>"
    

## Next Steps

To learn what parameters are available and what responses can be expected
while querying Brave Search, please review the following pages:

  * [Query Parameters](/app/documentation/news-search/query)
  * [Request Headers](/app/documentation/news-search/request-headers)
  * [Response Headers](/app/documentation/news-search/response-headers)
  * [Response Objects](/app/documentation/news-search/responses)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave News Search API

## Query Parameters

#### # News Search API

This table lists the query parameters supported by the News Search API. Some
are required, but most are optional.

Parameter| Required| Type| Default| Description  
---|---|---|---|---  
q| true| string| |  The user’s search query term. Query can not be empty. Maximum of 400 characters and 50 words in the query.  
country| false| string| US|  The search query country, where the results come
from. The country string is limited to 2 character country codes of supported
countries. For a list of supported values, see [Country
Codes](/app/documentation/news-search/codes#country-codes).  
search_lang| false| string| en|  The search language preference. The 2 or more
character language code for which the search results are provided. For a list
of possible values, see [Language Codes](/app/documentation/news-
search/codes#language-codes).  
ui_lang| false| string| en-US|  User interface language preferred in response.
Usually of the format ‘<language_code>-<country_code>’. For more, see [RFC
9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-language). For a
list of supported values, see [UI Language Codes](/app/documentation/news-
search/codes#market-codes).  
count| false| number| 20|  The number of search results returned in response.
The maximum is `50`. The actual number delivered may be less than requested.
Combine this parameter with `offset` to paginate search results.  
offset| false| number| 0|  The zero based offset that indicates number of
search results per page (count) to skip before returning the result. The
maximum is `9`. The actual number delivered may be less than requested based
on the query. In order to paginate results use this parameter together with
`count`. For example, if your user interface displays 20 search results per
page, set `count` to `20` and offset to `0` to show the first page of results.
To get subsequent pages, increment `offset` by 1 (e.g. 0, 1, 2). The results
may overlap across multiple pages.  
spellcheck| false| bool| 1|  Whether to spellcheck provided query. If the
spellchecker is enabled, the modified query is always used for search. The
modified query can be found in `altered` key from the
[query](/app/documentation/news-search/responses#Query) response model.  
safesearch| false| string| moderate|  Filters search results for adult
content. The following values are supported:

  * `off` \- No filtering.
  * `moderate` \- Filter out explicit content.
  * `strict` \- Filter out explicit and suggestive content.

  
freshness| false| string| |  Filters search results by when they were discovered. The following values are supported: \- `pd`: Discovered within the last 24 hours. \- `pw`: Discovered within the last 7 Days. \- `pm`: Discovered within the last 31 Days. \- `py`: Discovered within the last 365 Days… \- `YYYY-MM-DDtoYYYY-MM-DD`: timeframe is also supported by specifying the date range e.g. `2022-04-01to2022-07-30`.  
extra_snippets| false| bool| |  A snippet is an excerpt from a page you get as a result of the query, and extra_snippets allow you to get up to 5 additional, alternative excerpts. Only available under `Free AI`, `Base AI`, `Pro AI`, `Base Data`, `Pro Data` and `Custom plans`.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave News Search API

## Request Headers

#### News Search API Request Headers

This table lists the request headers supported by the News Search API, most of
which are optional.

Header| Required| Name| Description  
---|---|---|---  
Accept| false| Accept|  The default supported media type is `application/json`  
Accept-Encoding| false| Accept Encoding|  The supported compression type is
`gzip`.  
Api-Version| false| Web Search API Version|  The Brave Web Search API version
to use. This is denoted by the format `YYYY-MM-DD`. The latest version is used
by default, and the previous ones can be found in the [API Changelog](./api-
changelog).  
Cache-Control| false| Cache Control|  Search will return cached web search
results by default. To prevent caching set the Cache-Control header to `no-
cache`. This is currently done as best effort.  
User-Agent| false| User Agent|  The user agent of the client sending the
request. Search can utilize the user agent to provide a different experience
depending on the client sending the request. The user agent should follow the
commonly used browser agent strings on each platform. For more information on
curating user agents, see [RFC 9110](https://www.rfc-
editor.org/rfc/rfc9110.html#name-user-agent). User agent string examples by
platform:

  * **Android** : Mozilla/5.0 (Linux; Android 13; Pixel 7 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36
  * **iOS** : Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Mobile/15E148 Safari/604.1
  * **macOS** : Mozilla/5.0 (Macintosh; Intel Mac OS X 12_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
  * **Windows** : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/

  
X-Subscription-Token| true| Authentication token|  The secret token for the
subscribed plan to authenticate the request. Can be obtained from [API
Keys](/app/keys).

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave News Search API

## Response Headers

#### Global

This table lists the response headers supported by the News Search API.

Header| Name| Description  
---|---|---  
X-RateLimit-Limit| Rate Limit|  Rate limits associated with the requested
plan. An example rate limit `X-RateLimit-Limit: 1, 15000` means 1 request per
second and 15000 requests per month.  
X-RateLimit-Policy| Rate Limit Policy|  Rate limit policies currently
associated with the requested plan. An example policy `X-RateLimit-Policy:
1;w=1, 15000;w=2592000` means a limit of 1 request over a 1 second window and
15000 requests over a month window. The windows are always given in seconds.  
X-RateLimit-Remaining| Rate Limit Remaining|  Remaining quota units associated
with the expiring limits. An example remaining limit `X-RateLimit-Remaining:
1, 1000` indicates the API is able to be accessed once during the current
second, and 1000 times over the current month. **Note** : Only successful
requests are counted and billed.  
X-RateLimit-Reset| Rate Limit Reset|  The number of seconds until the quota
associated with the expiring limits resets. An example reset limit
`X-RateLimit-Reset: 1, 1419704` means a single request can be done again in a
second and in 1419704 seconds the full monthly quota associated with the plan
will be available again.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave News Search API

## Response Objects

#### # NewsSearchApiResponse

Top level response model for successful News Search API requests. The API can
also respond back with an error response based on invalid subscription keys
and rate limit events.

Field| Type| Description  
---|---|---  
type| "news"| The type of search API result. The value is always news.  
query| Query| News search query string.  
results| list [ NewsResult ]| The list of news results for the given query.  
  
#### # Query

A model representing information gathered around the requested query.

Field| Type| Description  
---|---|---  
original| string| The original query that was requested.  
altered| string| The altered query by the spellchecker. This is the query that
is used to search if any.  
cleaned| string| The cleaned noramlized query by the spellchecker. This is the
query that is used to search if any.  
spellcheck_off| bool| Whether the spell checker is enabled or disabled.  
show_strict_warning| bool| The value is True if the lack of results is due to
a 'strict' safesearch setting. Adult content relevant to the query was found,
but was blocked by safesearch.  
  
#### # NewsResult

A model representing a news result for the requested query.

Field| Type| Description  
---|---|---  
type| news_result| The type of news search API result. The value is always
news_result.  
url| string| The source url of the news article.  
title| string| The title of the news article.  
description| string| The description for the news article.  
age| string| A human readable representation of the page age.  
page_age| string| The page age found from the source web page.  
page_fetched| string| The iso date time when the page was last fetched. The
format is YYYY-MM-DDTHH:MM:SSZ  
breaking| bool| Whether the result includes breaking news.  
thumbnail| Thumbnail| The thumbnail for the news article.  
meta_url| MetaUrl| Aggregated information on the url associated with the news
search result.  
extra_snippets| list [ string ]| A list of extra alternate snippets for the
news search result.  
  
#### # Thumbnail

Aggregated details representing the news thumbnail

Field| Type| Description  
---|---|---  
src| string| The served url of the thumbnail associated with the news article.  
original| string| The original url of the thumbnail associated with the news
article.  
  
#### # MetaUrl

Aggregated information about a url.

Field| Type| Description  
---|---|---  
scheme| string| The protocol scheme extracted from the url.  
netloc| string| The network location part extracted from the url.  
hostname| string| The lowercased domain name extracted from the url.  
favicon| string| The favicon used for the url.  
path| string| The hierarchical path of the url useful as a display string.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave News Search API

## Codes

#### [#](/app/documentation/news-search/query#country) Country Codes

This table lists the country codes supported by the `country` parameter.

Country| Code  
---|---  
All Regions| ALL  
Argentina| AR  
Australia| AU  
Austria| AT  
Belgium| BE  
Brazil| BR  
Canada| CA  
Chile| CL  
Denmark| DK  
Finland| FI  
France| FR  
Germany| DE  
Hong Kong| HK  
India| IN  
Indonesia| ID  
Italy| IT  
Japan| JP  
Korea| KR  
Malaysia| MY  
Mexico| MX  
Netherlands| NL  
New Zealand| NZ  
Norway| NO  
Peoples Republic of China| CN  
Poland| PL  
Portugal| PT  
Republic of the Philippines| PH  
Russia| RU  
Saudi Arabia| SA  
South Africa| ZA  
Spain| ES  
Sweden| SE  
Switzerland| CH  
Taiwan| TW  
Turkey| TR  
United Kingdom| GB  
United States| US  
  
#### [#](/app/documentation/news-search/query#language) Language Codes

This table lists the language codes supported by the `search_lang` parameter.

Language| Code  
---|---  
Arabic| ar  
Basque| eu  
Bengali| bn  
Bulgarian| bg  
Catalan| ca  
Chinese Simplified| zh-hans  
Chinese Traditional| zh-hant  
Croatian| hr  
Czech| cs  
Danish| da  
Dutch| nl  
English| en  
English United Kingdom| en-gb  
Estonian| et  
Finnish| fi  
French| fr  
Galician| gl  
German| de  
Gujarati| gu  
Hebrew| he  
Hindi| hi  
Hungarian| hu  
Icelandic| is  
Italian| it  
Japanese| jp  
Kannada| kn  
Korean| ko  
Latvian| lv  
Lithuanian| lt  
Malay| ms  
Malayalam| ml  
Marathi| mr  
Norwegian Bokmål| nb  
Polish| pl  
Portuguese Brazil| pt-br  
Portuguese Portugal| pt-pt  
Punjabi| pa  
Romanian| ro  
Russian| ru  
Serbian Cyrylic| sr  
Slovak| sk  
Slovenian| sl  
Spanish| es  
Swedish| sv  
Tamil| ta  
Telugu| te  
Thai| th  
Turkish| tr  
Ukrainian| uk  
Vietnamese| vi  
  
#### [#](/app/documentation/news-search/query#market-code) Market Codes

This table lists the country language codes supported by the `ui_lang`
parameter.

Country| Language| Code  
---|---|---  
Argentina| Spanish| es-AR  
Australia| English| en-AU  
Austria| German| de-AT  
Belgium| Dutch| nl-BE  
Belgium| French| fr-BE  
Brazil| Portuguese| pt-BR  
Canada| English| en-CA  
Canada| French| fr-CA  
Chile| Spanish| es-CL  
Denmark| Danish| da-DK  
Finland| Finnish| fi-FI  
France| French| fr-FR  
Germany| German| de-DE  
Hong Kong SAR| Traditional Chinese| zh-HK  
India| English| en-IN  
Indonesia| English| en-ID  
Italy| Italian| it-IT  
Japan| Japanese| ja-JP  
Korea| Korean| ko-KR  
Malaysia| English| en-MY  
Mexico| Spanish| es-MX  
Netherlands| Dutch| nl-NL  
New Zealand| English| en-NZ  
Norway| Norwegian| no-NO  
People's republic of China| Chinese| zh-CN  
Poland| Polish| pl-PL  
Republic of the Philippines| English| en-PH  
Russia| Russian| ru-RU  
South Africa| English| en-ZA  
Spain| Spanish| es-ES  
Sweden| Swedish| sv-SE  
Switzerland| French| fr-CH  
Switzerland| German| de-CH  
Taiwan| Traditional Chinese| zh-TW  
Turkey| Turkish| tr-TR  
United Kingdom| English| en-GB  
United States| English| en-US  
United States| Spanish| es-US

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search)

[ Get Started](/app/documentation/news-search/get-started)[ Query
Parameters](/app/documentation/news-search/query)[ Request
Headers](/app/documentation/news-search/request-headers)[ Response
Headers](/app/documentation/news-search/response-headers)[ Response
Objects](/app/documentation/news-search/responses)[
Codes](/app/documentation/news-search/codes)[ API
Changelog](/app/documentation/news-search/api-changelog)

[ Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave Search API Changelog - News

This changelog lists all updates to the Brave News Search API in chronological
order.

### 2023-08-15

  * Add news search.

================
File: plugins/repos/twat_search/resources/brave/brave_video.md
================
[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave Video Search API

Brave Search API is a REST API to query Brave Search and get back search
results from the web. The following sections describe how to curate requests,
including parameters and headers, to Brave Search API and get a JSON response
back.

> To try the API on a Free plan, you’ll still need to subscribe — you simply
> won’t be charged. Once subscribed, you can get an API key in the [API Keys
> section](/app/keys).

## Endpoints

Brave Search API exposes multiple endpoints for specific types of data, based
on the level of your subscription. If you don’t see the endpoint you’re
interested in, you may need to change your subscription.

Brave Video Search API is currently available at the following endpoint and
exposes an API to get videos from the web relevant to the query.

    
    
    https://api.search.brave.com/res/v1/videos/search
    

## Example

Get started immediately with CURL. An example request will look something like
this:

    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/videos/search?q=munich&count=10&country=us&search_lang=en&spellcheck=1" \
      -H "Accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "X-Subscription-Token: <YOUR_API_KEY>"
    

## Next Steps

To learn what parameters are available and what responses can be expected
while querying Brave Search, please review the following pages:

  * [Query Parameters](/app/documentation/video-search/query)
  * [Request Headers](/app/documentation/video-search/request-headers)
  * [Response Headers](/app/documentation/video-search/response-headers)
  * [Response Objects](/app/documentation/video-search/responses)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Video Search API

## Query Parameters

#### # Video Search API

This table lists the query parameters supported by the Video Search API. Some
are required, but most are optional.

Parameter| Required| Type| Default| Description  
---|---|---|---|---  
q| true| string| |  The user’s search query term. Query can not be empty. Maximum of 400 characters and 50 words in the query.  
country| false| string| US|  The search query country, where the results come
from. The country string is limited to 2 character country codes of supported
countries. For a list of supported values, see [Country
Codes](/app/documentation/video-search/codes#country-codes).  
search_lang| false| string| en|  The search language preference. The 2 or more
character language code for which the search results are provided. For a list
of possible values, see [Language Codes](/app/documentation/video-
search/codes#language-codes).  
ui_lang| false| string| en-US|  User interface language preferred in response.
Usually of the format ‘<language_code>-<country_code>’. For more, see [RFC
9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-language). For a
list of supported values, see [UI Language Codes](/app/documentation/video-
search/codes#market-codes).  
count| false| number| 20|  The number of search results returned in response.
The maximum is `50`. The actual number delivered may be less than requested.
Combine this parameter with `offset` to paginate search results.  
offset| false| number| 0|  The zero based offset that indicates number of
search results per page (count) to skip before returning the result. The
maximum is `9`. The actual number delivered may be less than requested based
on the query. In order to paginate results use this parameter together with
`count`. For example, if your user interface displays 20 search results per
page, set `count` to `20` and offset to `0` to show the first page of results.
To get subsequent pages, increment `offset` by 1 (e.g. 0, 1, 2). The results
may overlap across multiple pages.  
spellcheck| false| bool| 1|  Whether to spellcheck provided query. If the
spellchecker is enabled, the modified query is always used for search. The
modified query can be found in `altered` key from the
[query](/app/documentation/video-search/responses#Query) response model.  
safesearch| false| string| moderate|  Filters search results for adult
content. The following values are supported:

  * `off` \- No filtering.
  * `moderate` \- Filter out explicit content.
  * `strict` \- Filter out explicit and suggestive content.

  
freshness| false| string| |  Filters search results by when they were discovered. The following values are supported: \- `pd`: Discovered within the last 24 hours. \- `pw`: Discovered within the last 7 Days. \- `pm`: Discovered within the last 31 Days. \- `py`: Discovered within the last 365 Days… \- `YYYY-MM-DDtoYYYY-MM-DD`: timeframe is also supported by specifying the date range e.g. `2022-04-01to2022-07-30`.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Video Search API

## Request Headers

#### Video Search API Request Headers

This table lists the request headers supported by the Video Search API, most
of which are optional.

Header| Required| Name| Description  
---|---|---|---  
Accept| false| Accept|  The default supported media type is `application/json`  
Accept-Encoding| false| Accept Encoding|  The supported compression type is
`gzip`.  
Api-Version| false| Web Search API Version|  The Brave Web Search API version
to use. This is denoted by the format `YYYY-MM-DD`. The latest version is used
by default, and the previous ones can be found in the [API Changelog](./api-
changelog).  
Cache-Control| false| Cache Control|  Search will return cached web search
results by default. To prevent caching set the Cache-Control header to `no-
cache`. This is currently done as best effort.  
User-Agent| false| User Agent|  The user agent of the client sending the
request. Search can utilize the user agent to provide a different experience
depending on the client sending the request. The user agent should follow the
commonly used browser agent strings on each platform. For more information on
curating user agents, see [RFC 9110](https://www.rfc-
editor.org/rfc/rfc9110.html#name-user-agent). User agent string examples by
platform:

  * **Android** : Mozilla/5.0 (Linux; Android 13; Pixel 7 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36
  * **iOS** : Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Mobile/15E148 Safari/604.1
  * **macOS** : Mozilla/5.0 (Macintosh; Intel Mac OS X 12_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
  * **Windows** : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/

  
X-Subscription-Token| true| Authentication token|  The secret token for the
subscribed plan to authenticate the request. Can be obtained from [API
Keys](/app/keys).

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Video Search API

## Response Headers

#### Global

This table lists the response headers supported by the Video Search API.

Header| Name| Description  
---|---|---  
X-RateLimit-Limit| Rate Limit|  Rate limits associated with the requested
plan. An example rate limit `X-RateLimit-Limit: 1, 15000` means 1 request per
second and 15000 requests per month.  
X-RateLimit-Policy| Rate Limit Policy|  Rate limit policies currently
associated with the requested plan. An example policy `X-RateLimit-Policy:
1;w=1, 15000;w=2592000` means a limit of 1 request over a 1 second window and
15000 requests over a month window. The windows are always given in seconds.  
X-RateLimit-Remaining| Rate Limit Remaining|  Remaining quota units associated
with the expiring limits. An example remaining limit `X-RateLimit-Remaining:
1, 1000` indicates the API is able to be accessed once during the current
second, and 1000 times over the current month. **Note** : Only successful
requests are counted and billed.  
X-RateLimit-Reset| Rate Limit Reset|  The number of seconds until the quota
associated with the expiring limits resets. An example reset limit
`X-RateLimit-Reset: 1, 1419704` means a single request can be done again in a
second and in 1419704 seconds the full monthly quota associated with the plan
will be available again.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Video Search API

## Response Objects

#### # VideoSearchApiResponse

Top level response model for successful Video Search API requests. The API can
also respond back with an error response based on invalid subscription keys
and rate limit events.

Field| Type| Description  
---|---|---  
type| "videos"| The type of search API result. The value is always video.  
query| Query| Video search query string.  
results| list [ VideoResult ]| The list of video results for the given query.  
  
#### # Query

A model representing information gathered around the requested query.

Field| Type| Description  
---|---|---  
original| string| The original query that was requested.  
altered| string| The altered query by the spellchecker. This is the query that
is used to search if any.  
cleaned| string| The cleaned noramlized query by the spellchecker. This is the
query that is used to search if any.  
spellcheck_off| bool| Whether the spell checker is enabled or disabled.  
show_strict_warning| string| The value is True if the lack of results is due
to a 'strict' safesearch setting. Adult content relevant to the query was
found, but was blocked by safesearch.  
  
#### # VideoResult

A model representing a video result for the requested query.

Field| Type| Description  
---|---|---  
type| video_result| The type of video search API result. The value is always
video_result.  
url| string| The source url of the video.  
title| string| The title of the video.  
description| string| The description for the video.  
age| string| A human readable representation of the page age.  
page_age| string| The page age found from the source web page.  
page_fetched| string| The iso date time when the page was last fetched. The
format is YYYY-MM-DDTHH:MM:SSZ  
thumbnail| Thumbnail| The thumbnail for the video.  
video| VideoData| Metadata for the video.  
meta_url| MetaUrl| Aggregated information on the url associated with the video
search result.  
  
#### # Thumbnail

Aggregated details representing the video thumbnail

Field| Type| Description  
---|---|---  
src| string| The served url of the thumbnail associated with the video.  
original| string| The original url of the thumbnail associated with the video.  
  
#### # VideoData

A model representing metadata gathered for a video.

Field| Type| Description  
---|---|---  
duration| string| A time string representing the duration of the video.  
views| int| The number of views of the video.  
creator| string| The creator of the video.  
publisher| string| The publisher of the video.  
requires_subscription| bool| Whether the video requires a subscription.  
tags| list [ string ]| A list of tags relevant to the video.  
author| Profile| A list of profiles associated with the video.  
  
#### # Profile

A profile of an entity associated with the video.

Field| Type| Description  
---|---|---  
name| string| The name of the profile.  
long_name| string| The long name of the profile.  
url| string| The original url where the profile is available.  
img| string| The served image url representing the profile.  
  
#### # MetaUrl

Aggregated information about a url.

Field| Type| Description  
---|---|---  
scheme| string| The protocol scheme extracted from the url.  
netloc| string| The network location part extracted from the url.  
hostname| string| The lowercased domain name extracted from the url.  
favicon| string| The favicon used for the url.  
path| string| The hierarchical path of the url useful as a display string.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Video Search API

## Codes

#### [#](/app/documentation/video-search/query#country) Country Codes

This table lists the country codes supported by the `country` parameter.

Country| Code  
---|---  
All Regions| ALL  
Argentina| AR  
Australia| AU  
Austria| AT  
Belgium| BE  
Brazil| BR  
Canada| CA  
Chile| CL  
Denmark| DK  
Finland| FI  
France| FR  
Germany| DE  
Hong Kong| HK  
India| IN  
Indonesia| ID  
Italy| IT  
Japan| JP  
Korea| KR  
Malaysia| MY  
Mexico| MX  
Netherlands| NL  
New Zealand| NZ  
Norway| NO  
Peoples Republic of China| CN  
Poland| PL  
Portugal| PT  
Republic of the Philippines| PH  
Russia| RU  
Saudi Arabia| SA  
South Africa| ZA  
Spain| ES  
Sweden| SE  
Switzerland| CH  
Taiwan| TW  
Turkey| TR  
United Kingdom| GB  
United States| US  
  
#### [#](/app/documentation/video-search/query#language) Language Codes

This table lists the language codes supported by the `search_lang` parameter.

Language| Code  
---|---  
Arabic| ar  
Basque| eu  
Bengali| bn  
Bulgarian| bg  
Catalan| ca  
Chinese Simplified| zh-hans  
Chinese Traditional| zh-hant  
Croatian| hr  
Czech| cs  
Danish| da  
Dutch| nl  
English| en  
English United Kingdom| en-gb  
Estonian| et  
Finnish| fi  
French| fr  
Galician| gl  
German| de  
Gujarati| gu  
Hebrew| he  
Hindi| hi  
Hungarian| hu  
Icelandic| is  
Italian| it  
Japanese| jp  
Kannada| kn  
Korean| ko  
Latvian| lv  
Lithuanian| lt  
Malay| ms  
Malayalam| ml  
Marathi| mr  
Norwegian Bokmål| nb  
Polish| pl  
Portuguese Brazil| pt-br  
Portuguese Portugal| pt-pt  
Punjabi| pa  
Romanian| ro  
Russian| ru  
Serbian Cyrylic| sr  
Slovak| sk  
Slovenian| sl  
Spanish| es  
Swedish| sv  
Tamil| ta  
Telugu| te  
Thai| th  
Turkish| tr  
Ukrainian| uk  
Vietnamese| vi  
  
#### [#](/app/documentation/video-search/query#market-code) Market Codes

This table lists the country language codes supported by the `ui_lang`
parameter.

Country| Language| Code  
---|---|---  
Argentina| Spanish| es-AR  
Australia| English| en-AU  
Austria| German| de-AT  
Belgium| Dutch| nl-BE  
Belgium| French| fr-BE  
Brazil| Portuguese| pt-BR  
Canada| English| en-CA  
Canada| French| fr-CA  
Chile| Spanish| es-CL  
Denmark| Danish| da-DK  
Finland| Finnish| fi-FI  
France| French| fr-FR  
Germany| German| de-DE  
Hong Kong SAR| Traditional Chinese| zh-HK  
India| English| en-IN  
Indonesia| English| en-ID  
Italy| Italian| it-IT  
Japan| Japanese| ja-JP  
Korea| Korean| ko-KR  
Malaysia| English| en-MY  
Mexico| Spanish| es-MX  
Netherlands| Dutch| nl-NL  
New Zealand| English| en-NZ  
Norway| Norwegian| no-NO  
People's republic of China| Chinese| zh-CN  
Poland| Polish| pl-PL  
Republic of the Philippines| English| en-PH  
Russia| Russian| ru-RU  
South Africa| English| en-ZA  
Spain| Spanish| es-ES  
Sweden| Swedish| sv-SE  
Switzerland| French| fr-CH  
Switzerland| German| de-CH  
Taiwan| Traditional Chinese| zh-TW  
Turkey| Turkish| tr-TR  
United Kingdom| English| en-GB  
United States| English| en-US  
United States| Spanish| es-US

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search) [ Summarizer
Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search)

[ Get Started](/app/documentation/video-search/get-started)[ Query
Parameters](/app/documentation/video-search/query)[ Request
Headers](/app/documentation/video-search/request-headers)[ Response
Headers](/app/documentation/video-search/response-headers)[ Response
Objects](/app/documentation/video-search/responses)[
Codes](/app/documentation/video-search/codes)[ API
Changelog](/app/documentation/video-search/api-changelog)

[ News Search](/app/documentation/news-search) [
Suggest](/app/documentation/suggest) [
Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave Search API Changelog - Videos

This changelog lists all updates to the Brave Video Search API in
chronological order.

### 2023-08-15

  * Add video search.

================
File: plugins/repos/twat_search/resources/brave/brave.md
================
[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Web Search API - Changelog

This changelog outlines all significant changes to the Brave Web Search API in
chronological order.

### 2023-01-01

  * Add Brave Web Search API resource.

### 2023-04-14

  * Change `SearchResult` restaurant property to `location`.

### 2023-10-11

  * Add `spellcheck` flag.

# Local Search API - Changelog

This changelog outlines all significant changes to the Brave Local Search API
in chronological order.

### 2024-06-11

  * Add Brave Local Search API resource.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Web Search API

## Codes

#### [#](/app/documentation/web-search/query#country) Country Codes

This table lists the country codes supported by the `country` parameter.

Country| Code  
---|---  
All Regions| ALL  
Argentina| AR  
Australia| AU  
Austria| AT  
Belgium| BE  
Brazil| BR  
Canada| CA  
Chile| CL  
Denmark| DK  
Finland| FI  
France| FR  
Germany| DE  
Hong Kong| HK  
India| IN  
Indonesia| ID  
Italy| IT  
Japan| JP  
Korea| KR  
Malaysia| MY  
Mexico| MX  
Netherlands| NL  
New Zealand| NZ  
Norway| NO  
Peoples Republic of China| CN  
Poland| PL  
Portugal| PT  
Republic of the Philippines| PH  
Russia| RU  
Saudi Arabia| SA  
South Africa| ZA  
Spain| ES  
Sweden| SE  
Switzerland| CH  
Taiwan| TW  
Turkey| TR  
United Kingdom| GB  
United States| US  
  
#### [#](/app/documentation/web-search/query#language) Language Codes

This table lists the language codes supported by the `search_lang` parameter.

Language| Code  
---|---  
Arabic| ar  
Basque| eu  
Bengali| bn  
Bulgarian| bg  
Catalan| ca  
Chinese Simplified| zh-hans  
Chinese Traditional| zh-hant  
Croatian| hr  
Czech| cs  
Danish| da  
Dutch| nl  
English| en  
English United Kingdom| en-gb  
Estonian| et  
Finnish| fi  
French| fr  
Galician| gl  
German| de  
Gujarati| gu  
Hebrew| he  
Hindi| hi  
Hungarian| hu  
Icelandic| is  
Italian| it  
Japanese| jp  
Kannada| kn  
Korean| ko  
Latvian| lv  
Lithuanian| lt  
Malay| ms  
Malayalam| ml  
Marathi| mr  
Norwegian Bokmål| nb  
Polish| pl  
Portuguese Brazil| pt-br  
Portuguese Portugal| pt-pt  
Punjabi| pa  
Romanian| ro  
Russian| ru  
Serbian Cyrylic| sr  
Slovak| sk  
Slovenian| sl  
Spanish| es  
Swedish| sv  
Tamil| ta  
Telugu| te  
Thai| th  
Turkish| tr  
Ukrainian| uk  
Vietnamese| vi  
  
#### [#](/app/documentation/web-search/query#market-code) Market Codes

This table lists the country language codes supported by the `ui_lang`
parameter.

Country| Language| Code  
---|---|---  
Argentina| Spanish| es-AR  
Australia| English| en-AU  
Austria| German| de-AT  
Belgium| Dutch| nl-BE  
Belgium| French| fr-BE  
Brazil| Portuguese| pt-BR  
Canada| English| en-CA  
Canada| French| fr-CA  
Chile| Spanish| es-CL  
Denmark| Danish| da-DK  
Finland| Finnish| fi-FI  
France| French| fr-FR  
Germany| German| de-DE  
Hong Kong SAR| Traditional Chinese| zh-HK  
India| English| en-IN  
Indonesia| English| en-ID  
Italy| Italian| it-IT  
Japan| Japanese| ja-JP  
Korea| Korean| ko-KR  
Malaysia| English| en-MY  
Mexico| Spanish| es-MX  
Netherlands| Dutch| nl-NL  
New Zealand| English| en-NZ  
Norway| Norwegian| no-NO  
People's republic of China| Chinese| zh-CN  
Poland| Polish| pl-PL  
Republic of the Philippines| English| en-PH  
Russia| Russian| ru-RU  
South Africa| English| en-ZA  
Spain| Spanish| es-ES  
Sweden| Swedish| sv-SE  
Switzerland| French| fr-CH  
Switzerland| German| de-CH  
Taiwan| Traditional Chinese| zh-TW  
Turkey| Turkish| tr-TR  
United Kingdom| English| en-GB  
United States| English| en-US  
United States| Spanish| es-US

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

# Brave Web Search API

## Introduction

Brave Web Search API is a REST API to query Brave Search and get back search
results from the web. The following sections describe how to curate requests,
including parameters and headers, to Brave Web Search API and get a JSON
response back.

> To try the API on a Free plan, you’ll still need to subscribe — you simply
> won’t be charged. Once subscribed, you can get an API key in the [API Keys
> section](/app/keys).

## Endpoints

Brave Search API exposes multiple endpoints for specific types of data, based
on the level of your subscription. If you don’t see the endpoint you’re
interested in, you may need to change your subscription.

    
    
    https://api.search.brave.com/res/v1/web/search
    

## Example

A request has to be made to the web search endpoint. An example CURL request
is given below.

    
    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/web/search?q=brave+search" \
      -H "Accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "X-Subscription-Token: <YOUR_API_KEY>"
    

The response specification for Web Search API can be seen in the
[WebSearchApiResponse](/app/documentation/web-
search/responses#WebSearchApiResponse) model.

## Next Steps

To learn what parameters are available and what responses can be expected
while querying Brave Web Search API, please review the following pages:

  * [Query Parameters](/app/documentation/web-search/query#WebSearchAPIQueryParameters)
  * [Request Headers](/app/documentation/web-search/request-headers#WebSearchAPIRequestHeaders)
  * [Response Headers](/app/documentation/web-search/response-headers)
  * [Response Objects](/app/documentation/web-search/responses)

# Brave Local Search API

## Introduction

Brave Local Search API provides enrichments for location search results.

> Access to Local API is available through the [Pro
> plans](/app/subscriptions/subscribe?tab=normal).

## Endpoints

Brave Local Search API is currently available at the following endpoints and
exposes an API to get extra information about a location, including pictures
and related web results.

    
    
    https://api.search.brave.com/res/v1/local/pois
    

The endpoint supports batching and retrieval of extra information of up to 20
locations with a single request.

The local API also includes an endpoint to get an AI generated description for
a location.

    
    
    https://api.search.brave.com/res/v1/local/descriptions
    

## Example

An initial request has to be made to web search endpoint with a given query.
An example CURL request is given below.

    
    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/web/search?q=greek+restaurants+in+san+francisco" \
      -H "Accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "X-Subscription-Token: <YOUR_API_KEY>"
    

If the query returns a list of locations, as in this case, each location
result has an [id field](/app/documentation/web-
search/responses#LocationResult), which is a temporary ID that can be used to
retrieve extra information about the location. An example from the locations
result is given below.

    
    
    {
      "locations": {
        "results": [
          {
            "id": "1520066f3f39496780c5931d9f7b26a6",
            "title": "Pangea Banquet Mediterranean Food"
          },
          {
            "id": "d00b153c719a427ea515f9eacf4853a2",
            "title": "Park Mediterranean Grill"
          },
          {
            "id": "4b943b378725432aa29f019def0f0154",
            "title": "The Halal Mediterranean Co."
          }
        ]
      }
    }
    

The `id` value can be used to further fetch extra information about the
location. An example request is given below.

    
    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/local/pois?ids=1520066f3f39496780c5931d9f7b26a6&ids=d00b153c719a427ea515f9eacf4853a2" \
      -H "accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "x-subscription-token: <YOUR_API_KEY>"
    

An AI generated description associated with a location can be further fetched
using the example below.

    
    
    
    curl -s --compressed "https://api.search.brave.com/res/v1/local/descriptions?ids=1520066f3f39496780c5931d9f7b26a6&ids=d00b153c719a427ea515f9eacf4853a2" \
      -H "accept: application/json" \
      -H "Accept-Encoding: gzip" \
      -H "x-subscription-token: <YOUR_API_KEY>"
    

The response specification for Local Search API can be seen in the
[LocalPoiSearchApiResponse](/app/documentation/web-
search/responses#LocalPoiSearchApiResponse) and
[LocalDescriptionsSearchApiResponse](/app/documentation/web-
search/responses#LocalDescriptionsSearchApiResponse) models.

## Next Steps

To learn what parameters are available and what responses can be expected
while querying Brave Web Search API, please review the following pages:

  * [Query Parameters](/app/documentation/web-search/query#LocalSearchAPIQueryParameters)
  * [Request Headers](/app/documentation/web-search/request-headers#LocalSearchAPIRequestHeaders)
  * [Response Headers](/app/documentation/web-search/response-headers)
  * [Response Objects](/app/documentation/web-search/responses)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Web Search API

## Query Parameters

#### # Web Search API

This table lists the query parameters supported by the Web Search API. Some
are required, but most are optional.

Parameter| Required| Type| Default| Description  
---|---|---|---|---  
q| true| string| |  The user’s search query term. Query can not be empty. Maximum of 400 characters and 50 words in the query.  
country| false| string| US|  The search query country, where the results come
from. The country string is limited to 2 character country codes of supported
countries. For a list of supported values, see [Country
Codes](/app/documentation/web-search/codes#country-codes).  
search_lang| false| string| en|  The search language preference. The 2 or more
character language code for which the search results are provided. For a list
of possible values, see [Language Codes](/app/documentation/web-
search/codes#language-codes).  
ui_lang| false| string| en-US|  User interface language preferred in response.
Usually of the format ‘<language_code>-<country_code>’. For more, see [RFC
9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-language). For a
list of supported values, see [UI Language Codes](/app/documentation/web-
search/codes#market-codes).  
count| false| number| 20|  The number of search results returned in response.
The maximum is `20`. The actual number delivered may be less than requested.
Combine this parameter with `offset` to paginate search results.  
offset| false| number| 0|  The zero based offset that indicates number of
search results per page (count) to skip before returning the result. The
maximum is `9`. The actual number delivered may be less than requested based
on the query. In order to paginate results use this parameter together with
`count`. For example, if your user interface displays 20 search results per
page, set `count` to `20` and offset to `0` to show the first page of results.
To get subsequent pages, increment `offset` by 1 (e.g. 0, 1, 2). The results
may overlap across multiple pages.  
safesearch| false| string| moderate|  Filters search results for adult
content. The following values are supported:

  * `off`: No filtering is done.
  * `moderate`: Filters explicit content, like images and videos, but allows adult domains in the search results.
  * `strict`: Drops all adult content from search results.

  
freshness| false| string| |  Filters search results by when they were discovered. The following values are supported: \- `pd`: Discovered within the last 24 hours. \- `pw`: Discovered within the last 7 Days. \- `pm`: Discovered within the last 31 Days. \- `py`: Discovered within the last 365 Days… \- `YYYY-MM-DDtoYYYY-MM-DD`: timeframe is also supported by specifying the date range e.g. `2022-04-01to2022-07-30`.  
text_decorations| false| bool| 1|  Whether display strings (e.g. result
snippets) should include decoration markers (e.g. highlighting characters).  
spellcheck| false| bool| 1|  Whether to spellcheck provided query. If the
spellchecker is enabled, the modified query is always used for search. The
modified query can be found in `altered` key from the
[query](/app/documentation/web-search/responses#Query) response model.  
result_filter| false| string| |  A comma delimited string of result types to include in the search response. Not specifying this parameter will return back all result types in search response where data is available and a plan with the corresponding option is subscribed. The response always includes query and type to identify any query modifications and response type respectively. Available result filter values are: \- discussions \- faq \- infobox \- news \- query \- summarizer \- videos \- web \- locations Example result filter param `result_filter=discussions`, `videos` returns only `discussions`, and videos responses. Another example where only location results are required, set the `result_filter` param to `result_filter=locations`.  
goggles_id| false| string| |  Goggles act as a custom re-ranking on top of Brave’s search index. For more details, refer to the [Goggles repository](https://github.com/brave/goggles-quickstart). This parameter is deprecated. Please use the goggles parameter.  
goggles| false| string| |  Goggles act as a custom re-ranking on top of Brave’s search index. The parameter supports both a url where the Goggle is hosted or the definition of the goggle. For more details, refer to the [Goggles repository](https://github.com/brave/goggles-quickstart).  
units| false| string| |  The measurement units. If not provided, units are derived from search country. Possible values are: \- `metric`: The standardized measurement system \- `imperial`: The British Imperial system of units.  
extra_snippets| false| bool| |  A snippet is an excerpt from a page you get as a result of the query, and extra_snippets allow you to get up to 5 additional, alternative excerpts. Only available under `Free AI`, `Base AI`, `Pro AI`, `Base Data`, `Pro Data` and `Custom plans`.  
summary| false| bool| |  This parameter enables summary key generation in web search results. This is required for summarizer to be enabled.  
  

You can also optimise your search query by using [search
operators](https://search.brave.com/help/operators).

#### # Local Search API

This table lists the query parameters supported by the Local Search API. Some
are required, but most are optional.

Parameter| Required| Type| Default| Description  
---|---|---|---|---  
ids| true| list[string]| |  Unique identifier for the location. Ids can not be empty. Maximum of 20 ids per request. The parameter can be repeated to query for multiple ids.  
search_lang| false| string| en|  The search language preference. The 2 or more
character language code for which the search results are provided. For a list
of possible values, see [Language Codes](/app/documentation/web-
search/codes#language-codes).  
ui_lang| false| string| en-US|  User interface language preferred in response.
Usually of the format ‘<language_code>-<country_code>’. For more, see [RFC
9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-language). For a
list of supported values, see [UI Language Codes](/app/documentation/web-
search/codes#market-codes).  
units| false| string| |  The measurement units. If not provided, units are derived from search country. Possible values are: \- `metric`: The standardized measurement system \- `imperial`: The British Imperial system of units.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Web Search API

## Request Headers

#### # Web Search API Request Headers

This table lists the request headers supported by the Web Search API. Most are
optional, but note that sending more information in headers (such as client
location) will improve search results.

Header| Required| Name| Description  
---|---|---|---  
Accept| false| Accept|  The default supported media type is `application/json`  
Accept-Encoding| false| Accept Encoding|  The supported compression type is
`gzip`.  
Api-Version| false| Web Search API Version|  The Brave Web Search API version
to use. This is denoted by the format `YYYY-MM-DD`. The latest version is used
by default, and the previous ones can be found in the [API Changelog](./api-
changelog).  
Cache-Control| false| Cache Control|  Search will return cached web search
results by default. To prevent caching set the Cache-Control header to `no-
cache`. This is currently done as best effort.  
User-Agent| false| User Agent|  The user agent of the client sending the
request. Search can utilize the user agent to provide a different experience
depending on the client sending the request. The user agent should follow the
commonly used browser agent strings on each platform. For more information on
curating user agents, see [RFC 9110](https://www.rfc-
editor.org/rfc/rfc9110.html#name-user-agent). User agent string examples by
platform:

  * **Android** : Mozilla/5.0 (Linux; Android 13; Pixel 7 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36
  * **iOS** : Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Mobile/15E148 Safari/604.1
  * **macOS** : Mozilla/5.0 (Macintosh; Intel Mac OS X 12_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
  * **Windows** : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/

  
X-Loc-Lat| false| Latitude|  The latitude of the client’s geographical
location in degrees, to provide relevant local results. The latitiude must be
greater than or equal to -90.0 degrees and less than or equal to +90.0
degrees.  
X-Loc-Long| false| Longitude|  The longitude of the client’s geographical
location in degrees, to provide relevant local results. The longitude must be
greater than or equal to -180.0 degrees and less than or equal to +180.0
degrees.  
X-Loc-Timezone| false| Timezone|  The IANA timezone for the client’s device,
for example `America/New_York`. For complete list of IANA timezones and
location mappings see [IANA Database](https://www.iana.org/time-zones) and
[Geonames Database](https://download.geonames.org/export/dump/).  
X-Loc-City| false| City Name|  The generic name of the client city.  
X-Loc-State| false| State Code|  The code representing the client’s
state/region, can be up to 3 characters long. The region is the first-level
subdivision (the broadest or least specific) of the [ISO
3166-2](https://en.wikipedia.org/wiki/ISO_3166-2) code.  
X-Loc-State-Name| false| State Name|  The name of the client’s state/region.
The region is the first-level subdivision (the broadest or least specific) of
the [ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2) code.  
X-Loc-Country| false| Country Code|  The two letter code for the client’s
country. For a list of country codes, see [ISO 3166-1
alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2)  
X-Loc-Postal-Code| false| Postal Code|  The postal code of the client’s
location.  
X-Subscription-Token| true| Authentication token|  The secret token for the
subscribed plan to authenticate the request. Can be obtained from [API
Keys](/app/keys).  
  
#### # Local Search API Request Headers

This table lists the request headers supported by the Local Search API, most
of which are optional.

Header| Required| Name| Description  
---|---|---|---  
Accept| false| Accept|  The default supported media type is `application/json`  
Accept-Encoding| false| Accept Encoding|  The supported compression type is
`gzip`.  
Api-Version| false| Web Search API Version|  The Brave Web Search API version
to use. This is denoted by the format `YYYY-MM-DD`. The latest version is used
by default, and the previous ones can be found in the [API Changelog](./api-
changelog).  
Cache-Control| false| Cache Control|  Search will return cached web search
results by default. To prevent caching set the Cache-Control header to `no-
cache`. This is currently done as best effort.  
User-Agent| false| User Agent|  The user agent of the client sending the
request. Search can utilize the user agent to provide a different experience
depending on the client sending the request. The user agent should follow the
commonly used browser agent strings on each platform. For more information on
curating user agents, see [RFC 9110](https://www.rfc-
editor.org/rfc/rfc9110.html#name-user-agent). User agent string examples by
platform:

  * **Android** : Mozilla/5.0 (Linux; Android 13; Pixel 7 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36
  * **iOS** : Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Mobile/15E148 Safari/604.1
  * **macOS** : Mozilla/5.0 (Macintosh; Intel Mac OS X 12_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
  * **Windows** : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/

  
X-Subscription-Token| true| Authentication token|  The secret token for the
subscribed plan to authenticate the request. Can be obtained from [API
Keys](/app/keys).

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Web Search API

## Response Headers

#### Global

This table lists the response headers supported by the Web Search API.

Header| Name| Description  
---|---|---  
X-RateLimit-Limit| Rate Limit|  Rate limits associated with the requested
plan. An example rate limit `X-RateLimit-Limit: 1, 15000` means 1 request per
second and 15000 requests per month.  
X-RateLimit-Policy| Rate Limit Policy|  Rate limit policies currently
associated with the requested plan. An example policy `X-RateLimit-Policy:
1;w=1, 15000;w=2592000` means a limit of 1 request over a 1 second window and
15000 requests over a month window. The windows are always given in seconds.  
X-RateLimit-Remaining| Rate Limit Remaining|  Remaining quota units associated
with the expiring limits. An example remaining limit `X-RateLimit-Remaining:
1, 1000` indicates the API is able to be accessed once during the current
second, and 1000 times over the current month. **Note** : Only successful
requests are counted and billed.  
X-RateLimit-Reset| Rate Limit Reset|  The number of seconds until the quota
associated with the expiring limits resets. An example reset limit
`X-RateLimit-Reset: 1, 1419704` means a single request can be done again in a
second and in 1419704 seconds the full monthly quota associated with the plan
will be available again.

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard)

[![Brave](https://cdn.search.brave.com/search-
api/web/v1/client/_app/immutable/assets/brave-
logo.BytqdRrN.svg)](/app/dashboard) [ Documentation](/app/documentation)

[ Web Search](/app/documentation/web-search)

[ Get Started](/app/documentation/web-search/get-started)[ Query
Parameters](/app/documentation/web-search/query)[ Request
Headers](/app/documentation/web-search/request-headers)[ Response
Headers](/app/documentation/web-search/response-headers)[ Response
Objects](/app/documentation/web-search/responses)[
Codes](/app/documentation/web-search/codes)[ API
Changelog](/app/documentation/web-search/api-changelog)

[ Summarizer Search](/app/documentation/summarizer-search) [ Image
Search](/app/documentation/image-search) [ Video
Search](/app/documentation/video-search) [ News
Search](/app/documentation/news-search) [ Suggest](/app/documentation/suggest)
[ Spellcheck](/app/documentation/spellcheck) [
General](/app/documentation/general)

[Login](/login) [Register](/register)

###### Brave Web Search API

## Response Objects

#### # WebSearchApiResponse

Top level response model for successful Web Search API requests. The response
will include the relevant keys based on the plan subscribed, query relevance
or applied result_filter as a query parameter. The API can also respond back
with an error response based on invalid subscription keys and rate limit
events.

Field| Type| Required| Description  
---|---|---|---  
type| "search"| true|  The type of web search API result. The value is always
search.  
discussions| Discussions| false|  Discussions clusters aggregated from forum
posts that are relevant to the query.  
faq| FAQ| false|  Frequently asked questions that are relevant to the search
query.  
infobox| GraphInfobox| false|  Aggregated information on an entity showable as
an infobox.  
locations| Locations| false|  Places of interest (POIs) relevant to location
sensitive queries.  
mixed| MixedResponse| false|  Preferred ranked order of search results.  
news| News| false|  News results relevant to the query.  
query| Query| false|  Search query string and its modifications that are used
for search.  
videos| Videos| false|  Videos relevant to the query.  
web| Search| false|  Web search results relevant to the query.  
summarizer| Summarizer| false|  Summary key to get summary results for the
query.  
  
#### # LocalPoiSearchApiResponse

Top level response model for successful Local Search API request to get extra
information for locations. The response will include a list of location
results corresponding to the ids in the request. The API can also respond back
with an error response in cases like too many ids being requested, invalid
subscription keys, and rate limit events. Access to Local Search API requires
a subscription to a Pro plan.

Field| Type| Required| Description  
---|---|---|---  
type| "local_pois"| true|  The type of local POI search API result. The value
is always local_pois.  
results| list [ LocationResult ]| false|  Location results matching the ids in
the request.  
  
#### # LocalDescriptionsSearchApiResponse

Top level response model for successful Local Search API request to get AI
generated description for locations. The response includes a list of generated
descriptions corresponding to the ids in the request. The API can also respond
back with an error response in cases like too many ids being requested,
invalid subscription keys, and rate limit events. Access to Local Search API
requires a subscription to a Pro plan.

Field| Type| Required| Description  
---|---|---|---  
type| "local_descriptions"| true|  The type of local description search API
result. The value is always local_descriptions.  
results| list [ LocationDescription ]| false|  Location descriptions matching
the ids in the request.  
  
#### # Query

A model representing information gathered around the requested query.

Field| Type| Required| Description  
---|---|---|---  
original| string| true|  The original query that was requested.  
show_strict_warning| bool| false|  Whether there is more content available for
query, but the response was restricted due to safesearch.  
altered| string| false|  The altered query for which the search was performed.  
safesearch| bool| false|  Whether safesearch was enabled.  
is_navigational| bool| false|  Whether the query is a navigational query to a
domain.  
is_geolocal| bool| false|  Whether the query has location relevance.  
local_decision| string| false|  Whether the query was decided to be location
sensitive.  
local_locations_idx| int| false|  The index of the location.  
is_trending| bool| false|  Whether the query is trending.  
is_news_breaking| bool| false|  Whether the query has news breaking articles
relevant to it.  
ask_for_location| bool| false|  Whether the query requires location
information for better results.  
language| Language| false|  The language information gathered from the query.  
spellcheck_off| bool| false|  Whether the spellchecker was off.  
country| string| false|  The country that was used.  
bad_results| bool| false|  Whether there are bad results for the query.  
should_fallback| bool| false|  Whether the query should use a fallback.  
lat| string| false|  The gathered location latitutde associated with the
query.  
long| string| false|  The gathered location longitude associated with the
query.  
postal_code| string| false|  The gathered postal code associated with the
query.  
city| string| false|  The gathered city associated with the query.  
state| string| false|  The gathered state associated with the query.  
header_country| string| false|  The country for the request origination.  
more_results_available| bool| false|  Whether more results are available for
the given query.  
custom_location_label| string| false|  Any custom location labels attached to
the query.  
reddit_cluster| string| false|  Any reddit cluster associated with the query.  
  
#### # Discussions

A model representing a discussion cluster relevant to the query.

Field| Type| Required| Description  
---|---|---|---  
type| "search"| true|  The type identifying a discussion cluster. Currently
the value is always search.  
results| list [ DiscussionResult ]| true|  A list of discussion results.  
mutated_by_goggles| bool| true|  Whether the discussion results are changed by
a Goggle. False by default.  
  
#### # DiscussionResult (SearchResult)

A discussion result. These are forum posts and discussions that are relevant
to the search query.

Field| Type| Required| Description  
---|---|---|---  
type| "discussion"| true|  The discussion result type identifier. The value is
always discussion.  
data| ForumData| false|  The enriched aggregated data for the relevant forum
post.  
  
#### # ForumData

Defines a result from a discussion forum.

Field| Type| Required| Description  
---|---|---|---  
forum_name| string| true|  The name of the forum.  
num_answers| int| false|  The number of answers to the post.  
score| string| false|  The score of the post on the forum.  
title| string| false|  The title of the post on the forum.  
question| string| false|  The question asked in the forum post.  
top_comment| string| false|  The top-rated comment under the forum post.  
  
#### # FAQ

Frequently asked questions relevant to the search query term.

Field| Type| Required| Description  
---|---|---|---  
type| "faq"| true|  The FAQ result type identifier. The value is always faq.  
results| list [ QA ]| true|  A list of aggregated question answer results
relevant to the query.  
  
#### # QA

A question answer result.

Field| Type| Required| Description  
---|---|---|---  
question| string| true|  The question being asked.  
answer| string| true|  The answer to the question.  
title| string| true|  The title of the post.  
url| string| true|  The url pointing to the post.  
meta_url| MetaUrl| false|  Aggregated information about the url.  
  
#### # MetaUrl

Aggregated information about a url.

Field| Type| Required| Description  
---|---|---|---  
scheme| string| true|  The protocol scheme extracted from the url.  
netloc| string| true|  The network location part extracted from the url.  
hostname| string| false|  The lowercased domain name extracted from the url.  
favicon| string| true|  The favicon used for the url.  
path| string| true|  The hierarchical path of the url useful as a display
string.  
  
#### # Search

A model representing a collection of web search results.

Field| Type| Required| Description  
---|---|---|---  
type| "search"| true|  A type identifying web search results. The value is
always search.  
results| list [ SearchResult ]| true|  A list of search results.  
family_friendly| bool| true|  Whether the results are family friendly.  
  
#### # SearchResult (Result)

Aggregated information on a web search result, relevant to the query.

Field| Type| Required| Description  
---|---|---|---  
type| "search_result"| true|  A type identifying a web search result. The
value is always search_result.  
subtype| "generic"| true|  A sub type identifying the web search result type.  
is_live| bool| true|  Whether the web search result is currently live. Default
value is False.  
deep_results| DeepResult| false|  Gathered information on a web search result.  
schemas| list [ list ]| false|  A list of schemas (structured data) extracted
from the page. The schemas try to follow
[schema.org](https://schema.org/docs/schemas.html) and will return anything we
can extract from the HTML that can fit into these models.  
meta_url| MetaUrl| false|  Aggregated information on the url associated with
the web search result.  
thumbnail| Thumbnail| false|  The thumbnail of the web search result.  
age| string| false|  A string representing the age of the web search result.  
language| string| true|  The main language on the web search result.  
location| LocationResult| false|  The location details if the query relates to
a restaurant.  
video| VideoData| false|  The video associated with the web search result.  
movie| MovieData| false|  The movie associated with the web search result.  
faq| FAQ| false|  Any frequently asked questions associated with the web
search result.  
qa| QAPage| false|  Any question answer information associated with the web
search result page.  
book| Book| false|  Any book information associated with the web search result
page.  
rating| Rating| false|  Rating found for the web search result page.  
article| Article| false|  An article found for the web search result page.  
product| ProductReview| false|  The main product and a review that is found on
the web search result page.  
product_cluster| list [ ProductReview ]| false|  A list of products and
reviews that are found on the web search result page.  
cluster_type| string| false|  A type representing a cluster. The value can be
product_cluster.  
cluster| list [ Result ]| false|  A list of web search results.  
creative_work| CreativeWork| false|  Aggregated information on the creative
work found on the web search result.  
music_recording| MusicRecording| false|  Aggregated information on music
recording found on the web search result.  
review| Review| false|  Aggregated information on the review found on the web
search result.  
software| Software| false|  Aggregated information on a software product found
on the web search result page.  
recipe| Recipe| false|  Aggregated information on a recipe found on the web
search result page.  
organization| Organization| false|  Aggregated information on a organization
found on the web search result page.  
content_type| string| false|  The content type associated with the search
result page.  
extra_snippets| list [ string ]| false|  A list of extra alternate snippets
for the web search result.  
  
#### # Result

A model representing a web search result.

Field| Type| Required| Description  
---|---|---|---  
title| string| true|  The title of the web page.  
url| string| true|  The url where the page is served.  
is_source_local| bool| true|  
is_source_both| bool| true|  
description| string| false|  A description for the web page.  
page_age| string| false|  A date representing the age of the web page.  
page_fetched| string| false|  A date representing when the web page was last
fetched.  
profile| Profile| false|  A profile associated with the web page.  
language| string| false|  A language classification for the web page.  
family_friendly| bool| true|  Whether the web page is family friendly.  
  
#### # AbstractGraphInfobox (Result)

Shared aggregated information on an entity from a knowledge graph.

Field| Type| Required| Description  
---|---|---|---  
type| "infobox"| true|  The infobox result type identifier. The value is
always infobox.  
position| int| true|  The position on a search result page.  
label| string| false|  Any label associated with the entity.  
category| string| false|  Category classification for the entity.  
long_desc| string| false|  A longer description for the entity.  
thumbnail| Thumbnail| false|  The thumbnail associated with the entity.  
attributes| list [ list [ string ] ]| false|  A list of attributes about the
entity.  
profiles| list [ Profile ] | list [ DataProvider ]| false|  The profiles associated with the entity.  
website_url| string| false|  The official website pertaining to the entity.  
ratings| list [ Rating ]| false|  Any ratings given to the entity.  
providers| list [ DataProvider ]| false|  A list of data sources for the
entity.  
distance| Unit| false|  A unit representing quantity relevant to the entity.  
images| list [ Thumbnail ]| false|  A list of images relevant to the entity.  
movie| MovieData| false|  Any movie data relevant to the entity. Appears only
when the result is a movie.  
  
#### # GenericInfobox (AbstractGraphInfobox)

Aggregated information on a generic entity from a knowledge graph.

Field| Type| Required| Description  
---|---|---|---  
subtype| "generic"| true|  The infobox subtype identifier. The value is always
generic.  
found_in_urls| list [ string ]| false|  List of urls where the entity was
found.  
  
#### # EntityInfobox (AbstractGraphInfobox)

Aggregated information on an entity from a knowledge graph.

Field| Type| Required| Description  
---|---|---|---  
subtype| "entity"| true|  The infobox subtype identifier. The value is always
entity.  
  
#### # QAInfobox (AbstractGraphInfobox)

A question answer infobox.

Field| Type| Required| Description  
---|---|---|---  
subtype| "code"| true|  The infobox subtype identifier. The value is always
code.  
data| QAPage| true|  The question and relevant answer.  
meta_url| MetaUrl| false|  Detailed information on the page containing the
question and relevant answer.  
  
#### # InfoboxWithLocation (AbstractGraphInfobox)

An infobox with location.

Field| Type| Required| Description  
---|---|---|---  
subtype| "location"| true|  The infobox subtype identifier. The value is
always location.  
is_location| bool| true|  Whether the entity a location.  
coordinates| list [ float ]| false|  The coordinates of the location.  
zoom_level| int| true|  The map zoom level.  
location| LocationResult| false|  The location result.  
  
#### # InfoboxPlace (AbstractGraphInfobox)

An infobox for a place, such as a business.

Field| Type| Required| Description  
---|---|---|---  
subtype| "place"| true|  The infobox subtype identifier. The value is always
place.  
location| LocationResult| true|  The location result.  
  
#### # GraphInfobox

Aggregated information on an entity shown as an infobox.

Field| Type| Required| Description  
---|---|---|---  
type| "graph"| true|  The type identifier for infoboxes. The value is always
graph.  
results| GenericInfoboxQAInfoboxInfoboxPlaceInfoboxWithLocationEntityInfobox|
true|  A list of infoboxes associated with the query.  
  
#### # QAPage

Aggreated result from a question answer page.

Field| Type| Required| Description  
---|---|---|---  
question| string| true|  The question that is being asked.  
answer| Answer| true|  An answer to the question.  
  
#### # Answer

A response representing an answer to a question on a forum.

Field| Type| Required| Description  
---|---|---|---  
text| string| true|  The main content of the answer.  
author| string| false|  The name of the author of the answer.  
upvoteCount| int| false|  Number of upvotes on the answer.  
downvoteCount| int| false|  The number of downvotes on the answer.  
  
#### # Thumbnail

Aggregated details representing a picture thumbnail.

Field| Type| Required| Description  
---|---|---|---  
src| string| true|  The served url of the picture thumbnail.  
original| string| false|  The original url of the image.  
  
#### # LocationWebResult (Result)

A model representing a web result related to a location.

Field| Type| Required| Description  
---|---|---|---  
meta_url| MetaUrl| true|  Aggregated information about the url.  
  
#### # LocationResult (Result)

A result that is location relevant.

Field| Type| Required| Description  
---|---|---|---  
type| "location_result"| true|  Location result type identifier. The value is
always location_result.  
id| string| false|  A Temporary id associated with this result, which can be
used to retrieve extra information about the location. It remains valid for 8
hours…  
provider_url| string| true|  The complete url of the provider.  
coordinates| list [ float ]| false|  A list of coordinates associated with the
location. This is a lat long represented as a floating point.  
zoom_level| int| true|  The zoom level on the map.  
thumbnail| Thumbnail| false|  The thumbnail associated with the location.  
postal_address| PostalAddress| false|  The postal address associated with the
location.  
opening_hours| OpeningHours| false|  The opening hours, if it is a business,
associated with the location .  
contact| Contact| false|  The contact of the business associated with the
location.  
price_range| string| false|  A display string used to show the price
classification for the business.  
rating| Rating| false|  The ratings of the business.  
distance| Unit| false|  The distance of the location from the client.  
profiles| list [ DataProvider ]| false|  Profiles associated with the
business.  
reviews| Reviews| false|  Aggregated reviews from various sources relevant to
the business.  
pictures| PictureResults| false|  A bunch of pictures associated with the
business.  
action| Action| false|  An action to be taken.  
serves_cuisine| list [ string ]| false|  A list of cuisine categories served.  
categories| list [ string ]| false|  A list of categories.  
icon_category| string| false|  An icon category.  
results| LocationWebResult| false|  Web results related to this location.  
timezone| string| false|  IANA timezone identifier.  
timezone_offset| string| false|  The utc offset of the timezone.  
  
#### # LocationDescription

AI generated description of a location result.

Field| Type| Required| Description  
---|---|---|---  
type| "local_description"| true|  The type of a location description. The
value is always local_description.  
id| string| true|  A Temporary id of the location with this description.  
description| string| false|  AI generated description of the location with the
given id.  
  
#### # Locations

A model representing location results.

Field| Type| Required| Description  
---|---|---|---  
type| "locations"| true|  Location type identifier. The value is always
locations.  
results| list [ LocationResult ]| true|  An aggregated list of location
sensitive results.  
  
#### # MixedResponse

The ranking order of results on a search result page.

Field| Type| Required| Description  
---|---|---|---  
type| "mixed"| true|  The type representing the model mixed. The value is
always mixed.  
main| list [ ResultReference ]| false|  The ranking order for the main section
of the search result page.  
top| list [ ResultReference ]| false|  The ranking order for the top section
of the search result page.  
side| list [ ResultReference ]| false|  The ranking order for the side section
of the search result page.  
  
#### # ResultReference

The ranking order of results on a search result page.

Field| Type| Required| Description  
---|---|---|---  
type| string| true|  The type of the result.  
index| int| false|  The 0th based index where the result should be placed.  
all| bool| true|  Whether to put all the results from the type at specific
position.  
  
#### # Videos

A model representing video results.

Field| Type| Required| Description  
---|---|---|---  
type| videos| true|  The type representing the videos. The value is always
videos.  
results| list [ VideoResult ]| true|  A list of video results.  
mutated_by_goggles| bool| false|  Whether the video results are changed by a
Goggle. False by default.  
  
#### # News

A model representing news results.

Field| Type| Required| Description  
---|---|---|---  
type| news| true|  The type representing the news. The value is always news.  
results| list [ NewsResult ]| true|  A list of news results.  
mutated_by_goggles| bool| false|  Whether the news results are changed by a
Goggle. False by default.  
  
#### # NewsResult (Result)

A model representing news results.

Field| Type| Required| Description  
---|---|---|---  
meta_url| MetaUrl| false|  The aggregated information on the url representing
a news result  
source| string| false|  The source of the news.  
breaking| bool| true|  Whether the news result is currently a breaking news.  
is_live| bool| true|  Whether the news result is currently live.  
thumbnail| Thumbnail| false|  The thumbnail associated with the news result.  
age| string| false|  A string representing the age of the news article.  
extra_snippets| list [ string ]| false|  A list of extra alternate snippets
for the news search result.  
  
#### # PictureResults

A model representing a list of pictures.

Field| Type| Required| Description  
---|---|---|---  
viewMoreUrl| string| false|  A url to view more pictures.  
results| list [ Thumbnail ]| true|  A list of thumbnail results.  
  
#### # Action

A model representing an action to be taken.

Field| Type| Required| Description  
---|---|---|---  
type| string| true|  The type representing the action.  
url| string| true|  A url representing the action to be taken.  
  
#### # PostalAddress

A model representing a postal address of a location

Field| Type| Required| Description  
---|---|---|---  
type| "PostalAddress"| true|  The type identifying a postal address. The value
is always PostalAddress.  
country| string| false|  The country associated with the location.  
postalCode| string| false|  The postal code associated with the location.  
streetAddress| string| false|  The street address associated with the
location.  
addressRegion| string| false|  The region associated with the location. This
is usually a state.  
addressLocality| string| false|  The address locality or subregion associated
with the location.  
displayAddress| string| true|  The displayed address string.  
  
#### # OpeningHours

Opening hours of a bussiness at a particular location.

Field| Type| Required| Description  
---|---|---|---  
current_day| list [ DayOpeningHours ]| false|  The current day opening hours.
Can have two sets of opening hours.  
days| list [ list [ DayOpeningHours ] ]| false|  The opening hours for the
whole week.  
  
#### # DayOpeningHours

A model representing the opening hours for a particular day for a business at
a particular location.

Field| Type| Required| Description  
---|---|---|---  
abbr_name| string| true|  A short string representing the day of the week.  
full_name| string| true|  A full string representing the day of the week.  
opens| string| true|  A 24 hr clock time string for the opening time of the
business on a particular day.  
closes| string| true|  A 24 hr clock time string for the closing time of the
business on a particular day.  
  
#### # Contact

A model representing contact information for an entity.

Field| Type| Required| Description  
---|---|---|---  
email| string| false|  The email address.  
telephone| string| false|  The telephone number.  
  
#### # DataProvider

A model representing the data provider associated with the entity.

Field| Type| Required| Description  
---|---|---|---  
type| "external"| true|  The type representing the source of data. This is
usually external.  
name| string| true|  The name of the data provider. This can be a domain.  
url| string| true|  The url where the information is coming from.  
long_name| string| false|  The long name for the data provider.  
img| string| false|  The served url for the image data.  
  
#### # Profile

A profile of an entity.

Field| Type| Required| Description  
---|---|---|---  
name| string| true|  The name of the profile.  
long_name| string| true|  The long name of the profile.  
url| string| false|  The original url where the profile is available.  
img| string| false|  The served image url representing the profile.  
  
#### # Unit

A model representing a unit of measurement.

Field| Type| Required| Description  
---|---|---|---  
value| float| true|  The quantity of the unit.  
units| string| true|  The name of the unit associated with the quantity.  
  
#### # MovieData

Aggregated data for a movie result.

Field| Type| Required| Description  
---|---|---|---  
name| string| false|  Name of the movie.  
description| string| false|  A short plot summary for the movie.  
url| string| false|  A url serving a movie profile page.  
thumbnail| Thumbnail| false|  A thumbnail for a movie poster.  
release| string| false|  The release date for the movie.  
directors| list [ Person ]| false|  A list of people responsible for directing
the movie.  
actors| list [ Person ]| false|  A list of actors in the movie.  
rating| Rating| false|  Rating provided to the movie from various sources.  
duration| string| false|  The runtime of the movie. The format is HH:MM:SS.  
genre| list [ string ]| false|  List of genres in which the movie can be
classified.  
query| string| false|  The query that resulted in the movie result.  
  
#### # Thing

A model describing a generic thing.

Field| Type| Required| Description  
---|---|---|---  
type| "thing"| true|  A type identifying a thing. The value is always thing.  
name| string| true|  The name of the thing.  
url| string| false|  A url for the thing.  
thumbnail| Thumbnail| false|  Thumbnail associated with the thing.  
  
#### # Person (Thing)

A model describing a person entity.

Field| Type| Required| Description  
---|---|---|---  
type| "person"| true|  A type identifying a person. The value is always
person.  
  
#### # Rating

The rating associated with an entity.

Field| Type| Required| Description  
---|---|---|---  
ratingValue| float| true|  The current value of the rating.  
bestRating| float| true|  Best rating received.  
reviewCount| int| false|  The number of reviews associated with the rating.  
profile| Profile| false|  The profile associated with the rating.  
is_tripadvisor| bool| true|  Whether the rating is coming from Tripadvisor.  
  
#### # Book

A model representing a book result.

Field| Type| Required| Description  
---|---|---|---  
title| string| true|  The title of the book.  
author| list [ Person ]| true|  The author of the book.  
date| string| false|  The publishing date of the book.  
price| Price| false|  The price of the book.  
pages| int| false|  The number of pages in the book.  
publisher| Person| false|  The publisher of the book.  
rating| Rating| false|  A gathered rating from different sources associated
with the book.  
  
#### # Price

A model representing the price for an entity.

Field| Type| Required| Description  
---|---|---|---  
price| string| true|  The price value in a given currency.  
price_currency| string| true|  The current of the price value.  
  
#### # Article

A model representing an article.

Field| Type| Required| Description  
---|---|---|---  
author| list [ Person ]| false|  The author of the article.  
date| string| false|  The date when the article was published.  
publisher| Organization| false|  The name of the publisher for the article.  
thumbnail| Thumbnail| false|  A thumbnail associated with the article.  
isAccessibleForFree| bool| false|  Whether the article is free to read or is
behind a paywall.  
  
#### # ContactPoint (Thing)

A way to contact an entity.

Field| Type| Required| Description  
---|---|---|---  
type| "contact_point"| true|  A type string identifying a contact point. The
value is always contact_point.  
telephone| string| false|  The telephone number of the entity.  
email| string| false|  The email address of the entity.  
  
#### # Organization (Thing)

An entity responsible for another entity.

Field| Type| Required| Description  
---|---|---|---  
type| "organization"| true|  A type string identifying an organization. The
value is always organization.  
contact_points| list [ ContactPoint ]| false|  A list of contact points for
the organization.  
  
#### # HowTo

Aggregated information on a how to.

Field| Type| Required| Description  
---|---|---|---  
text| string| true|  The how to text.  
name| string| false|  A name for the how to.  
url| string| false|  A url associated with the how to.  
image| list [ string ]| false|  A list of image urls associated with the how
to.  
  
#### # Recipe

Aggregated information on a recipe.

Field| Type| Required| Description  
---|---|---|---  
title| string| true|  The title of the recipe.  
description| string| true|  The description of the recipe.  
thumbnail| Thumbnail| true|  A thumbnail associated with the recipe.  
url| string| true|  The url of the web page where the recipe was found.  
domain| string| true|  The domain of the web page where the recipe was found.  
favicon| string| true|  The url for the favicon of the web page where the
recipe was found.  
time| string| false|  The total time required to cook the recipe.  
prep_time| string| false|  The preparation time for the recipe.  
cook_time| string| false|  The cooking time for the recipe.  
ingredients| string| false|  Ingredients required for the recipe.  
instructions| list [ HowTo ]| false|  List of instructions for the recipe.  
servings| int| false|  How many people the recipe serves.  
calories| int| false|  Calorie count for the recipe.  
rating| Rating| false|  Aggregated information on the ratings associated with
the recipe.  
recipeCategory| string| false|  The category of the recipe.  
recipeCuisine| string| false|  The cuisine classification for the recipe.  
video| VideoData| false|  Aggregated information on the cooking video
associated with the recipe.  
  
#### # Product

A model representing a product.

Field| Type| Required| Description  
---|---|---|---  
type| "Product"| true|  A string representing a product type. The value is
always product.  
name| string| true|  The name of the product.  
category| string| false|  The category of the product.  
price| string| true|  The price of the product.  
thumbnail| Thumbnail| true|  A thumbnail associated with the product.  
description| string| false|  The description of the product.  
offers| list [ Offer ]| false|  A list of offers available on the product.  
rating| Rating| false|  A rating associated with the product.  
  
#### # Offer

An offer associated with a product.

Field| Type| Required| Description  
---|---|---|---  
url| string| true|  The url where the offer can be found.  
priceCurrency| string| true|  The currency in which the offer is made.  
price| string| true|  The price of the product currently on offer.  
  
#### # Review

A model representing a review for an entity.

Field| Type| Required| Description  
---|---|---|---  
type| "review"| true|  A string representing review type. This is always
review.  
name| string| true|  The review title for the review.  
thumbnail| Thumbnail| true|  The thumbnail associated with the reviewer.  
description| string| true|  A description of the review (the text of the
review itself).  
rating| Rating| true|  The ratings associated with the review.  
  
#### # Reviews

The reviews associated with an entity.

Field| Type| Required| Description  
---|---|---|---  
results| list [ TripAdvisorReview ]| true|  A list of trip advisor reviews for
the entity.  
viewMoreUrl| string| true|  A url to a web page where more information on the
result can be seen.  
reviews_in_foreign_language| bool| true|  Any reviews available in a foreign
language.  
  
#### # TripAdvisorReview

A model representing a Tripadvisor review.

Field| Type| Required| Description  
---|---|---|---  
title| string| true|  The title of the review.  
description| string| true|  A description seen in the review.  
date| string| true|  The date when the review was published.  
rating| Rating| true|  A rating given by the reviewer.  
author| Person| true|  The author of the review.  
review_url| string| true|  A url link to the page where the review can be
found.  
language| string| true|  The language of the review.  
  
#### # CreativeWork

A creative work relevant to the query. An example can be enriched metadata for
an app.

Field| Type| Required| Description  
---|---|---|---  
name| string| true|  The name of the creative work.  
thumbnail| Thumbnail| true|  A thumbnail associated with the creative work.  
rating| Rating| false|  A rating that is given to the creative work.  
  
#### # MusicRecording

Result classified as a music label or a song.

Field| Type| Required| Description  
---|---|---|---  
name| string| true|  The name of the song or album.  
thumbnail| Thumbnail| false|  A thumbnail associated with the music.  
rating| Rating| false|  The rating of the music.  
  
#### # Software

A model representing a software entity.

Field| Type| Required| Description  
---|---|---|---  
name| string| false|  The name of the software product.  
author| string| false|  The author of software product.  
version| string| false|  The latest version of the software product.  
codeRepository| string| false|  The code repository where the software product
is currently available or maintained.  
homepage| string| false|  The home page of the software product.  
datePublisher| string| false|  The date when the software product was
published.  
is_npm| bool| false|  Whether the software product is available on npm.  
is_pypi| bool| false|  Whether the software product is available on pypi.  
stars| int| false|  The number of stars on the repository.  
forks| int| false|  The numbers of forks of the repository.  
ProgrammingLanguage| string| false|  The programming language spread on the
software product.  
  
#### # DeepResult

Aggregated deep results from news, social, videos and images.

Field| Type| Required| Description  
---|---|---|---  
news| list [ NewsResult ]| false|  A list of news results associated with the
result.  
buttons| list [ ButtonResult ]| false|  A list of buttoned results associated
with the result.  
videos| list [ VideoResult ]| false|  Videos associated with the result.  
images| list [ Image ]| false|  Images associated with the result.  
  
#### # VideoResult (Result)

A model representing a video result.

Field| Type| Required| Description  
---|---|---|---  
type| "video_result"| true|  The type identifying the video result. The value
is always video_result.  
video| VideoData| true|  Meta data for the video.  
meta_url| MetaUrl| false|  Aggregated information on the URL  
thumbnail| Thumbnail| false|  The thumbnail of the video.  
age| string| false|  A string representing the age of the video.  
  
#### # VideoData

A model representing metadata gathered for a video.

Field| Type| Required| Description  
---|---|---|---  
duration| string| false|  A time string representing the duration of the
video. The format can be HH:MM:SS or MM:SS.  
views| string| false|  The number of views of the video.  
creator| string| false|  The creator of the video.  
publisher| string| false|  The publisher of the video.  
thumbnail| Thumbnail| false|  A thumbnail associated with the video.  
tags| list [ string ]| false|  A list of tags associated with the video.  
author| Profile| false|  Author of the video.  
requires_subscription| bool| false|  Whether the video requires a subscription
to watch.  
  
#### # ButtonResult

A result which can be used as a button.

Field| Type| Required| Description  
---|---|---|---  
type| "button_result"| true|  A type identifying button result. The value is
always button_result.  
title| string| true|  The title of the result.  
url| string| true|  The url for the button result.  
  
#### # Image

A model describing an image

Field| Type| Required| Description  
---|---|---|---  
thumbnail| Thumbnail| true|  The thumbnail associated with the image.  
url| string| false|  The url of the image.  
properties| ImageProperties| false|  Metadata on the image.  
  
#### # Language

A model representing a language.

Field| Type| Required| Description  
---|---|---|---  
main| string| true|  The main language seen in the string.  
  
#### # ImageProperties

Metadata on an image.

Field| Type| Required| Description  
---|---|---|---  
url| string| true|  The original image URL.  
resized| string| true|  The url for a better quality resized image.  
placeholder| string| true|  The placeholder image url.  
height| int| false|  The image height.  
width| int| false|  The image width.  
format| string| false|  The image format.  
content_size| string| false|  The image size.  
  
#### # Summarizer

Details on getting the summary.

Field| Type| Required| Description  
---|---|---|---  
type| "summarizer"| true|  The value is always summarizer.  
key| string| true|  The key for the summarizer API.

================
File: plugins/repos/twat_search/resources/pplx/pplx_urls.txt
================
https://docs.perplexity.ai/guides/getting-started
https://docs.perplexity.ai/guides/model-cards
https://docs.perplexity.ai/guides/pricing
https://docs.perplexity.ai/guides/structured-outputs
https://docs.perplexity.ai/guides/prompt-guide
https://docs.perplexity.ai/guides/bots
https://docs.perplexity.ai/api-reference/chat-completions

================
File: plugins/repos/twat_search/resources/pplx/pplx.md
================
[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Guides

Initial Setup

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Guides

  * [Initial Setup](/guides/getting-started)
  * [Supported Models](/guides/model-cards)
  * [Pricing](/guides/pricing)
  * [Rate Limits and Usage Tiers](/guides/usage-tiers)
  * [Structured Outputs Guide](/guides/structured-outputs)
  * [Prompt Guide](/guides/prompt-guide)
  * [Perplexity Crawlers](/guides/bots)

Guides

# Initial Setup

Register and make a successful API request

##

​

Registration

  * Start by visiting the [API Settings page](https://www.perplexity.ai/pplx-api)

  * Register your credit card to get started

This step will not charge your credit card. It just stores payment information
for later API usage.

##

​

Generate an API key

  * Every API call needs a valid API key

The API key is a long-lived access token that can be used until it is manually
refreshed or deleted.

Send the API key as a bearer token in the Authorization header with each API
request.

When you run out of credits, your API keys will be blocked until you add to
your credit balance. You can avoid this by configuring “Automatic Top Up”,
which refreshes your balance whenever you drop below $2.

##

​

Make your API call

  * The API is conveniently OpenAI client-compatible for easy integration with existing applications.

cURL

    
    
    curl --location 'https://api.perplexity.ai/chat/completions' \
    --header 'accept: application/json' \
    --header 'content-type: application/json' \
    --header 'Authorization: Bearer {API_KEY}' \
    --data '{
      "model": "sonar-pro ",
      "messages": [
        {
          "role": "system",
          "content": "Be precise and concise."
        },
        {
          "role": "user",
          "content": "How many stars are there in our galaxy?"
        }
      ]
    }'
    

python

    
    
    from openai import OpenAI
    
    YOUR_API_KEY = "INSERT API KEY HERE"
    
    messages = [
        {
            "role": "system",
            "content": (
                "You are an artificial intelligence assistant and you need to "
                "engage in a helpful, detailed, polite conversation with a user."
            ),
        },
        {   
            "role": "user",
            "content": (
                "How many stars are in the universe?"
            ),
        },
    ]
    
    client = OpenAI(api_key=YOUR_API_KEY, base_url="https://api.perplexity.ai")
    
    # chat completion without streaming
    response = client.chat.completions.create(
        model="sonar-pro",
        messages=messages,
    )
    print(response)
    
    # chat completion with streaming
    response_stream = client.chat.completions.create(
        model="sonar-pro",
        messages=messages,
        stream=True,
    )
    for response in response_stream:
        print(response)
    

[Supported Models](/guides/model-cards)

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

On this page

  * Registration
  * Generate an API key
  * Make your API call

[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Guides

Supported Models

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Guides

  * [Initial Setup](/guides/getting-started)
  * [Supported Models](/guides/model-cards)
  * [Pricing](/guides/pricing)
  * [Rate Limits and Usage Tiers](/guides/usage-tiers)
  * [Structured Outputs Guide](/guides/structured-outputs)
  * [Prompt Guide](/guides/prompt-guide)
  * [Perplexity Crawlers](/guides/bots)

Guides

# Supported Models

Model| Context Length| Model Type  
---|---|---  
`sonar-deep-research`| 128k| Chat Completion  
`sonar-reasoning-pro`| 128k| Chat Completion  
`sonar-reasoning`| 128k| Chat Completion  
`sonar-pro`| 200k| Chat Completion  
`sonar`| 128k| Chat Completion  
`r1-1776`| 128k| Chat Completion  
  
  1. `sonar-reasoning-pro` and `sonar-pro` have a max output token limit of 8k
  2. The reasoning models output CoTs in their responses as well
  3. `r1-1776` is an offline chat model that does not use our search subsystem

[Initial Setup](/guides/getting-started)[Pricing](/guides/pricing)

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Guides

Pricing

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Guides

  * [Initial Setup](/guides/getting-started)
  * [Supported Models](/guides/model-cards)
  * [Pricing](/guides/pricing)
  * [Rate Limits and Usage Tiers](/guides/usage-tiers)
  * [Structured Outputs Guide](/guides/structured-outputs)
  * [Prompt Guide](/guides/prompt-guide)
  * [Perplexity Crawlers](/guides/bots)

Guides

# Pricing

Model| Input Tokens (Per Million Tokens)| Reasoning Tokens (Per Million
Tokens)| Output Tokens (Per Million Tokens)| Price per 1000 searches  
---|---|---|---|---  
`sonar-deep-research`| $2| $3| $8| $5  
`sonar-reasoning-pro`| $2| -| $8| $5  
`sonar-reasoning`| $1| -| $5| $5  
`sonar-pro`| $3| -| $15| $5  
`sonar`| $1| -| $1| $5  
`r1-1776`| $2| -| $8| -  
  
`r1-1776` is an offline chat model that does not use our search subsystem

##

​

Pricing Breakdown

Detailed Pricing Breakdown for Sonar Deep Research

**Input Tokens**

  1. Input tokens are priced at $2/1M tokens

  2. Input tokens comprise of Prompt tokens (user prompt) + Citation tokens (these are processed tokens from running searches)

**Search Queries**

  1. Deep Research runs multiple searches to conduct exhaustive research

  2. Searches are priced at $5/1000 searches

  3. A request that does 30 searches will cost $0.15 in this step.

**Reasoning Tokens**

  1. Reasoning is a distinct step in Deep Research since it does extensive automated reasoning through all the material it gathers during its research phase

  2. Reasoning tokens here are a bit different than the CoTs in the answer - these are tokens that we use to reason through the research material prior to generating the outputs via the CoTs.

  3. Reasoning tokens are priced at $3/1M tokens

**Output Tokens**

  1. Output tokens (Completion tokens) are priced at $8/1M tokens

**Total Price**

Your total price per request finally is a sum of the above 4 components

Detailed Pricing Breakdown for Sonar Reasoning Pro and Sonar Pro

**Input Tokens**

  1. Input tokens are priced at $2/1M tokens and $3/1M tokens respectively

  2. Input tokens comprise of Prompt tokens (user prompt) + Citation tokens (these are processed tokens from running searches)

**Search Queries**

  1. To give detailed answers, both the Pro APIs also run multiple searches on top of the user prompt where necessary for more exhaustive information retrieval

  2. Searches are priced at $5/1000 searches

  3. A request that does 3 searches will cost $0.015 in this step

**Output Tokens**

  1. Output tokens (Completion tokens) are priced at $8/1M tokens and $15/1M tokens respectively

**Total Price**

Your total price per request finally is a sum of the above 3 components

Detailed Pricing Breakdown for Sonar Reasoning and Sonar

**Input Tokens**

  1. Input tokens are priced at $1/1M tokens for both

  2. Input tokens comprise of Prompt tokens (user prompt)

**Search Queries**

  1. Each request does 1 search priced at $5/1000 searches

**Output Tokens**

  1. Output tokens (Completion tokens) are priced at $5/1M tokens and $1/1M tokens respectively

**Total Price**

Your total price per request finally is a sum of the above 2 components

[Supported Models](/guides/model-cards)[Rate Limits and Usage
Tiers](/guides/usage-tiers)

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Guides

Structured Outputs Guide

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Guides

  * [Initial Setup](/guides/getting-started)
  * [Supported Models](/guides/model-cards)
  * [Pricing](/guides/pricing)
  * [Rate Limits and Usage Tiers](/guides/usage-tiers)
  * [Structured Outputs Guide](/guides/structured-outputs)
  * [Prompt Guide](/guides/prompt-guide)
  * [Perplexity Crawlers](/guides/bots)

Guides

# Structured Outputs Guide

Structured outputs is currently a beta feature and only available to users in
Tier-3

##

​

Overview

We currently support two types of structured outputs: **JSON Schema** and
**Regex**. LLM responses will work to match the specified format, except for
the following cases:

  * The output exceeds `max_tokens`

Enabling the structured outputs can be done by adding a `response_format`
field in the request:

**JSON Schema**

  * `response_format: { type: "json_schema", json_schema: {"schema": object} }` .

  * The schema should be a valid JSON schema object.

**Regex** (only avilable for `sonar` right now)

  * `response_format: { type: "regex", regex: {"regex": str} }` .

  * The regex is a regular expression string.

We recommend to give the LLM some hints about the output format in the
prompts.

The first request with a new JSON Schema or Regex expects to incur delay on
the first token. Typically, it takes 10 to 30 seconds to prepare the new
schema. Once the schema has been prepared, the subsequent requests will not
see such delay.

##

​

Examples

###

​

1\. Get a response in JSON format

**Request**

python

    
    
    import requests
    from pydantic import BaseModel
    
    class AnswerFormat(BaseModel):
        first_name: str
        last_name: str
        year_of_birth: int
        num_seasons_in_nba: int
    
    url = "https://api.perplexity.ai/chat/completions"
    headers = {"Authorization": "Bearer YOUR_API_KEY"}
    payload = {
        "model": "sonar",
        "messages": [
            {"role": "system", "content": "Be precise and concise."},
            {"role": "user", "content": (
                "Tell me about Michael Jordan. "
                "Please output a JSON object containing the following fields: "
                "first_name, last_name, year_of_birth, num_seasons_in_nba. "
            )},
        ],
        "response_format": {
    		    "type": "json_schema",
            "json_schema": {"schema": AnswerFormat.model_json_schema()},
        },
    }
    response = requests.post(url, headers=headers, json=payload).json()
    print(response["choices"][0]["message"]["content"])
    

**Response**

    
    
    {"first_name":"Michael","last_name":"Jordan","year_of_birth":1963,"num_seasons_in_nba":15}
    

###

​

2\. Use a regex to output the format

**Request**

python

    
    
    import requests
    
    url = "https://api.perplexity.ai/chat/completions"
    headers = {"Authorization": "Bearer YOUR_API_KEY"}
    payload = {
        "model": "sonar",
        "messages": [
            {"role": "system", "content": "Be precise and concise."},
            {"role": "user", "content": "What is the IPv4 address of OpenDNS DNS server?"},
        ],
        "response_format": {
    		    "type": "regex",
            "regex": {"regex": r"(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"},
        },
    }
    response = requests.post(url, headers=headers, json=payload).json()
    print(response["choices"][0]["message"]["content"])
    

**Response**

    
    
    208.67.222.222
    

##

​

Best Practices

###

​

Generating responses in a JSON Format

For Python users, we recommend using the Pydantic library to [generate JSON
schema](https://docs.pydantic.dev/latest/api/base_model/#pydantic.BaseModel.model_json_schema).

**Unsupported JSON Schemas**

Recursive JSON schema is not supported. As a result of that, unconstrained
objects are not supported either. Here’s a few example of unsupported schemas:

    
    
    # UNSUPPORTED!
    
    from typing import Any
    
    class UnconstrainedDict(BaseModel):
       unconstrained: dict[str, Any]
    
    class RecursiveJson(BaseModel):
       value: str
       child: list["RecursiveJson"]
    

###

​

Generating responses using a regex

**Supported Regex**

  * Characters: `\d`, `\w`, `\s` , `.`
  * Character classes: `[0-9A-Fa-f]` , `[^x]`
  * Quantifiers: `*`, `?` , `+`, `{3}`, `{2,4}` , `{3,}`
  * Alternation: `|`
  * Group: `( ... )`
  * Non-capturing group: `(?: ... )`
  * Positive lookahead: `(?= ... )`
  * Negative lookahead: `(?! ... )`

**Unsupported Regex**

  * Contents of group: `\1`
  * Anchors: `^`, `$`, `\b`
  * Positive look-behind: `(?<= ... )`
  * Negative look-behind: `(?<! ... )`
  * Recursion: `(?R)`

[Rate Limits and Usage Tiers](/guides/usage-tiers)[Prompt
Guide](/guides/prompt-guide)

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

On this page

  * Overview
  * Examples
  * 1\. Get a response in JSON format
  * 2\. Use a regex to output the format
  * Best Practices
  * Generating responses in a JSON Format
  * Generating responses using a regex

[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Guides

Prompt Guide

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Guides

  * [Initial Setup](/guides/getting-started)
  * [Supported Models](/guides/model-cards)
  * [Pricing](/guides/pricing)
  * [Rate Limits and Usage Tiers](/guides/usage-tiers)
  * [Structured Outputs Guide](/guides/structured-outputs)
  * [Prompt Guide](/guides/prompt-guide)
  * [Perplexity Crawlers](/guides/bots)

Guides

# Prompt Guide

##

​

System Prompt

You can use the system prompt to provide instructions related to style, tone,
and language of the response.

The real-time search component of our models does not attend to the system
prompt.

**Example of a system prompt**

    
    
    You are a helpful AI assistant.
    
    Rules:
    1. Provide only the final answer. It is important that you do not include any explanation on the steps below.
    2. Do not show the intermediate steps information.
    
    Steps:
    1. Decide if the answer should be a brief sentence or a list of suggestions.
    2. If it is a list of suggestions, first, write a brief and natural introduction based on the original query.
    3. Followed by a list of suggestions, each suggestion should be split by two newlines.
    

##

​

User Prompt

You should use the user prompt to pass in the actual query for which you need
an answer for. The user prompt will be used to kick off a real-time web search
to make sure the answer has the latest and the most relevant information
needed.

**Example of a user prompt**

    
    
    What are the best sushi restaurants in the world currently?
    

[Structured Outputs Guide](/guides/structured-outputs)[Perplexity
Crawlers](/guides/bots)

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Guides

Perplexity Crawlers

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Guides

  * [Initial Setup](/guides/getting-started)
  * [Supported Models](/guides/model-cards)
  * [Pricing](/guides/pricing)
  * [Rate Limits and Usage Tiers](/guides/usage-tiers)
  * [Structured Outputs Guide](/guides/structured-outputs)
  * [Prompt Guide](/guides/prompt-guide)
  * [Perplexity Crawlers](/guides/bots)

Guides

# Perplexity Crawlers

We strive to improve our service every day by delivering the best search
experience possible. To achieve this, we collect data using web crawlers
(“robots”) and user agents that gather and index information from the
internet, operating either automatically or in response to user requests.
Webmasters can use the following robots.txt tags to manage how their sites and
content interact with Perplexity. Each setting works independently, and it may
take up to 24 hours for our systems to reflect changes.

User Agent| Description  
---|---  
PerplexityBot| `PerplexityBot` is designed to surface and link websites in
search results on Perplexity. It is not used to crawl content for AI
foundation models. To ensure your site appears in search results, we recommend
allowing `PerplexityBot` in your site’s `robots.txt` file and permitting
requests from our published IP ranges listed below.  
  
Full user-agent string: `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko;
compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)`  
  
Published IP addresses: <https://www.perplexity.com/perplexitybot.json>  
Perplexity‑User| `Perplexity-User` supports user actions within Perplexity.
When users ask Perplexity a question, it might visit a web page to help
provide an accurate answer and include a link to the page in its response.
`Perplexity-User` controls which sites these user requests can access. It is
not used for web crawling or to collect content for training AI foundation
models.  
  
Full user-agent string: `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko;
compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)`  
  
Published IP addresses: <https://www.perplexity.com/perplexity-user.json>  
  
Since a user requested the fetch, this fetcher generally ignores robots.txt
rules.  
  
[Prompt Guide](/guides/prompt-guide)

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

[Perplexity home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/SonarByPerplexity.svg)![dark
logo](https://mintlify.s3.us-
west-1.amazonaws.com/perplexity/logo/Sonar_Wordmark_Light.svg)](/home.mdx)

Search docs

  * [Playground](https://labs.perplexity.ai/)
  * [Playground](https://labs.perplexity.ai/)

Search...

Navigation

Perplexity API

Chat Completions

[Home](/home)[Guides](/guides/getting-started)[API Reference](/api-
reference/chat-completions)[Changelog](/changelog/changelog)[System
Status](/system-status/system-
status)[FAQ](/faq/faq)[Discussions](/discussions/discussions)

##### Perplexity API

  * [POSTChat Completions](/api-reference/chat-completions)

Perplexity API

# Chat Completions

Generates a model’s response for the given chat conversation.

POST

/

chat

/

completions

Try it

cURL

Python

JavaScript

PHP

Go

Java

    
    
    curl --request POST \
      --url https://api.perplexity.ai/chat/completions \
      --header 'Authorization: Bearer <token>' \
      --header 'Content-Type: application/json' \
      --data '{
      "model": "sonar",
      "messages": [
        {
          "role": "system",
          "content": "Be precise and concise."
        },
        {
          "role": "user",
          "content": "How many stars are there in our galaxy?"
        }
      ],
      "max_tokens": 123,
      "temperature": 0.2,
      "top_p": 0.9,
      "search_domain_filter": null,
      "return_images": false,
      "return_related_questions": false,
      "search_recency_filter": "<string>",
      "top_k": 0,
      "stream": false,
      "presence_penalty": 0,
      "frequency_penalty": 1,
      "response_format": null
    }'

200

422

    
    
    {
      "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "model": "sonar",
      "object": "chat.completion",
      "created": 1724369245,
      "citations": [
        "https://www.astronomy.com/science/astro-for-kids-how-many-stars-are-there-in-space/",
        "https://www.esa.int/Science_Exploration/Space_Science/Herschel/How_many_stars_are_there_in_the_Universe",
        "https://www.space.com/25959-how-many-stars-are-in-the-milky-way.html",
        "https://www.space.com/26078-how-many-stars-are-there.html",
        "https://en.wikipedia.org/wiki/Milky_Way"
      ],
      "choices": [
        {
          "index": 0,
          "finish_reason": "stop",
          "message": {
            "role": "assistant",
            "content": "The number of stars in the Milky Way galaxy is estimated to be between 100 billion and 400 billion stars. The most recent estimates from the Gaia mission suggest that there are approximately 100 to 400 billion stars in the Milky Way, with significant uncertainties remaining due to the difficulty in detecting faint red dwarfs and brown dwarfs."
          },
          "delta": {
            "role": "assistant",
            "content": ""
          }
        }
      ],
      "usage": {
        "prompt_tokens": 14,
        "completion_tokens": 70,
        "total_tokens": 84
      }
    }

#### Authorizations

​

Authorization

string

header

required

Bearer authentication header of the form `Bearer <token>`, where `<token>` is
your auth token.

#### Body

application/json

​

model

string

required

The name of the model that will complete your prompt. Refer to [Supported
Models](https://docs.perplexity.ai/guides/model-cards) to find all the models
offered.

​

messages

object[]

required

A list of messages comprising the conversation so far.

Show child attributes

​

messages.content

string

required

The contents of the message in this turn of conversation.

​

messages.role

enum<string>

required

The role of the speaker in this turn of conversation. After the (optional)
system message, user and assistant roles should alternate with `user` then
`assistant`, ending in `user`.

Available options:

`system`,

`user`,

`assistant`

​

max_tokens

integer

The maximum number of completion tokens returned by the API. The number of
tokens requested in `max_tokens` plus the number of prompt tokens sent in
messages must not exceed the context window token limit of model requested. If
left unspecified, then the model will generate tokens until either it reaches
its stop token or the end of its context window.

​

temperature

number

default:

0.2

The amount of randomness in the response, valued between 0 inclusive and 2
exclusive. Higher values are more random, and lower values are more
deterministic.

Required range: `0 < x < 2`

​

top_p

number

default:

0.9

The nucleus sampling threshold, valued between 0 and 1 inclusive. For each
subsequent token, the model considers the results of the tokens with top_p
probability mass. We recommend either altering top_k or top_p, but not both.

Required range: `0 < x < 1`

​

search_domain_filter

any[]

Given a list of domains, limit the citations used by the online model to URLs
from the specified domains. Currently limited to only 3 domains for
whitelisting and blacklisting. For **blacklisting** add a `-` to the beginning
of the domain string. **Only available in certain tiers** \- refer to our
usage tiers [here](https://docs.perplexity.ai/guides/usage-tiers).

​

return_images

boolean

default:

false

Determines whether or not a request to an online model should return images.
**Only available in certain tiers** \- refer to our usage tiers
[here](https://docs.perplexity.ai/guides/usage-tiers).

​

return_related_questions

boolean

default:

false

Determines whether or not a request to an online model should return related
questions.**Only available in certain tiers** \- refer to our usage tiers
[here](https://docs.perplexity.ai/guides/usage-tiers).

​

search_recency_filter

string

Returns search results within the specified time interval - does not apply to
images. Values include `month`, `week`, `day`, `hour`.

​

top_k

number

default:

0

The number of tokens to keep for highest top-k filtering, specified as an
integer between 0 and 2048 inclusive. If set to 0, top-k filtering is
disabled. We recommend either altering top_k or top_p, but not both.

Required range: `0 < x < 2048`

​

stream

boolean

default:

false

Determines whether or not to incrementally stream the response with [server-
sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-
sent_events/Using_server-sent_events#event_stream_format) with `content-type:
text/event-stream`.

​

presence_penalty

number

default:

0

A value between -2.0 and 2.0. Positive values penalize new tokens based on
whether they appear in the text so far, increasing the model's likelihood to
talk about new topics. Incompatible with `frequency_penalty`.

Required range: `-2 < x < 2`

​

frequency_penalty

number

default:

1

A multiplicative penalty greater than 0. Values greater than 1.0 penalize new
tokens based on their existing frequency in the text so far, decreasing the
model's likelihood to repeat the same line verbatim. A value of 1.0 means no
penalty. Incompatible with `presence_penalty`.

Required range: `x > 0`

​

response_format

object

Enable structured outputs with a JSON or Regex schema. Refer to the guide
[here](https://docs.perplexity.ai/guides/structured-outputs) for more
information on how to use this parameter. **Only available in certain tiers**
\- refer to our usage tiers [here](https://docs.perplexity.ai/guides/usage-
tiers).

#### Response

200

200422

application/json

application/jsontext/event-stream

OK

​

id

string

An ID generated uniquely for each response.

​

model

string

The model used to generate the response.

​

object

string

The object type, which always equals `chat.completion`.

​

created

integer

The Unix timestamp (in seconds) of when the completion was created.

​

citations

any[]

Citations for the generated answer.

​

choices

object[]

The list of completion choices the model generated for the input prompt.

Show child attributes

​

choices.index

integer

​

choices.finish_reason

enum<string>

The reason the model stopped generating tokens. Possible values include `stop`
if the model hit a natural stopping point, or `length` if the maximum number
of tokens specified in the request was reached.

Available options:

`stop`,

`length`

​

choices.message

object

The message generated by the model.

Show child attributes

​

choices.message.content

string

required

The contents of the message in this turn of conversation.

​

choices.message.role

enum<string>

required

The role of the speaker in this turn of conversation. After the (optional)
system message, user and assistant roles should alternate with `user` then
`assistant`, ending in `user`.

Available options:

`system`,

`user`,

`assistant`

​

choices.delta

object

The incrementally streamed next tokens. Only meaningful when `stream = true`.

Show child attributes

​

choices.delta.content

string

required

The contents of the message in this turn of conversation.

​

choices.delta.role

enum<string>

required

The role of the speaker in this turn of conversation. After the (optional)
system message, user and assistant roles should alternate with `user` then
`assistant`, ending in `user`.

Available options:

`system`,

`user`,

`assistant`

​

usage

object

Usage statistics for the completion request.

Show child attributes

​

usage.prompt_tokens

integer

The number of tokens provided in the request prompt.

​

usage.completion_tokens

integer

The number of tokens generated in the response output.

​

usage.total_tokens

integer

The total number of tokens used in the chat completion (prompt + completion).

[twitter](https://twitter.com/perplexity_ai)[linkedin](https://www.linkedin.com/company/perplexity-
ai/)[discord](https://discord.com/invite/perplexity-
ai)[website](https://labs.perplexity.ai/)

cURL

Python

JavaScript

PHP

Go

Java

    
    
    curl --request POST \
      --url https://api.perplexity.ai/chat/completions \
      --header 'Authorization: Bearer <token>' \
      --header 'Content-Type: application/json' \
      --data '{
      "model": "sonar",
      "messages": [
        {
          "role": "system",
          "content": "Be precise and concise."
        },
        {
          "role": "user",
          "content": "How many stars are there in our galaxy?"
        }
      ],
      "max_tokens": 123,
      "temperature": 0.2,
      "top_p": 0.9,
      "search_domain_filter": null,
      "return_images": false,
      "return_related_questions": false,
      "search_recency_filter": "<string>",
      "top_k": 0,
      "stream": false,
      "presence_penalty": 0,
      "frequency_penalty": 1,
      "response_format": null
    }'

200

422

    
    
    {
      "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "model": "sonar",
      "object": "chat.completion",
      "created": 1724369245,
      "citations": [
        "https://www.astronomy.com/science/astro-for-kids-how-many-stars-are-there-in-space/",
        "https://www.esa.int/Science_Exploration/Space_Science/Herschel/How_many_stars_are_there_in_the_Universe",
        "https://www.space.com/25959-how-many-stars-are-in-the-milky-way.html",
        "https://www.space.com/26078-how-many-stars-are-there.html",
        "https://en.wikipedia.org/wiki/Milky_Way"
      ],
      "choices": [
        {
          "index": 0,
          "finish_reason": "stop",
          "message": {
            "role": "assistant",
            "content": "The number of stars in the Milky Way galaxy is estimated to be between 100 billion and 400 billion stars. The most recent estimates from the Gaia mission suggest that there are approximately 100 to 400 billion stars in the Milky Way, with significant uncertainties remaining due to the difficulty in detecting faint red dwarfs and brown dwarfs."
          },
          "delta": {
            "role": "assistant",
            "content": ""
          }
        }
      ],
      "usage": {
        "prompt_tokens": 14,
        "completion_tokens": 70,
        "total_tokens": 84
      }
    }

================
File: plugins/repos/twat_search/resources/you/you_news.md
================
[You.com API home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/light.svg)![dark logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/dark.svg)](/)

Search or ask...

  * [Discord](https://discord.com/invite/youdotcom)
  * [Support](mailto:api@you.com)
  * [Support](mailto:api@you.com)

Search...

Navigation

API Guide

News API

[Welcome](/welcome)[Quickstart](/docs/quickstart)[API Reference](/api-
reference/smart)[API Guide](/api-modes/smart-api)

##### API Guide

  * [Smart API](/api-modes/smart-api)
  * [Research API](/api-modes/research-api)
  * [Search API](/api-modes/search-api)
  * [News API](/api-modes/news-api)

##### Custom

  * [Custom APIs](/api-modes/custom-api)

API Guide

# News API

##

​

Stay Informed with the Latest Global News

When interacting with news results, LLMs are currently facing challlenges:

## Lack of News-Specific Filtering

LLMs lack the capability to focus exclusively on real-time news updates,
limiting their relevance for time-sensitive information.

## Limited Recency Filtering

Models are generally unable to filter news based on specific timeframes,
preventing users from accessing the most relevant information for them.

With our **News API** , we ensure you have reliable insights into global news
and events, keeping you informed about the latest stories and developments
worldwide

## Access to Live News

Our API integrates live news data, providing long snippets from trusted
sources, complete with URLs for verification.

## Customizable Recency and Region

For news results, users can filter data by timeframes such as the past day,
week, or month, enabling access to the insights you need.

## Uniquely Long Snippets

Ensure your responses are trustworthy and contain the information you need.

##

​

Use Cases

## Stay Up to Date

  

Current Developments in Key Areas of Interest

query.py

    
    
    import requests
    
    url = "https://api.ydc-index.io/news"
    
    query = {"query":"Latest News on Chipmakers"}
    
    headers = {"X-API-Key": "YOUR_API_KEY"}
    
    response = requests.request("GET", url, headers=headers, params=query)
    
    print(response.text)
    

Response

    
    
    {
    "news": {
    "query": {
      "original": "Latest News on Chipmakers",
      "show_strict_warning": false,
      "spellcheck_off": false
    },
    "results": [
      {
        "age": "4 days ago",
        "description": "The country that was once the world’s largest producer of semiconductors has embarked on a quest to return to the top.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› opinion  › articles  › 2024-12-12  › japan-chipmakers-gamble-the-future-of-semiconductors-on-the-past",
          "scheme": "https"
        },
        "page_age": "2024-12-12T22:37:49",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/qlX4Mz061puuwMyZm1FuXEbSGXuCbqjHIypccDoXGqw/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2lYTWNFSDBjMFli/OC92MS8xMjAweDgw/MC5qcGc"
        },
        "title": "Japan Chipmakers Gamble the Future of Semiconductors on the Past - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/opinion/articles/2024-12-12/japan-chipmakers-gamble-the-future-of-semiconductors-on-the-past"
      },
      {
        "age": "2 weeks ago",
        "description": "Chinese companies should buy locally instead, four of the country's top industry associations said.",
        "meta_url": {
          "hostname": "www.reuters.com",
          "netloc": "reuters.com",
          "path": "› technology  › chinese-firms-should-diversify-chip-sources-internet-society-china-says-2024-12-03",
          "scheme": "https"
        },
        "page_age": "2024-12-04T12:28:42",
        "source_name": "reuters.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/BuXIjhV4mQs66s0tgItvuiy4x3EBC-ZpqgaeQsLvF-Y/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/cmV1dGVycy5jb20v/cmVzaXplci92Mi9S/WlRXVE9YSkNCUDVS/RzM3RURQRERMWTdM/US5qcGc_YXV0aD02/YTI3MDBmMjJhMGJj/Y2I4NzE0OTM4M2Qz/YTZlODkyODg4YWI4/YTk4MTUwMmI2NGQ1/MDNlNDZmZDY0YjVl/MjRkJmFtcDtoZWln/aHQ9MTAwNSZhbXA7/d2lkdGg9MTkyMCZh/bXA7cXVhbGl0eT04/MCZhbXA7c21hcnQ9/dHJ1ZQ"
        },
        "title": "US chips are 'no longer safe,' Chinese industry bodies say in latest trade salvo | Reuters",
        "type": "news_result",
        "url": "https://www.reuters.com/technology/chinese-firms-should-diversify-chip-sources-internet-society-china-says-2024-12-03/"
      },
      {
        "age": "2 weeks ago",
        "description": "Microchip Technology Inc. is pausing its application for US semiconductor subsidies, making it the first known company to step back from a program designed to revitalize American chipmaking.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › articles  › 2024-12-03  › microchip-pauses-chips-act-application-after-scaling-back-plans",
          "scheme": "https"
        },
        "page_age": "2024-12-03T22:47:04",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/va8gs8bDTD44dQtxAjHvM8dB9Trdl_puEs2DlPWJlRM/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2llQThZQ3hjNXFp/TS92MC8xMjAweDgw/MC5qcGc"
        },
        "title": "Microchip Pauses Chips Act Application Amid Inventory Woes - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/articles/2024-12-03/microchip-pauses-chips-act-application-after-scaling-back-plans"
      },
      {
        "age": "2 weeks ago",
        "description": "The Chinese government has slammed America’s introduction of fresh export controls on US-made semiconductors that Washington fears Beijing could use to make the next generation of weapons and artificial intelligence (AI) systems.",
        "meta_url": {
          "hostname": "www.cnn.com",
          "netloc": "cnn.com",
          "path": "› 2024  › 12  › 02  › tech  › china-us-chips-new-restrictions-intl-hnk  › index.html",
          "scheme": "https"
        },
        "page_age": "2024-12-03T02:18:13",
        "source_name": "CNN",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/Q7u2YWA5NPRkhPy3LpctiPU11jkmoP6uYjggAeJ4J6s/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9tZWRp/YS5jbm4uY29tL2Fw/aS92MS9pbWFnZXMv/c3RlbGxhci9wcm9k/L2FwMjQyOTE1NDEy/Njk1OTEtY29weS5q/cGc_Yz0xNng5JnE9/d184MDAsY19maWxs"
        },
        "title": "AI and semiconductors: China hits out at latest US effort to block Beijing’s access to chip technology | CNN Business",
        "type": "news_result",
        "url": "https://www.cnn.com/2024/12/02/tech/china-us-chips-new-restrictions-intl-hnk/index.html"
      },
      {
        "age": "2 weeks ago",
        "description": "The US will launch its third crackdown in three years on China's semiconductor industry on Monday, restricting exports to 140 companies.",
        "meta_url": {
          "hostname": "www.reuters.com",
          "netloc": "reuters.com",
          "path": "› technology  › latest-us-strike-chinas-chips-hits-semiconductor-toolmakers-2024-12-02",
          "scheme": "https"
        },
        "page_age": "2024-12-03T01:14:17",
        "source_name": "reuters.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/UQxYf9pHn51WR53MNkbaa9ieeQZStKI64CetvaW6Tt4/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/cmV1dGVycy5jb20v/cmVzaXplci92Mi9E/VlRNTERBWkRWSkk3/RTdDUzZJUEtEUVRV/RS5qcGc_YXV0aD0x/OGZkYzAxZmNlZmEx/ZDk4MjIxODFhYTY0/YzA2YTBjNWFjMDk4/YWIxMmM2NzBjZmRh/NDlmYmRkY2Y2MDNh/ZjY0JmFtcDtoZWln/aHQ9MTAwNSZhbXA7/d2lkdGg9MTkyMCZh/bXA7cXVhbGl0eT04/MCZhbXA7c21hcnQ9/dHJ1ZQ"
        },
        "title": "Latest US clampdown on China's chips hits semiconductor toolmakers | Reuters",
        "type": "news_result",
        "url": "https://www.reuters.com/technology/latest-us-strike-chinas-chips-hits-semiconductor-toolmakers-2024-12-02/"
      },
      {
        "age": "3 weeks ago",
        "description": "There’s been a sea change among tech investors during the past month: Software stocks are hot, while semiconductor makers are not.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › articles  › 2024-11-26  › software-is-in-chips-are-out-as-traders-position-for-trump-era",
          "scheme": "https"
        },
        "page_age": "2024-11-26T12:01:25",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/LxYcJMko0ujutqhct8xx6kD_wc32kwIH0BJJTPle4cM/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2k3aDhSbUZuMjcy/US92MC8xMjAweDgw/MC5qcGc"
        },
        "title": "Software Is In, Chips Are Out as Traders Position for Trump Era - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/articles/2024-11-26/software-is-in-chips-are-out-as-traders-position-for-trump-era"
      },
      {
        "age": "4 weeks ago",
        "description": "Understanding how AI impacts business. The latest news on advancements in artificial intelligence, how to use AI, and how to invest in AI.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› ai",
          "scheme": "https"
        },
        "page_age": "2024-11-21T21:05:04",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/m7Jfen96xPtekEXCptzbwN1D52cE6auJKE9sP5DajpU/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/Ymxvb21iZXJnLmNv/bS9haQ"
        },
        "title": "AI - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/ai"
      },
      {
        "age": "1 month ago",
        "description": "The Biden administration is trying to shore up its CHIPS Act funding agreements before Donald Trump takes office.",
        "meta_url": {
          "hostname": "www.businessinsider.com",
          "netloc": "businessinsider.com",
          "path": "› chips-act-funding-biden-administration-trump-tariffs-repeal-2024-11",
          "scheme": "https"
        },
        "page_age": "2024-11-19T09:15:01",
        "source_name": "Business Insider",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/_16b6eIPhruMXBpV8ovQHxtj5qc_0nTGW5SHlHYseAg/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pLmlu/c2lkZXIuY29tLzY3/M2JiM2Y0ZmEwMTQw/Y2RkNTY0Mjk5ZT93/aWR0aD0xMjAwJmZv/cm1hdD1qcGVn"
        },
        "title": "The Biden administration is scrambling to send billions to chipmakers before Trump takes over",
        "type": "news_result",
        "url": "https://www.businessinsider.com/chips-act-funding-biden-administration-trump-tariffs-repeal-2024-11"
      },
      {
        "age": "1 month ago",
        "description": "JONATHAN: NVIDIA IS HUGE SO WE ALL HAVE EXPOSURE TO IT ONE WAY OR ANOTHER. EXPLAIN THE STORY THIS MORNING, REPORTING THE CHIPMAKER IS FACING A DESIGN SNACK FOR THE BLACKWELL CHIPS. WE SEE THAT DROPPING IN AND OUT OF THE NEWS REPEATEDLY, SHAKING UP THE STOCK FROM TIME TO TIME.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › videos  › 2024-11-18  › chipmakers-hope-for-a-boost-from-nvidia-earnigns-video",
          "scheme": "https"
        },
        "page_age": "2024-11-18T14:17:33",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/BzYM6UHOgK9rvTWBxa_JK-buc5-4LfUelVxMrd_tl6A/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2lCejhkVm53dEVo/WS92My8tMXgtMS5q/cGc"
        },
        "title": "Watch Chipmakers Hope for a Boost From Nvidia Earnings - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/videos/2024-11-18/chipmakers-hope-for-a-boost-from-nvidia-earnigns-video"
      },
      {
        "age": "1 month ago",
        "description": "Nov 17 (Reuters) - Nvidia's (NVDA.O), ... reported on Sunday. The Blackwell graphics processing units overheat when connected together in server racks designed to hold up to 72 chips, the report said, citing sources familiar with the issue. The chipmaker has asked its suppliers ...",
        "meta_url": {
          "hostname": "www.reuters.com",
          "netloc": "reuters.com",
          "path": "› technology  › artificial-intelligence  › new-nvidia-ai-chips-face-issue-with-overheating-servers-information-reports-2024-11-17",
          "scheme": "https"
        },
        "page_age": "2024-11-17T22:36:10",
        "source_name": "reuters.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/16Jgf2p3zxsmMnQUzOrP7EhvynVj9rnuSDEGYINQ-HQ/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/cmV1dGVycy5jb20v/cmVzaXplci92Mi9W/VFhIVkFJM1ZCT1lU/QVFSRzdOQlZMNUVJ/SS5qcGc_YXV0aD1m/ODQxZmExMzY3ODMz/ZGRjMmJhMTQ5MzJi/YTQ5Yzc2ZmY2MTNm/NmI0MWI3ZjM3Y2Yx/ZDU5ZDZiZjA2ZjYw/ZjJhJmFtcDtoZWln/aHQ9MTAwNSZhbXA7/d2lkdGg9MTkyMCZh/bXA7cXVhbGl0eT04/MCZhbXA7c21hcnQ9/dHJ1ZQ"
        },
        "title": "New Nvidia AI chips overheating in servers, the Information reports | Reuters",
        "type": "news_result",
        "url": "https://www.reuters.com/technology/artificial-intelligence/new-nvidia-ai-chips-face-issue-with-overheating-servers-information-reports-2024-11-17/"
      },
      {
        "age": "November 8, 2024",
        "description": "China hardliners in Congress are calling on the world's foremost semiconductor equipment makers - KLA , LAM , Applied Materials , Tokyo Electron and ASML - to provide details of their sales to China.",
        "meta_url": {
          "hostname": "www.reuters.com",
          "netloc": "reuters.com",
          "path": "› technology  › us-lawmakers-press-top-chip-equipment-makers-details-china-sales-2024-11-08",
          "scheme": "https"
        },
        "page_age": "2024-11-08T18:05:21",
        "source_name": "reuters.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/UQxYf9pHn51WR53MNkbaa9ieeQZStKI64CetvaW6Tt4/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/cmV1dGVycy5jb20v/cmVzaXplci92Mi9E/VlRNTERBWkRWSkk3/RTdDUzZJUEtEUVRV/RS5qcGc_YXV0aD0x/OGZkYzAxZmNlZmEx/ZDk4MjIxODFhYTY0/YzA2YTBjNWFjMDk4/YWIxMmM2NzBjZmRh/NDlmYmRkY2Y2MDNh/ZjY0JmFtcDtoZWln/aHQ9MTAwNSZhbXA7/d2lkdGg9MTkyMCZh/bXA7cXVhbGl0eT04/MCZhbXA7c21hcnQ9/dHJ1ZQ"
        },
        "title": "US lawmakers press top chip equipment makers for details on China sales | Reuters",
        "type": "news_result",
        "url": "https://www.reuters.com/technology/us-lawmakers-press-top-chip-equipment-makers-details-china-sales-2024-11-08/"
      },
      {
        "age": "November 8, 2024",
        "description": "The Biden administration is racing to complete Chips Act agreements with companies like Intel Corp. and Samsung Electronics Co., aiming to shore up one of its signature initiatives before President-elect Donald Trump enters the White House.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › articles  › 2024-11-08  › trump-s-win-sets-off-race-to-complete-chips-act-subsidy-deals",
          "scheme": "https"
        },
        "page_age": "2024-11-08T01:42:07",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/ei0BLNvbyDZ0k8vGo3QdI7Pm8QtABfzqpfz0ucNHOa0/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2l4T25CbS4zdFhq/Yy92MC8xMjAweDgw/MC5qcGc"
        },
        "title": "Trump’s Win Has Biden Rushing to Finalize Chips Act Deals With Intel, Samsung - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/articles/2024-11-08/trump-s-win-sets-off-race-to-complete-chips-act-subsidy-deals"
      },
      {
        "age": "November 4, 2024",
        "description": "Intel has fallen so far so fast that the chipmaker's stock price is no longer having an impact on the Dow Jones Industrial Average.",
        "meta_url": {
          "hostname": "www.cnbc.com",
          "netloc": "cnbc.com",
          "path": "› 2024  › 11  › 04  › the-dow-needs-nvidia-because-intel-plunge-made-semis-underrepresented.html",
          "scheme": "https"
        },
        "page_age": "2024-11-04T20:59:19",
        "source_name": "CNBC",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/1uMlnbYCbjdLfqxeP9fQOibcd2dVPGJwIDWZ65jYr8k/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWFn/ZS5jbmJjZm0uY29t/L2FwaS92MS9pbWFn/ZS8xMDgwMzI5OTIt/MTcyNjEwNzgwMzAw/NS1nZXR0eWltYWdl/cy0yMTU1MTI3Mzc5/LU5WSURJQV9IVUFO/Ry5qcGVnP3Y9MTcz/MjE5ODI4OCZhbXA7/dz0xOTIwJmFtcDto/PTEwODA"
        },
        "title": "The Dow needs Nvidia to give chipmakers representation in index after Intel's plunge",
        "type": "news_result",
        "url": "https://www.cnbc.com/2024/11/04/the-dow-needs-nvidia-because-intel-plunge-made-semis-underrepresented.html"
      },
      {
        "age": "October 30, 2024",
        "description": "Global stock indexes edged lower on Wednesday as a disappointing forecast from Advanced Micro Devices weighed on chipmakers, while gold prices rose to a record high as uncertainty ahead of next week's U.S. presidential election drove safe-haven demand.",
        "meta_url": {
          "hostname": "www.reuters.com",
          "netloc": "reuters.com",
          "path": "› markets  › global-markets-wrapup-1-2024-10-30",
          "scheme": "https"
        },
        "page_age": "2024-10-30T21:05:08",
        "source_name": "reuters.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/nzLyG1gwaIEPPDPWvUB6P6_z5aoPvHTWttuhFJ-AZZ4/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/cmV1dGVycy5jb20v/cmVzaXplci92Mi9O/MkJQN0VHM1RGT0dI/REFMSlFGVDNJRTZa/WS5qcGc_YXV0aD0z/YmE1ZmUyZGZkYzkx/MjAzZmU0YzVjZWE2/MWNhNmJiNjUzYTc0/ZjhlMjQxMjEzNDAx/NTUzYTYyMWZiODdm/M2E3JmFtcDtoZWln/aHQ9MTAwNSZhbXA7/d2lkdGg9MTkyMCZh/bXA7cXVhbGl0eT04/MCZhbXA7c21hcnQ9/dHJ1ZQ"
        },
        "title": "Stocks fall with chipmakers, gold hits record high | Reuters",
        "type": "news_result",
        "url": "https://www.reuters.com/markets/global-markets-wrapup-1-2024-10-30/"
      },
      {
        "age": "October 30, 2024",
        "description": "OpenAI is working with Broadcom and TSMC to build its first in-house chip designed to support its artificial intelligence systems, while adding AMD chips alongside Nvidia chips to meet its surging infrastructure demands, sources told Reuters.",
        "meta_url": {
          "hostname": "www.reuters.com",
          "netloc": "reuters.com",
          "path": "› technology  › artificial-intelligence  › openai-builds-first-chip-with-broadcom-tsmc-scales-back-foundry-ambition-2024-10-29",
          "scheme": "https"
        },
        "page_age": "2024-10-30T15:30:09",
        "source_name": "reuters.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/JfpxYjJ_YO4fhT-Ejlkn0j1V4oqVpLdCQ7LIqyAlBOM/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/cmV1dGVycy5jb20v/cmVzaXplci92Mi83/M0I1NTRIS0JCTUZI/S0tNTE82RlZFVTNH/US5qcGc_YXV0aD04/NDlhODM1MDIwZGVj/YjBhMjI1ZDM5MGVk/YmE4NzI0YTNkMWYz/YzI2ZmE1YjM4OGMy/ZWU1Y2NkNWVkZmU3/YWYzJmFtcDtoZWln/aHQ9MTAwNSZhbXA7/d2lkdGg9MTkyMCZh/bXA7cXVhbGl0eT04/MCZhbXA7c21hcnQ9/dHJ1ZQ"
        },
        "title": "Exclusive: OpenAI builds first chip with Broadcom and TSMC, scales back foundry ambition | Reuters",
        "type": "news_result",
        "url": "https://www.reuters.com/technology/artificial-intelligence/openai-builds-first-chip-with-broadcom-tsmc-scales-back-foundry-ambition-2024-10-29/"
      },
      {
        "age": "October 16, 2024",
        "description": "A rally in semiconductor names and strong economic data sent stocks higher on Thursday.",
        "meta_url": {
          "hostname": "www.cnbc.com",
          "netloc": "cnbc.com",
          "path": "› 2024  › 10  › 16  › stock-market-today-live-updates.html",
          "scheme": "https"
        },
        "page_age": "2024-10-16T22:02:35",
        "source_name": "CNBC",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/3z4hAEdUIDVi62nmwGTixjpwr8RNndn7RzgS7AzkBpU/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWFn/ZS5jbmJjZm0uY29t/L2FwaS92MS9pbWFn/ZS8xMDgwNDQ3MjUt/MTcyODM5Nzg5MDg1/NC1nZXR0eWltYWdl/cy0yMTc3NDM3MDE5/LW1zMV8xNzQ0X3ln/cXlwem9oLmpwZWc_/dj0xNzI5MTgxMTAz/JmFtcDt3PTE5MjAm/YW1wO2g9MTA4MA"
        },
        "title": "Dow closes at fresh record, Nasdaq ends higher as chipmakers rally: Live updates",
        "type": "news_result",
        "url": "https://www.cnbc.com/2024/10/16/stock-market-today-live-updates.html"
      },
      {
        "age": "October 16, 2024",
        "description": "ASML’s surprise results have implications for the overall chip industry. But first...",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › newsletters  › 2024-10-16  › asml-s-surprise-results-signal-uncertain-future-for-some-chipmakers",
          "scheme": "https"
        },
        "page_age": "2024-10-16T11:05:28",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/CCtgpMYTTo_tqrCUjHj2hE4j33_v3pN_12gPUXztUOk/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2k1QnRoejdlb1Zt/TS92MS8xMjAweDgw/MC5qcGc"
        },
        "title": "ASML’s Surprise Results Signal Uncertain Future for Some Chipmakers - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/newsletters/2024-10-16/asml-s-surprise-results-signal-uncertain-future-for-some-chipmakers"
      },
      {
        "age": "October 16, 2024",
        "description": "ASML Holding NV has lost more than €60 billion ($65.3 billion) in market value since it reported weak orders for its chipmaking machines, forcing investors to reevaluate the health of the industry.",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › articles  › 2024-10-16  › asml-s-tumble-fuels-questions-about-chip-industry-s-prospects",
          "scheme": "https"
        },
        "page_age": "2024-10-16T10:36:26",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/FRs8PrWQma_nu8Z4wCUNX20wYoNYhRPA5K8oYZWYcL0/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2k2dUkwNDdXTWpC/OC92MC8xMjAweDgw/MC5qcGc"
        },
        "title": "ASML’s Plunge Shows the Diverging Fortunes of Chipmakers From AI - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/articles/2024-10-16/asml-s-tumble-fuels-questions-about-chip-industry-s-prospects"
      },
      {
        "age": "October 16, 2024",
        "description": "Chip stocks fell Tuesday. ASML slashed its guidance for 2025, and the sector mulled reports of more potential restrictions on chip exports from some US firms.",
        "meta_url": {
          "hostname": "ca.finance.yahoo.com",
          "netloc": "ca.finance.yahoo.com",
          "path": "› news  › chipmakers-tumble-asml-forecast-cut-030444530.html",
          "scheme": "https"
        },
        "page_age": "2024-10-16T03:04:44",
        "source_name": "ca.finance.yahoo.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/a2VYr800beR71JP283brHdStO-voqPux2jfJuPjnnuw/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9zLnlp/bWcuY29tL255L2Fw/aS9yZXMvMS4yL1ho/OU5CUEk1WEFLR2Np/S3puREk1TmctLS9Z/WEJ3YVdROWFHbG5h/R3hoYm1SbGNqdDNQ/VEV5TURBN2FEMDVN/REEtL2h0dHBzOi8v/bWVkaWEuemVuZnMu/Y29tL2VuL2J1c2lu/ZXNzX2luc2lkZXJf/YXJ0aWNsZXNfODg4/LzEzMjI5YmQ0OWEz/NjhmYzQ4YmViZWZl/ZjUwZTcwMjU4"
        },
        "title": "Chipmakers tumble as ASML forecast cut issues a growth warning to the sector",
        "type": "news_result",
        "url": "https://ca.finance.yahoo.com/news/chipmakers-tumble-asml-forecast-cut-030444530.html"
      },
      {
        "age": "October 15, 2024",
        "description": "Manage your newsletter preferences anytime here. ASML Holding’s shares plunged the most in 26 years after it booked only about half the orders analysts expected, a startling slowdown for the Dutch company, maker of the world’s most advanced chip-making machines and one of the bellwethers ...",
        "meta_url": {
          "hostname": "www.bloomberg.com",
          "netloc": "bloomberg.com",
          "path": "› news  › newsletters  › 2024-10-15  › asml-shares-plunge-amid-strange-times-for-chipmakers-like-intel-samsung",
          "scheme": "https"
        },
        "page_age": "2024-10-15T22:14:46",
        "source_name": "Bloomberg L.P.",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/iQL7-057RrQw0oDsHMrzhHX5ksoVJeP0D_tecxHpO1E/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9hc3Nl/dHMuYndieC5pby9p/bWFnZXMvdXNlcnMv/aXFqV0hCRmRmeElV/L2lhaWNjSmhPMTc4/US92MC8xMjAweDgw/MC5qcGc"
        },
        "title": "ASML Shares Plunge Amid Strange Times for Chipmakers Like Intel, Samsung - Bloomberg",
        "type": "news_result",
        "url": "https://www.bloomberg.com/news/newsletters/2024-10-15/asml-shares-plunge-amid-strange-times-for-chipmakers-like-intel-samsung"
      }
    ],
    "type": "news"
    }
    }
    

Country-specific Industry Overviews

query.py

    
    
    import requests
    
    url = "https://api.ydc-index.io/news"
    
    querystring = {"query":"News on the Chemical Industry", "country":"IN"}
    
    headers = {"X-API-Key": "YOUR_API_KEY"}
    
    response = requests.request("GET", url, headers=headers, params=querystring)
    
    print(response.text)
    

reponse

    
    
    {
    "news": {
    "query": {
      "original": "Chemical Industry",
      "show_strict_warning": false,
      "spellcheck_off": false
    },
    "results": [
      {
        "age": "2 weeks ago",
        "description": "On December 3 2024 Sudarshan Chemical Industries a midcap company in the dyes and pigments industry saw a 5 02 increase in its stock outperforming the sector by 2 7 This marks the fourth consecutive day of gains with a total increase of 11 34 in the past four days The stock is currently trading ...",
        "meta_url": {
          "hostname": "www.marketsmojo.com",
          "netloc": "marketsmojo.com",
          "path": "› news  › stocks-in-action  › sudarshan-chemical-industries-stock-sees-positive-trend-outperforms-sector-and-market-308679",
          "scheme": "https"
        },
        "page_age": "2024-12-03T05:30:02",
        "source_name": "marketsmojo.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/YiUpk2EMKEHCCGwC57406-c57F3JKfEZr6r3jUKE_jk/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pLm1h/cmtldHNtb2pvLmNv/bS9uZXdzaW1nLzIw/MjQvMTIvU3VkYXJz/aGFuQ2hlbWljX3By/aWNlUmVsYXRlZGZh/Y3RvcnNfMjMxNjI5/LnBuZw"
        },
        "title": "Sudarshan Chemical Industries' Stock Sees Positive Trend, Outperforms Sector and Market",
        "type": "news_result",
        "url": "https://www.marketsmojo.com/news/stocks-in-action/sudarshan-chemical-industries-stock-sees-positive-trend-outperforms-sector-and-market-308679"
      },
      {
        "age": "2 weeks ago",
        "description": "Chemical industry in Tamil Nadu faces challenges, seeks government support for cracker project, growth projections shared at Chemvision 2024.",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› news  › national  › tamil-nadu  › chemical-industry-calls-for-cracker-project-to-boost-growth-in-tamil-nadu  › article68927363.ece",
          "scheme": "https"
        },
        "page_age": "2024-12-02T15:30:31",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/MbaLnkSns7korT9MlnYjmNHGa1WUAhQ3UWPKsATH8sw/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly90aC1p/LnRoZ2ltLmNvbS9w/dWJsaWMvaW5jb21p/bmcvNnppYXh2L2Fy/dGljbGU2ODkzOTQ1/Ni5lY2UvYWx0ZXJu/YXRlcy9MQU5EU0NB/UEVfMTIwMC9CVlJf/NDQ1NS5KUEc"
        },
        "title": "Chemical industry calls for cracker project to boost growth in Tamil Nadu - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/news/national/tamil-nadu/chemical-industry-calls-for-cracker-project-to-boost-growth-in-tamil-nadu/article68927363.ece"
      },
      {
        "age": "2 weeks ago",
        "description": "Paradeep Phosphates Limited has taken the planned shutdown of the ammonia and urea plants at Goa",
        "meta_url": {
          "hostname": "www.indianchemicalnews.com",
          "netloc": "indianchemicalnews.com",
          "path": "› general  › briefs-paradeep-phosphates-and-upl-24269",
          "scheme": "https"
        },
        "page_age": "2024-12-02T14:35:03",
        "source_name": "indianchemicalnews.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/Vl4q-rbHKGG5hwNVyuoaLx_Igp-MdbrerzkvrrHSwGQ/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/aW5kaWFuY2hlbWlj/YWxuZXdzLmNvbS9w/dWJsaWMvdXBsb2Fk/cy9uZXdzLzIwMjQv/MTIvMjQyNjkvVVBM/X2xvZ29fb3JpZ2lu/YWwuanBn"
        },
        "title": "Briefs: Paradeep Phosphates and UPL",
        "type": "news_result",
        "url": "https://www.indianchemicalnews.com/general/briefs-paradeep-phosphates-and-upl-24269"
      },
      {
        "age": "2 weeks ago",
        "description": "Press release - Exactitude Consultancy - Crop Protection Chemicals Market Detailed Industry Report Analysis 2024-2032 | BASF, Syngenta, Bayer Crop Science - published on openPR.com",
        "meta_url": {
          "hostname": "www.openpr.com",
          "netloc": "openpr.com",
          "path": "› news  › 3766044  › crop-protection-chemicals-market-detailed-industry-report",
          "scheme": "https"
        },
        "page_age": "2024-12-02T10:46:15",
        "source_name": "openpr.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/pAlDpCwJjcEf6Ypm-K1Tl8JBCNjyRa73ZfUkRYZwVXU/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9jZG4u/b3Blbi1wci5jb20v/TC9jL0xjMDIxNTQ3/NjdfZy5qcGc"
        },
        "title": "Crop Protection Chemicals Market Detailed Industry Report Analysis 2024-2032 | BASF, Syngenta, Bayer Crop Science",
        "type": "news_result",
        "url": "https://www.openpr.com/news/3766044/crop-protection-chemicals-market-detailed-industry-report"
      },
      {
        "age": "5 days ago",
        "description": "CII Visakhapatnam to organise conference on Industrial & Chemical Safety from December 12",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› news  › cities  › Visakhapatnam  › cii-visakhapatnam-to-organise-conference-on-industrial-chemical-safety-from-december-12  › article68973107.ece",
          "scheme": "https"
        },
        "page_age": "2024-12-11T12:18:13",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/llajowBpmgWaVpuUx0Kw-DZmR0pl3GHEUnH8UFCou-g/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/dGhlaGluZHUuY29t/L3RoZW1lL2ltYWdl/cy9vZy1pbWFnZS5w/bmc"
        },
        "title": "CII Visakhapatnam to organise conference on Industrial & Chemical Safety from December 12 - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/news/cities/Visakhapatnam/cii-visakhapatnam-to-organise-conference-on-industrial-chemical-safety-from-december-12/article68973107.ece"
      },
      {
        "age": "3 weeks ago",
        "description": "CII Tamil Nadu Chemvision 2024 highlights sustainability, safety, investment opportunities, and capability building in the chemical industry.",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› news  › national  › tamil-nadu  › cii-tamil-nadu-chemvision-2024-to-be-held-on-november-29  › article68923289.ece",
          "scheme": "https"
        },
        "page_age": "2024-11-28T17:55:13",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/llajowBpmgWaVpuUx0Kw-DZmR0pl3GHEUnH8UFCou-g/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/dGhlaGluZHUuY29t/L3RoZW1lL2ltYWdl/cy9vZy1pbWFnZS5w/bmc"
        },
        "title": "CII Tamil Nadu Chemvision 2024 to be held on November 29 - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/news/national/tamil-nadu/cii-tamil-nadu-chemvision-2024-to-be-held-on-november-29/article68923289.ece"
      },
      {
        "age": "1 month ago",
        "description": "Fire officials said the incident occurred when chemical sludge, which was awaiting transport to a cement factory, caught fire. Firefighters doused the flames before fire could spread to the main unit. The reactor and boilers, which are key components of the chemical industry, were not damaged ...",
        "meta_url": {
          "hostname": "timesofindia.indiatimes.com",
          "netloc": "timesofindia.indiatimes.com",
          "path": "› city  › hyderabad  › massive-fire-erupts-at-chemical-facility-near-hyderabad-no-casualties-reported  › articleshow  › 115369330.cms",
          "scheme": "https"
        },
        "page_age": "2024-11-16T19:01:00",
        "source_name": "timesofindia.indiatimes.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/2lwy_RjV_cNso39qFr6pfrjPykcR3cRKhHnm_Z6LzJI/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9zdGF0/aWMudG9paW1nLmNv/bS90aHVtYi9tc2lk/LTExNTM2OTMyNyx3/aWR0aC0xMDcwLGhl/aWdodC01ODAsaW1n/c2l6ZS01NjQ2MCxy/ZXNpemVtb2RlLTc1/LG92ZXJsYXktdG9p/X3N3LHB0LTMyLHlf/cGFkLTQwL3Bob3Rv/LmpwZw"
        },
        "title": "Massive Fire Erupts at Chemical Facility Near Hyderabad, No Casualties Reported | - Times of India",
        "type": "news_result",
        "url": "https://timesofindia.indiatimes.com/city/hyderabad/massive-fire-erupts-at-chemical-facility-near-hyderabad-no-casualties-reported/articleshow/115369330.cms"
      },
      {
        "age": "October 24, 2024",
        "description": "সম্প্রতি যে সংস্থা দেশের সেরা কোম্পানির পুরস্কার (তালিকাভুক্ত কোম্পানির মধ্যে) পেয়েছে, সেটিই সিঙ্গুরে ২২০ কোটি ...",
        "meta_url": {
          "hostname": "bangla.hindustantimes.com",
          "netloc": "bangla.hindustantimes.com",
          "path": "› pictures  › big-investment-for-singur-himadri-speciality-chemical-limited-to-invest-rs-220-crore-for-31729757880342.html",
          "scheme": "https"
        },
        "page_age": "2024-10-24T08:29:45",
        "source_name": "bangla.hindustantimes.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/P2qC9UIGEGMbr_jLcGcbrqITQpUIY6NaltTu8DBiEB8/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWFn/ZXMuaGluZHVzdGFu/dGltZXMuY29tL2Jh/bmdsYS9pbWcvMjAy/NC8xMC8yNC82MDB4/MzM4L0lORElBLVNU/RUVMLTBfMTcyOTc1/Nzg3MTk5NF8xNzI5/NzU4NTg1MDQ4LkpQ/Rw"
        },
        "title": "₹220 cr investment at Singur: সিঙ্গুরে ২২০ কোটি টাকার বিনিয়োগ আসছে! লগ্নি করছে দেশের সেরা হওয়া সংস্থা, কী হবে? - Big investment for Singur, Himadri Speciality Chemical limited to invest ₹220 crore for - ছবিঘর নিউজ",
        "type": "news_result",
        "url": "https://bangla.hindustantimes.com/pictures/big-investment-for-singur-himadri-speciality-chemical-limited-to-invest-rs-220-crore-for-31729757880342.html"
      },
      {
        "age": "September 16, 2024",
        "description": "Texas chemical plant explosion: Firefighters were battling a pipeline fire in suburban Houston that sparked grass fires and burned power poles on Monday",
        "meta_url": {
          "hostname": "www.hindustantimes.com",
          "netloc": "hindustantimes.com",
          "path": "› world-news  › us-news  › texas-chemical-plant-explosion-evacuations-ordered-in-la-porte-amid-roaring-pipeline-fire-101726503834727.html",
          "scheme": "https"
        },
        "page_age": "2024-09-16T16:28:49",
        "source_name": "Hindustan Times",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/faaxK__lNPrgy_h2x4S1uacJIB8EVkgmD37qsgKPHl8/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/aGluZHVzdGFudGlt/ZXMuY29tL2h0LWlt/Zy9pbWcvMjAyNC8w/OS8xNi8xNjAweDkw/MC9URVhBUy1DQVRU/TEUtLTFfMTY4MTQ1/MTc1NjI3MF8xNjgx/NDUxNzU2MjcwXzE3/MjY1MDM5MTIyNDUu/SlBH"
        },
        "title": "Texas chemical plant explosion: Evacuations ordered in La Porte amid roaring pipeline fire - Hindustan Times",
        "type": "news_result",
        "url": "https://www.hindustantimes.com/world-news/us-news/texas-chemical-plant-explosion-evacuations-ordered-in-la-porte-amid-roaring-pipeline-fire-101726503834727.html"
      },
      {
        "age": "September 13, 2024",
        "description": "Gas leak reported at chemical company in Ambernath, Thane district; fire brigade officials on scene, awaiting further details.",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› news  › national  › maharashtra  › gas-leak-at-chemical-factory-in-thanes-ambernath  › article68637090.ece",
          "scheme": "https"
        },
        "page_age": "2024-09-13T01:43:07",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/llajowBpmgWaVpuUx0Kw-DZmR0pl3GHEUnH8UFCou-g/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/dGhlaGluZHUuY29t/L3RoZW1lL2ltYWdl/cy9vZy1pbWFnZS5w/bmc"
        },
        "title": "Gas leak at chemical factory in Thane’s Ambernath - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/news/national/maharashtra/gas-leak-at-chemical-factory-in-thanes-ambernath/article68637090.ece"
      },
      {
        "age": "August 31, 2024",
        "description": "Discover the vast chemical armory of plants like garlic, used for centuries in diets and medicine for human health.",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› sci-tech  › the-chemical-treasury-in-garlic  › article68559174.ece",
          "scheme": "https"
        },
        "page_age": "2024-08-31T15:40:00",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/llajowBpmgWaVpuUx0Kw-DZmR0pl3GHEUnH8UFCou-g/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly93d3cu/dGhlaGluZHUuY29t/L3RoZW1lL2ltYWdl/cy9vZy1pbWFnZS5w/bmc"
        },
        "title": "The chemical treasury in garlic - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/sci-tech/the-chemical-treasury-in-garlic/article68559174.ece"
      },
      {
        "age": "August 26, 2024",
        "description": "And are there such more requests from the industry pending for the ministry to look at? Ajay Joshi: We will see a lot more anti-dumping duties, as well as a lot more minimum import support price initiatives that will be rendered from the Indian government to Indian chemical players.",
        "meta_url": {
          "hostname": "m.economictimes.com",
          "netloc": "m.economictimes.com",
          "path": "› markets  › expert-view  › more-anti-dumping-duties-to-support-local-indian-chemical-companies-ajay-joshi  › articleshow  › 112801519.cms",
          "scheme": "https"
        },
        "page_age": "2024-08-26T10:40:02",
        "source_name": "m.economictimes.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/36FKJ7dAKQRbme3CqO47lQcFO4VHz03KPSoqn-Eqxi0/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWcu/ZXRpbWcuY29tL3Ro/dW1iL21zaWQtMTEy/ODAxNTI4LHdpZHRo/LTEyMDAsaGVpZ2h0/LTYzMCxpbWdzaXpl/LTQwNTQsb3Zlcmxh/eS1ldG1hcmtldHMv/YXJ0aWNsZXNob3cu/anBn"
        },
        "title": "chemical companies: More anti-dumping duties to support local Indian chemical companies: Ajay Joshi - The Economic Times",
        "type": "news_result",
        "url": "https://m.economictimes.com/markets/expert-view/more-anti-dumping-duties-to-support-local-indian-chemical-companies-ajay-joshi/articleshow/112801519.cms"
      },
      {
        "age": "August 24, 2024",
        "description": "Producers of PFAS chemicals and semiconductors, a key part of most electronics, have formed a group that develops industry-friendly science aimed at heading off regulation as facilities release high levels of toxic waste, documents seen by the Guardian show.",
        "meta_url": {
          "hostname": "www.theguardian.com",
          "netloc": "theguardian.com",
          "path": "› environment  › article  › 2024  › aug  › 24  › pfas-toxic-waste-pollution-regulation-lobbying",
          "scheme": "https"
        },
        "page_age": "2024-08-24T13:00:33",
        "source_name": "The Guardian",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/0W30teYhY65K0E8fmkhQaOgOuJ6qSCa1DspIcofMITQ/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pLmd1/aW0uY28udWsvaW1n/L21lZGlhL2QwMzc3/ODczZjBjM2VlNjcw/YzljODNhNDgzMjU2/YzQ1MjljMzZjY2Qv/MF8yMzJfNjk2MF80/MTc2L21hc3Rlci82/OTYwLmpwZz93aWR0/aD0xMjAwJmhlaWdo/dD02MzAmcXVhbGl0/eT04NSZhdXRvPWZv/cm1hdCZmaXQ9Y3Jv/cCZvdmVybGF5LWFs/aWduPWJvdHRvbSUy/Q2xlZnQmb3Zlcmxh/eS13aWR0aD0xMDBw/Jm92ZXJsYXktYmFz/ZTY0PUwybHRaeTl6/ZEdGMGFXTXZiM1ps/Y214aGVYTXZkR2N0/WkdWbVlYVnNkQzV3/Ym1jJmVuYWJsZT11/cHNjYWxlJnM9NTRl/M2VkMjAzNzNmNjJh/Njg0ZTNhYzM0NTA5/ODgxOTI"
        },
        "title": "Industry acts to head off regulation on PFAS pollution from semiconductors | PFAS | The Guardian",
        "type": "news_result",
        "url": "https://www.theguardian.com/environment/article/2024/aug/24/pfas-toxic-waste-pollution-regulation-lobbying"
      },
      {
        "age": "August 6, 2024",
        "description": "Imported chemicals, reagents, and ... chemicals and are vital to experimental research across nearly every domain of scientific research. They comprise oxidisers, corrosive acids, and compressed gas, that are used by researchers to conduct experiments and even make new products. Outside of research settings, the medical diagnostics industry is run on ...",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› news  › national  › why-was-customs-duty-hike-imposed-for-lab-chemicals-explained  › article68489881.ece",
          "scheme": "https"
        },
        "page_age": "2024-08-06T03:00:00",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/XnRTEq-045V-3-yqQmZzNL2CXz7nfE9uB3A4EvvK_ek/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly90aC1p/LnRoZ2ltLmNvbS9w/dWJsaWMvaW5jb21p/bmcvM2QycmNkL2Fy/dGljbGU2ODQ4OTg5/NC5lY2UvYWx0ZXJu/YXRlcy9MQU5EU0NB/UEVfMTIwMC9JTUdf/UE8yMl9MYWJfMl8x/X0xVQlRLVkxOLmpw/Zw"
        },
        "title": "Why was a customs duty hike imposed for lab chemicals? | Explained - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/news/national/why-was-customs-duty-hike-imposed-for-lab-chemicals-explained/article68489881.ece"
      },
      {
        "age": "July 30, 2024",
        "description": "Scientists alarmed by 150% hike in customs duty on laboratory chemicals, sparking concerns over research funding and accessibility.",
        "meta_url": {
          "hostname": "www.thehindu.com",
          "netloc": "thehindu.com",
          "path": "› sci-tech  › science  › 150-customs-duty-on-lab-chemicals-alarms-scientists  › article68465158.ece",
          "scheme": "https"
        },
        "page_age": "2024-07-30T23:18:00",
        "source_name": "The Hindu",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/efUVKzr_CvwMR3CIlcnSh_yOGKOwJkoPdM38OGSPIlg/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly90aC1p/LnRoZ2ltLmNvbS9w/dWJsaWMvaW5jb21p/bmcvNHUzdWs1L2Fy/dGljbGU2ODQ2NjE4/NS5lY2UvYWx0ZXJu/YXRlcy9MQU5EU0NB/UEVfMTIwMC9QTzE5/X0xhYl9zYW1wbGVz/LmpwZw"
        },
        "title": "150% customs duty on lab chemicals alarms scientists - The Hindu",
        "type": "news_result",
        "url": "https://www.thehindu.com/sci-tech/science/150-customs-duty-on-lab-chemicals-alarms-scientists/article68465158.ece"
      },
      {
        "age": "July 15, 2024",
        "description": "Union Budget 2024: The agrochemical sector in India is pushing for an increase in import duties to combat the influx of chemicals from China. Industry experts are advocating for tariffs to level the playing field for domestic players and address the trade deficit.",
        "meta_url": {
          "hostname": "m.economictimes.com",
          "netloc": "m.economictimes.com",
          "path": "› news  › economy  › policy  › budget-2024-agro-chemical-sector-seeks-hike-in-import-duties  › articleshow  › 111736462.cms",
          "scheme": "https"
        },
        "page_age": "2024-07-15T18:35:05",
        "source_name": "m.economictimes.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/D0Ec-qSM1b3xUV5kKHPGOuFehxSUBw2KwKBFAyJ6zC8/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWcu/ZXRpbWcuY29tL3Ro/dW1iL21zaWQtMTEx/NzM2NDg4LHdpZHRo/LTEyMDAsaGVpZ2h0/LTYzMCxpbWdzaXpl/LTEwMTEwNCxvdmVy/bGF5LWVjb25vbWlj/dGltZXMvcGhvdG8u/anBn"
        },
        "title": "Budget 2024: Agro-chemical sector seeks hike in import duties - The Economic Times",
        "type": "news_result",
        "url": "https://m.economictimes.com/news/economy/policy/budget-2024-agro-chemical-sector-seeks-hike-in-import-duties/articleshow/111736462.cms"
      },
      {
        "age": "June 26, 2024",
        "description": "“These stocks have either reversed from a long-term support or made a multiyear breakout retest which make them quite safe as compared to the stocks which are witnessing a breakout which can fail if the markets correct,” said InCreds VP, Gaurav Bissa, in a client note.",
        "meta_url": {
          "hostname": "m.economictimes.com",
          "netloc": "m.economictimes.com",
          "path": "› markets  › stocks  › news  › brokerage-view-chemical-stocks-ripe-for-fresh-up-cycle  › articleshow  › 111270903.cms",
          "scheme": "https"
        },
        "page_age": "2024-06-26T09:05:03",
        "source_name": "m.economictimes.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/GqRtEiyviXDP6MguofYwO3jl4EWYZYJl6Wyj_EaGV30/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWcu/ZXRpbWcuY29tL3Ro/dW1iL21zaWQtMTEx/MjcwODg2LHdpZHRo/LTEyMDAsaGVpZ2h0/LTYzMCxpbWdzaXpl/LTQ4NjQwLG92ZXJs/YXktZXRtYXJrZXRz/L3Bob3RvLmpwZw"
        },
        "title": "chemical stocks: Brokerage View: Chemical stocks ripe for fresh up-cycle - The Economic Times",
        "type": "news_result",
        "url": "https://m.economictimes.com/markets/stocks/news/brokerage-view-chemical-stocks-ripe-for-fresh-up-cycle/articleshow/111270903.cms"
      },
      {
        "age": "June 3, 2024",
        "description": "Regulating chemicals one-by-one has allowed the tobacco industry to skirt menthol bans by creating new additives with similar effects but unclear safety profiles",
        "meta_url": {
          "hostname": "www.scientificamerican.com",
          "netloc": "scientificamerican.com",
          "path": "› article  › how-tobacco-companies-use-chemistry-to-get-around-menthol-bans",
          "scheme": "https"
        },
        "page_age": "2024-06-03T13:00:00",
        "source_name": "Scientific American",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/72rWK2FjUyLGTVKABZyKNSK-9aPPMcc4KVbkuSR30xw/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9zdGF0/aWMuc2NpZW50aWZp/Y2FtZXJpY2FuLmNv/bS9kYW0vbS8xNGEw/NDEyODM4ZGRkMTMv/b3JpZ2luYWwvMlJF/NDBLNl9XRUIuanBn/P3c9MTIwMA"
        },
        "title": "How Tobacco Companies Use Chemistry to Get around Menthol Bans | Scientific American",
        "type": "news_result",
        "url": "https://www.scientificamerican.com/article/how-tobacco-companies-use-chemistry-to-get-around-menthol-bans/"
      },
      {
        "age": "May 30, 2024",
        "description": "The plastic industry is pitching chemical recycling as a great new hope in the battle against the plastic pollution crisis. Experts say not so fast",
        "meta_url": {
          "hostname": "www.cnn.com",
          "netloc": "cnn.com",
          "path": "› 2024  › 05  › 30  › climate  › chemical-recycling-plastic-pollution-climate  › index.html",
          "scheme": "https"
        },
        "page_age": "2024-05-30T08:00:15",
        "source_name": "CNN",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/JKOPReSeEiqJBpPyB9W_qKvTxxtYi24wM8W2UP76JII/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9tZWRp/YS5jbm4uY29tL2Fw/aS92MS9pbWFnZXMv/c3RlbGxhci9wcm9k/L2dldHR5aW1hZ2Vz/LTEyNTg1MTE4NDUu/anBnP2M9MTZ4OSZx/PXdfODAwLGNfZmls/bA"
        },
        "title": "The plastics industry says chemical recycling could help banish pollution. It’s ‘an illusion,’ critics say | CNN",
        "type": "news_result",
        "url": "https://www.cnn.com/2024/05/30/climate/chemical-recycling-plastic-pollution-climate/index.html"
      },
      {
        "age": "May 25, 2024",
        "description": "India's Petroleum, Chemical, and Petrochemical Investment Regions (PCPIRs) are expected to attract investments worth USD 420 billion, reflecting the sector's robust potential. Additionally, the establishment of seven Central Institutes of Petrochemicals Engineering & Technology (CIPET) and the Institute of Pesticide Formulation Technology (IPFT) will drive skill development, ensuring a skilled workforce to support the industry...",
        "meta_url": {
          "hostname": "m.economictimes.com",
          "netloc": "m.economictimes.com",
          "path": "› industry  › indl-goods  › svs  › chem-  › -fertilisers  › indias-chemicals-market-to-hit-29-7-bn-in-2024-set-for-steady-growth-with-3-26-cagr-through-2029  › articleshow  › 110418837.cms",
          "scheme": "https"
        },
        "page_age": "2024-05-25T07:30:02",
        "source_name": "m.economictimes.com",
        "thumbnail": {
          "src": "https://you.com/proxy?url=https://imgs.news.you.com/KNb5Pm02ITThd3icECdte7bUxKtEz53cELFDWN7YRh8/rs:fit:200:200:1:0/g:ce/aHR0cHM6Ly9pbWcu/ZXRpbWcuY29tL3Ro/dW1iL21zaWQtMTEw/NDE4ODU2LHdpZHRo/LTEyMDAsaGVpZ2h0/LTYzMCxpbWdzaXpl/LTI0MDc0LG92ZXJs/YXktZWNvbm9taWN0/aW1lcy9hcnRpY2xl/c2hvdy5qcGc"
        },
        "title": "chemicals market: India's chemicals market to hit $29.7 bn in 2024, set for steady growth with 3.26% CAGR through 2029 - The Economic Times",
        "type": "news_result",
        "url": "https://m.economictimes.com/industry/indl-goods/svs/chem-/-fertilisers/indias-chemicals-market-to-hit-29-7-bn-in-2024-set-for-steady-growth-with-3-26-cagr-through-2029/articleshow/110418837.cms"
      }
    ],
    "type": "news"
    }
    }
    

##

​

Explore further

## [Quickstart](/docs/quickstart)## [API Reference](/api-reference)

[Search API](/api-modes/search-api)[Custom APIs](/api-modes/custom-api)

[twitter](https://twitter.com/youdotcom)[linkedin](https://www.linkedin.com/company/youdotcom)

[Powered by Mintlify](https://mintlify.com/preview-
request?utm_campaign=poweredBy&utm_medium=docs&utm_source=documentation.you.com)

On this page

  * Stay Informed with the Latest Global News
  * Use Cases
  * Explore further

[You.com API home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/light.svg)![dark logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/dark.svg)](/)

Search or ask...

  * [Discord](https://discord.com/invite/youdotcom)
  * [Support](mailto:api@you.com)
  * [Support](mailto:api@you.com)

Search...

Navigation

API Reference

News

[Welcome](/welcome)[Quickstart](/docs/quickstart)[API Reference](/api-
reference/smart)[API Guide](/api-modes/smart-api)

##### API Reference

  * [POSTSmart API](/api-reference/smart)
  * [POSTResearch API](/api-reference/research)
  * [GETSearch](/api-reference/search)
  * [GETNews](/api-reference/news)

API Reference

# News

GET

/

news

Try it

cURL

Python

JavaScript

PHP

Go

Java

    
    
    curl --request GET \
      --url https://chat-api.you.com/news \
      --header 'X-API-Key: <api-key>'

200

    
    
    {
      "news": {
        "results": [
          {
            "url": "https://news.you.com",
            "title": "Breaking News about the World's Greatest Search Engine!",
            "description": "Search on YDC for the news",
            "type": "news",
            "age": "18 hours ago",
            "page_age": "2 days",
            "breaking": false,
            "page_fetched": "2023-10-12T23:00:00Z",
            "thumbnail": {
              "original": "https://reuters.com/news.jpg"
            },
            "meta_url": {
              "scheme": "https",
              "netloc": "reuters.com",
              "hostname": "www.reuters.com",
              "path": "› 2023  › 10  › 18  › politics  › inflation  › index.html"
            }
          }
        ]
      }
    }

**Before You Get Started**

To register for usage of our News API, please reach out via email at
[api@you.com](mailto:api@you.com).

#### Authorizations

​

X-API-Key

string

header

required

#### Query Parameters

​

query

string

required

Search query used to retrieve relevant results from index

​

count

integer

Specifies the maximum number of web results to return. Range `1 ≤
num_web_results ≤ 20`.

​

offset

integer

Indicates the `offset` for pagination. The `offset` is calculated in multiples
of `num_web_results`. For example, if `num_web_results = 5` and `offset = 1`,
results 5–10 will be returned. Range `0 ≤ offset ≤ 9`.

​

country

string

Country Code, one of `['AR', 'AU', 'AT', 'BE', 'BR', 'CA', 'CL', 'DK', 'FI',
'FR', 'DE', 'HK', 'IN', 'ID', 'IT', 'JP', 'KR', 'MY', 'MX', 'NL', 'NZ', 'NO',
'CN', 'PL', 'PT', 'PH', 'RU', 'SA', 'ZA', 'ES', 'SE', 'CH', 'TW', 'TR', 'GB',
'US']`.

​

search_lang

string

Language codes, one of `['ar', 'eu', 'bn', 'bg', 'ca', 'Simplified',
'Traditional', 'hr', 'cs', 'da', 'nl', 'en', 'United', 'et', 'fi', 'fr', 'gl',
'de', 'gu', 'he', 'hi', 'hu', 'is', 'it', 'jp', 'kn', 'ko', 'lv', 'lt', 'ms',
'ml', 'mr', 'Bokmål', 'pl', 'Brazil', 'Portugal', 'pa', 'ro', 'ru', 'Cyrylic',
'sk', 'sl', 'es', 'sv', 'ta', 'te', 'th', 'tr', 'uk', 'vi']`.

​

ui_lang

string

User interface language for the response, one of `['es-AR', 'en-AU', 'de-AT',
'nl-BE', 'fr-BE', 'pt-BR', 'en-CA', 'fr-CA', 'es-CL', 'da-DK', 'fi-FI', 'fr-
FR', 'de-DE', 'SAR', 'en-IN', 'en-ID', 'it-IT', 'ja-JP', 'ko-KR', 'en-MY',
'es-MX', 'nl-NL', 'English', 'no-NO', 'of', 'pl-PL', 'the', 'ru-RU',
'English', 'es-ES', 'sv-SE', 'fr-CH', 'de-CH', 'Chinese', 'tr-TR', 'English',
'English', 'Spanish']`.

​

safesearch

string

Configures the safesearch filter for content moderation. `off` \- no filtering
applied.`moderate` \- moderate content filtering (default). `strict` \- strict
content filtering.

​

spellcheck

boolean

Determine whether the `query` requires spell-checking. default is `true`.

​

recency

enum<string>

Specify the desired recency for the requested articles.

Available options:

`day`,

`week`,

`month`,

`year`

#### Response

200 - application/json

A JSON object containing array of news results

​

news

object

Show child attributes

​

news.results

object[]

Show child attributes

​

news.results.url

string

​

news.results.title

string

​

news.results.description

string

​

news.results.type

string

​

news.results.age

string

​

news.results.page_age

string

​

news.results.breaking

boolean

​

news.results.page_fetched

string

​

news.results.thumbnail

object

Show child attributes

​

news.results.thumbnail.original

string

​

news.results.meta_url

object

Show child attributes

​

news.results.meta_url.scheme

string

​

news.results.meta_url.netloc

string

​

news.results.meta_url.hostname

string

​

news.results.meta_url.path

string

[Search](/api-reference/search)

[twitter](https://twitter.com/youdotcom)[linkedin](https://www.linkedin.com/company/youdotcom)

[Powered by Mintlify](https://mintlify.com/preview-
request?utm_campaign=poweredBy&utm_medium=docs&utm_source=documentation.you.com)

cURL

Python

JavaScript

PHP

Go

Java

    
    
    curl --request GET \
      --url https://chat-api.you.com/news \
      --header 'X-API-Key: <api-key>'

200

    
    
    {
      "news": {
        "results": [
          {
            "url": "https://news.you.com",
            "title": "Breaking News about the World's Greatest Search Engine!",
            "description": "Search on YDC for the news",
            "type": "news",
            "age": "18 hours ago",
            "page_age": "2 days",
            "breaking": false,
            "page_fetched": "2023-10-12T23:00:00Z",
            "thumbnail": {
              "original": "https://reuters.com/news.jpg"
            },
            "meta_url": {
              "scheme": "https",
              "netloc": "reuters.com",
              "hostname": "www.reuters.com",
              "path": "› 2023  › 10  › 18  › politics  › inflation  › index.html"
            }
          }
        ]
      }
    }

================
File: plugins/repos/twat_search/resources/you/you_news.txt
================
https://documentation.you.com/api-modes/news-api
https://documentation.you.com/api-reference/news

================
File: plugins/repos/twat_search/resources/you/you.md
================
Loading...

[You.com API home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/light.svg)![dark logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/dark.svg)](/)

Search or ask...

  * [Discord](https://discord.com/invite/youdotcom)
  * [Support](mailto:api@you.com)
  * [Support](mailto:api@you.com)

Search...

Navigation

Initial Setup

Quickstart

[Welcome](/welcome)[Quickstart](/docs/quickstart)[API Reference](/api-
reference/smart)[API Guide](/api-modes/smart-api)

##### Initial Setup

  * [Quickstart](/docs/quickstart)

##### More Examples

  * [Open Source Examples](/docs/opensource-examples)

Initial Setup

# Quickstart

##

​

Introduction

Welcome to the Quickstart Guide for integrating comprehensive, high-quality
answers with precise and reliable citations using our [Smart](/api-
modes/smart-api), [Research](/api-modes/research-api), [Search](/api-
modes/search-api) and [News](/api-modes/news-api) APIs. This guide will walk
you through the initial setup and provide you with sample code to perform
searches and retrieve results.

##

​

Step 1: Set Up Your API Key

**Before You Get Started**

To use the You.com Smart, Research, Search and News LLM endpoints, you can get
an API key through the self-serve portal at
[api.you.com](https://api.you.com). For support, please reach out via email at
[api@you.com](mailto:api@you.com).

Replace `X-API-Key` in the code with your actual API key:

API Key

    
    
    YOUR_API_KEY = "your_actual_api_key_here"
    

##

​

Step 2: Write the Search Function

Create a function to interact with the Research API:

  * Smart API
  * Research API
  * Search API
  * News API

Learn more about the [Smart API ](/api-modes/smart-api).

smart_api.py

    
    
    import requests
    
    def get_smart_results(query):
        headers = {"X-API-Key": YOUR_API_KEY}
        params = {"query": query, "instructions": instructions}
        return requests.get(
            "https://chat-api.you.com/smart?query={query}",
            params=params,
            headers=headers,
        ).json()
    

Use the function to search for AI snippets related to a specific topic:

  * Smart API
  * Research API
  * Search API
  * News API

smart_results.py

    
    
    get_smart_results("Who won the Nobel Prize in Physics in 2024?")
    

Answer

answer

    
    
    {
    "answer":"#### The 2024 Nobel Prize in Physics\n\n
    The 2024 Nobel Prize in Physics was awarded jointly to **John J. Hopfield**
    and **Geoffrey E. Hinton** \"for foundational discoveries and inventions that
    enable machine learning with artificial neural networks\".
    [[1]](https://www.nobelprize.org/prizes/physics/2024/summary/) [[2]](https://www.nobelprize.org/prizes/physics/2024/press-release/)\n\n      
    Hopfield and Hinton are pioneers in the field of artificial intelligence and
    machine learning. Their work in the 1980s laid the groundwork for the development of
    modern neural networks and deep learning algorithms, which are now widely used in
    various applications.
    [[3]](https://www.nobelprize.org/prizes/physics/2024/popular-information/) [[4]](https://spectrum.ieee.org/nobel-prize-in-physics)\n\n
    Specifically, Hopfield created a structure that can store and reconstruct information, while Hinton
    invented a method that can autonomously find properties in data, which are key
    innovations that make artificial intelligence work.
    [[5]](https://www.weforum.org/stories/2024/10/nobel-prize-winners-2024/)\n\n        
    The Nobel Prize committee recognized their \"foundational discoveries
    and inventions that enable machine learning with artificial neural networks\",
    highlighting how their work has been instrumental in the rapid progress of AI and machine
    learning in recent years.
    [[1]](https://www.nobelprize.org/prizes/physics/2024/summary/) [[2]](https://www.nobelprize.org/prizes/physics/2024/press-release/)"     
    
    "search_results":[
    {
    "url":"https://www.nobelprize.org/prizes/physics/2024/summary/",
    "name":"The Nobel Prize in Physics 2024 - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics 2024 was awarded jointly to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/physics/2024/press-release/",
    "name":"Press release: The Nobel Prize in Physics 2024 - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics 2024 was awarded jointly to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/physics/",
    "name":"Nobel Prize in Physics",
    "snippet":"The Nobel Prize medal. ... A slide rule that physics laureate Toshihide Maskawa used as a high school student.",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/all-nobel-prizes-2024/",
    "name":"All Nobel Prizes 2024 - NobelPrize.org",
    "snippet":"Ill. Niklas Elmehed © Nobel Prize Outreach · This year’s laureates used tools from physics to construct methods that helped lay the foundation for today’s powerful machine learning. John Hopfield created a structure that can store and reconstruct information.",
    "metadata":"None"
    },
    {
    "url":"https://www.aps.org/about/news/2024/10/nobel-physics-2024-winners",
    "name":"APS congratulates 2024 Nobel Prize winners",
    "snippet":"The latest news and announcements about APS and the global physics community.",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/physics/2024/popular-information/",
    "name":"The Nobel Prize in Physics 2024 - Popular science background - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics 2024 was awarded jointly to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "metadata":"None"
    },
    {
    "url":"https://time.com/7065011/nobel-prize-2024-winners/",
    "name":"These Are the Winners of the 2024 Nobel Prizes",
    "snippet":"Victor Ambros and Gary Ruvkun were on Monday awarded the 2024 Nobel Prize in Physiology or Medicine for their discovery of microRNAs, a class of small molecules essential for gene regulation. Their research has uncovered how these microRNAs influence cellular behavior and contribute to various ...",
    "metadata":"None"
    },
    {
    "url":"https://www.reuters.com/science/hopfield-hinton-win-2024-nobel-prize-physics-2024-10-08/",
    "name":"Nobel physics prize 2024 won by AI pioneers John Hopfield and Geoffrey Hinton | Reuters",
    "snippet":"[1/6]John J Hopfield and Geoffrey E Hinton are awarded this year's Nobel Prize in Physics, announced at a press conference at the Royal Swedish Academy of Sciences in Stockholm, Sweden October 8, 2024.",
    "metadata":"None"
    },
    {
    "url":"https://www.weforum.org/stories/2024/10/nobel-prize-winners-2024/",
    "name":"These are the Nobel Prize winners of 2024 | World Economic Forum",
    "snippet":"AI pioneers John Hopfield and Geoffrey Hinton were both awarded the Physics prize, for using tools to develop methods that are the foundation of today’s machine learning. Widely credited as \"a godfather of AI\", British-Canadian Hinton invented a method that can autonomously find properties ...",
    "metadata":"None"
    },
    {
    "url":"https://new.nsf.gov/news/nsf-congratulates-laureates-2024-nobel-prize-physics",
    "name":"NSF congratulates laureates of the 2024 Nobel Prize in physics | NSF - National Science Foundation",
    "snippet":"Two researchers used fundamental knowledge of the physical properties of materials to create key innovations that make artificial intelligence work ... The U.S. National Science Foundation congratulates John J. Hopfield and Geoffrey E. Hinton for their Nobel Prize in physics.",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/lists/all-nobel-prizes-in-physics/",
    "name":"All Nobel Prizes in Physics - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics has been awarded 118 times to 227 Nobel Prize laureates between 1901 and 2024. John Bardeen is the only laureate who has been awarded the Nobel Prize in Physics twice, in 1956 and 1972. This means that a total of 226 individuals have received the Nobel Prize in Physics.",
    "metadata":"None"
    },
    {
    "url":"https://en.wikipedia.org/wiki/List_of_Nobel_laureates_in_Physics",
    "name":"List of Nobel laureates in Physics - Wikipedia",
    "snippet":"The Nobel Prize in Physics has ... as of 2024. The first prize in physics was awarded in 1901 to Wilhelm Conrad Röntgen, of Germany, who received 150,782 SEK. John Bardeen is the only laureate to win the prize twice—in 1956 and 1972. William Lawrence Bragg was the youngest Nobel laureate in physics; he won the prize ...",
    "metadata":"None"
    },
    {
    "url":"https://www.reddit.com/r/math/comments/1fyzz6t/the_nobel_prize_in_physics_2024_was_awarded_to/",
    "name":"r/math on Reddit: The Nobel Prize in Physics 2024 was awarded to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "snippet":"I think the Boltzmann machine is a really beautiful model, even from the mathematical point of view. I’m still a little bit shocked when I learned that the Nobel Prize in Physics 2024 goes to ML/DL, as much as I also like (theoretical) computer science.",
    "metadata":"None"
    },
    {
    "url":"https://spectrum.ieee.org/nobel-prize-in-physics",
    "name":"Why the Nobel Prize in Physics Went to AI Research",
    "snippet":"The Nobel Prize Committee for Physics caught the academic community off-guard by handing the 2024 award to John J. Hopfield and Geoffrey E. Hinton for their foundational work in neural networks. The pair won the prize for their seminal papers, both published in the 1980s, that described rudimentary ...",
    "metadata":"None"
    },
    {
    "url":"https://www.nytimes.com/2024/10/08/science/nobel-prize-physics.html",
    "name":"Nobel Physics Prize Awarded for Pioneering A.I. Research by 2 Scientists - The New York Times",
    "snippet":"With work on machine learning that uses artificial neural networks, John J. Hopfield and Geoffrey E. Hinton “showed a completely new way for us to use computers,” the committee said.",
    "metadata":"None"
    },
    {
    "url":"https://www.reuters.com/world/nobel-prize-2024-live-physics-award-be-announced-2024-10-08/",
    "name":"Nobel Physics Prize 2024: Winners are machine learning pioneers Hopfield and Hinton - as it happened | Reuters",
    "snippet":"The award-giving body said the pair used tools from physics to develop methods \"that are the foundation of today\\'s powerful machine learning.\"",
    "metadata":"None"
    },
    {
    "url":"https://finshots.in/archive/whats-up-with-ai-and-the-2024-physics-nobel-prize/",
    "name":"What's up with AI and the 2024 Physics Nobel Prize?",
    "snippet":"An explainer of why two pioneers of AI, John Hopfield and Geoffrey Hinton, were awarded the 2024 Nobel Prize in Physics.",
    "metadata":"None"
    },
    {
    "url":"https://www.pbs.org/newshour/science/watch-live-the-winner-of-the-2024-nobel-prize-in-physics-is",
    "name":"WATCH: AI pioneers John Hopfield and Geoffrey Hinton win 2024 Nobel Prize in physics | PBS News",
    "snippet":"Hinton, who is known as the Godfather of artificial intelligence, is a citizen of Canada and Britain who works at the University of Toronto and Hopfield is an American working at Princeton.",
    "metadata":"None"
    },
    {
    "url":"https://en.wikipedia.org/wiki/Nobel_Prize_in_Physics",
    "name":"Nobel Prize in Physics - Wikipedia",
    "snippet":"The first Nobel Prize in Physics was awarded to German physicist Wilhelm Röntgen in recognition of the extraordinary services he rendered by the discovery of X-rays. This award is administered by the Nobel Foundation and is widely regarded as the most prestigious award that a scientist can receive in physics. It is presented in Stockholm at an annual ceremony on the 10th of December, the anniversary of Nobel's death. As of 2024...",
    "metadata":"None"
    },
    {
    "url":"https://www.jagranjosh.com/general-knowledge/list-of-2024-nobel-prize-winners-in-all-categories-1728376617-1",
    "name":"Nobel Prize 2024 Winners List: Recipient Name, Achievement from All Categories",
    "snippet":"Discover the complete list of Nobel Prize 2024 winners. Stay updated on the latest achievements and contributions recognized by the Nobel Committee this year.",
    "metadata":"None"
    },
    {
    "url":"https://www.artsci.utoronto.ca/news/geoffrey-hinton-wins-2024-nobel-prize-physics",
    "name":"Geoffrey Hinton wins 2024 Nobel Prize in Physics | Faculty of Arts & Science",
    "snippet":"Geoffrey Hinton, a University Professor Emeritus of the Department of Computer Science at the University of Toronto, has been awarded the 2024 Nobel Prize in Physics.",
    "metadata":"None"
    },
    {
    "url":"https://www.ap.org/news-highlights/spotlights/2024/pioneers-in-artificial-intelligence-win-the-nobel-prize-in-physics/",
    "name":"Pioneers in artificial intelligence win the Nobel Prize in physics | The Associated Press",
    "snippet":"This photo combo shows the 2024 Nobel Prize winners in Physics, professor John Hopfield, left, of Princeton University, and professor Geoffrey Hinton, of the University of Toronto, Tuesday, Oct. 8, 2024. (Princeton University via AP and Noah Berger/AP Photo) STOCKHOLM (AP) — Two pioneers of artificial intelligence — John Hopfield and Geoffrey Hinton — won ...",
    "metadata":"None"
    },
    {
    "url":"https://twitter.com/NobelPrize",
    "name":"The Nobel Prize (@NobelPrize) · X",
    "snippet":"The latest tweets from The Nobel Prize (@NobelPrize)",
    "metadata":"None"
    },
    {
    "url":"https://www.aljazeera.com/news/2024/10/8/john-hopfield-and-geoffrey-hinton-win-nobel-prize-in-physics-2024",
    "name":"AI scientists John Hopfield, Geoffrey Hinton win 2024 physics Nobel Prize | Science and Technology News | Al Jazeera",
    "snippet":"John Hopfield and Geoffrey Hinton have won the Nobel Prize in physics 2024 for their pioneering work in the field of machine learning.",
    "metadata":"None"
    }
    ]
    }
    
    
    

##

​

Explore our APIs

Unlock new possibilities with our suite of advanced APIs tailored to meet your
needs and explore more use cases.

## [Smart API](/api-modes/smart-api)## [Research API](/api-modes/research-
api)## [News API](/api-modes/news-api)## [Search API](/api-modes/search-api)

[Open Source Examples](/docs/opensource-examples)

[twitter](https://twitter.com/youdotcom)[linkedin](https://www.linkedin.com/company/youdotcom)

[Powered by Mintlify](https://mintlify.com/preview-
request?utm_campaign=poweredBy&utm_medium=docs&utm_source=documentation.you.com)

On this page

  * Introduction
  * Step 1: Set Up Your API Key
  * Step 2: Write the Search Function
  * Explore our APIs

[You.com API home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/light.svg)![dark logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/dark.svg)](/)

Search or ask...

  * [Discord](https://discord.com/invite/youdotcom)
  * [Support](mailto:api@you.com)
  * [Support](mailto:api@you.com)

Search...

Navigation

Initial Setup

Quickstart

[Welcome](/welcome)[Quickstart](/docs/quickstart)[API Reference](/api-
reference/smart)[API Guide](/api-modes/smart-api)

##### Initial Setup

  * [Quickstart](/docs/quickstart)

##### More Examples

  * [Open Source Examples](/docs/opensource-examples)

Initial Setup

# Quickstart

##

​

Introduction

Welcome to the Quickstart Guide for integrating comprehensive, high-quality
answers with precise and reliable citations using our [Smart](/api-
modes/smart-api), [Research](/api-modes/research-api), [Search](/api-
modes/search-api) and [News](/api-modes/news-api) APIs. This guide will walk
you through the initial setup and provide you with sample code to perform
searches and retrieve results.

##

​

Step 1: Set Up Your API Key

**Before You Get Started**

To use the You.com Smart, Research, Search and News LLM endpoints, you can get
an API key through the self-serve portal at
[api.you.com](https://api.you.com). For support, please reach out via email at
[api@you.com](mailto:api@you.com).

Replace `X-API-Key` in the code with your actual API key:

API Key

    
    
    YOUR_API_KEY = "your_actual_api_key_here"
    

##

​

Step 2: Write the Search Function

Create a function to interact with the Research API:

  * Smart API
  * Research API
  * Search API
  * News API

Learn more about the [Smart API ](/api-modes/smart-api).

smart_api.py

    
    
    import requests
    
    def get_smart_results(query):
        headers = {"X-API-Key": YOUR_API_KEY}
        params = {"query": query, "instructions": instructions}
        return requests.get(
            "https://chat-api.you.com/smart?query={query}",
            params=params,
            headers=headers,
        ).json()
    

Use the function to search for AI snippets related to a specific topic:

  * Smart API
  * Research API
  * Search API
  * News API

smart_results.py

    
    
    get_smart_results("Who won the Nobel Prize in Physics in 2024?")
    

Answer

answer

    
    
    {
    "answer":"#### The 2024 Nobel Prize in Physics\n\n
    The 2024 Nobel Prize in Physics was awarded jointly to **John J. Hopfield**
    and **Geoffrey E. Hinton** \"for foundational discoveries and inventions that
    enable machine learning with artificial neural networks\".
    [[1]](https://www.nobelprize.org/prizes/physics/2024/summary/) [[2]](https://www.nobelprize.org/prizes/physics/2024/press-release/)\n\n      
    Hopfield and Hinton are pioneers in the field of artificial intelligence and
    machine learning. Their work in the 1980s laid the groundwork for the development of
    modern neural networks and deep learning algorithms, which are now widely used in
    various applications.
    [[3]](https://www.nobelprize.org/prizes/physics/2024/popular-information/) [[4]](https://spectrum.ieee.org/nobel-prize-in-physics)\n\n
    Specifically, Hopfield created a structure that can store and reconstruct information, while Hinton
    invented a method that can autonomously find properties in data, which are key
    innovations that make artificial intelligence work.
    [[5]](https://www.weforum.org/stories/2024/10/nobel-prize-winners-2024/)\n\n        
    The Nobel Prize committee recognized their \"foundational discoveries
    and inventions that enable machine learning with artificial neural networks\",
    highlighting how their work has been instrumental in the rapid progress of AI and machine
    learning in recent years.
    [[1]](https://www.nobelprize.org/prizes/physics/2024/summary/) [[2]](https://www.nobelprize.org/prizes/physics/2024/press-release/)"     
    
    "search_results":[
    {
    "url":"https://www.nobelprize.org/prizes/physics/2024/summary/",
    "name":"The Nobel Prize in Physics 2024 - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics 2024 was awarded jointly to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/physics/2024/press-release/",
    "name":"Press release: The Nobel Prize in Physics 2024 - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics 2024 was awarded jointly to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/physics/",
    "name":"Nobel Prize in Physics",
    "snippet":"The Nobel Prize medal. ... A slide rule that physics laureate Toshihide Maskawa used as a high school student.",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/all-nobel-prizes-2024/",
    "name":"All Nobel Prizes 2024 - NobelPrize.org",
    "snippet":"Ill. Niklas Elmehed © Nobel Prize Outreach · This year’s laureates used tools from physics to construct methods that helped lay the foundation for today’s powerful machine learning. John Hopfield created a structure that can store and reconstruct information.",
    "metadata":"None"
    },
    {
    "url":"https://www.aps.org/about/news/2024/10/nobel-physics-2024-winners",
    "name":"APS congratulates 2024 Nobel Prize winners",
    "snippet":"The latest news and announcements about APS and the global physics community.",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/physics/2024/popular-information/",
    "name":"The Nobel Prize in Physics 2024 - Popular science background - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics 2024 was awarded jointly to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "metadata":"None"
    },
    {
    "url":"https://time.com/7065011/nobel-prize-2024-winners/",
    "name":"These Are the Winners of the 2024 Nobel Prizes",
    "snippet":"Victor Ambros and Gary Ruvkun were on Monday awarded the 2024 Nobel Prize in Physiology or Medicine for their discovery of microRNAs, a class of small molecules essential for gene regulation. Their research has uncovered how these microRNAs influence cellular behavior and contribute to various ...",
    "metadata":"None"
    },
    {
    "url":"https://www.reuters.com/science/hopfield-hinton-win-2024-nobel-prize-physics-2024-10-08/",
    "name":"Nobel physics prize 2024 won by AI pioneers John Hopfield and Geoffrey Hinton | Reuters",
    "snippet":"[1/6]John J Hopfield and Geoffrey E Hinton are awarded this year's Nobel Prize in Physics, announced at a press conference at the Royal Swedish Academy of Sciences in Stockholm, Sweden October 8, 2024.",
    "metadata":"None"
    },
    {
    "url":"https://www.weforum.org/stories/2024/10/nobel-prize-winners-2024/",
    "name":"These are the Nobel Prize winners of 2024 | World Economic Forum",
    "snippet":"AI pioneers John Hopfield and Geoffrey Hinton were both awarded the Physics prize, for using tools to develop methods that are the foundation of today’s machine learning. Widely credited as \"a godfather of AI\", British-Canadian Hinton invented a method that can autonomously find properties ...",
    "metadata":"None"
    },
    {
    "url":"https://new.nsf.gov/news/nsf-congratulates-laureates-2024-nobel-prize-physics",
    "name":"NSF congratulates laureates of the 2024 Nobel Prize in physics | NSF - National Science Foundation",
    "snippet":"Two researchers used fundamental knowledge of the physical properties of materials to create key innovations that make artificial intelligence work ... The U.S. National Science Foundation congratulates John J. Hopfield and Geoffrey E. Hinton for their Nobel Prize in physics.",
    "metadata":"None"
    },
    {
    "url":"https://www.nobelprize.org/prizes/lists/all-nobel-prizes-in-physics/",
    "name":"All Nobel Prizes in Physics - NobelPrize.org",
    "snippet":"The Nobel Prize in Physics has been awarded 118 times to 227 Nobel Prize laureates between 1901 and 2024. John Bardeen is the only laureate who has been awarded the Nobel Prize in Physics twice, in 1956 and 1972. This means that a total of 226 individuals have received the Nobel Prize in Physics.",
    "metadata":"None"
    },
    {
    "url":"https://en.wikipedia.org/wiki/List_of_Nobel_laureates_in_Physics",
    "name":"List of Nobel laureates in Physics - Wikipedia",
    "snippet":"The Nobel Prize in Physics has ... as of 2024. The first prize in physics was awarded in 1901 to Wilhelm Conrad Röntgen, of Germany, who received 150,782 SEK. John Bardeen is the only laureate to win the prize twice—in 1956 and 1972. William Lawrence Bragg was the youngest Nobel laureate in physics; he won the prize ...",
    "metadata":"None"
    },
    {
    "url":"https://www.reddit.com/r/math/comments/1fyzz6t/the_nobel_prize_in_physics_2024_was_awarded_to/",
    "name":"r/math on Reddit: The Nobel Prize in Physics 2024 was awarded to John J. Hopfield and Geoffrey E. Hinton \"for foundational discoveries and inventions that enable machine learning with artificial neural networks\"",
    "snippet":"I think the Boltzmann machine is a really beautiful model, even from the mathematical point of view. I’m still a little bit shocked when I learned that the Nobel Prize in Physics 2024 goes to ML/DL, as much as I also like (theoretical) computer science.",
    "metadata":"None"
    },
    {
    "url":"https://spectrum.ieee.org/nobel-prize-in-physics",
    "name":"Why the Nobel Prize in Physics Went to AI Research",
    "snippet":"The Nobel Prize Committee for Physics caught the academic community off-guard by handing the 2024 award to John J. Hopfield and Geoffrey E. Hinton for their foundational work in neural networks. The pair won the prize for their seminal papers, both published in the 1980s, that described rudimentary ...",
    "metadata":"None"
    },
    {
    "url":"https://www.nytimes.com/2024/10/08/science/nobel-prize-physics.html",
    "name":"Nobel Physics Prize Awarded for Pioneering A.I. Research by 2 Scientists - The New York Times",
    "snippet":"With work on machine learning that uses artificial neural networks, John J. Hopfield and Geoffrey E. Hinton “showed a completely new way for us to use computers,” the committee said.",
    "metadata":"None"
    },
    {
    "url":"https://www.reuters.com/world/nobel-prize-2024-live-physics-award-be-announced-2024-10-08/",
    "name":"Nobel Physics Prize 2024: Winners are machine learning pioneers Hopfield and Hinton - as it happened | Reuters",
    "snippet":"The award-giving body said the pair used tools from physics to develop methods \"that are the foundation of today\\'s powerful machine learning.\"",
    "metadata":"None"
    },
    {
    "url":"https://finshots.in/archive/whats-up-with-ai-and-the-2024-physics-nobel-prize/",
    "name":"What's up with AI and the 2024 Physics Nobel Prize?",
    "snippet":"An explainer of why two pioneers of AI, John Hopfield and Geoffrey Hinton, were awarded the 2024 Nobel Prize in Physics.",
    "metadata":"None"
    },
    {
    "url":"https://www.pbs.org/newshour/science/watch-live-the-winner-of-the-2024-nobel-prize-in-physics-is",
    "name":"WATCH: AI pioneers John Hopfield and Geoffrey Hinton win 2024 Nobel Prize in physics | PBS News",
    "snippet":"Hinton, who is known as the Godfather of artificial intelligence, is a citizen of Canada and Britain who works at the University of Toronto and Hopfield is an American working at Princeton.",
    "metadata":"None"
    },
    {
    "url":"https://en.wikipedia.org/wiki/Nobel_Prize_in_Physics",
    "name":"Nobel Prize in Physics - Wikipedia",
    "snippet":"The first Nobel Prize in Physics was awarded to German physicist Wilhelm Röntgen in recognition of the extraordinary services he rendered by the discovery of X-rays. This award is administered by the Nobel Foundation and is widely regarded as the most prestigious award that a scientist can receive in physics. It is presented in Stockholm at an annual ceremony on the 10th of December, the anniversary of Nobel's death. As of 2024...",
    "metadata":"None"
    },
    {
    "url":"https://www.jagranjosh.com/general-knowledge/list-of-2024-nobel-prize-winners-in-all-categories-1728376617-1",
    "name":"Nobel Prize 2024 Winners List: Recipient Name, Achievement from All Categories",
    "snippet":"Discover the complete list of Nobel Prize 2024 winners. Stay updated on the latest achievements and contributions recognized by the Nobel Committee this year.",
    "metadata":"None"
    },
    {
    "url":"https://www.artsci.utoronto.ca/news/geoffrey-hinton-wins-2024-nobel-prize-physics",
    "name":"Geoffrey Hinton wins 2024 Nobel Prize in Physics | Faculty of Arts & Science",
    "snippet":"Geoffrey Hinton, a University Professor Emeritus of the Department of Computer Science at the University of Toronto, has been awarded the 2024 Nobel Prize in Physics.",
    "metadata":"None"
    },
    {
    "url":"https://www.ap.org/news-highlights/spotlights/2024/pioneers-in-artificial-intelligence-win-the-nobel-prize-in-physics/",
    "name":"Pioneers in artificial intelligence win the Nobel Prize in physics | The Associated Press",
    "snippet":"This photo combo shows the 2024 Nobel Prize winners in Physics, professor John Hopfield, left, of Princeton University, and professor Geoffrey Hinton, of the University of Toronto, Tuesday, Oct. 8, 2024. (Princeton University via AP and Noah Berger/AP Photo) STOCKHOLM (AP) — Two pioneers of artificial intelligence — John Hopfield and Geoffrey Hinton — won ...",
    "metadata":"None"
    },
    {
    "url":"https://twitter.com/NobelPrize",
    "name":"The Nobel Prize (@NobelPrize) · X",
    "snippet":"The latest tweets from The Nobel Prize (@NobelPrize)",
    "metadata":"None"
    },
    {
    "url":"https://www.aljazeera.com/news/2024/10/8/john-hopfield-and-geoffrey-hinton-win-nobel-prize-in-physics-2024",
    "name":"AI scientists John Hopfield, Geoffrey Hinton win 2024 physics Nobel Prize | Science and Technology News | Al Jazeera",
    "snippet":"John Hopfield and Geoffrey Hinton have won the Nobel Prize in physics 2024 for their pioneering work in the field of machine learning.",
    "metadata":"None"
    }
    ]
    }
    
    
    

##

​

Explore our APIs

Unlock new possibilities with our suite of advanced APIs tailored to meet your
needs and explore more use cases.

## [Smart API](/api-modes/smart-api)## [Research API](/api-modes/research-
api)## [News API](/api-modes/news-api)## [Search API](/api-modes/search-api)

[Open Source Examples](/docs/opensource-examples)

[twitter](https://twitter.com/youdotcom)[linkedin](https://www.linkedin.com/company/youdotcom)

[Powered by Mintlify](https://mintlify.com/preview-
request?utm_campaign=poweredBy&utm_medium=docs&utm_source=documentation.you.com)

On this page

  * Introduction
  * Step 1: Set Up Your API Key
  * Step 2: Write the Search Function
  * Explore our APIs

[You.com API home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/light.svg)![dark logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/dark.svg)](/)

Search or ask...

  * [Discord](https://discord.com/invite/youdotcom)
  * [Support](mailto:api@you.com)
  * [Support](mailto:api@you.com)

Search...

Navigation

API Guide

Search API

[Welcome](/welcome)[Quickstart](/docs/quickstart)[API Reference](/api-
reference/smart)[API Guide](/api-modes/smart-api)

##### API Guide

  * [Smart API](/api-modes/smart-api)
  * [Research API](/api-modes/research-api)
  * [Search API](/api-modes/search-api)
  * [News API](/api-modes/news-api)

##### Custom

  * [Custom APIs](/api-modes/custom-api)

API Guide

# Search API

##

​

Accurate and Real-time Web Data

When trying to build applications that rely on integrating real-time web data,
solutions are very limited. LLMs are generally unable to extract and deliver
only snippets from sources without adding additional AI-generated content.

Our **Search API** provides you with direct snippets and URLs to stay
informed, ensuring an accurate and up-to-date understanding of the world.

## Access to Trusted Data

Our API integrates live web data, providing results from trusted sources
complete with URLs for verification.

## Uniquely Long Snippets

Ensure your responses are trustworthy and contain the information you need.

##

​

Use Cases

## Information from Trusted Sources

  

Scientific Articles

query.py

    
    
    import requests
    
    url = "https://api.ydc-index.io/search"
    
    query = {"query":"Search for Scientific Research Articles on Nanomotors for Cleaning Polluted Water"}
    
    headers = {"X-API-Key": "YOUR_API_KEY"}
    
    response = requests.request("GET", url, headers=headers, params=query)
    
    print(response.text)
    

Response

    
    
    {
    "hits": [
      {
        "description": "Self-propelled nanomotors hold considerable promise for developing innovative environmental applications.Self-propelled nanomotors hold considerable promise for developing innovative environmental applications. This review highlights the recent progress ...",
        "snippets": [
          "In addition, those nanoparticles cannot transport ions and pollutants from one place to another. Catalytically powered micro- and nanomotors have attracted a lot of attention over the last few years in multidisciplinary fields of chemistry and physics.5 Since the pioneering works a decade ago, synthetic nanomotors demonstrated the ability to efficiently convert chemical energy into motion like nature uses biochemistry to power biological motors.6,7 Fundamental research is being conducted in this field and a number of interesting applications are opening up in several different fields, such as",
          "The surface modification of some types of nanomotors allows them to capture oil from contaminated waters. Research by Pumera and co-workers described a sodium dodecyl sulfate (SDS)-loaded polysulfone (PSf) capsule that was used to shepherd several oil droplets and to merge them, cleaning the surface of the water.36 The driving force of self-propulsion is based in the Marangoni effect.",
          "These “self-powered remediation systems” could be seen as a new generation of “smart devices” for cleaning water in small pipes or cavities difficult to reach with traditional methods. With constant improvement and considering the key challenges, we expect that artificial nanomachines could play an important role in environmental applications in the near future. Pollution of water by contaminants and chemical threats is a prevalent topic in scientific, economic, political and, consequently, in the public media.",
          "Researchers and engineers are devoting considerable effort to produce more efficient technological solutions for cleaning environmental pollutants."
        ],
        "title": "Catalytic nanomotors for environmental monitoring and water remediation - PMC",
        "url": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4080807/"
      },
      {
        "description": "The most critical challenge of the twenty-first century is to provide sufficient clean, cheap water for all. This is made worse by population increase…",
        "snippets": [
          "The most critical challenge of the twenty-first century is to provide sufficient clean, cheap water for all. This is made worse by population increase, climate change, and declining water quality. Technology innovation, such as nanotechnology, is essential for enabling integrated water management to increase treatment effectiveness and expand water supplies using unconventional water sources.",
          "Nanotechnology can improve access to clean, safe drinking water by providing innovative nanomaterials for treating surface water, groundwater, and wastewater contaminated by hazardous metal ions, inorganic and organic solutes, and microorganisms. As a result, the development of nanotechnology provided ground-breaking solutions to issues in engineering, physics, chemistry, and others.",
          "Considering the essential need to examine and handle the developing hazardous wastes with lower costs, less energy, and more efficiency, this review shines a light on the current advancements in nanotechnology. Numerous industries, such as scientific research, the medical field, and the food industry, have paid close attention to the expanding significance of nanotechnology and the unique qualities of nanobubbles."
        ],
        "title": "Smart and innovative nanotechnology applications for water purification - ScienceDirect",
        "url": "https://www.sciencedirect.com/science/article/pii/S2773207X23000271"
      },
      {
        "description": "We describe the use of catalytically self-propelled microjets (dubbed micromotors) for degrading organic pollutants in water via the Fenton oxidation process. The tubular micromotors are composed of rolled-up functional nanomembranes consisting of Fe/Pt ...",
        "snippets": [
          "Great efforts have been made to efficiently propel and accurately control micro- and nanomotors by different mechanisms.29−37 Most self-propelled systems are based on the conversion of chemical energy into mechanical motion.38 Nonetheless, there are also other ways to produce self-motion at the micro- and nanoscale, for instance electromagnetic fields,22,39,40 local electrical fields,41 thermal gradients,42,43 photoinduced motion,44−46 or the Marangoni effect.28 This variety of propulsion mechanisms gave rise to a rich diversity of designs of nanomotors such as nanorods,47,48 spherical particles,34,49 microhelices,22,39,40 polymeric capsules,28,50 and tubular microjets.51−53",
          "Paxton W. F.; Kistler K. C.; Olmeda C. C.; Sen A.; St Angelo S. K.; Cao Y. Y.; Mallouk T. E.; Lammert P. E.; Crespi V. H. Catalytic Nanomotors: Autonomous Movement of Striped Nanorods. J. Am. Chem. Soc. 2004, 126, 13424–13431.",
          "Self-propelled microjets have been fabricated by roll-up nanotechnology of thin films51,52 and later produced in porous templates combined with electrodeposition methods.53 However, in the latter case, parameters such as shape, length, and diameter are limited by the commercially available templates, reducing the versatility in the design of those nanomotors.",
          "Differently, roll-up nanotechnology of functional nanomembranes allows a reproducible mass production method54 of micro/nanomotors with custom-made dimensions, flexible in material composition and design."
        ],
        "title": "Self-Propelled Micromotors for Cleaning Polluted Water - PMC",
        "url": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3872448/"
      },
      {
        "description": "Nano- and micromotors are machines designed to self-propel and—in the process of propelling themselves—perform specialized tasks like cleaning polluted waters. These motors offer distinct advantages over conventionally static decontamination methods, owing to their ability to move around ...",
        "snippets": [
          "Nano- and micromotors are machines designed to self-propel and—in the process of propelling themselves—perform specialized tasks like cleaning polluted waters. These motors offer distinct advantages over conventionally static decontamination methods, owing to their ability to move around and self-mix—which h Recent Review Articles",
          "In the last decade, considerable research efforts have been expended on exploring various mechanisms by which these motors can self-propel and remove pollutants, proving that the removal of oil droplets, heavy metals, and organic compounds using these synthetic motors is possible.",
          "A fundamental understanding of these removal mechanisms, with their attendant advantages and disadvantages, can help researchers fine-tune motor design in the future so that technical issues can be resolved before they are scaled-up for a wide variety of environmental applications."
        ],
        "title": "Nano- and micromotors for cleaning polluted waters: focused review on pollutant removal mechanisms - Nanoscale (RSC Publishing)",
        "url": "https://pubs.rsc.org/en/content/articlelanding/2017/nr/c7nr05494g"
      },
      {
        "description": "Sustainable nanotechnology has made substantial contributions in providing contaminant-free water to humanity. In this Review, we present the compelling need for providing access to clean water through nanotechnology-enabled solutions and the large disparities in ensuring their implementation.",
        "snippets": [
          "Topics discussed include: introduction; considerations for cellulose nanomaterial-based development for engineering applications (structures and nomenclature inconsistencies, comparisons to carbon nanotubes [CNT], cellulose nanomaterial manufg.); use of cellulose nanomaterials for water treatment technologies (nano-remediation strategies [as pollutant adsorbents, as scaffolds]); cellulose nanomaterials for water purifn.",
          "A review is given. Arsenic groundwater pollution has been reported for the Red River delta of Northern Vietnam and the Mekong delta of Southern Vietnam and Cambodia. Although the health of ∼10 million people is at risk from the drinking tube well water, little information is available on the health effects of As exposure in the residents of these regions.",
          "The countrywide survey on regional distribution of As pollution has not been conducted in these countries. As far as we know, symptoms of chronic As exposure have not yet been reported, probably due to the relative short-term usage of the tube wells in the regions.",
          "However, oxidative DNA damage has been obsd. in the residents of Cambodia and so further continuous usage of the tube well might cause severe damage to the health of the residents. We review literature concerning As pollution of groundwater and its health effects on residents of Vietnam and Cambodia."
        ],
        "title": "Clean Water through Nanotechnology: Needs, Gaps, and Fulfillment | ACS Nano",
        "url": "https://pubs.acs.org/doi/10.1021/acsnano.9b01730"
      },
      {
        "description": "Surface water is extremely susceptible to pollution stemming from human activities, such as the expansion of urban and suburban areas, industries, cit…",
        "snippets": [
          "In fact, sources of surface water have become the most common discharge sites for wastewater, which may contain microorganisms, pharmaceutical waste, heavy metals, and harmful pollutants. As a reference standard for clean water, the water quality standards and index of Malaysia were used.",
          "This prompts the use of nanotechnology applications to control surface water pollution and quality, as surface water is the main source of water consumption for humans, animals, and plants. This paper reviewed the application of nanotechnology for the detection and treatment of surface water pollution to ensure the sustainability of a green environment.",
          "Nanotechnologies for the detection and treatment of surface water pollution."
        ],
        "title": "A review of nanotechnological applications to detect and control surface water pollution - ScienceDirect",
        "url": "https://www.sciencedirect.com/science/article/pii/S2352186421006805"
      },
      {
        "description": "PDF | Nano- and micromotors are machines designed to self-propel and—in the process of propelling themselves—perform specialized tasks like cleaning... | Find, read and cite all the research you need on ResearchGate",
        "snippets": [
          "While offering autonomous propulsion, conventional micro-/nanomachines usually rely on the decomposition of external chemical fuels (e.g., H2 O2 ), which greatly hinders their applications in biologically relevant media. Recent developments have resulted in various micro-/nanomotors that can be powered by biocompatible fuels.",
          "Here, recent developments on fuel-free micro-/nanomotors (powered by various external stimuli such as light, magnetic, electric, or ultrasonic fields) are summarized, ranging from fabrication to propulsion mechanisms. The applications of these fuel-free micro-/nanomotors are also discussed, including nanopatterning, targeted drug/gene delivery, cell manipulation, and precision nanosurgery.",
          "Fuel-free synthetic micro-/nanomotors, which can move without external chemical fuels, represent another attractive solution for practical applications owing to their biocompatibility and sustainability.",
          "micromotor’s surface upon the nanomotor–oil interaction and"
        ],
        "title": "(PDF) Nano- and Micromotors for Cleaning Polluted Waters: Focused Review on Pollutant Removal Mechanisms",
        "url": "https://www.researchgate.net/publication/319642204_Nano-_and_Micromotors_for_Cleaning_Polluted_Waters_Focused_Review_on_Pollutant_Removal_Mechanisms"
      },
      {
        "description": "Important challenges in the global water situation, mainly resulting from worldwide population growth and climate change, require novel innovative water technologies in order to ensure a supply of drinking water and reduce global water pollution. Against ...",
        "snippets": [
          "The use of magnetic nanoparticles (magnetite Fe3O4) for separation of water pollutants has already been established in ground water remediation, in particular for the removal of arsenic.28 The conventionally applied “pump-and-treat” technology for groundwater treatment comprises pumping up the groundwater to the surface and further treatment, usually by activated carbon for final purification. The considerably extended operating hours and higher environmental clean-up costs can be reduced by applying in situ technologies.",
          "Even industrialized countries like the USA, providing highly innovative technologies for saving and purifying water, show the difficulty of exhausted water reservoirs due to the fact that more water is extracted than refilled. In the People’s Republic of China, 550 of the 600 largest cities suffer from a water shortage, since the biggest rivers are immensely polluted and even their use for irrigation has to be omitted, not to mention treatment for potable water.",
          "Photocatalysis is an advanced oxidation process that is employed in the field of water and wastewater treatment, in particular for oxidative elimination of micropollutants and microbial pathogens.48,49 As reported in the literature,50–52 most organic pollutants can be degraded by heterogeneous photocatalysis.",
          "Solids that are used to adsorb gases or dissolved substances are called adsorbents, and the adsorbed molecules are usually referred to collectively as the adsorbate.4 Due to their high specific surface area, nanoadsorbents show a considerably higher rate of adsorption for organic compounds compared with granular or powdered activated carbon. They have great potential for novel, more efficient, and faster decontamination processes aimed at removal of organic and inorganic pollutants like heavy metals and micropollutants."
        ],
        "title": "Innovations in nanotechnology for water treatment - PMC",
        "url": "https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4294021/"
      },
      {
        "description": "Nowadays, global water scarcity is becoming a pressing issue, and the discharge of various pollutants leads to the biological pollution of water bodies, which further leads to the poisoning of living organisms. Consequently, traditional water treatment methods are proving inadequate in addressing ...",
        "snippets": [
          "Nowadays, global water scarcity is becoming a pressing issue, and the discharge of various pollutants leads to the biological pollution of water bodies, which further leads to the poisoning of living organisms. Consequently, traditional water treatment methods are proving inadequate in addressing the growing demands of various industries.",
          "Effects of radius and length on the nanomotor rotors in aqueous solution driven by the rotating electric field. J. Phys. Chem. C 123 (50), 30649–30656. doi:10.1021/acs.jpcc.9b07345 ... Fuller, R., Landrigan, P. J., Balakrishnan, K., Bathan, G., Bose-O'Reilly, S., Brauer, M., et al. (2022). Pollution and health: a progress update.",
          "In recent years, micro/nanorobots and micro/nanomotor technologies have shown great advantages such as low cost, high efficiency and environmental friendliness in environmental remediation and water purification applications, which have gained widespread attention and have great potential for development and application.",
          "Micro/nanorobots (MNRs) or micro/nanomotors (MNMs), usually refer to microscopic substances with actuation capability between 1 and 1 mm in size, which can be both organic or inorganic, even artificially edited and modified microorganisms from nature."
        ],
        "title": "Frontiers | Micro/nanorobots for remediation of water resources and aquatic life",
        "url": "https://www.frontiersin.org/articles/10.3389/fbioe.2023.1312074/full"
      },
      {
        "description": "Nano/micromotor technology is evolving as an effective method for water treatment applications in comparison to existing static mechanisms. The dynamic nature of the nano/micromotor particles enable faster mass transport and a uniform mixing ...",
        "snippets": [
          "Other applications include self-powered porous spore@Fe3O4 biohybrid micromotors20 for the removal of toxic lead ions; mesoporous CoNi@Pt nanomotors, T/Fe/Cr micromotors and Fe3O4 nanoparticles are utilized for degradation of organic pollutants26–28; SW-Fe2O3/MnO2 micromotors used for oxidation of anthraquinone dyes/chlorophenols29; and MnFe2O4/oleic acid micromotors and Mg/Ti/Ni/Au Janus micromotors for oil removal22,30.",
          "On the other hand, the fabrication process of these nanomotors is complex and for driving requires a specific wavelength light source, a costly metal catalyst (Pt, Au), and hazardous media (i.e., hydrogen peroxide). They also have weak mechanical properties, such as limited reusability as an absorbent in aqueous media.",
          "There are very few reusable nanomotors that have been reported in past years. For instance, V. Singh et al.59 employed reusable ZrNPs/graphene/Pt hybrid micromotors for the removal of organophosphate compounds; D. Vilela et al.60 reported GOx-microbot-based reusable micromotors for lead-ion decontamination (2-cycle reuse); J.",
          "The extraction and recovery of toxic pollutants were successfully performed for ten cycles. In contrast to typical nanomotors, this design could be utilized to adjust the surface property of the TM nanorobots by changing the type of functional groups (e.g., -OH, -NH2, and -COOH) according to practical needs."
        ],
        "title": "Pick up and dispose of pollutants from water via temperature-responsive micellar copolymers on magnetite nanorobots - PMC",
        "url": "https://pmc.ncbi.nlm.nih.gov/articles/PMC8888651/"
      }
    ],
    "latency": 0.6449015140533447
    }
    

##

​

Explore further

## [Quickstart](/docs/quickstart)## [API Reference](/api-reference)

[Research API](/api-modes/research-api)[News API](/api-modes/news-api)

[twitter](https://twitter.com/youdotcom)[linkedin](https://www.linkedin.com/company/youdotcom)

[Powered by Mintlify](https://mintlify.com/preview-
request?utm_campaign=poweredBy&utm_medium=docs&utm_source=documentation.you.com)

On this page

  * Accurate and Real-time Web Data
  * Use Cases
  * Explore further

[You.com API home page![light logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/light.svg)![dark logo](https://mintlify.s3.us-
west-1.amazonaws.com/you/logo/dark.svg)](/)

Search or ask...

  * [Discord](https://discord.com/invite/youdotcom)
  * [Support](mailto:api@you.com)
  * [Support](mailto:api@you.com)

Search...

Navigation

API Reference

Search

[Welcome](/welcome)[Quickstart](/docs/quickstart)[API Reference](/api-
reference/smart)[API Guide](/api-modes/smart-api)

##### API Reference

  * [POSTSmart API](/api-reference/smart)
  * [POSTResearch API](/api-reference/research)
  * [GETSearch](/api-reference/search)
  * [GETNews](/api-reference/news)

API Reference

# Search

GET

/

search

Try it

cURL

Python

JavaScript

PHP

Go

Java

    
    
    curl --request GET \
      --url https://chat-api.you.com/search \
      --header 'X-API-Key: <api-key>'

200

    
    
    {
      "hits": [
        {
          "url": "https://you.com",
          "title": "The World's Greatest Search Engine!",
          "description": "Search on YDC",
          "favicon_url": "https://someurl.com/favicon",
          "thumbnail_url": "https://www.somethumbnailsite.com/thumbnail.jpg",
          "snippets": [
            "I'm an AI assistant that helps you get more done. What can I help you with?"
          ]
        }
      ],
      "latency": 1
    }

**Before You Get Started**

To register for usage of our Search API, please reach out via email at
[api@you.com](mailto:api@you.com).

#### Authorizations

​

X-API-Key

string

header

required

#### Query Parameters

​

query

string

required

Search query used to retrieve relevant results from index

​

num_web_results

integer

Specifies the maximum number of web results to return. Range `1 ≤
num_web_results ≤ 20`.

​

offset

integer

Indicates the `offset` for pagination. The `offset` is calculated in multiples
of `num_web_results`. For example, if `num_web_results = 5` and `offset = 1`,
results 5–10 will be returned. Range `0 ≤ offset ≤ 9`.

​

country

string

Country Code, one of `['AR', 'AU', 'AT', 'BE', 'BR', 'CA', 'CL', 'DK', 'FI',
'FR', 'DE', 'HK', 'IN', 'ID', 'IT', 'JP', 'KR', 'MY', 'MX', 'NL', 'NZ', 'NO',
'CN', 'PL', 'PT', 'PH', 'RU', 'SA', 'ZA', 'ES', 'SE', 'CH', 'TW', 'TR', 'GB',
'US']`.

​

safesearch

string

Configures the safesearch filter for content moderation. `off` \- no filtering
applied.`moderate` \- moderate content filtering (default). `strict` \- strict
content filtering.

#### Response

200 - application/json

A JSON object containing array of search hits and request latency

​

hits

object[]

Show child attributes

​

hits.url

string

The URL of the specific search result.

​

hits.title

string

The title or name of the search result.

​

hits.description

string

A brief description of the content of the search result.

​

hits.favicon_url

string

The URL of the favicon of the search result's domain.

​

hits.thumbnail_url

string

URL of the thumbnail.

​

hits.snippets

string[]

An array of text snippets from the search result, providing a preview of the
content.

​

latency

number

Indicates the time (in seconds) taken by the API to generate the response.

[Research API](/api-reference/research)[News](/api-reference/news)

[twitter](https://twitter.com/youdotcom)[linkedin](https://www.linkedin.com/company/youdotcom)

[Powered by Mintlify](https://mintlify.com/preview-
request?utm_campaign=poweredBy&utm_medium=docs&utm_source=documentation.you.com)

cURL

Python

JavaScript

PHP

Go

Java

    
    
    curl --request GET \
      --url https://chat-api.you.com/search \
      --header 'X-API-Key: <api-key>'

200

    
    
    {
      "hits": [
        {
          "url": "https://you.com",
          "title": "The World's Greatest Search Engine!",
          "description": "Search on YDC",
          "favicon_url": "https://someurl.com/favicon",
          "thumbnail_url": "https://www.somethumbnailsite.com/thumbnail.jpg",
          "snippets": [
            "I'm an AI assistant that helps you get more done. What can I help you with?"
          ]
        }
      ],
      "latency": 1
    }

================
File: plugins/repos/twat_search/resources/you/you.txt
================
https://api.you.com/
https://documentation.you.com/docs/quickstart
https://documentation.you.com/docs/quickstart#step-2%3A-write-the-search-function
https://documentation.you.com/api-modes/search-api
https://documentation.you.com/api-reference/search

================
File: plugins/repos/twat_search/resources/pricing.md
================
### Key Points

- Research suggests You.com API costs around $8 to $8.50 for 1000 searches, depending on the plan.
- It seems likely that Brave API offers the lowest cost at $3 for 1000 searches with the Base plan.
- The evidence leans toward HasData being the cheapest at $0.99 for 1000 searches (prorated), but it's a web scraping API, not a traditional search API, which might be unexpected for users expecting direct search functionality.
- Costs for other APIs like Tavily, SerpAPI, Sonar Perplexity, and Critique vary, with some plans having fixed monthly fees that affect the cost for 1000 searches.

---

### Costs for 1000 Searches

Below is a breakdown of the costs for 1000 searches for each API, based on the available plans and pricing models. Note that some costs are prorated for comparison, while others reflect the actual monthly fee if 1000 searches are within the plan's limit.

- **You.com API**:

  - Explorer AI: $8.50 (prorated, $100/month for 11, 765 calls).
  - Discoverer AI: $8.00 (prorated, $250/month for 31, 250 calls).
  - To perform 1000 searches, you’d likely choose Explorer AI, costing $100 monthly, but for comparison, the per-search cost is used.

- **Brave API**:

  - Free: $0 for up to 2, 000 requests/month, so 1000 searches cost $0 if within limit.
  - Base: $3.00 for 1000 requests (pay-per-use, $3 per 1, 000 requests).
  - Pro: $5.00 for 1000 requests (pay-per-use, $5 per 1, 000 requests).
  - The Base plan is the most cost-effective at $3 for 1000 searches.

- **Tavily API**:

  - Free: $0 for up to 1, 000 credits/month, so 1000 searches cost $0 if within limit.
  - Pay As You Go: $8.00 for 1000 searches ($0.008 per credit, each search is one credit).
  - Bootstrap: $6.67 for 1000 searches (prorated, $100/month for 15, 000 credits, effectively $0.00667 per search), but actual cost is $100 monthly for up to 15, 000 searches.
  - For 1000 searches, Pay As You Go at $8 is more cost-effective than the $100 monthly fee for Bootstrap.

- **SerpAPI**:

  - Developer: $15.00 for 1000 searches (prorated, $75/month for 5, 000 searches).
  - Production: $10.00 for 1000 searches (prorated, $150/month for 15, 000 searches).
  - To do 1000 searches, you pay $75 for Developer or $150 for Production monthly, so costs are $75 or $150, respectively, but prorated for comparison.

- **Sonar Perplexity**:

  - All models list $5.00 for 1000 searches as the search cost, but this likely excludes token costs (input, reasoning, output), which can add significant expense based on usage. Actual cost could be higher, estimated around $5 to $10+ depending on token usage.

- **HasData**:

  - Startup: $2.45 for 1000 searches (prorated, $49/month for 200, 000 credits, 10 credits per search).
  - Business: $0.99 for 1000 searches (prorated, $99/month for 1, 000, 000 credits, 10 credits per search).
  - Note: HasData is a web scraping API, not a traditional search API, which might be unexpected for users. Actual monthly cost is $49 or $99, respectively, for the plans.

- **Critique API**:
  - Pro plan has no monthly fee but usage costs $0.50 per million tokens. Cost for 1000 searches is variable, estimated $0.50 to $5 based on assumed token usage (e.g., 1, 000 to 10, 000 tokens per search), but exact cost depends on actual usage and is hard to quantify without specifics.

---

### Unexpected Detail

An unexpected detail is that HasData, at $0.99 for 1000 searches (prorated), is the cheapest option, but it's primarily for web scraping, not direct search queries, which might not meet expectations for users seeking traditional search APIs like Google or Bing results.

---

---

### Comprehensive Analysis of Search/SERP API Pricing Models in Traditional and LLM-Enabled Services

This analysis, conducted as of 03:54 PM PST on Wednesday, February 26, 2025, provides a detailed examination of the pricing structures for various Search/SERP APIs, both traditional and LLM-enabled, focusing on the cost for 1000 searches. The report aims to assist developers and businesses in understanding cost implications, feature differentiation, and suitability for different use cases, ensuring a thorough comparison across twelve major providers.

#### API Overview and Pricing Models

The APIs under consideration include You.com API, Brave API, Tavily API, SerpAPI, Sonar Perplexity, HasData, and Critique API. Each offers unique functionalities, with pricing based on calls, requests, credits, or tokens, reflecting their approach to search and data retrieval.

##### You.com API

You.com API is designed for AI-powered search, providing access to search and news endpoints. The pricing plans are as follows:

- **Trial AI**: Offers 1, 000 calls per month for 60 days at no cost, ideal for initial testing. For 1000 searches, the cost is $0, but only for the trial period.
- **Explorer AI**: Costs $100 per month for 11, 765 calls, translating to approximately $0.0085 per call. For 1000 searches, the prorated cost is $8.50, but to perform 1000 searches, you pay $100 monthly, as it's a fixed fee for up to 11, 765 calls.
- **Discoverer AI**: Priced at $250 per month for 31, 250 calls, approximately $0.008 per call. For 1000 searches, the prorated cost is $8.00, but the monthly fee is $250 for up to 31, 250 calls.

A "call" here likely refers to a search query, given the context of search endpoints. This API is LLM-enabled, focusing on delivering factual and up-to-date information with citations, making it suitable for applications requiring advanced search capabilities. For comparison, the cost for 1000 searches is $100 for Explorer AI or $250 for Discoverer AI, but prorated costs are $8.50 and $8.00, respectively.

##### Brave API

Brave API offers a traditional search engine API with the following plans:

- **Free**: Allows 1 request per second, up to 2, 000 requests per month at $0, requiring a credit card via Stripe. For 1000 searches, the cost is $0 if within the 2, 000 request limit.
- **Base**: Costs $3.00 per 1, 000 requests, with a rate of 20 requests per second and a monthly limit of 20, 000, 000 requests. This equates to $0.003 per request, so for 1000 searches, the cost is $3.00.
- **Pro**: Priced at $5.00 per 1, 000 requests, with 50 requests per second and unlimited requests, equating to $0.005 per request, so for 1000 searches, the cost is $5.00.

The Base plan is cost-effective for high-volume traditional search needs at $3 for 1000 searches, while the Pro plan offers higher throughput for unlimited usage at $5. The Free plan is $0 for up to 2, 000 requests, making it viable for low-volume users.

##### Tavily API

Tavily API is optimized for AI agents, providing real-time, accurate, and factual search results. Its pricing structure is credit-based:

- **Free**: Provides 1, 000 API credits per month, no credit card required, suitable for testing. For 1000 searches, the cost is $0, but only for up to 1, 000 credits, assuming each search is one credit.
- **Pay As You Go**: Costs $0.008 per credit, flexible for variable usage. Assuming each search is one credit, for 1000 searches, the cost is $8.00.
- **Bootstrap**: Fixed at $100 per month for 15, 000 API credits, approximately $0.00667 per credit. For 1000 searches, the prorated cost is $6.67, but the actual monthly fee is $100 for up to 15, 000 credits, so for 1000 searches, the cost is $100 if subscribed to this plan.

An API credit likely corresponds to a search query, with costs varying by search depth and extraction needs. This makes Tavily suitable for LLM integrations, with Pay As You Go at $8 for 1000 searches being more cost-effective than the $100 monthly fee for Bootstrap for low volumes.

##### SerpAPI

SerpAPI focuses on accessing search results from engines like Google, with clear search-based pricing:

- **Developer**: $75 per month for 5, 000 searches, equating to $0.015 per search. For 1000 searches, the prorated cost is $15.00, but to perform 1000 searches, you pay $75 monthly for up to 5, 000 searches, so the cost is $75.
- **Production**: $150 per month for 15, 000 searches, equating to $0.01 per search. For 1000 searches, the prorated cost is $10.00, but the monthly fee is $150 for up to 15, 000 searches, so the cost is $150.

Additional options include Regular Speed (no extra cost), Ludicrous Speed (+$75/month for Developer, +$150/month for Production), and Ludicrous Speed Max (+$225/month for Developer, +$450/month for Production), likely enhancing response times. This API is traditional, ideal for scraping search engine results with structured data. For 1000 searches, costs are $75 for Developer or $150 for Production, but prorated at $15 and $10, respectively.

##### Sonar Perplexity

Sonar Perplexity, from Perplexity AI, offers an LLM-enabled search API with a complex pricing model based on tokens and searches:

- For all models (Sonar, Sonar Deep Research, etc.), the table lists "Price per 1000 searches" as $5, which is the search cost. However, the detailed pricing includes input tokens ($1-$3 per million), reasoning tokens ($3 per million for Deep Research), output tokens ($1-$15 per million), and searches at $5 per 1, 000 searches.
- A typical request might do 30 searches costing $0.15 for searches, with token costs adding to the total. Estimating, for 1000 searches, if each request does 30 searches, it's about 33.33 requests, and total cost includes token usage, which can vary. The search cost alone is $5 for 1000 searches, but actual cost could be $5 to $10+ depending on token usage (e.g., input 100 tokens, reasoning 150, output 200 per search, leading to higher costs).

This API is designed for real-time, fact-based search, with flexibility for complex queries. For 1000 searches, the cost is at least $5 for searches, with additional token costs making the total likely higher.

##### HasData

HasData is primarily a web scraping API, with pricing based on credits:

- **Startup**: $49 per month for 200, 000 API credits, 15 concurrent requests. Each search costs 10 credits, so 200, 000 credits allow 20, 000 searches. For 1000 searches (10, 000 credits), the prorated cost is $2.45 ($49/200, 000 \* 10, 000), but the actual monthly cost is $49 for up to 20, 000 searches.
- **Business**: $99 per month for 1, 000, 000 API credits, 30 concurrent requests. Each search costs 10 credits, so 1, 000, 000 credits allow 100, 000 searches. For 1000 searches (10, 000 credits), the prorated cost is $0.99 ($99/1, 000, 000 \* 10, 000), but the actual monthly cost is $99 for up to 100, 000 searches.

Credit consumption varies by result type, but for comparison, for 1000 searches, the prorated costs are $2.45 for Startup and $0.99 for Business. However, actual cost is $49 or $99 monthly, respectively, making it $49 or $99 for 1000 searches if subscribed. Note: HasData is not a traditional search API, focusing on extracting data from websites, which might surprise users expecting direct search functionality.

##### Critique API

Critique API appears to be a platform for publishing and using APIs, with pricing:

- **Starter**: Free, 10 API calls per minute, no token limit, includes streaming search endpoint. For 1000 searches, cost is $0 if within rate limits.
- **Pro**: $0.00 per month (usage costs only), 100 API calls per minute, $0.50 per million tokens total, with priority support and websocket streaming. Given the streaming search endpoint, it may offer search capabilities, but details on what constitutes an "API call" or "token" are unclear. Assuming each search uses tokens, for 1000 searches, if each uses 1, 000 to 10, 000 tokens, total tokens are 1, 000, 000 to 10, 000, 000, costing $0.50 to $5, respectively. Exact cost depends on usage and is hard to quantify without specifics.

#### Comparative Analysis

To compare, we standardize costs for 1000 searches, noting both prorated costs for comparison and actual costs if subscribed to plans:

| API Provider | Plan | Cost for 1000 Searches (Prorated) | Actual Cost for 1000 Searches (If Within Plan) |
| --- | --- | --- | --- |
| You.com API | Explorer AI | $8.50 | $100 (monthly fee for 11, 765 calls) |
| You.com API | Discoverer AI | $8.00 | $250 (monthly fee for 31, 250 calls) |
| Brave API | Free | $0.00 (up to 2, 000 requests) | $0 (if within limit) |
| Brave API | Base | $3.00 | $3 (pay-per-use) |
| Brave API | Pro | $5.00 | $5 (pay-per-use) |
| Tavily API | Free | $0.00 (up to 1, 000 credits) | $0 (if within limit) |
| Tavily API | Pay As You Go | $8.00 | $8 (pay-per-use) |
| Tavily API | Bootstrap | $6.67 | $100 (monthly fee for 15, 000 credits) |
| SerpAPI | Developer | $15.00 | $75 (monthly fee for 5, 000 searches) |
| SerpAPI | Production | $10.00 | $150 (monthly fee for 15, 000 searches) |
| Sonar Perplexity | Any | $5.00 (search cost only) | $5+ (includes token costs, estimated $5-$10+) |
| HasData | Startup | $2.45 | $49 (monthly fee for 20, 000 searches) |
| HasData | Business | $0.99 | $99 (monthly fee for 100, 000 searches) |
| Critique API | Pro | $0.50-$5 (estimated, token-based) | Variable (depends on token usage) |

#### Considerations and Recommendations

- **Traditional Search Needs**: For cost efficiency, Brave API's Base plan at $3 for 1000 searches is competitive, especially for pay-per-use models. The Free plan at $0 is viable for low volumes up to 2, 000 requests.
- **LLM-Enabled Search**: Sonar Perplexity offers a base search cost of $5 for 1000 searches, but token costs can increase the total, estimated at $5 to $10+. You.com API at $8.50 to $8.00 (prorated) is another option, with actual costs at $100 to $250 monthly.
- **Web Scraping for Search**: HasData at $0.99 prorated for 1000 searches (Business plan) is the cheapest, but its focus on scraping may not meet direct search API expectations. Actual cost is $99 monthly for up to 100, 000 searches, making it $99 for 1000 searches if subscribed.
- **Fixed vs. Pay-Per-Use**: Plans with fixed monthly fees (e.g., You.com, SerpAPI, HasData, Tavily Bootstrap) may be cost-effective for high volumes but expensive for low usage like 1000 searches. Pay-per-use models (Brave, Tavily Pay As You Go) are better for variable or low volumes.

Users should consider their specific use case, such as volume, complexity, and integration needs, and verify details on official websites like [You API Plans](https://api.you.com/plans), [Brave Search API](https://brave.com/search/api/), [Tavily](https://tavily.com/), [SerpApi](https://serpapi.com/), [Sonar Perplexity Docs](https://docs.perplexity.ai/), and [HasData Prices](https://hasdata.com/prices).

#### Key Citations

- [You API Plans Web LLM & Web Search Pricing](https://api.you.com/plans)
- [Brave Search API Power your search and AI apps](https://brave.com/search/api/)
- [Tavily Fast, reliable access with high rate limits](https://tavily.com/)
- [SerpApi Get Google results from anywhere in the world](https://serpapi.com/)
- [Sonar by Perplexity Power your products with real-time research](https://docs.perplexity.ai/)
- [HasData Our Prices Experience the incredible accuracy](https://hasdata.com/prices)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/__init__.py
================


================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/aol.py
================
class Aol(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://search.aol.com/aol/search?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/ask.py
================
class Ask(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://www.ask.com/web?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/bing.py
================
class Bing(Falla):
    def __init__(self) -> None:
        super().__init__(name="Bing")
    def get_url(self, query: str) -> str:
        query_string = urllib.parse.urlencode(params)
    def get_title(self, elm: Tag) -> str:
        title_elem = elm.find("h2")
            return title_elem.get_text().strip()
    def get_link(self, elm: Tag) -> str:
        link_elem = elm.find("a")
        if link_elem and isinstance(link_elem, Tag) and "href" in link_elem.attrs:
            return cast(str, link_elem.attrs["href"])
    def get_snippet(self, elm: Tag) -> str:
        snippet_elem = elm.find("div", {"class": "b_caption"})
        if snippet_elem and isinstance(snippet_elem, Tag):
            p_elem = snippet_elem.find("p")
            if p_elem and isinstance(p_elem, Tag):
                return p_elem.get_text().strip()

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/dogpile.py
================
class DogPile(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://www.dogpile.com/serp?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/duckduckgo.py
================
logger = logging.getLogger(__name__)
class DuckDuckGo(Falla):
    def __init__(self) -> None:
        super().__init__(name="DuckDuckGo")
    def get_url(self, query: str) -> str:
        query_string = urllib.parse.urlencode(params)
    async def _fetch_page(self, url: str) -> str:
            await self._initialize_browser()
                logger.error("Browser context not initialized")
            page = await self.browser_context.new_page()
            await page.goto(url, timeout=30000, wait_until="domcontentloaded")
                    challenge_exists = await page.evaluate(f"""() => {{
                        logger.warning(f"Challenge detected on DuckDuckGo using selector: {selector}")
                        await page.screenshot(path="duckduckgo_challenge.png")
                        html_content = await page.content()
                        with open("duckduckgo_challenge.html", "w", encoding="utf-8") as f:
                            f.write(html_content)
                        logger.info("Saved challenge page content for debugging")
                logger.debug(f"Error checking for challenges: {e}")
                    logger.info(f"Waiting for selector: {self.wait_for_selector}")
                    await page.wait_for_selector(self.wait_for_selector, timeout=5000)
                    await asyncio.sleep(2)
                logger.warning(f"Timeout waiting for selector in {self.name}: {e}")
            return await page.content()
            logger.error(f"Error fetching page with Playwright: {e}")
                return await self._fetch_page(url)
                await page.close()
    def get_title(self, elm: Tag) -> str:
        title_elem = elm.select_one(".result__title")
            return title_elem.get_text().strip()
    def get_link(self, elm: Tag) -> str:
        link_elem = elm.select_one(".result__a")
            href = str(link_elem.attrs["href"])
            if href.startswith("/"):
    def get_snippet(self, elm: Tag) -> str:
        snippet_elem = elm.select_one(".result__snippet")
            return snippet_elem.get_text().strip()
    def get_each_elements(self, elements: list) -> list[dict[str, str]]:
                link = self.get_link(elm)
                    title = self.get_title(elm) or ""
                    snippet = self.get_snippet(elm) or ""
                    elements_list.append(
                logger.debug(f"Error processing element: {e}")

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/falla.py
================
logger = logging.getLogger(__name__)
class Falla:
    def __init__(self, name: str) -> None:
    async def _initialize_browser(self) -> None:
            p = await async_playwright().start()
            self.browser = await p.chromium.launch(headless=True)
            self.browser_context = await self.browser.new_context(
    async def _close_browser(self) -> None:
            await self.browser_context.close()
            await self.browser.close()
    async def _fetch_page(self, url: str) -> str:
            await self._initialize_browser()
                logger.error("Browser context not initialized")
            page = await self.browser_context.new_page()
            await page.goto(url, timeout=30000, wait_until="domcontentloaded")
                    logger.info(f"Waiting for selector: {self.wait_for_selector}")
                    await page.wait_for_selector(self.wait_for_selector, timeout=5000)
                    await asyncio.sleep(2)
                logger.warning(f"Timeout waiting for selector in {self.name}")
            return await page.content()
            logger.error(f"Error fetching page with Playwright: {e}")
                return await self._fetch_page(url)
                await page.close()
    def get_element_from_type(
            result = bs_obj.find(type_elem, attr_elem)
            return result if isinstance(result, bs4.element.Tag) else None
            logger.debug(f"Error getting element: {e}")
    def get_tag(
            result = element.find(_type, _attr)
            logger.debug(f"Error getting tag: {e}")
    async def parse_entry_point_async(self, entry_point: str) -> BeautifulSoup | None:
                content = await self._fetch_page(entry_point)
                return BeautifulSoup(content, "html.parser")
                return BeautifulSoup(self.scrapy_splash_request(entry_point), "html.parser")
            return BeautifulSoup(self.get_html_content(entry_point), "html.parser")
            logger.error(f"Error parsing entry point: {e}")
    def parse_entry_point(self, entry_point: str) -> BeautifulSoup | None:
                    loop = asyncio.get_running_loop()
                    if loop.is_running():
                        content_future = asyncio.create_task(self._fetch_page_sync(entry_point))
                        content = loop.run_until_complete(content_future)
                        content = loop.run_until_complete(self._fetch_page_sync(entry_point))
                    content = asyncio.run(self._fetch_page_sync(entry_point))
                    logger.warning(f"No content returned from {entry_point}")
            logger.error(f"Error parsing entry point: {e}", exc_info=True)
    async def _fetch_page_sync(self, url: str) -> str:
            async with async_playwright() as p:
                browser = await p.chromium.launch(headless=True)
                    context = await browser.new_context(
                    page = await context.new_page()
                    await page.goto(url, timeout=60000, wait_until="domcontentloaded")
                            await page.wait_for_selector(self.wait_for_selector, timeout=10000)
                    content = await page.content()
                    logger.info(f"Got content length: {len(content)}")
                    if len(content) > 0:
                        logger.debug(f"Content preview: {content[:200]}...")
                        logger.warning("Received empty content")
                    await browser.close()
                logger.info(f"Retrying in {wait_time} seconds (attempt {self.current_retry + 1}/{self.max_retries})")
                await asyncio.sleep(wait_time)
                return await self._fetch_page_sync(url)
    def get_each_elements(self, elements: bs4.element.ResultSet) -> list[dict[str, str]]:
                title = self.get_title(elm) or ""
                link = self.get_link(elm) or ""
                snippet = self.get_snippet(elm) or ""
                    elements_list.append(
                logger.debug(f"Error processing element: {e}")
    def scrapy_splash_request(self, entry_point: str) -> str:
            return requests.get(splash_url, timeout=30).text
            logger.error(f"Error using Splash: {e}")
    def get_html_content(self, url: str) -> str:
            response = requests.get(url, headers=headers, timeout=15)
            response.raise_for_status()  # Raise an exception for 4XX/5XX responses
            logger.error(f"Error getting HTML content: {e}")
    async def fetch_async(self, entry_point: str) -> list[dict[str, str]]:
            html_parser = await self.parse_entry_point_async(entry_point)
                elements = html_parser.find_all(self.container_element[0], attrs=self.container_element[1])
                return self.get_each_elements(elements)
            logger.error(f"Error fetching results: {e}")
    def fetch(self, entry_point: str) -> list[dict[str, str]]:
            html_parser = self.parse_entry_point(entry_point)
    def get_url(self, query: str) -> str:
        raise NotImplementedError(msg)
    def get_title(self, elm: bs4.element.Tag) -> str:
    def get_link(self, elm: bs4.element.Tag) -> str:
    def get_snippet(self, elm: bs4.element.Tag) -> str:
    def search(self, query: str, pages: str = "") -> list[dict[str, str]]:
        url = self.get_url(query)
        return self.fetch(url)
    async def search_async(self, query: str, pages: str = "") -> list[dict[str, str]]:
            return await self.fetch_async(url)
            await self._close_browser()
    async def close(self) -> None:

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/fetch_page.py
================
logging.basicConfig(
    handlers=[logging.StreamHandler(sys.stderr)],
logger = logging.getLogger(__name__)
async def fetch_page(url: str, wait_for_selector: str | None = None) -> str:
    logger.info(f"Fetching {url}")
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True)
            context = await browser.new_context(
            page = await context.new_page()
            await page.goto(url, timeout=30000, wait_until="domcontentloaded")
                    await page.wait_for_selector(wait_for_selector, timeout=5000)
                    logger.warning(f"Timeout waiting for selector: {e}")
            content = await page.content()
            logger.info(f"Got content length: {len(content)}")
            await browser.close()
async def main_async():
    parser = argparse.ArgumentParser(description="Fetch a web page using Playwright")
    parser.add_argument("url", help="URL to fetch")
    parser.add_argument("--wait-for", help="CSS selector to wait for")
    args = parser.parse_args()
    await fetch_page(args.url, args.wait_for)
def main():
    asyncio.run(main_async())
    main()

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/gibiru.py
================
class Gibiru(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://gibiru.com/results.html?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/google.py
================
logger = logging.getLogger(__name__)
class Google(Falla):
    def __init__(self) -> None:
        super().__init__(name="Google")
    def get_url(self, query: str) -> str:
        query_string = urllib.parse.urlencode(params)
    def get_title(self, elm: Tag) -> str:
        title_elem = elm.find("h3")
        if title_elem and isinstance(title_elem, Tag):
            return title_elem.get_text().strip()
    def get_link(self, elm: Tag) -> str:
        link_elem = elm.find("a")
        if link_elem and isinstance(link_elem, Tag) and "href" in link_elem.attrs:
            href = cast(str, link_elem.attrs["href"])
            if href.startswith("/url?"):
                href = href.split("&")[0].replace("/url?q=", "")
                return urllib.parse.unquote(href)
    def get_snippet(self, elm: Tag) -> str:
        snippet_elem = elm.find("div", {"class": "VwiC3b"})
        if not snippet_elem or not isinstance(snippet_elem, Tag):
            snippet_elem = elm.find("span", {"class": "aCOpRe"})
            snippet_elem = elm.find("div", {"class": "BNeawe s3v9rd AP7Wnd"})
            text_containers = elm.find_all(["div", "span", "p"], class_=True)
                    isinstance(container, Tag)
                    and container.get_text().strip()
                    and hasattr(container, "name")
        if snippet_elem and isinstance(snippet_elem, Tag):
            return snippet_elem.get_text().strip()
    def get_each_elements(self, elements: ResultSet) -> list[dict[str, str]]:
            if not isinstance(elm, Tag):
                title = self.get_title(elm)
                link = self.get_link(elm)
                snippet = self.get_snippet(elm)
                    elements_list.append(
                logger.debug(f"Error processing element: {e}")
    async def search_async(self, query: str, pages: str = "") -> list[dict[str, str]]:
            url = self.get_url(query)
            logger.info(f"Starting Google search for: {query}")
            logger.info(f"URL: {url}")
            await self._initialize_browser()
                logger.error("Browser context not initialized")
            page = await self.browser_context.new_page()
                await page.goto(url, timeout=60000, wait_until="domcontentloaded")
                logger.info("Page loaded, waiting for selector...")
                        await page.wait_for_selector(self.wait_for_selector, timeout=10000)
                        logger.info(f"Selector '{self.wait_for_selector}' found")
                    logger.warning(f"Timeout waiting for selector: {e}")
                    captcha_exists = await page.evaluate(f"""() => {{
                        logger.warning("CAPTCHA detected on Google search page")
                        await page.screenshot(path="google_captcha.png")
                    logger.error(f"Error checking for CAPTCHA: {e}")
                html_content = await page.content()
                logger.info(f"Retrieved HTML content, length: {len(html_content)}")
                debug_file = f"google_debug_{query.replace(' ', '_')}.html"
                with open(debug_file, "w", encoding="utf-8") as f:
                    f.write(html_content)
                logger.info(f"Wrote HTML content to {Path(debug_file).resolve()}")
                result_count = await page.evaluate("""() => {
                logger.info(f"JavaScript found {result_count} results with 'div.g' selector")
                        count = await page.evaluate(f"""() => {{
                        logger.info(f"Found {count} elements with selector: {selector}")
                        logger.error(f"Error evaluating selector {selector}: {e}")
                await page.close()
            results = await self.fetch_async(url)
            logger.info(f"Got {len(results)} results from fetch_async")
                logger.warning("No results found with standard method, trying alternate parsing")
                if Path(debug_file).exists():
                    with open(debug_file, encoding="utf-8") as f:
                        html_content = f.read()
                    soup = BeautifulSoup(html_content, "html.parser")
                        logger.info(f"Trying alternate selector: {selector_type}.{selector_attrs}")
                        elements = soup.find_all(selector_type, attrs=selector_attrs)
                        logger.info(f"Found {len(elements)} elements")
                            logger.info(f"Using alternate container: {self.container_element}")
                                if not isinstance(elem, Tag):
                                title = self.get_title(elem) or ""
                                link = self.get_link(elem) or ""
                                snippet = self.get_snippet(elem) or ""
                                    logger.info(f"Found result: {title} - {link}")
                                    manual_results.append(
                                logger.info(f"Found {len(manual_results)} results with alternate parsing")
            logger.error(f"Search failed: {e}", exc_info=True)
    def search(self, query: str, pages: str = "") -> list[dict[str, str]]:
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/mojeek.py
================
class Mojeek(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://www.mojeek.com/search?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/qwant.py
================
logger = logging.getLogger(__name__)
class Qwant(Falla):
    def __init__(self) -> None:
        super().__init__(name="Qwant")
    def get_url(self, query: str) -> str:
        query_string = urllib.parse.urlencode(params)
    async def _fetch_page(self, url: str) -> str:
            await self._initialize_browser()
                logger.error("Browser context not initialized")
            page = await self.browser_context.new_page()
            await page.goto(url, timeout=30000, wait_until="domcontentloaded")
                consent_button = page.locator('button:has-text("Accept all")')
                if await consent_button.count() > 0:
                    logger.info("Consent page detected, clicking 'Accept all'")
                    await consent_button.click()
                    await page.wait_for_load_state("networkidle")
                logger.debug(f"No consent page or error handling consent: {e}")
                    logger.info(f"Waiting for selector: {self.wait_for_selector}")
                    await page.wait_for_selector(self.wait_for_selector, timeout=5000)
                    await asyncio.sleep(2)
                logger.warning(f"Timeout waiting for selector in {self.name}: {e}")
            return await page.content()
            logger.error(f"Error fetching page with Playwright: {e}")
                return await self._fetch_page(url)
                await page.close()
    def get_title(self, elm: Tag) -> str:
        title_elem = elm.select_one(".result__title")
            return title_elem.get_text().strip()
        title_elem = elm.select_one("a.url")
        title_elem = elm.select_one("h3")
    def get_link(self, elm: Tag) -> str:
        link_elem = elm.select_one(".result__url")
            return str(link_elem.attrs["href"])
        link_elem = elm.select_one("a.url")
        link_elem = elm.select_one("a[href]")
    def get_snippet(self, elm: Tag) -> str:
        snippet_elem = elm.select_one(".result__desc")
            return snippet_elem.get_text().strip()
        snippet_elem = elm.select_one(".result__snippet")
        snippet_elem = elm.select_one("p")
    def search(self, query: str, pages: str = "") -> list[dict[str, str]]:
        url = self.get_url(query)
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/searchencrypt.py
================
class SearchEncrypt(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://www.searchencrypt.com/search/?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/startpage.py
================
class StartPage(Falla):
    def __init__(self):
    def search(self, search_text, pages=""):
        url = "https://www.startpage.com/sp/search?q=" + search_text.replace(" ", "+") + pages
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/yahoo.py
================
logger = logging.getLogger(__name__)
class Yahoo(Falla):
    def __init__(self) -> None:
        super().__init__(name="Yahoo")
    def get_url(self, query: str) -> str:
        query_string = urllib.parse.urlencode(params)
    async def _fetch_page(self, url: str) -> str:
            await self._initialize_browser()
                logger.error("Browser context not initialized")
            page = await self.browser_context.new_page()
            await page.goto(url, timeout=30000, wait_until="domcontentloaded")
                consent_button = page.locator('button:has-text("Accept all")')
                if await consent_button.count() > 0:
                    logger.info("Consent page detected, clicking 'Accept all'")
                    await consent_button.click()
                    await page.wait_for_load_state("networkidle")
                logger.debug(f"No consent page or error handling consent: {e}")
                    logger.info(f"Waiting for selector: {self.wait_for_selector}")
                    await page.wait_for_selector(self.wait_for_selector, timeout=5000)
                    await asyncio.sleep(2)
                logger.warning(f"Timeout waiting for selector in {self.name}: {e}")
            return await page.content()
            logger.error(f"Error fetching page with Playwright: {e}")
                return await self._fetch_page(url)
                await page.close()
    def get_title(self, elm: Tag) -> str:
        title_elem = elm.select_one("h3.title a")
            return title_elem.get_text().strip()
        title_elem = elm.select_one("h3")
    def get_link(self, elm: Tag) -> str:
        link_elem = elm.select_one("h3.title a")
            return str(link_elem.attrs["href"])
        link_elem = elm.select_one("a[href]")
    def get_snippet(self, elm: Tag) -> str:
        snippet_elem = elm.select_one("div.compText")
            return snippet_elem.get_text().strip()
        snippet_elem = elm.select_one("p")
    def search(self, query: str, pages: str = "") -> list[dict[str, str]]:
        url = self.get_url(query)
        return self.fetch(url)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/core/yandex.py
================
class Yandex(Falla):
    def __init__(self) -> None:
        super().__init__(name="Yandex")
    def get_url(self, query: str) -> str:
        query_string = urllib.parse.urlencode(params)
    def get_title(self, elm: Tag) -> str:
        title_elem = elm.select_one("div.organic__url-text")
            return title_elem.get_text().strip()
    def get_link(self, elm: Tag) -> str:
        link_elem = elm.select_one("a.link")
            return cast(str, link_elem.attrs["href"])
    def get_snippet(self, elm: Tag) -> str:
        snippet_elem = elm.select_one("div.text-container")
            return snippet_elem.get_text().strip()

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/__init__.py
================
PACKAGE_PATH = Path(__file__).parent

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/main.py
================
async def async_main(args: argparse.Namespace) -> None:
        results = await get_results_async(args.engine, args.query, args.num)
            for _i, _result in enumerate(results, 1):
        logging.error(f"Error: {e}")
        sys.exit(1)
def main() -> None:
    parser = argparse.ArgumentParser(
    parser.add_argument("-e", "--engine", help="Search engine to use", required=False)
    parser.add_argument("-q", "--query", help="Search query", required=False)
    parser.add_argument("-n", "--num", help="Number of results to return", type=int, default=10)
    parser.add_argument("-l", "--list", help="List available engines", action="store_true")
    parser.add_argument("-v", "--version", help="Show version", action="store_true")
    parser.add_argument("-j", "--json", help="Output results as JSON", action="store_true")
    parser.add_argument("--async", dest="use_async", help="Use async version", action="store_true")
    args = parser.parse_args()
    logging.basicConfig(
        for _engine in list_engines():
        parser.print_help()
            asyncio.run(async_main(args))
            results = get_results(args.engine, args.query, args.num)
    main()

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/requirements.txt
================
# this_file: src/twat_search/web/engines/lib_falla/requirements.txt
# Falla search engine scraper dependencies

beautifulsoup4>=4.12.2
lxml>=4.9.3
playwright>=1.40.0
# Core dependencies
requests>=2.31.0

# For splash_scrap mode (optional)
# If you want to use the splash scraper, you'll need to run a splash container:
# docker run -p 8050:8050 scrapinghub/splash

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/settings.py
================
SPLASH_SCRAP_URL = os.environ.get("FALLA_SPLASH_SCRAP_URL", "http://localhost:8050")
FALLA_DIR = Path(__file__).parent

================
File: plugins/repos/twat_search/src/twat_search/web/engines/lib_falla/utils.py
================
logger = logging.getLogger(__name__)
class Bcolors:
def list_engines() -> list[str]:
    core_dir = Path(__file__).parent / "core"
    for file in core_dir.glob("*.py"):
        if file.name not in ["__init__.py", "falla.py"] and not file.name.startswith("__"):
            engines.append(engine_name)
    return sorted(engines)
def get_engine_class(engine: str) -> type[Falla]:
    engine_lower = engine.lower()
        available = ", ".join(ENGINES_MAP.keys())
        raise ValueError(msg)
def get_results(engine: str, query: str, num_results: int = 10) -> list[dict[str, str]]:
    engine_class = get_engine_class(engine)
    engine_instance = engine_class()
        results = engine_instance.search(query)
        if num_results and len(results) > num_results:
        logger.error(f"Error getting results from {engine}: {e}")
async def get_results_async(engine: str, query: str, num_results: int = 10) -> list[dict[str, str]]:
        results = await engine_instance.search_async(query)
        await engine_instance.close()

================
File: plugins/repos/twat_search/src/twat_search/web/engines/__init__.py
================
    __all__.extend(
    __all__.extend(["TavilySearchEngine", "tavily"])
    __all__.extend(["PerplexitySearchEngine", "pplx"])
    __all__.extend(["CritiqueSearchEngine", "critique"])
    __all__.extend(["DuckDuckGoSearchEngine", "duckduckgo"])
    __all__.extend(["BingScraperSearchEngine", "bing_scraper"])
    logger.warning(f"Failed to import bing_scraper: {e}")
    __all__.extend(["GoogleScraperEngine", "google_scraper"])
    logger.warning(f"Failed to import google_scraper module: {e}")
    __all__.extend(["SerpApiSearchEngine", "google_serpapi"])
    if is_falla_available():
        logger.warning("Falla search engines are not available")
    logger.warning(f"Failed to import falla module: {e}")
def is_engine_available(engine_name: str) -> bool:
    standardized_name = standardize_engine_name(engine_name)
def get_engine_function(
    return available_engine_functions.get(standardized_name)
def get_available_engines() -> list[str]:
    return list(available_engine_functions.keys())

================
File: plugins/repos/twat_search/src/twat_search/web/engines/base.py
================
logger = logging.getLogger(__name__)
class SearchEngine(abc.ABC):
    def __init__(self, config: EngineConfig, **kwargs: Any) -> None:
        self.num_results = kwargs.get("num_results", 5)
        self.country = kwargs.get("country")
        self.language = kwargs.get("language")
        self.safe_search = kwargs.get("safe_search", True)
        self.time_frame = kwargs.get("time_frame")
        self.timeout = kwargs.get("timeout", 10)
        self.retries = kwargs.get("retries", 2)
        self.retry_delay = kwargs.get("retry_delay", 1.0)
        self.use_random_user_agent = kwargs.get("use_random_user_agent", True)
            raise EngineError(self.engine_code, msg)
            logger.debug(f"Engine '{self.engine_code}' requires an API key")
            logger.debug(f"Config API key: {self.config.api_key is not None}")
                key_prefix = key_value[:4] if len(key_value) > 4 else "****"
                key_suffix = key_value[-4:] if len(key_value) > 4 else "****"
                logger.debug(f"Config API key value: {key_prefix}...{key_suffix}")
                env_value = os.environ.get(env_var)
                logger.debug(f"Environment variable {env_var}: {env_value is not None}")
                    env_prefix = env_value[:4] if len(env_value) > 4 else "****"
                    env_suffix = env_value[-4:] if len(env_value) > 4 else "****"
                    logger.debug(f"Env var {env_var} value: {env_prefix}...{env_suffix}")
            logger.debug(f"Final API key check before validation: {self.config.api_key is not None}")
                    f"Please set it via one of these env vars: {', '.join(self.env_api_key_names)}"
                logger.error(f"API key validation failed: {msg}")
            logger.debug(f"API key validation passed for {self.engine_code}")
    def _get_num_results(self, param_name: str = "num_results", min_value: int = 1) -> int:
        value = self.kwargs.get(param_name)
                return max(min_value, int(value))
                logger.warning(f"Invalid value for '{param_name}' ({value!r}) in {self.engine_code}, using default.")
        default = self.config.default_params.get(param_name) or self.config.default_params.get("num_results")
                return max(min_value, int(default))
                logger.warning(
    def max_results(self) -> int:
        return self._get_num_results(param_name="num_results", min_value=1)
    def limit_results(self, results: list[SearchResult]) -> list[SearchResult]:
        logger.debug(f"limit_results: Got {len(results)} results, self.num_results={self.num_results}")
        logger.debug(f"limit_results: Using max_results={max_results}")
        if max_results > 0 and len(results) > max_results:
            logger.debug(f"limit_results: Limiting to {len(limited)} results")
        logger.debug(f"limit_results: Returning all {len(results)} results (no limiting needed)")
    async def make_http_request(
        if self.use_random_user_agent and "user-agent" not in {k.lower() for k in headers}:
            headers["User-Agent"] = random.choice(USER_AGENTS)
        for attempt in range(1, actual_retries + 2):  # +2 because we want actual_retries+1 attempts
                async with httpx.AsyncClient(timeout=actual_timeout) as client:
                    response = await client.request(
                    response.raise_for_status()
                jitter = random.uniform(0.5, 1.5)
                await asyncio.sleep(actual_delay)
            last_error = httpx.RequestError("Unknown error occurred during HTTP request")
        raise EngineError(self.engine_code, msg) from last_error
    async def search(self, query: str) -> list[SearchResult]:
    def _get_api_key(self) -> str:
            raise EngineError(
                f"API key is required. Set it via one of these env vars: {', '.join(self.env_api_key_names)}",
def register_engine(engine_class: type[SearchEngine]) -> type[SearchEngine]:
        if not hasattr(engine_class, "engine_code") or not engine_class.engine_code:
            raise AttributeError(error_msg)
        if not hasattr(engine_class, "env_api_key_names"):
                engine_class.friendly_engine_name = ENGINE_FRIENDLY_NAMES.get(
        logger.error(f"Failed to register engine {engine_class.__name__}: {e}")
def get_engine(engine_name: str, config: EngineConfig, **kwargs: Any) -> SearchEngine:
    engines = get_registered_engines()
    std_engine_name = standardize_engine_name(engine_name)
    engine_class = engines.get(std_engine_name)
        engine_class = engines.get(engine_name)
        available_engines = ", ".join(sorted(engines.keys()))
        logger.error(msg)
        raise SearchError(msg)
            logger.warning(msg)
            raise EngineError(engine_name, msg)
        engine_instance = engine_class(config, **kwargs)
        logger.debug(f"Successfully initialized engine: {engine_name}")
        if "API key" in str(e):
            env_vars = getattr(engine_class, "env_api_key_names", [])
            env_vars_str = ", ".join(env_vars) if env_vars else "No environment variables defined"
            raise EngineError(engine_name, msg) from e
def get_registered_engines() -> dict[str, type[SearchEngine]]:
    return _engine_registry.copy()

================
File: plugins/repos/twat_search/src/twat_search/web/engines/bing_scraper.py
================
logger = logging.getLogger(__name__)
    class BingScraper:  # type: ignore
        def __init__(
        def search(self, query: str, num_results: int = 10) -> list[Any]:
class BingScraperResult(BaseModel):
class BingScraperSearchEngine(SearchEngine):
        super().__init__(config, **kwargs)
        self.max_retries: int = kwargs.get(
        ) or self.config.default_params.get("max_retries", 3)
        self.delay_between_requests: float = kwargs.get(
        ) or self.config.default_params.get("delay_between_requests", 1.0)
            unused_params.append(f"country='{country}'")
            unused_params.append(f"language='{language}'")
            unused_params.append(f"safe_search={safe_search}")
            unused_params.append(f"time_frame='{time_frame}'")
            logger.debug(
                f"Parameters {', '.join(unused_params)} set but not used by Bing Scraper",
    def _convert_result(self, result: Any) -> SearchResult | None:
            logger.warning("Empty result received from Bing Scraper")
        if not hasattr(result, "title") or not hasattr(result, "url"):
            logger.warning(f"Invalid result format: {result}")
            validated = BingScraperResult(
                description=result.description if hasattr(result, "description") else None,
            return SearchResult(
                    "url": str(result.url),
                    "description": result.description if hasattr(result, "description") else None,
            logger.warning(f"Validation error for result: {exc}")
            logger.warning(f"Unexpected error converting result: {exc}")
    async def search(self, query: str) -> list[SearchResult]:
            raise EngineError(self.engine_code, "Search query cannot be empty")
        logger.info(f"Searching Bing with query: '{query}'")
            scraper = BingScraper(
            raw_results = scraper.search(query, num_results=self.num_results)
                logger.info("No results returned from Bing Scraper")
                f"Received {len(raw_results)} raw results from Bing Scraper",
                search_result = self._convert_result(result)
                    results.append(search_result)
                    if len(results) >= self.num_results:
            logger.info(
                f"Returning {len(results)} validated results from Bing Scraper",
            return self.limit_results(results)
            logger.error(error_msg)
            raise EngineError(self.engine_code, error_msg) from exc
async def bing_scraper(
    config = EngineConfig(enabled=True)
    engine = BingScraperSearchEngine(
    return await engine.search(query)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/brave.py
================
class BraveResult(BaseModel):
class BraveNewsResult(BaseModel):
class BaseBraveEngine(SearchEngine):
    def __init__(
        super().__init__(
            raise EngineError(self.engine_code, "API key is required")
    async def search(self, query: str) -> list[SearchResult]:
        count_param = min(self.max_brave_results, self.num_results)
        async with httpx.AsyncClient() as client:
                response = await client.get(
                response.raise_for_status()
                data = response.json()
                section = data.get(self.response_key, {})
                if section.get("results"):
                            parsed = self.result_model(**result)
                            results.append(self.convert_result(parsed, result))
                            if len(results) >= self.num_results:
                return self.limit_results(results)
                raise EngineError(
    def convert_result(self, parsed: BaseModel, raw: dict[str, Any]) -> SearchResult:
        snippet = getattr(parsed, "description", "")
            publisher = getattr(parsed, "publisher", None)
            published_time = getattr(parsed, "published_time", None)
        url = getattr(parsed, "url", None)
            url = HttpUrl("https://brave.com")
        return SearchResult(
            title=getattr(parsed, "title", ""),
    async def _make_api_call(self, query: str) -> dict[str, Any]:
            raise EngineError(self.engine_code, "Search query cannot be empty")
            "count": min(
        if hasattr(self, "freshness") and self.freshness:
        response = await self.make_http_request(
            data: dict[str, Any] = response.json()
    def limit_results(self, results: list[SearchResult]) -> list[SearchResult]:
class BraveSearchEngine(BaseBraveEngine):
        data = await self._make_api_call(query)
            web_result = data.get("web", {})
            items = web_result.get("results", [])
            for _idx, item in enumerate(items, start=1):
                title = item.get("title", "")
                url = item.get("url", "")
                description = item.get("description", "")
                results.append(
                    SearchResult(
class BraveNewsSearchEngine(BaseBraveEngine):
            items = data.get("results", [])
                if item.get("age") or item.get("source", {}).get("name"):
                    source_name = item.get("source", {}).get("name", "")
                    age = item.get("age", "")
                elif item.get("age"):
                    description = f"{description} - {item.get('age')}"
async def brave(
    config = EngineConfig(api_key=api_key, enabled=True)
    engine = BraveSearchEngine(
    return await engine.search(query)
async def brave_news(
    engine = BraveNewsSearchEngine(

================
File: plugins/repos/twat_search/src/twat_search/web/engines/critique.py
================
class CritiqueResult(BaseModel):
    url: str = Field(default="")  # URL of the result source
    title: str = Field(default="")  # Title of the result
    summary: str = Field(default="")  # Summary or snippet from the result
    source: str = Field(default="")  # Source of the result
class CritiqueResponse(BaseModel):
    results: list[CritiqueResult] = Field(default_factory=list)
class CritiqueSearchEngine(SearchEngine):
    def __init__(
        super().__init__(config)
        self.num_results = self._get_num_results(param_name="num_results", min_value=1)
        self.image_url = image_url or kwargs.get("image_url")
        self.image_base64 = image_base64 or kwargs.get("image_base64")
        self.source_whitelist = source_whitelist or kwargs.get(
        self.source_blacklist = source_blacklist or kwargs.get(
        self.output_format = output_format or kwargs.get("output_format")
        api_key_from_kwargs = kwargs.get("api_key")
            raise EngineError(
                f"API key is required for critique. Set it via one of these env vars: {', '.join(self.env_api_key_names)} or use the --api-key parameter",
    async def _convert_image_url_to_base64(self, image_url: str) -> str:
            async with httpx.AsyncClient() as client:
                response = await client.get(image_url, timeout=30)
                response.raise_for_status()
                encoded = base64.b64encode(response.content).decode("utf-8")
            raise EngineError(self.engine_code, f"Error processing image: {e}")
    async def _build_payload(self, query: str) -> dict[str, Any]:
            payload["image"] = await self._convert_image_url_to_base64(self.image_url)
    def _build_result(self, item: CritiqueResult, rank: int) -> SearchResult:
                HttpUrl(item.url)
                else HttpUrl(
            url_obj = HttpUrl("https://critique-labs.ai")
        return SearchResult(
            raw=item.model_dump(),
    def _parse_results(self, data: dict[str, Any]) -> list[SearchResult]:
        critique_data = CritiqueResponse(
            results=data.get("results", []),
            response=data.get("response"),
            structured_output=data.get("structured_output"),
            results.append(
                SearchResult(
                    url=HttpUrl("https://critique-labs.ai"),
        for idx, item in enumerate(critique_data.results, 1):
                results.append(self._build_result(item, idx))
    async def search(self, query: str) -> list[SearchResult]:
        payload = await self._build_payload(query)
                response = await client.post(
                data = response.json()
                results = self._parse_results(data)
                return self.limit_results(results)
async def critique(
    config = EngineConfig(api_key=api_key, enabled=True)
    engine = CritiqueSearchEngine(
    return await engine.search(query)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/duckduckgo.py
================
logger = logging.getLogger(__name__)
class DuckDuckGoResult(BaseModel):
class DuckDuckGoSearchEngine(SearchEngine):
    def __init__(
        super().__init__(config, **kwargs)
        ) = self._map_init_params(
            logger.debug(
    def _map_init_params(
        num_results = kwargs.get(
        ) or config.default_params.get("num_results", 10)
        region = kwargs.get("region", country) or config.default_params.get(
        lang = language or config.default_params.get("language", None)
        timelimit = kwargs.get("timelimit", time_frame) or config.default_params.get(
        if timelimit and not kwargs.get("timelimit"):
            timelimit = time_mapping.get(timelimit.lower(), timelimit)
        safesearch = kwargs.get("safesearch", safe_search)
        if isinstance(safesearch, str):
            safesearch = safesearch.lower() not in ["off", "false"]
        proxy = kwargs.get("proxy") or config.default_params.get("proxy", None)
        timeout = kwargs.get(
        ) or config.default_params.get("timeout", 10)
    def _convert_result(self, raw: dict[str, Any]) -> SearchResult | None:
            ddg_result = DuckDuckGoResult(
            return SearchResult(
            logger.warning(f"Validation error for result: {exc}")
    async def search(self, query: str) -> list[SearchResult]:
            ddgs = DDGS(proxy=self.proxy, timeout=self.timeout)
            raw_results = ddgs.text(
                converted = self._convert_result(raw)
                    results.append(converted)
                    if len(results) >= self.num_results:
            raise EngineError(
async def duckduckgo(
    config = EngineConfig(enabled=True)
    engine = DuckDuckGoSearchEngine(
    return await engine.search(query)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/falla.py
================
logger = logging.getLogger(__name__)
def is_falla_available() -> bool:
            _check_engine = Google()
            logger.warning(f"Falla engines are not available: {e}")
class FallaSearchEngine(SearchEngine):
    def __init__(self, config: Any, **kwargs: Any) -> None:
        super().__init__(config, **kwargs)
        self._falla_engine = self._create_engine_instance()
    def _create_engine_instance(self) -> Falla:
        return self._falla_engine_class()
    async def search(self, query: str, **_kwargs: Any) -> list[SearchResult]:
            raw_results = cast(list[dict[str, str]], await self._falla_engine.search_async(query=query))
            for i, item in enumerate(raw_results):
                url = item.get("link", "")
                    result = SearchResult(
                        title=item.get("title", ""),
                        url=HttpUrl(url),  # Convert to HttpUrl type
                        snippet=item.get("snippet", ""),
                    results.append(result)
                    logger.warning(f"Invalid URL '{url}' in {self.engine_code} result: {url_err}")
            return self.limit_results(results)
            logger.error(f"Search failed: {e!s}")
            raise EngineError(self.engine_code, msg) from e
class GoogleFallaEngine(FallaSearchEngine):
class BingFallaEngine(FallaSearchEngine):
class DuckDuckGoFallaEngine(FallaSearchEngine):
class YahooFallaEngine(FallaSearchEngine):
class AskFallaEngine(FallaSearchEngine):
class AolFallaEngine(FallaSearchEngine):
class DogpileFallaEngine(FallaSearchEngine):
class GibiruFallaEngine(FallaSearchEngine):
class MojeekFallaEngine(FallaSearchEngine):
class QwantFallaEngine(FallaSearchEngine):
class YandexFallaEngine(FallaSearchEngine):
async def google_falla(query: str, **kwargs: Any) -> list[SearchResult]:
    return await search(
async def bing_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def duckduckgo_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def yahoo_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def ask_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def aol_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def dogpile_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def gibiru_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def mojeek_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def qwant_falla(query: str, **kwargs: Any) -> list[SearchResult]:
async def yandex_falla(query: str, **kwargs: Any) -> list[SearchResult]:

================
File: plugins/repos/twat_search/src/twat_search/web/engines/google_scraper.py
================
    class GoogleSearchResult:  # type: ignore
        def __init__(self, url: str, title: str, description: str):
    def google_search(*args, **kwargs):  # type: ignore
logger = logging.getLogger(__name__)
class GoogleScraperResult(BaseModel):
class GoogleScraperEngine(SearchEngine):
    def __init__(
        super().__init__(config, **kwargs)
        self.num_results: int = num_results or self.config.default_params.get(
        self.language: str = language or self.config.default_params.get(
        self.region: str | None = country or self.config.default_params.get(
            else self.config.default_params.get("safe", "active")
        self.sleep_interval: float = kwargs.get(
        ) or self.config.default_params.get("sleep_interval", 0.0)
        self.ssl_verify: bool | None = kwargs.get(
        ) or self.config.default_params.get("ssl_verify", None)
        self.proxy: str | None = kwargs.get("proxy") or self.config.default_params.get(
        self.unique: bool = kwargs.get("unique") or self.config.default_params.get(
            unused_params.append(f"time_frame='{time_frame}'")
            logger.debug(
                f"Parameters {', '.join(unused_params)} set but not used by Google Scraper",
    def _convert_result(self, result: GoogleSearchResult) -> SearchResult | None:
            logger.warning("Empty result received from Google Scraper")
            title = getattr(result, "title", "") or ""
            description = getattr(result, "description", "") or ""
            validated = GoogleScraperResult(
                url=HttpUrl(result.url),
            return SearchResult(
                    "url": str(result.url),
            logger.warning(f"Validation error for result: {exc}")
            logger.warning(f"Unexpected error converting result: {exc}")
    async def search(self, query: str) -> list[SearchResult]:
            raise EngineError(self.engine_code, "Search query cannot be empty")
        logger.info(f"Searching Google with query: '{query}'")
            raw_results = list(
                google_search(
                logger.info("No results returned from Google Scraper")
                f"Received {len(raw_results)} raw results from Google Scraper",
            logger.error(error_msg)
            raise EngineError(self.engine_code, error_msg) from exc
            for search_result in (self._convert_result(cast(GoogleSearchResult, result)) for result in raw_results)
        logger.info(
            f"Returning {len(results)} validated results from Google Scraper",
        return self.limit_results(results)
async def google_scraper(
    return await search(

================
File: plugins/repos/twat_search/src/twat_search/web/engines/hasdata.py
================
class HasDataGoogleResult(BaseModel):
    def from_api_result(cls, result: dict[str, Any]) -> HasDataGoogleResult:
        return cls(
            title=result.get("title", ""),
            url=HttpUrl(result.get("link", "")),
            snippet=result.get("snippet", ""),
class HasDataBaseEngine(SearchEngine):
    def __init__(
        super().__init__(config)
        self.location = location or kwargs.get("location") or self.config.default_params.get("location")
            device_type or kwargs.get("device_type") or self.config.default_params.get("device_type", "desktop")
            raise EngineError(
                f"HasData API key is required. Set it via one of these env vars: {', '.join(self.env_api_key_names)}",
    async def search(self, query: str) -> list[SearchResult]:
        async with httpx.AsyncClient() as client:
                response = await client.get(
                response.raise_for_status()
                data = response.json()
                organic_results = data.get("organicResults", [])
                for i, result in enumerate(organic_results):
                        parsed = HasDataGoogleResult.from_api_result(result)
                        results.append(
                            SearchResult(
class HasDataGoogleEngine(HasDataBaseEngine):
class HasDataGoogleLightEngine(HasDataBaseEngine):
async def hasdata_google_full(
    config = EngineConfig(api_key=api_key, enabled=True)
    engine = HasDataGoogleEngine(
    return await engine.search(query)
async def hasdata_google(
    engine = HasDataGoogleLightEngine(

================
File: plugins/repos/twat_search/src/twat_search/web/engines/pplx.py
================
logger = logging.getLogger(__name__)
class PerplexityResult(BaseModel):
    answer: str = Field(default="")
    url: str = Field(default="https://perplexity.ai")
    title: str = Field(default="Perplexity AI Response")  # Default title
class PerplexitySearchEngine(SearchEngine):
    def __init__(
        super().__init__(config, **kwargs)
        self.model = self.kwargs.get("model") or self.config.default_params.get("model", "sonar")
    async def search(self, query: str) -> list[SearchResult]:
            raise EngineError(self.engine_code, "Search query cannot be empty")
            "search_recency_filter": self.kwargs.get("time_frame", ""),
            response = await self.make_http_request(
            data = response.json()
            raise EngineError(
                f"API request failed: {str(e)!s}",
                f"Unexpected error: {str(e)!s}",
        for choice in data.get("choices", []):
            answer = choice.get("message", {}).get("content", "")
                pr = PerplexityResult(answer=answer, url=url, title=title)
                url_obj = HttpUrl(pr.url)  # Validate URL format
                results.append(
                    SearchResult(
                if len(results) >= self.max_results:
                logger.warning(f"Invalid result from Perplexity: {e}")
        return self.limit_results(results)
async def pplx(
    logger.debug(f"pplx function called with api_key parameter: {api_key is not None}")
            env_value = os.environ.get(env_var)
            logger.debug(f"Environment variable {env_var} in pplx function: {env_value is not None}")
                logger.debug(f"Using API key from environment variable {env_var}")
        key_prefix = api_key[:4] if len(api_key) > 4 else "****"
        key_suffix = api_key[-4:] if len(api_key) > 4 else "****"
        logger.debug(f"Final API key value: {key_prefix}...{key_suffix}")
        logger.error(
    config = EngineConfig(
    logger.debug(f"Created EngineConfig with api_key: {config.api_key is not None}")
        logger.debug("Creating PerplexitySearchEngine instance")
        engine = PerplexitySearchEngine(
        logger.debug("PerplexitySearchEngine instance created successfully")
        logger.debug("Executing search with PerplexitySearchEngine")
        results = await engine.search(query)
        logger.debug(f"Search completed, got {len(results)} results")
        logger.error(f"Error in pplx function: {e}")

================
File: plugins/repos/twat_search/src/twat_search/web/engines/serpapi.py
================
class SerpApiResult(BaseModel):
class SerpApiResponse(BaseModel):
class SerpApiSearchEngine(SearchEngine):
    def __init__(
        super().__init__(config)
            "num": kwargs.get("num", num_results) or self.config.default_params.get("num", 10),
            "google_domain": kwargs.get("google_domain")
            or self.config.default_params.get("google_domain", "google.com"),
            "gl": kwargs.get("gl", country) or self.config.default_params.get("gl"),
            "hl": kwargs.get("hl", language) or self.config.default_params.get("hl"),
            "safe": _convert_safe(kwargs.get("safe", safe_search)) or self.config.default_params.get("safe"),
            "time_period": kwargs.get("time_period", time_frame) or self.config.default_params.get("time_period"),
            raise EngineError(
                f"SerpApi API key is required. Set it via one of these env vars: {', '.join(self.env_api_key_names)}",
    async def search(self, query: str) -> list[SearchResult]:
        params.update({k: v for k, v in self._params.items() if v is not None})
        async with httpx.AsyncClient() as client:
                response = await client.get(
                response.raise_for_status()
                data = response.json()
                serpapi_response = SerpApiResponse(**data)
                        results.append(
                            SearchResult(
                                raw=result.model_dump(),  # Include raw result for debugging
                return self.limit_results(results)
def _convert_safe(safe: bool | str | None) -> str | None:
    if isinstance(safe, bool):
async def google_serpapi(
    config = EngineConfig(
    engine = SerpApiSearchEngine(
    return await engine.search(query)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/tavily.py
================
class TavilySearchResult(BaseModel):
class TavilySearchResponse(BaseModel):
class TavilySearchEngine(SearchEngine):
    def __init__(
        super().__init__(config, **kwargs)
            self.num_results = kwargs.get("num_results") or self.config.default_params.get(
        self.search_depth = search_depth or self.config.default_params.get("search_depth", "basic")
        self.include_domains = include_domains or self.config.default_params.get("include_domains", None)
        self.exclude_domains = exclude_domains or self.config.default_params.get("exclude_domains", None)
        self.include_answer = include_answer or self.config.default_params.get("include_answer", False)
        self.max_tokens = max_tokens or self.config.default_params.get("max_tokens", None)
        self.search_type = search_type or self.config.default_params.get("search_type", "search")
            raise EngineError(
                f"Tavily API key is required. Set it via one of these env vars: {', '.join(self.env_api_key_names)}",
    def _build_payload(self, query: str) -> dict:
    def _convert_result(self, item: dict, rank: int) -> SearchResult | None:
            validated_url = HttpUrl(item.get("url", ""))
            return SearchResult(
                title=item.get("title", ""),
                snippet=textwrap.shorten(
                    item.get("content", "").strip(),
    async def search(self, query: str) -> list[SearchResult]:
        payload = self._build_payload(query)
        async with httpx.AsyncClient() as client:
                response = await client.post(
                response.raise_for_status()
                data = response.json()
                raise EngineError(self.engine_code, f"HTTP error: {e}")
                raise EngineError(self.engine_code, f"Request error: {e}")
                raise EngineError(self.engine_code, f"Error: {e!s}")
            parsed_response = TavilySearchResponse.model_validate(data)
            items = [item.model_dump() for item in parsed_response.results]
            items = data.get("results", [])
        for idx, item in enumerate(items, start=1):
            converted = self._convert_result(item, idx)
                results.append(converted)
async def tavily(
    config = EngineConfig(
    engine = TavilySearchEngine(
    return await engine.search(query)

================
File: plugins/repos/twat_search/src/twat_search/web/engines/you.py
================
logger = logging.getLogger(__name__)
class YouSearchHit(BaseModel):
    snippet: str = Field(alias="description")
class YouSearchResponse(BaseModel):
    search_id: str | None = Field(None, alias="searchId")
class YouNewsArticle(BaseModel):
class YouNewsResponse(BaseModel):
class YouBaseEngine(SearchEngine):
    def __init__(
        super().__init__(config, **kwargs)
        self.country_code = kwargs.get("country") or self.config.default_params.get(
    async def _make_api_call(self, query: str) -> Any:
            params["safesearch"] = str(self.safe_search).lower()
        logger.debug(f"Making You.com API request to {self.base_url} with params: {params}")
            response = await self.make_http_request(
            return response.json()
            raise EngineError(self.engine_code, f"API request failed: {str(e)!s}")
            raise EngineError(self.engine_code, f"Unexpected error: {str(e)!s}")
class YouSearchEngine(YouBaseEngine):
    async def search(self, query: str) -> list[SearchResult]:
            raise EngineError(self.engine_code, "Search query cannot be empty")
        data = await self._make_api_call(query)
            you_response = YouSearchResponse(**data)
                    results.append(
                        SearchResult(
                            raw=hit.model_dump(by_alias=True),
                    if len(results) >= self.max_results:
                    logger.warning(f"Invalid result from You.com: {e}")
            return self.limit_results(results)
            raise EngineError(
class YouNewsSearchEngine(YouBaseEngine):
            you_response = YouNewsResponse(**data)
                            raw=article.model_dump(by_alias=True),
                    logger.warning(f"Invalid news result from You.com: {e}")
async def you(
    config = EngineConfig(
    engine = YouSearchEngine(
    return await engine.search(query)
async def you_news(
    engine = YouNewsSearchEngine(

================
File: plugins/repos/twat_search/src/twat_search/web/__init__.py
================
    __all__.extend(["Config", "EngineConfig", "SearchResult", "search"])
    __all__.extend(["brave", "brave_news"])
    __all__.extend(["pplx"])
    __all__.extend(["tavily"])
    __all__.extend(["you", "you_news"])
    __all__.extend(["critique"])
    __all__.extend(["duckduckgo"])
    __all__.extend(["bing_scraper"])
    __all__.extend(["serpapi"])

================
File: plugins/repos/twat_search/src/twat_search/web/api.py
================
logger = logging.getLogger(__name__)
def get_engine_params(
    std_engine_name = standardize_engine_name(engine_name)
    for k, v in kwargs.items():
        if k.startswith(std_engine_name + "_"):
            engine_specific[k[len(std_engine_name) + 1 :]] = v
        elif k.startswith(engine_name + "_"):  # For backward compatibility
            engine_specific[k[len(engine_name) + 1 :]] = v
    std_engines = [standardize_engine_name(e) for e in engines]
    non_prefixed = {k: v for k, v in kwargs.items() if not any(k.startswith(e + "_") for e in std_engines + engines)}
def init_engine_task(
    engine_config = config.engines.get(std_engine_name)
        engine_config = config.engines.get(engine_name)
            logger.warning(f"Engine '{engine_name}' not configured, using default configuration.")
            engine_config = EngineConfig(enabled=True)
    num_results = kwargs.get("num_results")
        logger.debug(f"Initializing engine '{engine_name}' with num_results={num_results}")
        engine_params = get_engine_params(
            engines=kwargs.get("engines", []),
            common_params=kwargs.get("common_params", {}),
        engine_instance: SearchEngine = get_engine(
        logger.info(f"🔍 Querying engine: {engine_name}")
        async def search_with_error_handling() -> list[SearchResult]:
                return await engine_instance.search(query)
                logger.error(f"Search with engine '{engine_name}' failed: {e}")
        return engine_name, search_with_error_handling()
        error_type = type(e).__name__
        error_msg = str(e)
            logger.error(
            logger.warning(f"Engine '{engine_name}' is disabled. Enable it in your configuration to use it.")
            logger.error(f"Failed to initialize engine '{engine_name}': {error_type}: {error_msg}")
async def search(
        config = config or Config()
        if engines is not None and len(engines) == 0:
            logger.error(msg)
            raise SearchError(msg)
        explicit_engines_requested = engines is not None and len(engines) > 0
        engines_to_try = engines or list(config.engines.keys())
        logger.debug(f"Search requested with num_results={num_results}")
        engines_to_try = [standardize_engine_name(e) for e in engines_to_try]
                            default_result_count = engine_config.default_params.get("result_count")
                task_result = init_engine_task(
                    engine_names.append(task_result[0])
                    tasks.append(task_result[1])
                    failed_engines.append(engine_name)
                logger.warning(f"Unexpected error initializing engine '{engine_name}': {e}")
            failed_engines_str = ", ".join(failed_engines)
            logger.warning(f"Failed to initialize engines: {failed_engines_str}")
                msg = f"No search engines could be initialized from requested engines: {', '.join(engines_to_try)}"
            logger.warning("Falling back to any available search engine...")
            for engine_name in get_registered_engines():
                        logger.info(f"Successfully fell back to engine: {engine_name}")
                    logger.debug(f"Failed to initialize fallback engine {engine_name}: {e}")
        results = await asyncio.gather(*tasks, return_exceptions=True)
        for engine_name, result in zip(engine_names, results, strict=False):
            if isinstance(result, Exception):
            elif isinstance(result, list):
                result_count = len(result)
                    logger.info(
                    flattened_results.extend(result)
                logger.warning(
                    f"⚠️ Engine '{engine_name}' returned unexpected type: {type(result)}",
        logger.error(f"Search failed: {e}")
    logger.info(f"Total results found across all engines: {len(flattened_results)}")

================
File: plugins/repos/twat_search/src/twat_search/web/cli.py
================
class CustomJSONEncoder(json_lib.JSONEncoder):
    def default(self, o: Any) -> Any:
            return str(o)
            return json_lib.JSONEncoder.default(self, o)
console = Console()
class SearchCLI:
    def __init__(self) -> None:
        self.logger = logging.getLogger("twat_search.cli")
        self.log_handler = RichHandler(rich_tracebacks=True)
        self._configure_logging()
        self.console = Console()
        available_engines = get_available_engines()
            self.logger.warning(
                f"{', '.join(missing_engines)}. "
    def _configure_logging(self, verbose: bool = False) -> None:
        logging.basicConfig(
        self.logger.setLevel(level)
        logging.getLogger("twat_search.web.api").setLevel(level)
        logging.getLogger("twat_search.web.engines").setLevel(level)
        logging.getLogger("httpx").setLevel(level)
    def _parse_engines(self, engines_arg: Any) -> list[str] | None:
        if isinstance(engines_arg, str):
            if engines_arg.strip().lower() == "free":
                self.logger.info(f"Using 'free' engines: {', '.join(engines)}")
            if engines_arg.strip().lower() == "best":
                self.logger.info(f"Using 'best' engines: {', '.join(engines)}")
            if engines_arg.strip().lower() == "all":
                engines = get_available_engines()
                self.logger.info(
                    f"Using 'all' available engines: {', '.join(engines)}",
            engines = [e.strip() for e in engines_arg.split(",") if e.strip()]
            return [standardize_engine_name(e) for e in engines]
        if isinstance(engines_arg, list | tuple):
            engines = [str(e).strip() for e in engines_arg if str(e).strip()]
            f"Unexpected engines type: {type(engines_arg)}. Using all available engines.",
    async def _run_search(
                standardized_name = standardize_engine_name(engine)
                    available.append(engine)
                    invalid_engines.append(engine)
                        f"Engine '{engine}' is not valid. Valid engines are: {', '.join(ALL_POSSIBLE_ENGINES)}",
                error_msg = f"None of the specified engines are valid: {', '.join(invalid_engines)}"
                self.logger.error(error_msg)
                _display_errors([error_msg])
            self.logger.debug(f"Attempting to search with engines: {engines}")
            results = await search(query=query, engines=engines, strict_mode=True, **kwargs)
            return _process_results(results)
            self.logger.error(f"Search failed: {e}")
            _display_errors([str(e)])
    async def _search_engine(
            engine_func = get_engine_function(engine)
                self.logger.warning(error_msg)
            registered_engines = get_registered_engines()
            engine_class = registered_engines.get(engine)
            friendly = engine_class.friendly_engine_name if engine_class else ENGINE_FRIENDLY_NAMES.get(engine, engine)
            friendly = ENGINE_FRIENDLY_NAMES.get(engine, engine)
            self.console.print(f"[bold]Searching {friendly}[/bold]: {query}")
            results = await engine_func(query=query, **params)
            processed_results = _process_results(results)
                _display_json_results(processed_results)
            _display_results(processed_results, verbose, plain)
            self.logger.error(f"{friendly} search failed: {e}")
    def q(
        self._configure_logging(verbose)
            self.logger.debug(f"Overriding num_results={num_results} with n={kwargs['n']}")
        self.logger.debug(f"Args - n: {kwargs.get('n')}, num_results: {num_results}")
        self.logger.debug(f"Using num_results={num_results}")
            self.logger.debug(
        engine_list = self._parse_engines(engines)
        common_params = {k: v for k, v in common_params.items() if v is not None}
        self.logger.debug(f"Right before executing search, num_results={num_results}")
            results = asyncio.run(
                self._run_search(
            with self.console.status(
            _display_json_results(results)
        _display_results(results, verbose, plain)
    def info(
            config = Config()
                self._display_engines_json(engine, config)
                self._display_engines_plain(engine, config)
                self._list_all_engines(config)
                self._show_engine_details(engine, config)
                self.logger.error(
    def _display_engines_plain(self, engine: str | None, config: Config) -> None:
                self.console.print(engine)
            for engine_name in sorted(config.engines.keys()):
                self.console.print(engine_name)
    def _list_all_engines(self, config: Config) -> None:
        table = Table(title="🔎 Available Search Engines")
        table.add_column("Engine", style="cyan", no_wrap=True)
        table.add_column("Enabled", style="magenta")
        table.add_column("API Key Required", style="yellow")
        sorted_engines = sorted(config.engines.items(), key=lambda x: x[0])
                hasattr(
                if engine_class and hasattr(engine_class, "env_api_key_names"):
                    api_key_required = bool(engine_class.env_api_key_names)
            table.add_row(
        self.console.print(table)
        self.console.print(
    def _show_engine_details(self, engine_name: str, config: Config) -> None:
            self.console.print("\nAvailable engines:")
                self.console.print(f"- {name}")
        api_key_required = hasattr(engine_config, "api_key") and engine_config.api_key is not None
            engine_class = registered_engines.get(engine_name)
            if not api_key_required and engine_class and hasattr(engine_class, "env_api_key_names"):
                if engine_class and hasattr(engine_class, "friendly_name")
                else ENGINE_FRIENDLY_NAMES.get(engine_name, engine_name)
                    value_status = "✅" if os.environ.get(env_name) else "❌"
                    self.console.print(f"  {env_name}: {value_status}")
            self.console.print("\n[bold]Default Parameters:[/bold]")
                for param, value in engine_config.default_params.items():
                    self.console.print(f"  {param}: {value}")
                self.console.print("  No default parameters specified")
                base_engine = engine_name.split("-")[0]
                engine_module = importlib.import_module(module_name)
                function_name = engine_name.replace("-", "_")
                if hasattr(engine_module, function_name):
                    func = getattr(engine_module, function_name)
                    self.console.print("\n[bold]Function Interface:[/bold]")
                        f"  [green]{function_name}()[/green] - {func.__doc__.strip().split('\\n')[0]}",
                    self.console.print("\n[bold]Example Usage:[/bold]")
            self.console.print("\n[bold]Basic Configuration:[/bold]")
    def _display_engines_json(self, engine: str | None, config: Config) -> None:
            result[engine] = _get_engine_info(
            for engine_name, engine_config in sorted(config.engines.items()):
                result[engine_name] = _get_engine_info(
    async def critique(
            params["source_whitelist"] = [domain.strip() for domain in source_whitelist.split(",")]
            params["source_blacklist"] = [domain.strip() for domain in source_blacklist.split(",")]
        params.update(kwargs)
        return await self._search_engine(
    async def brave(
        params = {k: v for k, v in params.items() if v is not None}
        return await self._search_engine("brave", query, params, json, verbose, plain)
    async def brave_news(
    async def serpapi(
        return await self._search_engine("serpapi", query, params, json, verbose, plain)
    async def tavily(
            params["include_domains"] = [s.strip() for s in include_domains.split(",") if s.strip()]
            params["exclude_domains"] = [s.strip() for s in exclude_domains.split(",") if s.strip()]
        return await self._search_engine("tavily", query, params, json, verbose, plain)
    async def pplx(
        self.logger.debug(f"CLI pplx method called with api_key: {api_key is not None}")
            env_value = os.environ.get(env_var)
            self.logger.debug(f"CLI pplx method - Environment variable {env_var}: {env_value is not None}")
        self.logger.debug(f"CLI pplx method - params: {params}")
        self.logger.debug(f"CLI pplx method - final params: {params}")
        return await self._search_engine("pplx", query, params, json, verbose, plain)
    async def you(
        return await self._search_engine("you", query, params, json, verbose, plain)
    async def you_news(
    async def duckduckgo(
    async def hasdata_google(
    async def hasdata_google_light(
def _check_engine_availability(engine_name: str) -> bool:
    return is_engine_available(engine_name)
def _get_engine_info(
    if hasattr(engine_config, "api_key") and engine_config.api_key is not None:
                {"name": env_name, "set": bool(os.environ.get(env_name))} for env_name in engine_class.env_api_key_names
        if hasattr(
        "enabled": engine_config.enabled if hasattr(engine_config, "enabled") else False,
def _process_results(results: list) -> list[dict[str, Any]]:
        engine_name = getattr(result, "source", None) or "unknown"
        engine_results.setdefault(engine_name, []).append(result)
    for engine, engine_results_list in engine_results.items():
            processed.append(
        for idx, result in enumerate(engine_results_list):
            url = str(result.url)
                    "snippet": result.snippet[:100] + "..." if len(result.snippet) > 100 else result.snippet,
                    "raw_result": getattr(result, "raw", None),
def _display_results(
        console.print("[bold red]No results found![/bold red]")
        urls = set()
                urls.add(result["url"])
        for url in sorted(urls):
            console.print(url)
    table = Table()  # Remove show_lines=True to eliminate row separator lines
        table.add_column("Status", style="magenta")
        table.add_column("Title", style="green")
        table.add_column("URL", style="blue", overflow="fold")
        table.add_column("URL", style="blue", overflow="fold", max_width=70)
            table.add_row(result["engine"], result["url"])
    console.print(table)
                console.print(result)
def _display_errors(error_messages: list[str]) -> None:
    table = Table(title="❌ Search Errors")
    table.add_column("Error", style="red")
        table.add_row(error)
def _display_json_results(processed_results: list[dict[str, Any]]) -> None:
        results_by_engine[engine]["results"].append(
                "snippet": result.get("snippet") if result.get("snippet") != "N/A" else None,
                "raw": result.get("raw_result"),
    console.print(json_lib.dumps(results_by_engine, indent=2, cls=CustomJSONEncoder))
def main() -> None:
    fire.core.Display = lambda lines, out: console.print(*lines)
    fire.Fire(SearchCLI)
    main()

================
File: plugins/repos/twat_search/src/twat_search/web/config.py
================
load_dotenv()
logger = logging.getLogger(__name__)
class EngineConfig(BaseModel):
    default_params: dict[str, Any] = Field(default_factory=dict)
    def __init__(self, **data: Any) -> None:
        super().__init__(**data)
            logger.debug(f"No API key provided for {self.engine_code}, checking environment variables")
                engine_class = get_registered_engines().get(self.engine_code)
                if engine_class and hasattr(engine_class, "env_api_key_names") and engine_class.env_api_key_names:
                    logger.debug(
                        env_value = os.environ.get(env_var)
                        logger.debug(f"Checking env var '{env_var}' in __init__: {env_value is not None}")
                            logger.debug(f"Using API key from environment variable '{env_var}'")
                logger.debug(f"Couldn't check API key requirements for {self.engine_code}: {e}")
    @field_validator("api_key")
    def validate_api_key(cls, v: str | None, info: Any) -> str | None:
        engine_code = info.data.get("engine_code")
        logger.debug(f"Validating API key for engine '{engine_code}', current value: {v is not None}")
                engine_class = get_registered_engines().get(engine_code)
                logger.debug(f"Found engine class for '{engine_code}': {engine_class is not None}")
                        logger.debug(f"Checking env var '{env_var}': {env_value is not None}")
                    logger.warning(
                        f"Please set one of these environment variables: {', '.join(engine_class.env_api_key_names)}",
                logger.debug(f"Couldn't check API key requirements for {engine_code}: {e}")
class Config(BaseModel):
    engines: dict[str, EngineConfig] = Field(default_factory=dict)
    def get_config_path(cls) -> Path:
            return Path(cls.config_path)
        env_path = os.environ.get("TWAT_SEARCH_CONFIG_PATH")
            return Path(env_path)
        home_config = Path.home() / ".twat" / "search_config.json"
        if home_config.exists():
        xdg_config = Path.home() / ".config" / "twat" / "search_config.json"
        if xdg_config.exists():
            config_data = json.loads(json.dumps(DEFAULT_CONFIG))  # Deep copy
        config_path = self.get_config_path()
        if config_path.exists():
                with open(config_path, encoding="utf-8") as f:
                    file_config = json.load(f)
                self._merge_config(config_data, file_config)
                logger.info(f"Loaded configuration from {config_path}")
                logger.error(f"Error loading configuration from {config_path}: {e}")
        _apply_env_overrides(config_data)
            self._merge_config(config_data, data)
        super().__init__(**config_data)
    def _merge_config(self, target: dict[str, Any], source: dict[str, Any]) -> None:
        for key, value in source.items():
            if key in target and isinstance(target[key], dict) and isinstance(value, dict):
                self._merge_config(target[key], value)
    def add_engine(
            engine_config.default_params.update(default_params)
            self.engines[engine_name] = EngineConfig(
def _apply_env_overrides(config_data: dict[str, Any]) -> None:
    for env_var, config_path in ENV_VAR_MAP.items():
            value = _parse_env_value(env_value)
            if isinstance(config_path, str):
                _set_nested_value(config_data, config_path, value)
                _set_nested_value(config_data, path, value)
def _parse_env_value(env_value: str) -> Any:
    if env_value.startswith("{") and env_value.endswith("}"):
            return json.loads(env_value)
    if env_value.lower() in ("true", "yes", "1"):
    if env_value.lower() in ("false", "no", "0"):
    if env_value.isdigit():
        return int(env_value)
    if env_value.replace(".", "", 1).isdigit():
        return float(env_value)
def _set_nested_value(config_data: dict[str, Any], path: list[str], value: Any) -> None:
    for i, path_part in enumerate(path):
        if i == len(path) - 1:
            if path_part not in current or not isinstance(current[path_part], dict):

================
File: plugins/repos/twat_search/src/twat_search/web/engine_constants.py
================
def standardize_engine_name(engine_name: str) -> str:
    return engine_name.replace("-", "_")

================
File: plugins/repos/twat_search/src/twat_search/web/exceptions.py
================
class SearchError(Exception):
    def __init__(self, message: str) -> None:
        super().__init__(message)
class EngineError(SearchError):
    def __init__(self, engine_name: str, message: str) -> None:
        super().__init__(f"Engine '{engine_name}': {message}")

================
File: plugins/repos/twat_search/src/twat_search/web/models.py
================
class SearchResult(BaseModel):
    @field_validator("source")
    def validate_non_empty(cls, v: str) -> str:
        if not v or not v.strip():
            raise ValueError(msg)
        return v.strip()
    @field_validator("title", "snippet")
    def ensure_string(cls, v: str) -> str:
        return v.strip() if v and v.strip() else ""

================
File: plugins/repos/twat_search/src/twat_search/web/utils.py
================
logger = logging.getLogger(__name__)
def load_environment(force_reload: bool = False) -> None:
    if not force_reload and os.environ.get(loaded_flag) == "1":
        logger.debug("Environment variables already loaded")
    load_dotenv()
    if logger.isEnabledFor(logging.DEBUG):  # Only in debug mode
        for key, value in os.environ.items():
                logger.debug(
        logger.info("Environment variables loaded")
class RateLimiter:
    def __init__(self, calls_per_second: float) -> None:
    async def wait(self) -> None:
        current_time = time.time()
            logger.debug(f"Rate limiter sleeping for {delay:.4f} seconds")
            time.sleep(delay)
        self.last_call_time = time.time()
    def wait_if_needed(self) -> None:
        if len(self.call_timestamps) >= self.calls_per_second:
            oldest_timestamp = min(self.call_timestamps)
        self.call_timestamps.append(time.time())
def extract_domain(url: str) -> str:
    parsed = urlparse(url)
        parsed = urlparse(f"https://{url}")
def extract_query_param(url: str, param: str) -> str | None:
    params = parse_qs(parsed.query)
    return params.get(param, [None])[0]

================
File: plugins/repos/twat_search/src/twat_search/__init__.py
================
    __all__.append("__version__")
    __all__.append("web")

================
File: plugins/repos/twat_search/src/twat_search/__main__.py
================
logging.basicConfig(
    handlers=[RichHandler(rich_tracebacks=True)],
logger = logging.getLogger(__name__)
console = Console()
SearchCLIType = TypeVar("SearchCLIType")
class TwatSearchCLI:
    def __init__(self) -> None:
            self.web: Any = web_cli.SearchCLI()
            logger.error(f"Web CLI not available: {e!s}")
            logger.error(
    def _cli_error(*args: Any, **kwargs: Any) -> int:
        console.print(
    def version() -> str:
def main() -> None:
    install(show_locals=True)
    ansi_decoder = AnsiDecoder()
    console = Console(theme=Theme({"prompt": "cyan", "question": "bold cyan"}))
    def display(lines, out):
        console.print(Group(*map(ansi_decoder.decode_line, lines)))
    fire.Fire(TwatSearchCLI, name="twat-search")
    main()

================
File: plugins/repos/twat_search/src/twat_search/__version__.py
================


================
File: plugins/repos/twat_search/tests/unit/web/engines/__init__.py
================


================
File: plugins/repos/twat_search/tests/unit/web/engines/test_base.py
================
class TestSearchEngine(SearchEngine):
    async def search(self, query: str) -> list[SearchResult]:
            SearchResult(
                url=HttpUrl("https://example.com/test"),
register_engine(TestSearchEngine)
class DisabledTestSearchEngine(SearchEngine):
        raise NotImplementedError(msg)
register_engine(DisabledTestSearchEngine)
def test_search_engine_is_abstract() -> None:
    assert hasattr(SearchEngine, "__abstractmethods__")
    with pytest.raises(TypeError):
        SearchEngine(EngineConfig())  # type: ignore
def test_search_engine_name_class_var() -> None:
    assert hasattr(SearchEngine, "engine_code")
def test_engine_registration() -> None:
    class NewEngine(SearchEngine):
    returned_class = register_engine(NewEngine)
    engine_instance = get_engine("new_engine", EngineConfig())
    assert isinstance(engine_instance, NewEngine)
def test_get_engine_with_invalid_name() -> None:
    with pytest.raises(SearchError, match="Unknown search engine"):
        get_engine("nonexistent_engine", EngineConfig())
def test_get_engine_with_disabled_engine() -> None:
    config = EngineConfig(enabled=False)
    with pytest.raises(SearchError, match="is disabled"):
        get_engine("disabled_engine", config)
def test_get_engine_with_config() -> None:
    config = EngineConfig(
    engine = get_engine("test_engine", config)
def test_get_engine_with_kwargs() -> None:
    engine = get_engine("test_engine", EngineConfig(), **kwargs)

================
File: plugins/repos/twat_search/tests/unit/web/__init__.py
================


================
File: plugins/repos/twat_search/tests/unit/web/test_api.py
================
logging.basicConfig(level=logging.DEBUG)
T = TypeVar("T")
class MockSearchEngine(SearchEngine):
    def __init__(self, config: EngineConfig, **kwargs: Any) -> None:
        super().__init__(config, **kwargs)
        self.should_fail = kwargs.get("should_fail", False)
    async def search(self, query: str) -> list[SearchResult]:
            raise Exception(msg)
        result_count = self.kwargs.get("result_count", 1)
            SearchResult(
                url=HttpUrl(f"https://example.com/{i + 1}"),
            for i in range(result_count)
register_engine(MockSearchEngine)
def mock_config() -> Config:
    config = Config()
        "mock": EngineConfig(
async def setup_teardown() -> AsyncGenerator[None]:
    tasks = [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
    with contextlib.suppress(asyncio.CancelledError):
        await asyncio.gather(*tasks)
async def test_search_with_mock_engine(
    results = await search("test query", engines=["mock"], config=mock_config)
    assert len(results) == 2
    assert all(isinstance(result, SearchResult) for result in results)
    assert all(result.source == "mock" for result in results)
async def test_search_with_additional_params(
    results = await search(
    assert len(results) == 3
async def test_search_with_engine_specific_params(
    assert len(results) == 4
async def test_search_with_no_engines(setup_teardown: None) -> None:
    with pytest.raises(SearchError, match="No search engines configured"):
        await search("test query", engines=[])
async def test_search_with_failing_engine(
    assert len(results) == 0
async def test_search_with_nonexistent_engine(
    with pytest.raises(SearchError, match="No search engines could be initialized"):
        await search("test query", engines=["nonexistent"], config=mock_config)
async def test_search_with_disabled_engine(
        await search("test query", engines=["mock"], config=mock_config)

================
File: plugins/repos/twat_search/tests/unit/web/test_config.py
================
def test_engine_config_defaults() -> None:
    config = EngineConfig()
def test_engine_config_values() -> None:
    config = EngineConfig(
def test_config_defaults(isolate_env_vars: None) -> None:
    config = Config()
    assert isinstance(config.engines, dict)
    assert len(config.engines) == 0
def test_config_with_env_vars(monkeypatch: MonkeyPatch, env_vars_for_brave: None) -> None:
def test_config_with_direct_initialization() -> None:
    custom_config = Config(
        engines={"test_engine": EngineConfig(api_key="direct_key", enabled=True, default_params={"count": 5})},
def test_config_env_vars_override_direct_config(monkeypatch: MonkeyPatch) -> None:
    monkeypatch.setenv("BRAVE_API_KEY", "env_key")
        engines={"brave": EngineConfig(api_key="direct_key", enabled=True, default_params={"count": 5})},

================
File: plugins/repos/twat_search/tests/unit/web/test_exceptions.py
================
def test_search_error() -> None:
    exception = SearchError(error_message)
    assert str(exception) == error_message
    assert isinstance(exception, Exception)
def test_engine_error() -> None:
    exception = EngineError(engine_name, error_message)
    assert str(exception) == f"Engine '{engine_name}': {error_message}"
    assert isinstance(exception, SearchError)
def test_engine_error_inheritance() -> None:
        raise EngineError(msg, "Test error")
        if isinstance(e, EngineError):
def test_search_error_as_base_class() -> None:
        raise SearchError(msg)
        exceptions.append(e)
        raise EngineError(msg, "API key missing")
    assert len(exceptions) == 2
    assert isinstance(exceptions[0], SearchError)
    assert isinstance(exceptions[1], EngineError)
    assert "General search error" in str(exceptions[0])
    assert "Engine 'brave': API key missing" in str(exceptions[1])

================
File: plugins/repos/twat_search/tests/unit/web/test_models.py
================
def test_search_result_valid_data() -> None:
    url = HttpUrl("https://example.com")
    result = SearchResult(
    assert str(result.url) == "https://example.com/"
def test_search_result_with_optional_fields() -> None:
def test_search_result_invalid_url() -> None:
    with pytest.raises(ValidationError):
        SearchResult.model_validate(
def test_search_result_empty_fields() -> None:
        SearchResult(
def test_search_result_serialization() -> None:
    result_dict = result.model_dump()
    assert str(result_dict["url"]) == "https://example.com/"
    result_json = result.model_dump_json()
    assert isinstance(result_json, str)
def test_search_result_deserialization() -> None:
    result = SearchResult.model_validate(data)

================
File: plugins/repos/twat_search/tests/unit/web/test_utils.py
================
def rate_limiter() -> RateLimiter:
    return RateLimiter(calls_per_second=5)
def test_rate_limiter_init() -> None:
    limiter = RateLimiter(calls_per_second=10)
def test_rate_limiter_wait_when_not_needed(rate_limiter: RateLimiter) -> None:
    with patch("time.sleep") as mock_sleep:
        rate_limiter.wait_if_needed()
        mock_sleep.assert_not_called()
        for _ in range(3):  # 4 total calls including the one above
def test_rate_limiter_wait_when_needed(rate_limiter: RateLimiter) -> None:
    now = time.time()
    rate_limiter.call_timestamps = [now - 0.01 * i for i in range(rate_limiter.calls_per_second)]
    with patch("time.sleep") as mock_sleep, patch("time.time", return_value=now):
        mock_sleep.assert_called_once()
def test_rate_limiter_cleans_old_timestamps(rate_limiter: RateLimiter) -> None:
    with patch("time.time", return_value=now):
    assert len(rate_limiter.call_timestamps) == len(recent_stamps) + 1  # +1 for the new call
@pytest.mark.parametrize("calls_per_second", [1, 5, 10, 100])
def test_rate_limiter_with_different_rates(calls_per_second: int) -> None:
    limiter = RateLimiter(calls_per_second=calls_per_second)
        for _ in range(calls_per_second):
            limiter.wait_if_needed()
        patch("time.sleep") as mock_sleep,
        patch("time.time", return_value=time.time()),

================
File: plugins/repos/twat_search/tests/unit/__init__.py
================


================
File: plugins/repos/twat_search/tests/unit/mock_engine.py
================
class MockSearchEngine(SearchEngine):
    def __init__(self, config: EngineConfig, **kwargs: Any) -> None:
        super().__init__(config, **kwargs)
        self.should_fail = kwargs.get("should_fail", False)
    async def search(self, query: str) -> list[SearchResult]:
            raise Exception(msg)
        result_count = self.kwargs.get("result_count", 1)
            SearchResult(
                url=HttpUrl(f"https://example.com/{i + 1}"),
            for i in range(result_count)
register_engine(MockSearchEngine)

================
File: plugins/repos/twat_search/tests/web/test_bing_scraper.py
================
class MockSearchResult:
    def __init__(self, title: str, url: str, description: str = "") -> None:
def engine_config() -> EngineConfig:
    return EngineConfig(enabled=True)
def engine(engine_config: EngineConfig) -> BingScraperSearchEngine:
    return BingScraperSearchEngine(config=engine_config, num_results=5)
def mock_results() -> list[MockSearchResult]:
        MockSearchResult(
class TestBingScraperEngine:
    @patch("twat_search.web.engines.bing_scraper.BingScraper")
    def test_init(self, mock_BingScraper: MagicMock, engine: Any) -> None:
        mock_BingScraper.assert_not_called()
    async def test_search_basic(
        mock_instance = MagicMock()
        results = await engine.search("test query")
        assert len(results) == 2
        assert isinstance(results[0], SearchResult)
        assert str(results[0].url) == "https://example.com/1"
        mock_BingScraper.assert_called_once_with(max_retries=3, delay_between_requests=1.0)
        mock_instance.search.assert_called_once_with("test query", num_results=5)
    async def test_custom_parameters(self, mock_BingScraper: MagicMock) -> None:
        engine = BingScraperSearchEngine(
            config=EngineConfig(enabled=True),
        await engine.search("test query")
        mock_BingScraper.assert_called_once_with(max_retries=5, delay_between_requests=2.0)
        mock_instance.search.assert_called_once_with("test query", num_results=10)
    async def test_invalid_url_handling(self, mock_BingScraper: MagicMock, engine: BingScraperSearchEngine) -> None:
        assert len(results) == 1
    @patch("twat_search.web.api.search")
    async def test_bing_scraper_convenience_function(self, mock_search: AsyncMock) -> None:
            SearchResult(
                url=HttpUrl("https://example.com"),
        results = await bing_scraper(
        mock_search.assert_called_once()
    async def test_empty_query(self, mock_BingScraper: MagicMock, engine: BingScraperSearchEngine) -> None:
        with pytest.raises(EngineError) as excinfo:
            await engine.search("")
        assert "Search query cannot be empty" in str(excinfo.value)
    async def test_no_results(self, mock_BingScraper: MagicMock, engine: BingScraperSearchEngine) -> None:
        assert isinstance(results, list)
        assert len(results) == 0
    async def test_network_error(self, mock_BingScraper: MagicMock, engine: BingScraperSearchEngine) -> None:
        mock_instance.search.side_effect = ConnectionError("Network timeout")
        assert "Network error connecting to Bing" in str(excinfo.value)
    async def test_parsing_error(self, mock_BingScraper: MagicMock, engine: BingScraperSearchEngine) -> None:
        mock_instance.search.side_effect = RuntimeError("Failed to parse HTML")
        assert "Error parsing Bing search results" in str(excinfo.value)
    async def test_invalid_result_format(self, mock_BingScraper: MagicMock, engine: BingScraperSearchEngine) -> None:
        class InvalidResult:
            def __init__(self):
        mock_instance.search.return_value = [InvalidResult()]

================
File: plugins/repos/twat_search/tests/conftest.py
================
@pytest.fixture(autouse=True)
def isolate_env_vars(monkeypatch: MonkeyPatch) -> None:
    for env_var in list(os.environ.keys()):
        if any(env_var.endswith(suffix) for suffix in ["_API_KEY", "_ENABLED", "_DEFAULT_PARAMS"]):
            monkeypatch.delenv(env_var, raising=False)
    monkeypatch.setenv("_TEST_ENGINE", "true")
def env_vars_for_brave(monkeypatch: MonkeyPatch) -> None:
        sys.path.insert(0, str(Path(__file__).parent.parent))
        class MockBraveEngine(SearchEngine):
        register_engine(MockBraveEngine)
    monkeypatch.setenv("BRAVE_API_KEY", "test_brave_key")
    monkeypatch.setenv("BRAVE_ENABLED", "true")
    monkeypatch.setenv("BRAVE_DEFAULT_PARAMS", '{"count": 10}')
    monkeypatch.delenv("_TEST_ENGINE", raising=False)

================
File: plugins/repos/twat_search/tests/test_twat_search.py
================
def test_version():

================
File: plugins/repos/twat_search/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_search/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/asottile/pyupgrade
    rev: v3.19.1
    hooks:
      - id: pyupgrade
        args: [--py311-plus]

  - repo: https://github.com/pycqa/isort
    rev: 6.0.0
    hooks:
      - id: isort
        args: ['--profile=black', '--line-length=120']

  - repo: https://github.com/MarcoGorelli/absolufy-imports
    rev: v0.3.1
    hooks:
      - id: absolufy-imports

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      - id: check-yaml
      - id: requirements-txt-fixer

  - repo: https://github.com/asottile/add-trailing-comma
    rev: v3.1.0
    hooks:
      - id: add-trailing-comma

  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.9.7
    hooks:
      - id: ruff
        types: [python]
        additional_dependencies: ['typing-extensions']
        args: [
          '--fix',
          '--ignore=ARG001,ARG002,ARG004,ARG005,B904,C901,DTZ005,E501,F401,F811,FBT001,FBT002,FBT003,I001,ISC001,N803,PLR0911,PLR0912,PLR0913,PLR0915,PLR2004,S311,S603,S607,T201,PLR1714,PYI056,PTH123,PLW0603,S110,PT013,SIM102,EM102'
        ]
      - id: ruff-format
        args: [--respect-gitignore]
        types: [python]

================
File: plugins/repos/twat_search/CHANGELOG.md
================
---
this_file: CHANGELOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

## Unreleased Changes

### Added
- Added error handling in `init_engine_task` to prevent task cancellation issues
- Improved error messages for better debugging
- Added detailed logging for search engine initialization and execution
- Added check for empty engines list in `search` function, raising a `SearchError` with appropriate message
- Added proper handling of mock engine parameters, ensuring correct result count

### Changed
- Enhanced error handling in `get_engine` function to raise `SearchError` with correct message
- Improved mock engine handling in `search` function to properly set `result_count`
- Modified `Config` class to check for `_TEST_ENGINE` environment variable
- Enhanced `_parse_env_value` function to handle JSON strings for engine default parameters
- Added `BRAVE_DEFAULT_PARAMS` to `ENV_VAR_MAP` for proper configuration
- Standardized engine names for more consistent lookups
- Improved error handling patterns across the codebase

### Fixed
- Fixed issue with environment variable parsing for engine default parameters
- Fixed handling of empty engines list in search function
- Fixed mock engine result count handling
- Fixed error handling in `get_engine` function to use the correct exception type
- Fixed environment variable application to engine configurations

## 0.1.2 (2024-05-15)

### Fixed
- Fixed linting errors in `google.py` and `test_google_falla_debug.py`
- Replaced insecure usage of temporary file directory `/tmp` with `tempfile.gettempdir()`
- Replaced `os.path` functions with `pathlib.Path` methods for better path handling
- Removed unused imports (`os` and `NavigableString`) from `google.py`

## 0.1.1 (2024-05-10)

### Added
- Added support for Falla-based search engines
- Added refactored search engine initialization
- Added wrapper coroutine to handle exceptions during search process
- Added detailed logging for engine initialization and search processes

### Changed
- Improved error handling for search engines
- Enhanced configuration system
- Standardized engine names for more consistent lookups

## 0.1.0 (2024-05-01)

### Added
- Initial release
- Support for multiple search engines
- Asynchronous search capabilities
- Rate limiting
- Strong typing with Pydantic validation

### Changed
- Updated the `config.py` file to correctly import BaseSettings from the pydantic-settings package
- Updated the `pyproject.toml` file to add pydantic-settings as a dependency
- Updated the example usage in `example.py`
- Completed the implementation of the web search functionality as specified in TODO.md
- Planned comprehensive tests for the package

### 2024-02-25 - Initial Development

- Created initial project structure and files
- Created a preliminary TODO.md, PROGRESS.md, and research.txt

---

================
File: plugins/repos/twat_search/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(["tree", "-a", "-I", ".git", "--gitignore", "-n", "-h", "-I", "*_cache"])
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_search/debug_fetch.py
================
logging.basicConfig(
    handlers=[logging.StreamHandler(sys.stdout)],
logger = logging.getLogger(__name__)
class DebugFetcher:
    def __init__(self):
        self.output_dir = Path("debug_output")
    async def fetch_page(self, url: str, engine: str) -> str:
        logger.info(f"Fetching {url}")
        async with async_playwright() as p:
            browser = await p.chromium.launch(headless=True)
                context = await browser.new_context(
                page = await context.new_page()
                await page.goto(url, timeout=60000, wait_until="networkidle")
                await page.screenshot(path=self.output_dir / f"{engine}_screenshot.png")
                content = await page.content()
                logger.info(f"Got content length: {len(content)}")
                await browser.close()
    def save_html(self, content: str, engine: str) -> None:
        path.write_text(content, encoding="utf-8")
        logger.info(f"Saved HTML content to {path}")
    def analyze_selectors(self, content: str, engine: str) -> None:
        soup = BeautifulSoup(content, "html.parser")
        with open(self.output_dir / f"{engine}_analysis.txt", "w", encoding="utf-8") as f:
            f.write(f"Analysis for {engine}\n")
            f.write("=" * 80 + "\n\n")
            f.write("Potential containers found:\n")
                elements = soup.select(selector)
                f.write(f"{selector}: {len(elements)} elements found\n")
                if elements and len(elements) > 0:
                    f.write(f"\nSample structure of {selector}:\n")
                    f.write(str(sample)[:500] + "...\n\n")
            f.write("\nOther potential elements:\n")
                elements = soup.select(tag)
                    f.write(f"{tag}: {len(elements)} elements found\n")
                    if len(elements) < 5:  # Only show samples for a small number
                        for i, elem in enumerate(elements):
                            f.write(f"  {i + 1}. {elem.get_text().strip()[:100]}\n")
    async def process_engine(self, engine: str, query: str) -> None:
            logger.error(f"Unsupported engine: {engine}")
        url = self.engines[engine].format(query.replace(" ", "+"))
        content = await self.fetch_page(url, engine)
        self.save_html(content, engine)
        self.analyze_selectors(content, engine)
    async def run(self, engines: list[str], query: str) -> None:
        self.output_dir.mkdir(exist_ok=True)
        logger.info(f"Processing engines: {', '.join(engines)}")
            await self.process_engine(engine, query)
async def main_async() -> None:
    parser = argparse.ArgumentParser(description="Fetch search engine HTML for debugging")
    parser.add_argument("query", help="Search query")
    parser.add_argument(
    args = parser.parse_args()
    fetcher = DebugFetcher()
    await fetcher.run(engines, args.query)
def main() -> None:
    asyncio.run(main_async())
    main()

================
File: plugins/repos/twat_search/falla_search.py
================
logging.basicConfig(
    handlers=[logging.StreamHandler(sys.stdout)],
logger = logging.getLogger(__name__)
class SimpleFallaSearch:
    def __init__(self, engine: str = "google"):
    def get_url(self, query: str) -> str:
        query = query.replace(" ", "+")
        raise ValueError(error_msg)
    async def fetch_page(self, url: str) -> str | None:
        logger.info(f"Fetching {url}")
            async with async_playwright() as p:
                browser = await p.chromium.launch(headless=True)
                    context = await browser.new_context(
                    page = await context.new_page()
                    await page.goto(url, timeout=30000, wait_until="domcontentloaded")
                            await page.wait_for_selector(selector, timeout=5000)
                            logger.warning(f"Timeout waiting for selector: {e}")
                    content = await page.content()
                    logger.info(f"Got content length: {len(content)}")
                    logger.info(f"Content preview: {content[:500]}")
                    await browser.close()
            logger.error(f"Error fetching page: {e}")
    def parse_results(self, html: str) -> list[dict[str, str]]:
        soup = BeautifulSoup(html, "html.parser")
        containers = soup.select(container_selector)
        logger.info(f"Found {len(containers)} result containers")
                title_elem = container.select_one(self.selectors[self.engine]["title"])
                title = title_elem.get_text().strip() if title_elem else ""
                link_elem = container.select_one(self.selectors[self.engine]["link"])
                link = str(link_elem.get("href")) if link_elem and link_elem.get("href") else ""
                snippet_elem = container.select_one(self.selectors[self.engine]["snippet"])
                snippet = snippet_elem.get_text().strip() if snippet_elem else ""
                    results.append({"title": title, "link": link, "snippet": snippet})
                logger.error(f"Error parsing result: {e}")
    async def search(self, query: str) -> list[dict[str, str]]:
        url = self.get_url(query)
        html = await self.fetch_page(url)
            logger.warning("No HTML content returned")
        return self.parse_results(html)
    def search_sync(self, query: str) -> list[dict[str, str]]:
        return asyncio.run(self.search(query))
async def main_async():
    parser = argparse.ArgumentParser(description="Search using Falla")
    parser.add_argument("query", help="Search query")
    parser.add_argument(
    args = parser.parse_args()
    search = SimpleFallaSearch(engine=args.engine)
    results = await search.search(args.query)
        for _i, _result in enumerate(results, 1):
def main():
    asyncio.run(main_async())
    main()

================
File: plugins/repos/twat_search/google_debug_Python_programming_language.html
================
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta http-equiv="origin-trial" content="A/kargTFyk8MR5ueravczef/wIlTkbVk1qXQesp39nV+xNECPdLBVeYffxrM8TmZT6RArWGQVCJ0LRivD7glcAUAAACQeyJvcmlnaW4iOiJodHRwczovL2dvb2dsZS5jb206NDQzIiwiZmVhdHVyZSI6IkRpc2FibGVUaGlyZFBhcnR5U3RvcmFnZVBhcnRpdGlvbmluZzIiLCJleHBpcnkiOjE3NDIzNDIzOTksImlzU3ViZG9tYWluIjp0cnVlLCJpc1RoaXJkUGFydHkiOnRydWV9"><meta http-equiv="content-type" content="text/html; charset=utf-8"><meta name="viewport" content="initial-scale=1"><title>https://www.google.com/search?q=Python+programming+language&amp;num=100&amp;hl=en&amp;gl=us&amp;sei=V3_GZ4bVMa-Mxc8PmfaSgAU</title></head>
<body style="font-family: arial, sans-serif; background-color: #fff; color: #000; padding:20px; font-size:18px; overscroll-behavior:contain;" onload="e=document.getElementById('captcha');if(e){e.focus();} if(solveSimpleChallenge) {solveSimpleChallenge(0,0);}">
<div style="max-width:400px;">
<hr noshade="" size="1" style="color:#ccc; background-color:#ccc;"><br>
<form id="captcha-form" action="index" method="post">
<noscript>
<div style="font-size:13px;">
  In order to continue, please enable javascript on your web browser.
</div>
</noscript>
<script type="text/javascript" async="" charset="utf-8" src="https://www.gstatic.com/recaptcha/releases/rW64dpMGAGrjU7JJQr9xxPl8/recaptcha__en.js" crossorigin="anonymous" integrity="sha384-stI8i0l4UnrupDOI6In0RQyfmG7x4RHoRSlzjf/yDu+T9clp2es/I4WNYpU4isM0"></script><script src="https://www.google.com/recaptcha/api.js" async="" defer=""></script>
<script>var submitCallback = function(response) {document.getElementById('captcha-form').submit();};</script>
<div id="recaptcha" class="g-recaptcha" data-sitekey="6LfwuyUTAAAAAOAmoS0fdqijC2PbbdH4kjq62Y1b" data-callback="submitCallback" data-s="2PCOkW9zecXmuwvrhgPcpcw_x_FYf41FDpdcdvcuH75pD6YHnEr3IMqaldQ6BBWhjf7hNEe43XOAQINFMpjBrZN9AYoPKVQ8EdY5ezYpG_U2ff1VWdJoen6rzZnuzrMcYnyRbMdmYQOpb2iOnyVec9iIHi9JlX2CpnKr6o613bYAGHxlnpzycDiMilWJBrDI7I-zz4ErCkNgczFfu4GU_okWdfs-Rwr_dDC3YEBmWyW1XIh7xeO615YYQY-T0Hs2_ZbYU9XD9qesC1gth_3l7awQQiCV_Hg"></div>

<input type="hidden" name="q" value="EhAqAqMSyMhKAIkbml3BFgvLGNj-mb4GIjCNaWGW1Jn0FAd7D6NUQ1h7QQJSSo_pe7URjP4b1z0SdZHf7dSw1kCExJN1AZBFZHUyAXJaAUM"><input type="hidden" name="continue" value="https://www.google.com/search?q=Python+programming+language&amp;num=100&amp;hl=en&amp;gl=us&amp;sei=V3_GZ4bVMa-Mxc8PmfaSgAU">
</form>
<hr noshade="" size="1" style="color:#ccc; background-color:#ccc;">

<div style="font-size:13px; line-break: anywhere;">
<b>About this page</b><br><br>

Our systems have detected unusual traffic from your computer network.  This page checks to see if it's really you sending the requests, and not a robot.  <a href="#" onclick="document.getElementById('infoDiv').style.display='block';">Why did this happen?</a><br><br>

<div id="infoDiv" style="display:none; background-color:#eee; padding:10px; margin:0 0 15px 0; line-height:1.4em;">
This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the <a href="//www.google.com/policies/terms/">Terms of Service</a>. The block will expire shortly after those requests stop.  In the meantime, solving the above CAPTCHA will let you continue to use our services.<br><br>This traffic may have been sent by malicious software, a browser plug-in, or a script that sends automated requests.  If you share your network connection, ask your administrator for help — a different computer using the same IP address may be responsible.  <a href="//support.google.com/websearch/answer/86640">Learn more</a><br><br>Sometimes you may be asked to solve the CAPTCHA if you are using advanced terms that robots are known to use, or sending requests very quickly.
</div>

IP address: 2a02:a312:c8c8:4a00:891b:9a5d:c116:bcb<br>Time: 2025-03-04T04:19:36Z<br>URL: https://www.google.com/search?q=Python+programming+language&amp;num=100&amp;hl=en&amp;gl=us&amp;sei=V3_GZ4bVMa-Mxc8PmfaSgAU<br>
</div>
</div>


</body></html>

================
File: plugins/repos/twat_search/google_debug_test_query.html
================
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta http-equiv="origin-trial" content="A/kargTFyk8MR5ueravczef/wIlTkbVk1qXQesp39nV+xNECPdLBVeYffxrM8TmZT6RArWGQVCJ0LRivD7glcAUAAACQeyJvcmlnaW4iOiJodHRwczovL2dvb2dsZS5jb206NDQzIiwiZmVhdHVyZSI6IkRpc2FibGVUaGlyZFBhcnR5U3RvcmFnZVBhcnRpdGlvbmluZzIiLCJleHBpcnkiOjE3NDIzNDIzOTksImlzU3ViZG9tYWluIjp0cnVlLCJpc1RoaXJkUGFydHkiOnRydWV9"><meta http-equiv="content-type" content="text/html; charset=utf-8"><meta name="viewport" content="initial-scale=1"><title>https://www.google.com/search?q=test+query&amp;num=100&amp;hl=en&amp;gl=us&amp;sei=4YbGZ-DbL4yNxc8P4uOmgQg</title></head>
<body style="font-family: arial, sans-serif; background-color: #fff; color: #000; padding:20px; font-size:18px; overscroll-behavior:contain;" onload="e=document.getElementById('captcha');if(e){e.focus();} if(solveSimpleChallenge) {solveSimpleChallenge(0,0);}">
<div style="max-width:400px;">
<hr noshade="" size="1" style="color:#ccc; background-color:#ccc;"><br>
<form id="captcha-form" action="index" method="post">
<noscript>
<div style="font-size:13px;">
  In order to continue, please enable javascript on your web browser.
</div>
</noscript>
<script type="text/javascript" async="" charset="utf-8" src="https://www.gstatic.com/recaptcha/releases/rW64dpMGAGrjU7JJQr9xxPl8/recaptcha__en.js" crossorigin="anonymous" integrity="sha384-stI8i0l4UnrupDOI6In0RQyfmG7x4RHoRSlzjf/yDu+T9clp2es/I4WNYpU4isM0"></script><script src="https://www.google.com/recaptcha/api.js" async="" defer=""></script>
<script>var submitCallback = function(response) {document.getElementById('captcha-form').submit();};</script>
<div id="recaptcha" class="g-recaptcha" data-sitekey="6LfwuyUTAAAAAOAmoS0fdqijC2PbbdH4kjq62Y1b" data-callback="submitCallback" data-s="FqngvDzHeZnU97lmKD1UxwWETdILMoDw8MPTPlsKvh_DUItYKzRxxheg4ttQqMSmncOae3bMD4rAgM0_4aTE6nfRoyJIv6bKITJtcEM3dPdhunKfIzoBMotBAt83t0iBpUPOp8sMUzY-6pvdwGGqEcND2GILtsz_7j6-htwn0-wVK96f9yq_Dl6-jNBenTMPhdmb1k4tJFZ066jMqLFl8AmvC1Kc1IKcKTqQDoSR7Kozs5nn5j8M4KI-hz0HLDFeFYFIZXASnCWF8c9DZQ1-QfrhWKqZ1vY"></div>

<input type="hidden" name="q" value="EhAqAqMSyMhKAIkbml3BFgvLGOKNmr4GIjBO7UBahpD_eHomWH46AYpWusU92R1i9lWqHXc-RRZMKtceXQDkLhqsMpEbCCRe_jsyAXJaAUM"><input type="hidden" name="continue" value="https://www.google.com/search?q=test+query&amp;num=100&amp;hl=en&amp;gl=us&amp;sei=4YbGZ-DbL4yNxc8P4uOmgQg">
</form>
<hr noshade="" size="1" style="color:#ccc; background-color:#ccc;">

<div style="font-size:13px; line-break: anywhere;">
<b>About this page</b><br><br>

Our systems have detected unusual traffic from your computer network.  This page checks to see if it's really you sending the requests, and not a robot.  <a href="#" onclick="document.getElementById('infoDiv').style.display='block';">Why did this happen?</a><br><br>

<div id="infoDiv" style="display:none; background-color:#eee; padding:10px; margin:0 0 15px 0; line-height:1.4em;">
This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the <a href="//www.google.com/policies/terms/">Terms of Service</a>. The block will expire shortly after those requests stop.  In the meantime, solving the above CAPTCHA will let you continue to use our services.<br><br>This traffic may have been sent by malicious software, a browser plug-in, or a script that sends automated requests.  If you share your network connection, ask your administrator for help — a different computer using the same IP address may be responsible.  <a href="//support.google.com/websearch/answer/86640">Learn more</a><br><br>Sometimes you may be asked to solve the CAPTCHA if you are using advanced terms that robots are known to use, or sending requests very quickly.
</div>

IP address: 2a02:a312:c8c8:4a00:891b:9a5d:c116:bcb<br>Time: 2025-03-04T04:51:46Z<br>URL: https://www.google.com/search?q=test+query&amp;num=100&amp;hl=en&amp;gl=us&amp;sei=4YbGZ-DbL4yNxc8P4uOmgQg<br>
</div>
</div>


</body></html>

================
File: plugins/repos/twat_search/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_search/pyproject.toml
================
[build-system]
requires = ['hatchling>=1.27.0', 'hatch-vcs>=0.4.0']
build-backend = 'hatchling.build'
[tool.hatch.build.targets.wheel]
packages = ['src/twat_search']
[tool.hatch.build.hooks.vcs]
version-file = 'src/twat_search/__version__.py'

[tool.hatch.version]
source = 'vcs'

[tool.hatch.version.raw-options]
version_scheme = 'post-release'

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.envs.default]
dependencies = [
    'pytest>=1.3.4',
    'pytest-cov>=1.0.0',
    'mypy>=1.15.0',
    'ruff>=0.9.7',
    'absolufy-imports>=0.3.1',
    'pre-commit>=4.1.0',
    'pyupgrade>=3.19.1',
    'isort>=6.0.0',
]

[tool.hatch.envs.default.scripts]
test = 'pytest {args:tests}'
test-cov = 'pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_search --cov=tests {args:tests}'
type-check = 'mypy src/twat_search tests'
lint = ['ruff check src/twat_search tests', 'ruff format src/twat_search tests']
[[tool.hatch.envs.all.matrix]]
python = ['3.10', '3.11', '3.12']

[tool.hatch.envs.lint]
detached = true
dependencies = [
    'mypy>=1.15.0',
    'ruff>=0.9.7',
    'absolufy-imports>=0.3.1',
    'pre-commit>=4.1.0',
    'pyupgrade>=3.19.1',
    'isort>=6.0.1',
]

[tool.hatch.envs.lint.scripts]
typing = 'mypy --install-types --non-interactive {args:src/twat_search tests}'
style = ['ruff check {args:.}', 'ruff format {args:.}']
fmt = ['ruff format {args:.}', 'ruff check --fix {args:.}']
all = ['style', 'typing']

[tool.hatch.envs.test]
dependencies = ['.[test]']

[tool.hatch.envs.test.scripts]
test = 'python -m pytest -n auto {args:tests}'
test-cov = 'python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_search --cov=tests {args:tests}'
bench = 'python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only'
bench-save = 'python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json'

[tool.ruff]
target-version = 'py310'
line-length = 120

[tool.ruff.lint]
select = [
    'A',
    'ARG',
    'B',
    'C',
    'DTZ',
    'E',
    'EM',
    'F',
    'FBT',
    'I',
    'ICN',
    'ISC',
    'N',
    'PLC',
    'PLE',
    'PLR',
    'PLW',
    'PT',
    'PTH',
    'PYI',
    'RET',
    'RSE',
    'RUF',
    'S',
    'SIM',
    'T',
    'TCH',
    'TID',
    'UP',
    'W',
    'YTT',
]
ignore = [
    'B027',
    'C901',
    'FBT003',
    'PLR0911',
    'PLR0912',
    'PLR0913',
    'PLR0915',
    'PLR1714',
    'PLW0603',
    'PT013',
    'PTH123',
    'PYI056',
    'S105',
    'S106',
    'S107',
    'S110','SIM102'
]
unfixable = [
    # Don't touch unused imports
    'F401',
]

[tool.ruff.lint.isort]
known-first-party = ['twat_search']

[tool.ruff.lint.flake8-tidy-imports]
ban-relative-imports = 'all'

[tool.ruff.lint.per-file-ignores]
# Tests can use magic values, assertions, and relative imports
'tests/**/*' = ['PLR2004', 'S101', 'TID252']

[tool.coverage.run]
source_pkgs = ['twat_search', 'tests']
branch = true
parallel = true
omit = ['src/twat_search/__about__.py']

[tool.coverage.paths]
twat_search = ['src/twat_search', '*/twat-search/src/twat_search']
tests = ['tests', '*/twat-search/tests']

[tool.coverage.report]
exclude_lines = [
    'no cov',
    'if __name__ == .__main__.:',
    'if TYPE_CHECKING:',
    'pass',
    'raise NotImplementedError',
    'raise ImportError',
    'except ImportError',
    'except KeyError',
    'except AttributeError',
    'except NotImplementedError',
]

[project]
name = 'twat-search'
description = 'Web search plugin for twat'
readme = 'README.md'
requires-python = '>=3.10'
license = { text = 'MIT' }
keywords = []
dynamic = ["version"]
classifiers = [
    'Development Status :: 4 - Beta',
    'Programming Language :: Python',
    'Programming Language :: Python :: 3.10',
    'Programming Language :: Python :: 3.11',
    'Programming Language :: Python :: 3.12',
    'Programming Language :: Python :: Implementation :: CPython',
    'Programming Language :: Python :: Implementation :: PyPy',
]
dependencies = [
    'twat>=1.8.1',
    'pydantic>=2.10.6',
    'pydantic-settings>=2.8.1',
    'httpx>=0.28.1',
    'python-dotenv>=1.0.1',
    'fire>=0.7.0',
    'rich>=13.9.4',
    'requests>=2.32.3',
    'beautifulsoup4>=4.13.0',
]

[[project.authors]]
name = 'Adam Twardoch'
email = 'adam+github@twardoch.com'

[project.urls]
Documentation = 'https://github.com/twardoch/twat-search#readme'
Issues = 'https://github.com/twardoch/twat-search/issues'
Source = 'https://github.com/twardoch/twat-search'
[project.entry-points."twat.plugins"]
search = 'twat_search'

[project.optional-dependencies]
test = [
    'pytest>=8.3.4',
    'pytest-cov>=6.0.0',
    'pytest-xdist>=3.6.1',
    'pytest-benchmark[histogram]>=5.1.0',
    'pytest-asyncio>=0.25.3',
]
dev = [
    'pre-commit>=4.1.0',
    'ruff>=0.9.7',
    'mypy>=1.15.0',
    'absolufy-imports>=0.3.1',
    'pyupgrade>=3.19.1',
    'isort>=6.0.1',
]
brave = []
duckduckgo = ['duckduckgo-search>=7.5.0']
bing_scraper = ['scrape-bing>=0.1.2.1']
tavily = ['tavily-python>=0.5.1']
pplx = []
serpapi = ['serpapi>=0.1.5']
hasdata = []
falla = [    "lxml>=5.3.1",
    "playwright>=1.50.0",
]
google_scraper = ['googlesearch-python>=1.3.0']
all = [
    'twat',
    'duckduckgo-search>=7.5.0',
    'scrape-bing>=0.1.2.1',
    'tavily-python>=0.5.1',
    'serpapi>=0.1.5',
    'googlesearch-python>=1.3.0',
    'beautifulsoup4>=4.13.0',
    'lxml>=5.3.1',
    'playwright>=1.50.0',
]

[project.scripts]
twat-search = 'twat_search.__main__:main'
twat-search-web = 'twat_search.web.cli:main'

================
File: plugins/repos/twat_search/README.md
================
---
this_file: README.md
---

# Twat Search

A multi-engine web search aggregator that provides a unified interface for searching across various search engines.

## Features

- **Multi-engine search**: Search across various providers including Brave, Google (via SerpAPI), Bing, and more
- **Asynchronous operation**: Uses `asyncio` for efficient concurrent searches
- **Rate limiting**: Built-in rate limiting to prevent API throttling
- **Strong typing**: Pydantic validation for all data models
- **Robust error handling**: Graceful handling of engine failures and empty engine lists
- **Flexible configuration**: Configure via environment variables, `.env` files, or directly in code
- **CLI interface**: Interactive command-line interface with rich output formatting
- **JSON output**: Standardized JSON output format for all search results
- **Modern path handling**: Uses `pathlib` for cross-platform path operations
- **Secure temp files**: Uses `tempfile` for secure temporary file operations

## Recent Improvements

- Enhanced error handling for search engines, including proper handling of empty engine lists
- Standardized engine names for more consistent lookups
- Detailed logging for search engine initialization and execution
- Improved environment variable handling for engine configuration
- Fixed issues with mock engine result count handling
- Added proper JSON string parsing for engine default parameters
- Fixed error handling in `get_engine` function to use the correct exception type
- Improved handling of engine-specific parameters in the search function
- Enhanced configuration system with better environment variable support

## Installation

```bash
pip install twat-search
```

## Usage

### Basic Usage

```python
import asyncio
from twat_search.web.api import search

async def main():
    results = await search("Python programming", engines=["brave", "google"])
    for result in results:
        print(f"Title: {result.title}")
        print(f"URL: {result.url}")
        print(f"Description: {result.description}")
        print("---")

asyncio.run(main())
```

### Configuration

You can configure the search engines using environment variables:

```bash
# Enable/disable engines
export BRAVE_ENABLED=true
export GOOGLE_ENABLED=false

# Set API keys
export SERPAPI_API_KEY=your_api_key
export TAVILY_API_KEY=your_api_key

# Configure engine parameters
export BRAVE_DEFAULT_PARAMS='{"count": 10, "country": "US"}'
```

Or directly in code:

```python
from twat_search.web.config import Config

config = Config(
    brave={"enabled": True, "default_params": {"count": 10}},
    google={"enabled": False}
)
```

### Command Line Interface

```bash
# Basic search
twat-search web q "Python programming"

# Specify engines
twat-search web q "Python programming" --engines brave,google

# Get JSON output
twat-search web q "Python programming" --json

# List available engines
twat-search web info --plain
```

## Error Handling

The package provides robust error handling with custom exception classes:

- `SearchError`: Base exception for all search-related errors
- `EngineError`: Raised when there's an issue with a specific engine

Example:

```python
from twat_search.web.api import search
from twat_search.web.exceptions import SearchError, EngineError

try:
    results = await search("Python programming")
except EngineError as e:
    print(f"Engine error: {e}")
except SearchError as e:
    print(f"Search error: {e}")
```

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Project Documentation

The project maintains several key documentation files:

- **README.md**: This file, containing an overview of the project, installation instructions, and usage examples.
- **CHANGELOG.md**: Documents all notable changes to the project, organized by version.
- **TODO.md**: Contains a prioritized list of tasks and improvements planned for the project.
- **LICENSE**: The project's license information.

## Development Workflow

When contributing to this project, please follow these guidelines:

1. Check the **TODO.md** file for prioritized tasks that need attention.
2. Run `./cleanup.py status` regularly to check for linting errors and test failures.
3. Document all changes in **CHANGELOG.md** under the appropriate version section.
4. Add comprehensive tests for new features and bug fixes.
5. Ensure all code passes linting and type checking before submitting.

### Code Quality Tools

The project uses several tools to maintain code quality:

- **Ruff**: For linting and formatting Python code
- **Mypy**: For static type checking
- **Pytest**: For running tests
- **Pre-commit hooks**: To ensure code quality before commits

Run these tools regularly during development:

```bash
# Format code
ruff format --respect-gitignore --target-version py312 .

# Lint code
ruff check --output-format=github --fix --unsafe-fixes .

# Run tests
python -m pytest
```

## Quick start guide

### Python API

```python
import asyncio
from twat_search.web import search

async def main():
    # Search across all configured engines
    results = await search("quantum computing applications")

    # Print results
    for result in results:
        print(f"[{result.source}] {result.title}")
        print(f"URL: {result.url}")
        print(f"Snippet: {result.snippet}\n")

# Run the async function
asyncio.run(main())
```

### Command line interface

```bash
# Search using all available engines
twat-search web q "climate change solutions"

# Search with specific engines
twat-search web q "machine learning frameworks" -e brave,tavily

# Get json output
twat-search web q "renewable energy" --json

# Use engine-specific command
twat-search web q -e brave "web development trends" --count 10
```

## Core architecture

### Module structure

```
twat_search/
└── web/
    ├── engines/            # Individual search engine implementations
    │   ├── __init__.py     # Engine registration and availability checks
    │   ├── base.py         # Base SearchEngine class definition
    │   ├── brave.py        # Brave search implementation
    │   ├── bing_scraper.py # Bing scraper implementation
    │   └── ...             # Other engine implementations
    │   └── lib_falla/      # Falla-based search engine implementations
    │       ├── core/       # Core Falla functionality
    │       │   ├── falla.py    # Base Falla class
    │       │   ├── google.py   # Google search implementation
    │       │   └── ...         # Other Falla-based implementations
    ├── __init__.py         # Module exports
    ├── api.py              # Main search API
    ├── cli.py              # Command-line interface
    ├── config.py           # Configuration handling
    ├── exceptions.py       # Custom exceptions
    ├── models.py           # Data models
    └── utils.py            # Utility functions
```

## Supported search engines

Twat Search provides a consistent interface to the following search engines:

| Engine | Module | API Key Required | Description | Package Extra |
| --- | --- | --- | --- | --- |
| Brave | `brave` | Yes | Web search via Brave Search API | `brave` |
| Brave News | `brave_news` | Yes | News search via Brave API | `brave` |
| You.com | `you` | Yes | Web search via You.com API | - |
| You.com News | `you_news` | Yes | News search via You.com API | - |
| Tavily | `tavily` | Yes | Research-focused search API | `tavily` |
| Perplexity | `pplx` | Yes | AI-powered search with detailed answers | `pplx` |
| SerpAPI | `serpapi` | Yes | Google search results via SerpAPI | `serpapi` |
| HasData Google | `hasdata-google` | Yes | Google search results via HasData API | `hasdata` |
| HasData Google Light | `hasdata-google-light` | Yes | Light version of HasData API | `hasdata` |
| Critique | `critique` | Yes | Visual and textual search capabilities | - |
| DuckDuckGo | `duckduckgo` | No | Privacy-focused search results | `duckduckgo` |
| Bing Scraper | `bing_scraper` | No | Web scraping of Bing search results | `bing_scraper` |
| Google Falla | `google_falla` | No | Google search via Playwright-based scraping | `falla` |

## Configuration management

### Environment variables

Configure engines using environment variables:

```bash
# Api keys
BRAVE_API_KEY=your_brave_api_key
TAVILY_API_KEY=your_tavily_api_key
PERPLEXITY_API_KEY=your_perplexity_api_key
YOU_API_KEY=your_you_api_key
SERPAPI_API_KEY=your_serpapi_api_key
CRITIQUE_API_KEY=your_critique_api_key
HASDATA_API_KEY=your_hasdata_api_key

# Engine enablement
BRAVE_ENABLED=true
TAVILY_ENABLED=true
PERPLEXITY_ENABLED=true
YOU_ENABLED=true
SERPAPI_ENABLED=true
CRITIQUE_ENABLED=true
DUCKDUCKGO_ENABLED=true
BING_SCRAPER_ENABLED=true
HASDATA_GOOGLE_ENABLED=true

# Default parameters (json format)
BRAVE_DEFAULT_PARAMS='{"count": 10, "safesearch": "off"}'
TAVILY_DEFAULT_PARAMS='{"max_results": 5, "search_depth": "basic"}'
PERPLEXITY_DEFAULT_PARAMS='{"model": "pplx-7b-online"}'
YOU_DEFAULT_PARAMS='{"safe_search": true, "count": 8}'
SERPAPI_DEFAULT_PARAMS='{"num": 10, "gl": "us"}'
HASDATA_GOOGLE_DEFAULT_PARAMS='{"location": "Austin,Texas,United States", "device_type": "desktop"}'
DUCKDUCKGO_DEFAULT_PARAMS='{"max_results": 10, "safesearch": "moderate", "time": "d"}'
BING_SCRAPER_DEFAULT_PARAMS='{"max_retries": 3, "delay_between_requests": 1.0}'

# Global default for all engines
NUM_RESULTS=5
```

You can store these in a `.env` file in your project directory, which will be automatically loaded by the library using `python-dotenv`.

### Programmatic configuration

Configure engines programmatically when using the Python API:

```python
from twat_search.web import Config, EngineConfig, search

# Create custom configuration
config = Config(
    engines={
        "brave": EngineConfig(
            api_key="your_brave_api_key",
            enabled=True,
            default_params={"count": 10, "country": "US"}
        ),
        "bing_scraper": EngineConfig(
            enabled=True,
            default_params={"max_retries": 3, "delay_between_requests": 1.0}
        ),
        "tavily": EngineConfig(
            api_key="your_tavily_api_key",
            enabled=True,
            default_params={"search_depth": "advanced"}
        )
    }
)

# Use the configuration
results = await search("quantum computing", config=config)
```

## Error Handling Framework

Twat Search provides a robust error handling framework to ensure graceful failure and clear error messages:

### Exception Hierarchy

- `SearchError`: Base exception for all search-related errors
- `EngineError`: Specific exception for engine-related issues

### Key Error Handling Features

- **Empty Engine List Detection**: The search function now checks for empty engine lists and raises a clear error
- **Standardized Error Messages**: All error messages follow a consistent format for better debugging
- **Detailed Logging**: Comprehensive logging throughout the search process
- **Graceful Engine Failures**: Individual engine failures don't crash the entire search process
- **Environment Variable Validation**: Proper parsing and validation of environment variables

### Example Error Scenarios

```python
# No engines configured
try:
    results = await search("query", engines=[])
except SearchError as e:
    print(e)  # "No search engines specified or available"

# Non-existent engine
try:
    results = await search("query", engines=["nonexistent_engine"])
except SearchError as e:
    print(e)  # "Engine 'nonexistent_engine': not found"

# Disabled engine
try:
    results = await search("query", engines=["disabled_engine"])
except SearchError as e:
    print(e)  # "Engine 'disabled_engine': is disabled"
```

## Recent Fixes and Improvements

The latest version includes several important fixes and improvements:

1. **Fixed Environment Variable Parsing**: Properly handles JSON strings in environment variables for engine default parameters
2. **Empty Engine List Handling**: Added explicit check and error for empty engine lists
3. **Mock Engine Parameter Handling**: Improved handling of mock engine parameters for testing
4. **Standardized Error Messages**: Enhanced error messages for better debugging
5. **Config Class Improvements**: Modified to properly check for test environment variables
6. **Enhanced Logging**: Added detailed logging throughout the search process
7. **Improved Exception Handling**: Better handling of exceptions during engine initialization and search

These improvements make the package more robust, easier to debug, and more reliable in various usage scenarios.

## Development Status

The project is actively maintained and regularly updated. All unit tests are passing, and the codebase is continuously improved based on user feedback and identified issues.

Current focus areas include:
- Fixing Falla-based search engines
- Addressing type errors in the codebase
- Improving integration with Playwright
- Enhancing documentation and standardizing JSON output formats

For a complete list of planned improvements, see the TODO.md file.

## License

This project is licensed under the MIT License - see the LICENSE file for details.

---

## Appendix: available engines and requirements

| Engine | Package Extra | API Key Required | Environment Variable | Notes |
| --- | --- | --- | --- | --- |
| Brave | `brave` | Yes | `BRAVE_API_KEY` | General web search engine |
| Brave News | `brave` | Yes | `BRAVE_API_KEY` | News-specific search |
| You.com | - | Yes | `YOU_API_KEY` | AI-powered web search |
| You.com News | - | Yes | `YOU_API_KEY` | News-specific search |
| Tavily | `tavily` | Yes | `TAVILY_API_KEY` | Research-focused search |
| Perplexity | `pplx` | Yes | `PPLX_API_KEY` | AI-powered search with detailed answers |
| SerpAPI | `serpapi` | Yes | `SERPAPI_API_KEY` | Google search results API |
| HasData Google | `hasdata` | Yes | `HASDATA_API_KEY` | Google search results API |
| HasData Google Light | `hasdata` | Yes | `HASDATA_API_KEY` | Lightweight Google search API |
| Critique | - | Yes | `CRITIQUE_API_KEY` | Supports image analysis |
| DuckDuckGo | `duckduckgo` | No | - | Privacy-focused search |
| Bing Scraper | `bing_scraper` | No | - | Uses web scraping techniques |

================
File: plugins/repos/twat_search/requirements.txt
================
beautifulsoup4>=4.12.2 
playwright>=1.40.0

================
File: plugins/repos/twat_search/test_async_falla.py
================
async def test_async():
    d = DuckDuckGo()
        return await d.search_async("FontLab")
        results = asyncio.run(test_async())

================
File: plugins/repos/twat_search/test_falla.py
================
async def test():
    d = DuckDuckGo()
        await d._initialize_browser()
        return await d._fetch_page("https://duckduckgo.com/?q=FontLab")
        await d._close_browser()
    with contextlib.suppress(Exception):
        result = asyncio.run(test())

================
File: plugins/repos/twat_search/test_google_falla_debug.py
================
logging.basicConfig(
    handlers=[logging.StreamHandler()],
logger = logging.getLogger("google_falla_debug")
src_path = Path(__file__).parent / "src"
sys.path.insert(0, str(src_path))
    logger.error(f"Failed to import Google engine: {e}")
    logger.error("Make sure the path is correct and all dependencies are installed.")
    sys.exit(1)
async def main():
        engine = Google()
        logger.info(f"Testing Google Falla engine with query: {query}")
        logger.info("Engine configuration:")
        logger.info(f"  - Name: {engine.name}")
        logger.info(f"  - Method: {engine.use_method}")
        logger.info(f"  - Container element: {engine.container_element}")
        logger.info(f"  - Wait for selector: {engine.wait_for_selector}")
        logger.info(f"  - Max retries: {engine.max_retries}")
        logger.info("\nRunning search...")
        results = await engine.search_async(query)
        logger.info(f"\nSearch completed. Found {len(results)} results.")
        for i, result in enumerate(results, 1):
            logger.info(f"\nResult {i}:")
            logger.info(f"  Title: {result.get('title', 'N/A')}")
            logger.info(f"  Link: {result.get('link', 'N/A')}")
            snippet = result.get("snippet", "N/A")
                logger.info(f"  Snippet: {snippet[:100]}...")
                logger.info("  Snippet: N/A")
        temp_dir = tempfile.gettempdir()  # Use tempfile module for secure temp directory
        html_files = [f for f in os.listdir(temp_dir) if f.startswith("google_debug_") and f.endswith(".html")]
            logger.info("\nDebug HTML files were saved:")
                logger.info(f"  - {Path(temp_dir) / html_file}")
            logger.warning("No debug HTML files were found.")
        logger.error(f"Error during search: {e}")
        logger.error(traceback.format_exc())
                await engine.close()
                logger.info("Engine resources cleaned up.")
                logger.error(f"Error during cleanup: {e}")
    exit_code = asyncio.run(main())
    sys.exit(exit_code)

================
File: plugins/repos/twat_search/test_simple.py
================
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
async def fetch_page(url):
    logger.info(f"Fetching {url}")
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True)
            context = await browser.new_context(
            page = await context.new_page()
            await page.goto(url, timeout=30000, wait_until="domcontentloaded")
            await page.wait_for_selector("body", timeout=5000)
            content = await page.content()
            logger.info(f"Got content length: {len(content)}")
            await browser.close()
def fetch_page_sync(url):
        return asyncio.run(fetch_page(url))
        logger.error(f"Error in fetch_page_sync: {e}")
    content = fetch_page_sync("https://www.google.com/search?q=FontLab")
    content = fetch_page_sync("https://html.duckduckgo.com/html/?q=FontLab")

================
File: plugins/repos/twat_search/test_sync_falla.py
================
def test_sync():
    d = DuckDuckGo()
        return d.search("FontLab")
    results = test_sync()

================
File: plugins/repos/twat_search/TODO.md
================
--- 
this_file: TODO.md
--- 

# TODO

Tip: Periodically run `./cleanup.py status` to see results of lints and tests. Use `uv pip ...` not `pip ...`


## Phase 1

- [ ] Fix Falla-based search engines
  - [ ] Resolve Playwright dependency issues (ModuleNotFoundError: No module named 'playwright')
  - [ ] Ensure proper error handling for browser automation
  - [ ] Add retry mechanism for flaky browser interactions
  - [ ] Update selectors in search engine implementations to match current page structures
  - [ ] Implement proper handling of CAPTCHA challenges and consent pages

- [ ] Address type errors in the codebase
  - [ ] Fix type error in `wait_for_selector` where `self.wait_for_selector` could be `None`
  - [ ] Fix type error in `find_all` where `attrs` parameter has incompatible type
  - [ ] Fix type error in `get_title`, `get_link`, and `get_snippet` where `elem` parameter has incompatible type
  - [ ] Fix type error in `elem.find` where `PageElement` has no attribute `find`
  - [ ] Fix FBT001/FBT002 errors (Boolean-typed positional arguments)
  - [ ] Fix unused argument warnings (ARG001/ARG002)
  - [ ] Fix line length issues (E501)

- [ ] Improve integration with Playwright
  - [ ] Add proper browser context management
  - [ ] Implement headless mode configuration
  - [ ] Add proxy support for browser-based engines
  - [ ] Implement specific exception handling for common Playwright errors

## Phase 2

- [ ] Improve code quality
  - [ ] Refactor CLI module to reduce complexity
  - [ ] Add more comprehensive docstrings
  - [ ] Standardize error handling patterns across all engines
  - [ ] Add try-except blocks for all external API calls
  - [ ] Create custom exception classes for different error scenarios
  - [ ] Add graceful fallbacks for common error cases

- [ ] Enhance documentation
  - [ ] Create detailed API documentation
  - [ ] Add examples for each search engine
  - [ ] Document configuration options comprehensively
  - [ ] Add detailed docstrings to all classes and methods

- [ ] Standardize JSON output formats
  - [ ] Ensure consistent field names across all engines
  - [ ] Add schema validation for engine outputs
  - [ ] Implement proper JSON serialization for all models
  - [ ] Utilize the existing `SearchResult` model consistently
  - [ ] Remove utility functions like `_process_results` and `_display_json_results`
  - [ ] Remove `CustomJSONEncoder` class
  - [ ] Update engine `search` methods to return list of `SearchResult` objects

- [ ] Improve test coverage
  - [ ] Add unit tests for all search engine implementations
  - [ ] Add integration tests for the entire search pipeline
  - [ ] Implement mock responses for external API calls in tests
  - [ ] Add performance benchmarks for search operations

## Low Priority

- [ ] Add more search engines
  - [ ] Add support for Kagi search
  - [ ] Implement Ecosia search
  - [ ] Implement Startpage search
  - [ ] Implement Qwant AI engine using the QwantAI package
  - [ ] Integrate with more specialized search APIs

- [ ] Implement caching mechanism
  - [ ] Add Redis-based result caching
  - [ ] Implement TTL for cached results
  - [ ] Add result deduplication across engines

- [ ] Performance optimizations
  - [ ] Profile and optimize slow code paths
  - [ ] Reduce memory usage for large result sets
  - [ ] Optimize concurrent search operations
  - [ ] Implement timeout handling for slow search engines

- [ ] Enhance result processing
  - [ ] Add result ranking based on relevance
  - [ ] Implement result filtering options
  - [ ] Add support for different result formats (HTML, Markdown, etc.)

- [ ] Improve CLI functionality
  - [ ] Add interactive mode for search operations
  - [ ] Implement result pagination in CLI output
  - [ ] Add support for saving search results to file
  - [ ] Implement search history functionality

## Completed

- [x] Set up Falla module
- [x] Create base search engine class
- [x] Implement Brave search engine
- [x] Implement Google search via SerpAPI
- [x] Implement Falla-based engines (Google, Bing, etc.)
- [x] Fix linting errors in google.py and test_google_falla_debug.py
- [x] Improve error handling in get_engine function
- [x] Fix handling of empty engines list in search function
- [x] Fix mock engine result count handling
- [x] Fix environment variable parsing for engine default parameters
- [x] Add BRAVE_DEFAULT_PARAMS to ENV_VAR_MAP
- [x] Modify Config class to check for _TEST_ENGINE environment variable
- [x] Replace `os.path.abspath()` with `Path.resolve()` in `google.py`
- [x] Replace `os.path.exists()` with `Path.exists()` in `google.py`
- [x] Replace insecure usage of temporary file directory `/tmp` with `tempfile.gettempdir()` in `test_google_falla_debug.py`
- [x] Replace `os.path.join()` with `Path` and the `/` operator in `test_google_falla_debug.py`
- [x] Remove unused imports (`os` and `NavigableString`) from `google.py`
- [x] Add descriptive error messages when engines are not found or disabled
- [x] Handle engine initialization failures gracefully
- [x] Improve error handling in `init_engine_task` function
- [x] Update `search` function to handle the changes to `init_engine_task`
- [x] Add standardization of engine names for more consistent lookups
- [x] Add wrapper coroutine to handle exceptions during search process
- [x] Add detailed logging for engine initialization and search processes
- [x] Return empty results on failure instead of raising exceptions

Tip: Periodically run `./cleanup.py status` to see results of lints and tests.

This is the test command that we are targeting: 

```bash
for engine in $(twat-search web info --plain); do echo; echo; echo; echo ">>> $engine"; twat-search web q -e $engine "Adam Twardoch" -n 1 --json --verbose; done;
```

## High Priority

### Phase 1: Fix Falla-based Search Engines

- [ ] Fix Yahoo and Qwant search engines
  - [ ] Update selectors in Yahoo implementation to match current page structure
  - [ ] Update selectors in Qwant implementation to match current page structure
  - [ ] Test with various search queries to ensure reliability
  - [ ] Handle consent pages and other interactive elements

- [ ] Fix Google and DuckDuckGo search engines
  - [ ] Update selectors in Google implementation to match current page structure
  - [ ] Update selectors in DuckDuckGo implementation to match current page structure
  - [ ] Implement proper handling of CAPTCHA challenges
  - [ ] Test with various search queries to ensure reliability

- [ ] Fix type errors in `google.py`:
  - [ ] Fix type error in `wait_for_selector` where `self.wait_for_selector` could be `None`
  - [ ] Fix type error in `find_all` where `attrs` parameter has incompatible type
  - [ ] Fix type error in `get_title`, `get_link`, and `get_snippet` where `elem` parameter has incompatible type
  - [ ] Fix type error in `elem.find` where `PageElement` has no attribute `find`

- [ ] Improve Falla integration with Playwright:
  - [ ] Ensure proper browser and context management for efficient resource usage
  - [ ] Implement specific exception handling for common Playwright errors
  - [ ] Add comprehensive type hinting throughout the codebase
  - [ ] Improve method docstrings for better code documentation

### Phase 2: Address Remaining Engine Issues

- [ ] Fix engines returning empty results
  - [ ] Identify and address common failure patterns
  - [ ] Add detailed logging for debugging
  - [ ] Create test script to isolate issues

- [ ] Fix linting errors in the codebase:
  - [ ] Address FBT001/FBT002 errors in CLI functions (Boolean-typed positional arguments)
  - [ ] Fix E501 line length issues in various files
  - [ ] Address F401 unused import errors in `__init__.py` and other files
  - [ ] Fix B904 exception handling issues (use `raise ... from err` pattern)
  - [ ] Address PLR2004 magic value comparison issues

## Medium Priority

### Improve Code Quality

- [ ] Implement comprehensive error handling:
  - [ ] Add try-except blocks for all external API calls
  - [ ] Create custom exception classes for different error scenarios
  - [ ] Add graceful fallbacks for common error cases

- [ ] Improve test coverage:
  - [ ] Add unit tests for all search engine implementations
  - [ ] Add integration tests for the entire search pipeline
  - [ ] Implement mock responses for external API calls in tests
  - [ ] Add performance benchmarks for search operations

- [ ] Enhance documentation:
  - [ ] Add detailed docstrings to all classes and methods
  - [ ] Create comprehensive API documentation
  - [ ] Add usage examples for all search engines
  - [ ] Document configuration options and environment variables

### Standardize JSON Output Format

- [ ] Standardize JSON output across all engines
  - [ ] Utilize the existing `SearchResult` model consistently
  - [ ] Remove utility functions like `_process_results` and `_display_json_results`
  - [ ] Remove `CustomJSONEncoder` class
  - [ ] Update engine `search` methods to return list of `SearchResult` objects

- [ ] Update API function return types
  - [ ] Change return type to `list[SearchResult]`
  - [ ] Ensure proper handling of results from engines

- [ ] Update CLI display functions
  - [ ] Use `model_dump` for JSON serialization
  - [ ] Implement simplified result display

## Low Priority

### Feature Enhancements

- [ ] Add support for additional search engines:
  - [ ] Implement Ecosia search
  - [ ] Implement Startpage search
  - [ ] Implement other alternative search engines
  - [ ] Implement Qwant AI engine using the QwantAI package (https://pypi.org/project/QwantAI/)

- [ ] Enhance result processing:
  - [ ] Implement result deduplication across engines
  - [ ] Add result ranking based on relevance
  - [ ] Implement result filtering options
  - [ ] Add support for different result formats (HTML, Markdown, etc.)

- [ ] Improve CLI functionality:
  - [ ] Add interactive mode for search operations
  - [ ] Implement result pagination in CLI output
  - [ ] Add support for saving search results to file
  - [ ] Implement search history functionality

- [ ] Add advanced search features:
  - [ ] Implement image search capabilities
  - [ ] Add support for news search
  - [ ] Implement video search functionality
  - [ ] Add support for academic/scholarly search

- [ ] Optimize performance:
  - [ ] Implement caching for search results
  - [ ] Optimize concurrent search operations
  - [ ] Reduce memory usage during search operations
  - [ ] Implement timeout handling for slow search engines

## Completed Tasks

- [x] Setup and dependencies
  - [x] Create new module `src/twat_search/web/engines/falla.py`
  - [x] Add necessary dependencies to `pyproject.toml`
  - [x] Define engine constants in `src/twat_search/web/engine_constants.py`
  - [x] Update `src/twat_search/web/engines/__init__.py` to import and register new engines
  - [x] Create utility function to check if Falla is installed and accessible

- [x] Create base `FallaSearchEngine` class
  - [x] Inherit from `SearchEngine`
  - [x] Implement `search` method with proper error handling and retries
  - [x] Create fallback implementation for when Falla is not available

- [x] Implement specific Falla-based engines
  - [x] `google-falla`: Google search using Falla
  - [x] `bing-falla`: Bing search using Falla
  - [x] `duckduckgo-falla`: DuckDuckGo search using Falla
  - [x] `yahoo-falla`: Yahoo search using Falla
  - [x] `qwant-falla`: Qwant search using Falla
  - [x] And other engines (Aol, Ask, Dogpile, Gibiru, Mojeek, Yandex)

- [x] Fix linting errors in the codebase:
  - [x] Replace `os.path.abspath()` with `Path.resolve()` in `google.py`
  - [x] Replace `os.path.exists()` with `Path.exists()` in `google.py`
  - [x] Replace insecure usage of temporary file directory `/tmp` with `tempfile.gettempdir()` in `test_google_falla_debug.py`
  - [x] Replace `os.path.join()` with `Path` and the `/` operator in `test_google_falla_debug.py`
  - [x] Remove unused imports (`os` and `NavigableString`) from `google.py`

- [x] Improve `get_engine` function and error handling
  - [x] Add descriptive error messages when engines are not found or disabled
  - [x] Handle engine initialization failures gracefully
  - [x] Improve error handling in `init_engine_task` function
  - [x] Update `search` function to handle the changes to `init_engine_task`
  - [x] Add standardization of engine names for more consistent lookups
  - [x] Add wrapper coroutine to handle exceptions during search process
  - [x] Add detailed logging for engine initialization and search processes
  - [x] Return empty results on failure instead of raising exceptions

================
File: plugins/repos/twat_search/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_speech/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_speech/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_speech/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 800]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 958]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1.1K]  LOG.md
├── [ 712]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [ 426]  package.toml
├── [7.3K]  pyproject.toml
├── [ 128]  src
│   └── [ 128]  twat_speech
│       └── [1.6K]  twat_speech.py
└── [ 128]  tests
    └── [ 157]  test_twat_speech.py

10 directories, 18 files

================
File: plugins/repos/twat_speech/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_speech/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_speech --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_speech/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-speech
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_speech/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_speech/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_speech/dist/.gitignore
================
*

================
File: plugins/repos/twat_speech/src/twat_speech/__version__.py
================


================
File: plugins/repos/twat_speech/src/twat_speech/twat_speech.py
================
logging.basicConfig(
logger = logging.getLogger(__name__)
class Config:
def process_data(
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        raise ValueError(msg)
def main() -> None:
        config = Config(name="default", value="test", options={"key": "value"})
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        logger.exception("An error occurred: %s", str(e))
    main()

================
File: plugins/repos/twat_speech/tests/test_twat_speech.py
================
def test_version():

================
File: plugins/repos/twat_speech/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_speech/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_speech/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_speech/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_speech/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v0.0.1] - 2025-02-15

### Added

- Initial release of the twat-speech package
- Basic project structure with modern Python packaging (PEP 621 compliance)
- Development workflow using Hatch
- Type hints and runtime type checking setup
- Test suite configuration
- CI/CD configuration in .github directory
- Basic documentation in README.md

### Changed

- Restructured source code: moved `twat_speech.py` into proper package structure
- Updated .gitignore with comprehensive Python-specific ignores

### Note

This is the first release of the package, establishing the foundation for future development.

## [Unreleased]

_No unreleased changes yet._

[Unreleased]: https://github.com/twardoch/twat-speech/compare/v0.0.1...HEAD
[v0.0.1]: https://github.com/twardoch/twat-speech/releases/tag/v0.0.1

================
File: plugins/repos/twat_speech/package.toml
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_speech/pyproject.toml
================
# this_file: twat_speech/pyproject.toml

# this_file: twat_speech/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_speech"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-speech"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-speech#readme"
Issues = "https://github.com/twardoch/twat-speech/issues"
Source = "https://github.com/twardoch/twat-speech"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
speech = "twat_speech"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_speech/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_speech --cov=tests {args:tests}"
type-check = "mypy src/twat_speech tests"
lint = ["ruff check src/twat_speech tests", "ruff format src/twat_speech tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_speech tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_speech", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_speech/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_speech = ["src/twat_speech", "*/twat-speech/src/twat_speech"]
tests = ["tests", "*/twat-speech/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "twat>=1.8.1",           # Main twat package
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_speech --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_speech/README.md
================
# 



## Features

- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-speech
```

## Usage

```python
import twat_speech
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_speech/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_task/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_task/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_task/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 768]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [1.0K]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 500]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [2.1K]  LOG.md
├── [ 716]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [7.5K]  pyproject.toml
├── [ 160]  src
│   ├── [ 408]  testprefect.py
│   └── [ 224]  twat_task
│       ├── [ 940]  __init__.py
│       ├── [5.0K]  __main__.py
│       └── [3.8K]  task.py
└── [ 128]  tests
    └── [ 151]  test_twat_task.py

10 directories, 20 files

================
File: plugins/repos/twat_task/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_task/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_task --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_task/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-task
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_task/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_task/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_task/dist/.gitignore
================
*

================
File: plugins/repos/twat_task/src/twat_task/__init__.py
================


================
File: plugins/repos/twat_task/src/twat_task/__main__.py
================
@task(retries=2)
def extract_audio_task(video_path: Path, audio_path: Path) -> None:
    time.sleep(2)  # Simulate API call
        "duration": randint(60, 3600),
        "bitrate": f"{randint(128, 320)}kbps",
    for _i in range(10):
        time.sleep(0.5)  # Simulate processing time
    audio_path.write_text(json.dumps(metadata))
def generate_transcript_task(audio_path: Path) -> str:
    metadata = json.loads(audio_path.read_text())
    time.sleep(1.5)
    for _i in range(chunks):
        time.sleep(0.3)  # Simulate API call and processing
        chunk_text = " ".join(choice(words) for _ in range(randint(5, 15)))
        transcript_parts.append(chunk_text)
    return " ".join(transcript_parts)
def process_video_flow(video_path: Path) -> tuple[Path, str]:
    audio = video_path.with_suffix(".mp3")
    if not audio.exists():
        extract_audio_task(video_path, audio)
    transcript = generate_transcript_task(audio)
class VideoTranscript(BaseModel):
    def audio_path(self) -> Path:
        audio, _ = process_video_flow(self.video_path)
    def text_transcript(self) -> str:
        _, transcript = process_video_flow(self.video_path)
    vt = VideoTranscript(video_path=Path("video.mp4"))

================
File: plugins/repos/twat_task/src/twat_task/__version__.py
================


================
File: plugins/repos/twat_task/src/twat_task/task.py
================
@task(retries=2)
def extract_audio_task(video_path: Path, audio_path: Path) -> None:
    time.sleep(2)  # Simulate API call
        "duration": randint(60, 3600),
        "bitrate": f"{randint(128, 320)}kbps",
    for _i in range(10):
        time.sleep(0.5)  # Simulate processing time
    audio_path.write_text(json.dumps(metadata))
def generate_transcript_task(audio_path: Path) -> str:
    metadata = json.loads(audio_path.read_text())
    time.sleep(1.5)
    for _i in range(chunks):
        time.sleep(0.3)  # Simulate API call and processing
        chunk_text = " ".join(choice(words) for _ in range(randint(5, 15)))
        transcript_parts.append(chunk_text)
    return " ".join(transcript_parts)
def process_video_flow(video_path: Path) -> tuple[Path, str]:
    audio = video_path.with_suffix(".mp3")
    if not audio.exists():
        extract_audio_task(video_path, audio)
    transcript = generate_transcript_task(audio)
class VideoTranscript(BaseModel):
    def audio_path(self) -> Path:
        audio, _ = process_video_flow(self.video_path)
    def text_transcript(self) -> str:
        _, transcript = process_video_flow(self.video_path)

================
File: plugins/repos/twat_task/src/testprefect.py
================
@flow(log_prints=True)
def hello_world(name: str = "world", goodbye: bool = False):
    hello_world.serve(

================
File: plugins/repos/twat_task/tests/test_twat_task.py
================
def test_version():

================
File: plugins/repos/twat_task/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_task/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_task/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_task/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_task/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to the `twat-task` project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v1.7.5] - 2025-02-15

### Changed

- Updated README.md with minor improvements
- Project documentation refinements

## [v1.7.3] - 2025-02-15

### Added

- Enhanced task functionality in `task.py`
- New features in main CLI interface

### Fixed

- Improvements to test Prefect integration
- Bug fixes in task execution flow

## [v1.7.0] - 2025-02-13

### Added

- New test Prefect integration file
- Enhanced project configuration

### Changed

- Updated gitignore rules
- Simplified package initialization
- Refined dependency management

## [v1.6.2] - 2025-02-06

### Fixed

- Updated project configuration in pyproject.toml

## [v1.6.1] - 2025-02-06

### Changed

- Refactored task implementation
- Streamlined main CLI interface
- Updated package initialization
- Enhanced test coverage

## [v1.6.0] - 2025-02-06

### Changed

- Significantly simplified README.md
- Focused documentation on core features

## [v1.1.0] - 2025-02-03

### Added

- Comprehensive README with project documentation
- Enhanced project structure
- Initial implementation of pipeline functionality
- Basic CLI interface
- Test framework setup

## [v1.0.0] - 2025-02-06

### Added

- Initial project setup
- GitHub Actions workflows for CI/CD
- Basic project structure
- License and configuration files
- Core package functionality
- Initial test suite

[v1.7.5]: https://github.com/twardoch/twat-task/compare/v1.7.3...v1.7.5
[v1.7.3]: https://github.com/twardoch/twat-task/compare/v1.7.0...v1.7.3
[v1.7.0]: https://github.com/twardoch/twat-task/compare/v1.6.2...v1.7.0
[v1.6.2]: https://github.com/twardoch/twat-task/compare/v1.6.1...v1.6.2
[v1.6.1]: https://github.com/twardoch/twat-task/compare/v1.6.0...v1.6.1
[v1.6.0]: https://github.com/twardoch/twat-task/compare/v1.1.0...v1.6.0
[v1.1.0]: https://github.com/twardoch/twat-task/compare/v1.0.0...v1.1.0
[v1.0.0]: https://github.com/twardoch/twat-task/releases/tag/v1.0.0

================
File: plugins/repos/twat_task/pyproject.toml
================
# this_file: twat_task/pyproject.toml

# this_file: twat_task/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_task"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-task"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = "Video processing task utilities using Prefect for workflow management"
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = ["task", "workflow", "prefect", "video", "audio", "transcription"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "prefect>=3.1.0",
    "pydantic>=2.0.0",
    "twat>=1.8.1",           # Main twat package
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-task#readme"
Issues = "https://github.com/twardoch/twat-task/issues"
Source = "https://github.com/twardoch/twat-task"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
task = "twat_task"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_task/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_task --cov=tests {args:tests}"
type-check = "mypy src/twat_task tests"
lint = ["ruff check src/twat_task tests", "ruff format src/twat_task tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_task tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_task", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_task/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_task = ["src/twat_task", "*/twat-task/src/twat_task"]
tests = ["tests", "*/twat-task/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "prefect>=3.1.0",
    "pydantic>=2.0.0",
    "twat>=1.8.1",           # Main twat package
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_task --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_task/README.md
================
# twat-task

(work in progress)

Video processing pipeline utilities using Prefect for workflow management. This package provides a set of tasks and flows for processing video files, including audio extraction and transcription.

## Features

- High-level interface for video processing pipelines
- Built-in tasks for common operations:
  - Audio extraction from video files
  - Speech-to-text transcription
- Workflow management using Prefect:
  - Automatic retries for failed tasks
  - Progress tracking and logging
  - Task caching and resumability
- Pydantic models for data validation
- Type hints and modern Python features

## Installation

```bash
pip install twat-task
```

TBD

## License

MIT License  
.

================
File: plugins/repos/twat_task/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_text/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_text/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_text/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 800]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 952]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1.3K]  LOG.md
├── [ 708]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [ 426]  package.toml
├── [7.3K]  pyproject.toml
├── [ 128]  src
│   └── [ 128]  twat_text
│       └── [1.6K]  twat_text.py
└── [ 128]  tests
    └── [ 151]  test_twat_text.py

10 directories, 18 files

================
File: plugins/repos/twat_text/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_text/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_text --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_text/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-text
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_text/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_text/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_text/dist/.gitignore
================
*

================
File: plugins/repos/twat_text/src/twat_text/__version__.py
================


================
File: plugins/repos/twat_text/src/twat_text/twat_text.py
================
logging.basicConfig(
logger = logging.getLogger(__name__)
class Config:
def process_data(
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        raise ValueError(msg)
def main() -> None:
        config = Config(name="default", value="test", options={"key": "value"})
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        logger.exception("An error occurred: %s", str(e))
    main()

================
File: plugins/repos/twat_text/tests/test_twat_text.py
================
def test_version():

================
File: plugins/repos/twat_text/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_text/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_text/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_text/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_text/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

*No unreleased changes yet.*

## [v0.0.1] - 2025-02-15

Initial release of the twat-text package.

### Added

- Basic project structure with modern Python packaging (PEP 621 compliance)
- Initial implementation of `twat_text.py` with:
  - Type hints and dataclass-based configuration
  - Basic logging setup
  - Process data function skeleton
- Development workflow setup with Hatch
- Basic test suite configuration
- MIT License
- Project documentation in README.md

### Changed

- Moved `src/twat_text.py` to `src/twat_text/twat_text.py` for better package structure
- Updated .gitignore to exclude `_private` directory

### Technical Details

- Added proper type hints using modern Python syntax (|)
- Implemented error handling with proper exception messages
- Set up logging with debug mode support
- Added `__future__.annotations` import for modern type hint handling

[Unreleased]: https://github.com/twardoch/twat-text/compare/v0.0.1...HEAD
[v0.0.1]: https://github.com/twardoch/twat-text/releases/tag/v0.0.1

================
File: plugins/repos/twat_text/package.toml
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_text/pyproject.toml
================
# this_file: twat_text/pyproject.toml

# this_file: twat_text/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_text"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-text"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
]

# Project Authors
# ---------------
[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"

# Project URLs
# ------------
# Links to project resources for documentation, issues, and source code
[project.urls]
Documentation = "https://github.com/twardoch/twat-text#readme"
Issues = "https://github.com/twardoch/twat-text/issues"
Source = "https://github.com/twardoch/twat-text"

# Twat Plugin Registration
# -----------------------
# Registers this package as a plugin for the twat ecosystem
[project.entry-points."twat.plugins"]
text = "twat_text"

# Version configuration using VCS (Git)
[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

# VCS hook configuration for version file generation
[tool.hatch.build.hooks.vcs]
version-file = "src/twat_text/__version__.py"

# Default development environment configuration
[tool.hatch.envs.default]
dependencies = [
    "pytest",                # Testing framework
    "pytest-cov",           # Coverage reporting
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Scripts available in the default environment
[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_text --cov=tests {args:tests}"
type-check = "mypy src/twat_text tests"
lint = ["ruff check src/twat_text tests", "ruff format src/twat_text tests"]

# Python version matrix for testing
[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]

# Linting environment configuration
[tool.hatch.envs.lint]
detached = true  # Run in isolated environment
dependencies = [
    "mypy>=1.15.0",         # Static type checker
    "ruff>=0.9.6",          # Fast Python linter
]

# Linting environment scripts
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_text tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]

# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88

# Ruff lint rules configuration
[tool.ruff.lint]
extend-select = [
    "A",     # flake8-builtins
    "ARG",   # flake8-unused-arguments
    "B",     # flake8-bugbear
    "C",     # flake8-comprehensions
    "DTZ",   # flake8-datetimez
    "E",     # pycodestyle errors
    "EM",    # flake8-errmsg
    "F",     # pyflakes
    "FBT",   # flake8-boolean-trap
    "I",     # isort
    "ICN",   # flake8-import-conventions
    "ISC",   # flake8-implicit-str-concat
    "N",     # pep8-naming
    "PLC",   # pylint convention
    "PLE",   # pylint error
    "PLR",   # pylint refactor
    "PLW",   # pylint warning
    "Q",     # flake8-quotes
    "RUF",   # Ruff-specific rules
    "S",     # flake8-bandit
    "T",     # flake8-debugger
    "TID",   # flake8-tidy-imports
    "UP",    # pyupgrade
    "W",     # pycodestyle warnings
    "YTT",   # flake8-2020
]
ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",   # Import block formatting
]

# File-specific Ruff configurations
[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

# MyPy (type checker) configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true

# Coverage.py configuration for test coverage
[tool.coverage.run]
source_pkgs = ["twat_text", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_text/__about__.py",
]

# Coverage path mappings
[tool.coverage.paths]
twat_text = ["src/twat_text", "*/twat-text/src/twat_text"]
tests = ["tests", "*/twat-text/tests"]

# Coverage report configuration
[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

# Optional dependencies
[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0",  # For performance testing
]

dev = [
    "pre-commit>=4.1.0",     # Git pre-commit hooks
    "ruff>=0.9.6",           # Fast Python linter
    "mypy>=1.15.0",          # Static type checker
]

all = [
    "twat>=1.8.1",           # Main twat package
]

# Test environment configuration
[tool.hatch.envs.test]
dependencies = [".[test]"]

# Test environment scripts
[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_text --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

# Pytest configuration
[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

# Pytest-benchmark configuration
[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_text/README.md
================
# 



## Features

- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-text
```

## Usage

```python
import twat_text
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_text/VERSION.txt
================
v2.6.1

================
File: plugins/repos/twat_video/.cursor/rules/0project.mdc
================
---
description: About this project
globs:
---
# About this project

`twat-fs` is a file system utility library focused on robust and extensible file upload capabilities with multiple provider support. It provides:

- Multi-provider upload system with smart fallback (catbox.moe default, plus Dropbox, S3, etc.)
- Automatic retry for temporary failures, fallback for permanent ones
- URL validation and clean developer experience with type hints
- Simple CLI: `python -m twat_fs upload_file path/to/file.txt`
- Easy installation: `uv pip install twat-fs` (basic) or `uv pip install 'twat-fs[all,dev]'` (all features)

## Development Notes
- Uses `uv` for Python package management
- Quality tools: ruff, mypy, pytest
- Clear provider protocol for adding new storage backends
- Strong typing and runtime checks throughout

================
File: plugins/repos/twat_video/.cursor/rules/cleanup.mdc
================
---
description: Run `cleanup.py` script before and after changes
globs: 
---
Before you do any changes or if I say "cleanup", run the `cleanup.py update` script in the main folder. Analyze the results, describe recent changes in @LOG.md and edit @TODO.md to update priorities and plan next changes. PERFORM THE CHANGES, then run the `cleanup.py status` script and react to the results.

When you edit @TODO.md, lead in lines with empty GFM checkboxes if things aren't done (`- [ ] `) vs. filled (`- [x] `) if done.

================
File: plugins/repos/twat_video/.cursor/rules/filetree.mdc
================
---
description: File tree of the project
globs: 
---
[ 864]  .
├── [  64]  .benchmarks
├── [  96]  .cursor
│   └── [ 224]  rules
│       ├── [ 821]  0project.mdc
│       ├── [ 516]  cleanup.mdc
│       ├── [ 997]  filetree.mdc
│       └── [2.0K]  quality.mdc
├── [  96]  .github
│   └── [ 128]  workflows
│       ├── [2.7K]  push.yml
│       └── [1.4K]  release.yml
├── [3.5K]  .gitignore
├── [ 470]  .pre-commit-config.yaml
├── [ 987]  CLEANUP.txt
├── [1.0K]  LICENSE
├── [1.2K]  LOG.md
├── [ 710]  README.md
├── [   7]  VERSION.txt
├── [ 13K]  cleanup.py
├── [ 160]  dist
├── [ 426]  package.toml
├── [5.4K]  pyproject.toml
├── [ 128]  src
│   └── [ 192]  twat_video
│       ├── [ 142]  __init__.py
│       └── [1.6K]  twat_video.py
└── [ 128]  tests
    └── [ 154]  test_twat_video.py

10 directories, 19 files

================
File: plugins/repos/twat_video/.cursor/rules/quality.mdc
================
---
description: Quality
globs: 
---
- **Verify Information**: Always verify information before presenting it. Do not make assumptions or speculate without clear evidence.
- **No Apologies**: Never use apologies.
- **No Whitespace Suggestions**: Don't suggest whitespace changes.
- **No Inventions**: Don't invent major changes other than what's explicitly requested.
- **No Unnecessary Confirmations**: Don't ask for confirmation of information already provided in the context.
- **Preserve Existing Code**: Don't remove unrelated code or functionalities. Pay attention to preserving existing structures.
- **No Implementation Checks**: Don't ask the user to verify implementations that are visible in the provided context.
- **No Unnecessary Updates**: Don't suggest updates or changes to files when there are no actual modifications needed.
- **No Current Implementation**: Don't show or discuss the current implementation unless specifically requested.
- **Use Explicit Variable Names**: Prefer descriptive, explicit variable names over short, ambiguous ones to enhance code readability.
- **Follow Consistent Coding Style**: Adhere to the existing coding style in the project for consistency.
- **Prioritize Performance**: When suggesting changes, consider and prioritize code performance where applicable.
- **Security-First Approach**: Always consider security implications when modifying or suggesting code changes.
- **Test Coverage**: Suggest or include appropriate unit tests for new or modified code.
- **Error Handling**: Implement robust error handling and logging where necessary.
- **Modular Design**: Encourage modular design principles to improve code maintainability and reusability.
- **Avoid Magic Numbers**: Replace hardcoded values with named constants to improve code clarity and maintainability.
- **Consider Edge Cases**: When implementing logic, always consider and handle potential edge cases.
- **Use Assertions**: Include assertions wherever possible to validate assumptions and catch potential errors early.

================
File: plugins/repos/twat_video/.github/workflows/push.yml
================
name: Build & Test

on:
  push:
    branches: [main]
    tags-ignore: ["v*"]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: write
  id-token: write

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  quality:
    name: Code Quality
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Ruff lint
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "check --output-format=github"

      - name: Run Ruff Format
        uses: astral-sh/ruff-action@v3
        with:
          version: "latest"
          args: "format --check --respect-gitignore"

  test:
    name: Run Tests
    needs: quality
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
        os: [ubuntu-latest]
      fail-fast: true
    runs-on: ${{ matrix.os }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: ${{ matrix.python-version }}
          enable-cache: true
          cache-suffix: ${{ matrix.os }}-${{ matrix.python-version }}

      - name: Install test dependencies
        run: |
          uv pip install --system --upgrade pip
          uv pip install --system ".[test]"

      - name: Run tests with Pytest
        run: uv run pytest -n auto --maxfail=1 --disable-warnings --cov-report=xml --cov-config=pyproject.toml --cov=src/twat_video --cov=tests tests/

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}-${{ matrix.os }}
          path: coverage.xml

  build:
    name: Build Distribution
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Upload distribution artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist-files
          path: dist/
          retention-days: 5

================
File: plugins/repos/twat_video/.github/workflows/release.yml
================
name: Release

on:
  push:
    tags: ["v*"]

permissions:
  contents: write
  id-token: write

jobs:
  release:
    name: Release to PyPI
    runs-on: ubuntu-latest
    environment:
      name: pypi
      url: https://pypi.org/p/twat-video
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install UV
        uses: astral-sh/setup-uv@v5
        with:
          version: "latest"
          python-version: "3.12"
          enable-cache: true

      - name: Install build tools
        run: uv pip install build hatchling hatch-vcs

      - name: Build distributions
        run: uv run python -m build --outdir dist

      - name: Verify distribution files
        run: |
          ls -la dist/
          test -n "$(find dist -name '*.whl')" || (echo "Wheel file missing" && exit 1)
          test -n "$(find dist -name '*.tar.gz')" || (echo "Source distribution missing" && exit 1)

      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_TOKEN }}

      - name: Create GitHub Release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/*
          generate_release_notes: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

================
File: plugins/repos/twat_video/.ruff_cache/.gitignore
================
# Automatically created by ruff.
*

================
File: plugins/repos/twat_video/.ruff_cache/CACHEDIR.TAG
================
Signature: 8a477f597d28d172789f06886806bc55

================
File: plugins/repos/twat_video/dist/.gitignore
================
*

================
File: plugins/repos/twat_video/src/twat_video/__init__.py
================


================
File: plugins/repos/twat_video/src/twat_video/__version__.py
================


================
File: plugins/repos/twat_video/src/twat_video/twat_video.py
================
logging.basicConfig(
logger = logging.getLogger(__name__)
class Config:
def process_data(
        logger.setLevel(logging.DEBUG)
        logger.debug("Debug mode enabled")
        raise ValueError(msg)
def main() -> None:
        config = Config(name="default", value="test", options={"key": "value"})
        result = process_data([], config=config)
        logger.info("Processing completed: %s", result)
        logger.exception("An error occurred: %s", str(e))
    main()

================
File: plugins/repos/twat_video/tests/test_twat_video.py
================
def test_version():

================
File: plugins/repos/twat_video/.gitignore
================
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: plugins/repos/twat_video/.pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: plugins/repos/twat_video/cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: plugins/repos/twat_video/LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: plugins/repos/twat_video/LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v0.0.1] - 2025-02-15

Initial release of the twat-video package.

### Added

- Basic project structure with modern Python packaging (PEP 621 compliant)
- Initial implementation of `twat_video.py` with:
  - Configuration class with type hints
  - Data processing functionality
  - Logging setup
- Added `videoextendprompt.py` script for:
  - Generating video continuation prompts based on image frame analysis
  - Support for multiple AI models (Gemini, GPT-4, Claude-3, Grok-2)
  - Image pair processing functionality
  - Markdown output generation
- Project setup with Hatch for development workflow
- Basic documentation in README.md
- MIT License

### Changed

- Moved `twat_video.py` to proper module structure
- Updated .gitignore to exclude private files

## [Initial Commit] - 2025-02-09

- Repository initialization
- Basic project structure setup

[v0.0.1]: https://github.com/twardoch/twat-video/releases/tag/v0.0.1

================
File: plugins/repos/twat_video/package.toml
================
# Package configuration
[package]
include_cli = true        # Include CLI boilerplate
include_logging = true    # Include logging setup
use_pydantic = true      # Use Pydantic for data validation
use_rich = true          # Use Rich for terminal output

[features]
mkdocs = false           # Enable MkDocs documentation
vcs = true              # Initialize Git repository
github_actions = true   # Add GitHub Actions workflows

================
File: plugins/repos/twat_video/pyproject.toml
================
# this_file: twat_video/pyproject.toml

# this_file: twat_video/pyproject.toml

# Build System Configuration
# -------------------------
# Specifies the build system and its requirements for packaging the project
# Specifies the build backend and its requirements for building the package
[build-system]
requires = [
    "hatchling>=1.27.0",     # Core build backend for Hatch
    "hatch-vcs>=0.4.0",      # Version Control System plugin for Hatch
]
build-backend = "hatchling.build"  # Use Hatchling as the build backend

# Wheel build configuration
# Specifies which packages to include in the wheel distribution
[tool.hatch.build.targets.wheel]
packages = ["src/twat_video"]

# Project Metadata Configuration
# ------------------------------
# Comprehensive project description, requirements, and compatibility information
[project]
name = "twat-video"
dynamic = ["version"]  # Version is determined dynamically from VCS
description = ""
readme = "README.md"
requires-python = ">=3.10"  # Minimum Python version required
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

# Runtime Dependencies
# -------------------
# External packages required for the project to function
dependencies = [
    "twat>=1.8.1",           # Main twat package
]


[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"


[project.urls]
Documentation = "https://github.com/twardoch/twat-video#readme"
Issues = "https://github.com/twardoch/twat-video/issues"
Source = "https://github.com/twardoch/twat-video"




[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

[tool.hatch.build.hooks.vcs]
version-file = "src/twat_video/__version__.py"



[tool.hatch.envs.default]
dependencies = [
    "pytest",
    "pytest-cov",
    "mypy>=1.0.0",
    "ruff>=0.1.0",
]


[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_video --cov=tests {args:tests}"
type-check = "mypy src/twat_video tests"
lint = ["ruff check src/twat_video tests", "ruff format src/twat_video tests"]


[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]


[tool.hatch.envs.lint]
detached = true
dependencies = [
    "mypy>=1.0.0",
    "ruff>=0.1.0",
]


[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat_video tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]


[tool.ruff]
target-version = "py310"
line-length = 88
lint.extend-select = [
    "A",
    "ARG",
    "B",
    "C",
    "DTZ",
    "E",
    "EM",
    "F",
    "FBT",
    "I",
    "ICN",
    "ISC",
    "N",
    "PLC",
    "PLE",
    "PLR",
    "PLW",
    "Q",
    "RUF",
    "S",
    "T",
    "TID",
    "UP",
    "W",
    "YTT",
]
lint.ignore = [
    "ARG001", # Unused function argument
    "E501",   # Line too long
    "I001",
]


[tool.ruff.per-file-ignores]
"tests/*" = ["S101"]


[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true


[tool.coverage.run]
source_pkgs = ["twat_video", "tests"]
branch = true
parallel = true
omit = [
    "src/twat_video/__about__.py",
]


[tool.coverage.paths]
twat_video = ["src/twat_video", "*/twat-video/src/twat_video"]
tests = ["tests", "*/twat-video/tests"]


[tool.coverage.report]
exclude_lines = [
    "no cov",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

[project.optional-dependencies]
test = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "pytest-xdist>=3.5.0",  # For parallel test execution
    "pytest-benchmark[histogram]>=4.0.0",  # For performance testing
]

dev = [
    "pre-commit>=3.6.0",
    "ruff>=0.1.0",
    "mypy>=1.0.0",
]

all = [
    "twat>=1.0.0",
]

[tool.hatch.envs.test]
dependencies = [".[test]"]

[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat_video --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: plugins/repos/twat_video/README.md
================
# 



## Features

- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat-video
```

## Usage

```python
import twat_video
```

## Development

This project uses [Hatch](https://hatch.pypa.io/) for development workflow management.

### Setup Development Environment

```bash
# Install hatch if you haven't already
pip install hatch

# Create and activate development environment
hatch shell

# Run tests
hatch run test

# Run tests with coverage
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: plugins/repos/twat_video/VERSION.txt
================
v2.6.1

================
File: src/twat/__init__.py
================
__version__ = metadata.version(__name__)
def _load_plugin(name: str) -> ModuleType | None:
        eps = metadata.entry_points(group="twat.plugins")
                plugin = ep.load()
def __getattr__(name: str) -> Any:
    plugin = _load_plugin(name)
    raise AttributeError(msg)
def run_plugin(plugin_name: str) -> NoReturn:
                if hasattr(plugin, "main"):
                    plugin.main()
                    sys.exit(0)
                    sys.exit(1)
def main() -> NoReturn:
    if len(sys.argv) < 2:
    run_plugin(plugin_name)

================
File: src/twat/__version__.py
================


================
File: tests/test_twat.py
================
def test_version():

================
File: tests/VERSION.txt
================
v1.7.12

================
File: .gitignore
================
plugins/repos/
*_autogen/
.DS_Store
__version__.py
__pycache__/
_Chutzpah*
_deps
_NCrunch_*
_pkginfo.txt
_Pvt_Extensions
_ReSharper*/
_TeamCity*
_UpgradeReport_Files/
!?*.[Cc]ache/
!.axoCover/settings.json
!.vscode/extensions.json
!.vscode/launch.json
!.vscode/settings.json
!.vscode/tasks.json
!**/[Pp]ackages/build/
!Directory.Build.rsp
.*crunch*.local.xml
.axoCover/*
.builds
.cr/personal
.fake/
.history/
.ionide/
.localhistory/
.mfractor/
.ntvs_analysis.dat
.paket/paket.exe
.sass-cache/
.vs/
.vscode
.vscode/*
.vshistory/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
[Bb]in/
[Bb]uild[Ll]og.*
[Dd]ebug/
[Dd]ebugPS/
[Dd]ebugPublic/
[Ee]xpress/
[Ll]og/
[Ll]ogs/
[Oo]bj/
[Rr]elease/
[Rr]eleasePS/
[Rr]eleases/
[Tt]est[Rr]esult*/
[Ww][Ii][Nn]32/
*_h.h
*_i.c
*_p.c
*_wpftmp.csproj
*- [Bb]ackup ([0-9]).rdl
*- [Bb]ackup ([0-9][0-9]).rdl
*- [Bb]ackup.rdl
*.[Cc]ache
*.[Pp]ublish.xml
*.[Rr]e[Ss]harper
*.a
*.app
*.appx
*.appxbundle
*.appxupload
*.aps
*.azurePubxml
*.bim_*.settings
*.bim.layout
*.binlog
*.btm.cs
*.btp.cs
*.build.csdef
*.cab
*.cachefile
*.code-workspace
*.coverage
*.coveragexml
*.d
*.dbmdl
*.dbproj.schemaview
*.dll
*.dotCover
*.DotSettings.user
*.dsp
*.dsw
*.dylib
*.e2e
*.exe
*.gch
*.GhostDoc.xml
*.gpState
*.ilk
*.iobj
*.ipdb
*.jfm
*.jmconfig
*.la
*.lai
*.ldf
*.lib
*.lo
*.log
*.mdf
*.meta
*.mm.*
*.mod
*.msi
*.msix
*.msm
*.msp
*.ncb
*.ndf
*.nuget.props
*.nuget.targets
*.nupkg
*.nvuser
*.o
*.obj
*.odx.cs
*.opendb
*.opensdf
*.opt
*.out
*.pch
*.pdb
*.pfx
*.pgc
*.pgd
*.pidb
*.plg
*.psess
*.publishproj
*.publishsettings
*.pubxml
*.pyc
*.rdl.data
*.rptproj.bak
*.rptproj.rsuser
*.rsp
*.rsuser
*.sap
*.sbr
*.scc
*.sdf
*.sln.docstates
*.sln.iml
*.slo
*.smod
*.snupkg
*.so
*.suo
*.svclog
*.tlb
*.tlh
*.tli
*.tlog
*.tmp
*.tmp_proj
*.tss
*.user
*.userosscache
*.userprefs
*.vbp
*.vbw
*.VC.db
*.VC.VC.opendb
*.VisualState.xml
*.vsp
*.vspscc
*.vspx
*.vssscc
*.xsd.cs
**/[Pp]ackages/*
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.HTMLClient/GeneratedArtifacts
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
*~
~$*
$tf/
AppPackages/
artifacts/
ASALocalRun/
AutoTest.Net/
Backup*/
BenchmarkDotNet.Artifacts/
bld/
BundleArtifacts/
ClientBin/
cmake_install.cmake
CMakeCache.txt
CMakeFiles
CMakeLists.txt.user
CMakeScripts
CMakeUserPresets.json
compile_commands.json
coverage*.info
coverage*.json
coverage*.xml
csx/
CTestTestfile.cmake
dlldata.c
DocProject/buildhelp/
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/*.HxC
DocProject/Help/*.HxT
DocProject/Help/html
DocProject/Help/Html2
ecf/
FakesAssemblies/
FodyWeavers.xsd
Generated_Code/
Generated\ Files/
healthchecksdb
install_manifest.txt
ipch/
Makefile
MigrationBackup/
mono_crash.*
nCrunchTemp_*
node_modules/
nunit-*.xml
OpenCover/
orleans.codegen.cs
Package.StoreAssociation.xml
paket-files/
project.fragment.lock.json
project.lock.json
publish/
PublishScripts/
rcf/
ScaffoldingReadMe.txt
ServiceFabricBackup/
StyleCopReport.xml
Testing
TestResult.xml
UpgradeLog*.htm
UpgradeLog*.XML
x64/
x86/
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Distribution / packaging
!dist/.gitkeep

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
.ruff_cache/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# IDE
.idea/
.vscode/
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Project specific
__version__.py
_private

================
File: .pre-commit-config.yaml
================
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
        args: [--respect-gitignore]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: debug-statements
      - id: check-case-conflict
      - id: mixed-line-ending
        args: [--fix=lf]

================
File: cleanup.py
================
LOG_FILE = Path("CLEANUP.txt")
os.chdir(Path(__file__).parent)
def new() -> None:
    if LOG_FILE.exists():
        LOG_FILE.unlink()
def prefix() -> None:
    readme = Path(".cursor/rules/0project.mdc")
    if readme.exists():
        log_message("\n=== PROJECT STATEMENT ===")
        content = readme.read_text()
        log_message(content)
def suffix() -> None:
    todo = Path("TODO.md")
    if todo.exists():
        log_message("\n=== TODO.md ===")
        content = todo.read_text()
def log_message(message: str) -> None:
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
    with LOG_FILE.open("a") as f:
        f.write(log_line)
def run_command(cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
        result = subprocess.run(
            log_message(result.stdout)
        log_message(f"Command failed: {' '.join(cmd)}")
        log_message(f"Error: {e.stderr}")
        return subprocess.CompletedProcess(cmd, 1, "", str(e))
def check_command_exists(cmd: str) -> bool:
        return which(cmd) is not None
class Cleanup:
    def __init__(self) -> None:
        self.workspace = Path.cwd()
    def _print_header(self, message: str) -> None:
        log_message(f"\n=== {message} ===")
    def _check_required_files(self) -> bool:
            if not (self.workspace / file).exists():
                log_message(f"Error: {file} is missing")
    def _generate_tree(self) -> None:
        if not check_command_exists("tree"):
            log_message("Warning: 'tree' command not found. Skipping tree generation.")
            rules_dir = Path(".cursor/rules")
            rules_dir.mkdir(parents=True, exist_ok=True)
            tree_result = run_command(["tree", "-a", "-I", ".git", "--gitignore", "-n", "-h", "-I", "*_cache"])
            with open(rules_dir / "filetree.mdc", "w") as f:
                f.write("---\ndescription: File tree of the project\nglobs: \n---\n")
                f.write(tree_text)
            log_message("\nProject structure:")
            log_message(tree_text)
            log_message(f"Failed to generate tree: {e}")
    def _git_status(self) -> bool:
        result = run_command(["git", "status", "--porcelain"], check=False)
        return bool(result.stdout.strip())
    def _venv(self) -> None:
        log_message("Setting up virtual environment")
            run_command(["uv", "venv"])
            if venv_path.exists():
                os.environ["VIRTUAL_ENV"] = str(self.workspace / ".venv")
                log_message("Virtual environment created and activated")
                log_message("Virtual environment created but activation failed")
            log_message(f"Failed to create virtual environment: {e}")
    def _install(self) -> None:
        log_message("Installing package with all extras")
            self._venv()
            run_command(["uv", "pip", "install", "-e", ".[test,dev]"])
            log_message("Package installed successfully")
            log_message(f"Failed to install package: {e}")
    def _run_checks(self) -> None:
        log_message("Running code quality checks")
            log_message(">>> Running code fixes...")
            run_command(
            log_message(">>>Running type checks...")
            run_command(["python", "-m", "mypy", "src", "tests"], check=False)
            log_message(">>> Running tests...")
            run_command(["python", "-m", "pytest", "tests"], check=False)
            log_message("All checks completed")
            log_message(f"Failed during checks: {e}")
    def status(self) -> None:
        prefix()  # Add README.md content at start
        self._print_header("Current Status")
        self._check_required_files()
        self._generate_tree()
        result = run_command(["git", "status"], check=False)
        self._print_header("Environment Status")
        self._install()
        self._run_checks()
        suffix()  # Add TODO.md content at end
    def venv(self) -> None:
        self._print_header("Virtual Environment Setup")
    def install(self) -> None:
        self._print_header("Package Installation")
    def update(self) -> None:
        self.status()
        if self._git_status():
            log_message("Changes detected in repository")
                run_command(["git", "add", "."])
                run_command(["git", "commit", "-m", commit_msg])
                log_message("Changes committed successfully")
                log_message(f"Failed to commit changes: {e}")
            log_message("No changes to commit")
    def push(self) -> None:
        self._print_header("Pushing Changes")
            run_command(["git", "push"])
            log_message("Changes pushed successfully")
            log_message(f"Failed to push changes: {e}")
def repomix(
            cmd.append("--compress")
            cmd.append("--remove-empty-lines")
            cmd.append("-i")
            cmd.append(ignore_patterns)
        cmd.extend(["-o", output_file])
        run_command(cmd)
        log_message(f"Repository content mixed into {output_file}")
        log_message(f"Failed to mix repository: {e}")
def print_usage() -> None:
    log_message("Usage:")
    log_message("  cleanup.py status   # Show current status and run all checks")
    log_message("  cleanup.py venv     # Create virtual environment")
    log_message("  cleanup.py install  # Install package with all extras")
    log_message("  cleanup.py update   # Update and commit changes")
    log_message("  cleanup.py push     # Push changes to remote")
def main() -> NoReturn:
    new()  # Clear log file
    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)
    cleanup = Cleanup()
            cleanup.status()
            cleanup.venv()
            cleanup.install()
            cleanup.update()
            cleanup.push()
        log_message(f"Error: {e}")
    repomix()
    sys.stdout.write(Path("CLEANUP.txt").read_text())
    sys.exit(0)  # Ensure we exit with a status code
    main()

================
File: LICENSE
================
MIT License

Copyright (c) 2025 Adam Twardoch

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: LOG.md
================
---
this_file: LOG.md
---

# Changelog

All notable changes to the `twat` project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.7.6] - 2025-02-15

### Changed

- Updated package dependencies in `pyproject.toml`
- Enhanced documentation in README.md
- Refined package configuration

## [1.7.5] - 2025-02-15

### Added

- Extensive documentation improvements in README.md
- Enhanced package initialization functionality

### Changed

- Major refactoring of `src/twat/__init__.py`
- Updated path handling in `src/twat/paths.py`
- Refined package dependencies

## [1.7.0] - 2025-02-13

### Added

- New gitignore patterns for better repository management

### Changed

- Improved project structure
- Enhanced development workflow

## [1.6.2] - 2025-02-06

### Added

- Extended package configuration in `pyproject.toml`
- Added new package metadata
- Improved dependency specifications

## [1.6.1] - 2025-02-06

### Changed

- Refactored initialization code in `src/__init__.py` and `src/twat/__init__.py`
- Improved path handling in `src/twat/paths.py`
- Enhanced test coverage

### Fixed

- Path resolution issues
- Package initialization bugs

## [1.6.0] - 2025-02-06

### Added

- Initial GitHub repository setup
- Comprehensive plugin system implementation
- Modern Python packaging with PEP 621 compliance

### Changed

- Updated project documentation
- Refined package structure

## [1.1.0] - 2025-02-03

### Added

- Implemented core package functionality in `src/__init__.py`
- Added initial test suite in `tests/test_package.py`
- Enhanced project configuration in `pyproject.toml`

### Changed

- Improved gitignore configuration
- Updated README with more detailed information

## [1.0.1] - 2025-02-03

### Fixed

- Package configuration issues in `pyproject.toml`
- Repository ignore patterns

## [1.0.0] - 2025-02-03

### Added

- Initial release
- Basic package structure
- Core module implementation
- Package initialization
- Version management system

[1.7.6]: https://github.com/twardoch/twat/compare/v1.7.5...v1.7.6
[1.7.5]: https://github.com/twardoch/twat/compare/v1.7.0...v1.7.5
[1.7.0]: https://github.com/twardoch/twat/compare/v1.6.2...v1.7.0
[1.6.2]: https://github.com/twardoch/twat/compare/v1.6.1...v1.6.2
[1.6.1]: https://github.com/twardoch/twat/compare/v1.6.0...v1.6.1
[1.6.0]: https://github.com/twardoch/twat/compare/v1.1.0...v1.6.0
[1.1.0]: https://github.com/twardoch/twat/compare/v1.0.1...v1.1.0
[1.0.1]: https://github.com/twardoch/twat/compare/v1.0.0...v1.0.1
[1.0.0]: https://github.com/twardoch/twat/releases/tag/v1.0.0

================
File: pyproject.toml
================
[build-system]
requires = ["hatchling>=1.21.0", "hatch-vcs>=0.3.0"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/twat"]
include = ["src/twat/**/*.py"]

[project]
name = "twat"
dynamic = ["version"]
description = ""
readme = "README.md"
requires-python = ">=3.10"
license = "MIT"
keywords = []
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

dependencies = [
    "importlib-metadata>=8.6.1",
    "importlib-resources>=6.5.2",
    "typing-extensions>=4.0.0; python_version < '3.10'",
    "pydantic>=2.10.6",
    "tomli>=2.2.1",
    "loguru>=0.7.3",
    "fire>=0.7.0",
    "python-dotenv>=1.0.1",
    "rich>=13.9.4",

]

[project.scripts]
twat = "twat:main"

[[project.authors]]
name = "Adam Twardoch"
email = "adam+github@twardoch.com"


[project.urls]
Documentation = "https://github.com/twardoch/twat#readme"
Issues = "https://github.com/twardoch/twat/issues"
Source = "https://github.com/twardoch/twat"


[tool.hatch.version]
source = "vcs"

[tool.hatch.version.raw-options]
version_scheme = "post-release"

[tool.hatch.build.hooks.vcs]
version-file = "src/twat/__version__.py"


[tool.hatch.envs.default]
dependencies = [
    "pytest",
    "pytest-cov",
    "mypy>=1.15.0",
    "ruff>=0.9.7",
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "ruff>=0.9.7",
    "mypy>=1.15.0",
]


[tool.hatch.envs.default.scripts]
test = "pytest {args:tests}"
test-cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat --cov=tests {args:tests}"
type-check = "mypy src/twat tests"
lint = ["ruff check src/twat tests", "ruff format src/twat tests"]


[[tool.hatch.envs.all.matrix]]
python = ["3.10", "3.11", "3.12"]


[tool.hatch.envs.lint]
detached = true
dependencies = ["mypy>=1.0.0", "ruff>=0.1.0"]


[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/twat tests}"
style = ["ruff check {args:.}", "ruff format {args:.}"]
fmt = ["ruff format {args:.}", "ruff check --fix {args:.}"]
all = ["style", "typing"]


[tool.ruff]
target-version = 'py310'
line-length = 120

[tool.ruff.lint]
select = [
    'A',
    'ARG',
    'B',
    'C',
    'DTZ',
    'E',
    'EM',
    'F',
    'FBT',
    'I',
    'ICN',
    'ISC',
    'N',
    'PLC',
    'PLE',
    'PLR',
    'PLW',
    'Q',
    'RUF',
    'S',
    'T',
    'TID',
    'UP',
    'W',
    'YTT',
]
ignore = [
    'ARG001',
    'ARG002',
    'ARG004',
    'ARG005',
    'B904',
    'C901',
    'DTZ005',
    'E501',
    'F401',
    'F811',
    'FBT001',
    'FBT002',
    'FBT003',
    'I001',
    'ISC001',
    'N803',
    'PLR0911',
    'PLR0912',
    'PLR0913',
    'PLR0915',
    'PLR2004',
    'S311',
    'S603',
    'S607',
    'T201',
]

[tool.ruff.lint.per-file-ignores]
"tests/*" = ['S101']


[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true


[tool.coverage.run]
source_pkgs = ["twat", "tests"]
branch = true
parallel = true
omit = ["src/twat/__about__.py"]


[tool.coverage.paths]
twat = ["src/twat", "*/twat/src/twat"]
tests = ["tests", "*/twat/tests"]


[tool.coverage.report]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

[project.optional-dependencies]
test = [
    "pytest>=8.3.4",
    "pytest-cov>=6.0.0",
    "pytest-xdist>=3.6.1",                # For parallel test execution
    "pytest-benchmark[histogram]>=5.1.0", # For performance testing
]

dev = ["pre-commit>=4.0.0", "ruff>=0.1.0", "mypy>=1.0.0"]

audio = ["twat-audio[all]>=0.0.1"]
cache = ["twat-cache[all]>=1.7.5"]
coding = ["twat-coding[all]>=0.0.1"]
ez = ["twat-ez[all]>=1.7.5"]
fs = ["twat-fs[all]>=1.7.5"]
genai = ["twat-genai[all]>=1.7.5"]
hatch = ["twat-hatch[all]>=1.7.5"]
image = ["twat-image[all]>=1.7.5"]
labs = ["twat-labs[all]>=1.7.5"]
llm = ["twat-llm[all]>=0.0.1"]
mp = ["twat-mp[all]>=1.7.5"]
search = ["twat-search[all]>=0.0.1"]
speech = ["twat-speech[all]>=0.0.1"]
task = ["twat-task[all]>=1.7.5"]
text = ["twat-text[all]>=0.0.1"]
video = ["twat-video[all]>=0.0.1"]

all = [
    "twat-audio[all]>=2.1.3",
    "twat-cache[all]>=1.8.1",
    "twat-coding[all]>=0.0.1",
    "twat-ez[all]>=2.1.3",
    "twat-fs[all]>=1.8.1",
    "twat-genai[all]>=2.1.3",
    "twat-hatch[all]>=2.1.3",
    "twat-image[all]>=2.1.3",
    "twat-labs[all]>=2.1.3",
    "twat-llm[all]>=2.1.3",
    "twat-mp[all]>=2.1.3",
    "twat-search[all]>=2.1.3",
    "twat-speech[all]>=2.1.3",
    "twat-task[all]>=2.1.3",
    "twat-text[all]>=2.1.3",
    "twat-video[all]>=2.1.3",
    "importlib-metadata>=8.6.1",
    "importlib-resources>=6.5.2",
    "typing-extensions>=4.0.0; python_version < '3.10'",
    "pydantic>=2.10.6",
    "tomli>=2.2.1",
    "loguru>=0.7.3",
    "fire>=0.7.0",
    "python-dotenv>=1.0.1",
    "rich>=13.9.4",
]

[tool.hatch.envs.test]
dependencies = [".[test]"]

[tool.hatch.envs.test.scripts]
test = "python -m pytest -n auto {args:tests}"
test-cov = "python -m pytest -n auto --cov-report=term-missing --cov-config=pyproject.toml --cov=src/twat --cov=tests {args:tests}"
bench = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only"
bench-save = "python -m pytest -v -p no:briefcase tests/test_benchmark.py --benchmark-only --benchmark-json=benchmark/results.json"

[tool.pytest.ini_options]
markers = ["benchmark: marks tests as benchmarks (select with '-m benchmark')"]
addopts = "-v -p no:briefcase"
testpaths = ["tests"]
python_files = ["test_*.py"]
filterwarnings = ["ignore::DeprecationWarning", "ignore::UserWarning"]
asyncio_mode = "auto"

[tool.pytest-benchmark]
min_rounds = 100
min_time = 0.1
histogram = true
storage = "file"
save-data = true
compare = [
    "min",    # Minimum time
    "max",    # Maximum time
    "mean",   # Mean time
    "stddev", # Standard deviation
    "median", # Median time
    "iqr",    # Inter-quartile range
    "ops",    # Operations per second
    "rounds", # Number of rounds
]

================
File: README.md
================
# twat

A plugin-based Python package system that provides a unified interface for various utilities.

## Features

- Plugin system for Python packages with standardized interfaces
- Dynamic plugin loading and discovery
- Modern Python packaging with PEP 621 compliance
- Type hints and runtime type checking
- Comprehensive test suite and documentation
- CI/CD ready configuration

## Installation

```bash
pip install twat
```

## Usage

### As a Library

```python
import twat

# Load a plugin
fs = twat.fs  # Loads the twat-fs plugin

# List available plugins
from importlib.metadata import entry_points
plugins = entry_points(group="twat.plugins")
```

### Command Line

```bash
# Run a plugin through the twat command
twat fs --help

# Or run the plugin directly if it provides a CLI
twat-fs --help
```

## Plugin Development Guide

To create a new plugin for the `twat` system, follow these guidelines. We'll use the `twat-fs` package as an example.

### 1. Package Structure

```
twat-yourplugin/
├── src/
│   └── twat_yourplugin/
│       ├── __init__.py      # Main package interface
│       ├── __main__.py      # CLI entry point
│       └── core.py          # Core functionality
├── tests/
├── pyproject.toml           # Package configuration
├── README.md
└── LICENSE
```

### 2. Package Configuration

Your `pyproject.toml` should include:

```toml
[project]
name = "twat-yourplugin"  # Use hyphen in package name
dependencies = [
    "twat",               # Always include the main package
    # Your other dependencies
]

[project.scripts]
twat-yourplugin = "twat_yourplugin.__main__:main"  # Optional CLI

[project.entry-points."twat.plugins"]
yourplugin = "twat_yourplugin"  # Register as a twat plugin
```

### 3. Package Interface

Your `__init__.py` should expose a clean public API:

```python
# this_file: src/twat_yourplugin/__init__.py
"""Package description."""

from importlib import metadata

__version__ = metadata.version(__name__)

# Export your public API
from .core import main_function, OtherClass

__all__ = ["main_function", "OtherClass"]
```

### 4. CLI Support (Optional)

If your plugin provides a command-line interface:

```python
# this_file: src/twat_yourplugin/__main__.py
"""Command-line interface."""

import sys
from typing import NoReturn

def main() -> NoReturn:
    """CLI entry point."""
    # Your CLI implementation
    sys.exit(0)

if __name__ == "__main__":
    main()
```

### 5. Example Implementation

Here's a minimal example based on `twat-fs`:

```python
# src/twat_yourplugin/__init__.py
"""Your plugin description."""

from importlib import metadata
from .core import upload_file, ProviderType

__version__ = metadata.version(__name__)
__all__ = ["upload_file", "ProviderType"]

# src/twat_yourplugin/__main__.py
"""CLI interface."""

import sys
from typing import NoReturn
import fire
from loguru import logger

from .core import upload_file

def main() -> NoReturn:
    """CLI entry point."""
    try:
        fire.Fire({
            "upload": upload_file,
            # other commands...
        })
        sys.exit(0)
    except Exception as e:
        logger.error(f"Error: {e}")
        sys.exit(1)

if __name__ == "__main__":
    main()
```

### 6. Best Practices

1. **Naming**:
   - Package name: `twat-yourplugin` (with hyphen)
   - Import name: `twat_yourplugin` (with underscore)
   - Plugin entry point: `yourplugin` (no prefix)

2. **Dependencies**:
   - Always include `twat` as a dependency
   - Use optional dependencies for provider-specific features
   - Include development tools in `dev` extra

3. **Documentation**:
   - Include docstrings for all public APIs
   - Provide a comprehensive README
   - Include usage examples

4. **Testing**:
   - Write unit tests for core functionality
   - Include integration tests if applicable
   - Test both Python API and CLI interface

5. **Type Hints**:
   - Use type hints throughout your code
   - Support Python 3.10+
   - Follow PEP 484 and PEP 585

### 7. Development Workflow

```bash
# Create development environment
pip install hatch
hatch shell

# Install in development mode
pip install -e '.[dev,test]'

# Run tests
hatch run test
hatch run test-cov

# Run linting
hatch run lint

# Format code
hatch run format
```

## License

MIT License  
.

================
File: TODO-ALL.md
================
# TODO for all `twat` repos

# `twat_fs`, `twat_cache` and `twat_hatch` have had the most work done. Do not make major changes to them. Instead, distill a "best" configuration for them, and apply it to the other packages.

## 1. Critical Issues

### 1.1. Package Import and Version Issues

* `twat_speech`: Fix missing `__version__` attribute
* `twat_text`: Fix missing `__version__` attribute
* `twat_llm`: Fix missing `__version__` attribute

### 1.2. Missing Dependencies

* `twat_genai`: Install missing `fal` package
* `twat_cache`: Install missing `pydantic` package
* `twat_image`: Install missing `numpy`,  `webcolors`,  `PIL` packages
* `twat_llm`: Install missing `cv2`,  `llm`,  `pathos`,  `PIL`,  `tenacity` packages

### 1.3. Type Annotation Issues

* All packages: Add return type annotations to test functions
* `twat_cache`: Fix multiple type annotation issues in engines
* `twat_ez`: Fix type annotations in `py_needs.py`
* `twat_genai`: Fix type annotations in async functions
* `twat_hatch`: Fix type annotations in CLI functions

## 2. High Priority Improvements

### 2.1. Code Quality

* `twat_cache`: 
  + Fix 39 linting errors
  + Address 344 type checking errors
  + Fix failing tests
* `twat_genai`:
  + Fix 43 linting errors
  + Address 29 type checking errors
  + Fix dependency issues
* `twat_hatch`:
  + Fix 27 linting errors
  + Address 5 type checking errors

### 2.2. Test Coverage

* `twat_mp`: Excellent test coverage, use as reference
* Most packages: Only have basic version test
* Add comprehensive test suites for:
  + `twat_cache`
  + `twat_genai`
  + `twat_fs`
  + `twat_image`
  + `twat_llm`

### 2.3. Documentation

* All packages: Add proper docstrings
* All packages: Create README.md with:
  + Installation instructions
  + Basic usage examples
  + API documentation
  + Development setup guide

## 3. Medium Priority Tasks

### 3.1. Code Structure

* `twat_task`: Fix property decorators

### 3.2. Security

* `twat_task`: Fix random number generation usage


## 4. Low Priority Enhancements

### 4.1. Developer Experience

* All packages: Add pre-commit hooks
* All packages: Standardize development tools

### 4.2. CI/CD

* All packages: Add GitHub Actions for:
  + Testing
  + Linting
  + Type checking
  + Documentation building
  + Package publishing

### 4.3. Monitoring & Logging

* All packages: Add structured logging
* All packages: Add error reporting

## 5. Package-Specific Notes

### 5.1. twat_cache

* Highest priority for fixes
* Major type system overhaul needed
* Test suite needs complete rewrite

### 5.2. twat_genai

* Second highest priority
* Focus on async code improvements
* Fix dependency management

### 5.3. twat_mp

* Use as reference implementation
* Well-tested with benchmarks
* Document performance patterns

### 5.4. twat_fs

* Core functionality package
* Needs better error handling
* Add upload progress tracking

### 5.5. twat_hatch

* Focus on CLI improvements
* Better subprocess handling
* Add project templates

### 5.6. twat_image

* Fix dependency chain
* Add image processing tests
* Improve error messages

### 5.7. twat_llm

* Complete dependency audit
* Add model validation
* Improve async handling

## 6. Action Plan

1. Week 1:
   - Fix all critical version issues
   - Resolve dependency problems
   - Add missing type annotations

2. Week 2:
   - Address major linting issues
   - Fix type checking errors
   - Get all tests passing

3. Week 3:
   - Improve test coverage
   - Add documentation
   - Fix security issues

4. Week 4:
   - Implement CI/CD
   - Add development tools
   - Performance optimizations

## 7. Metrics for Success

1. All packages should have:
   - Zero linting errors
   - Zero type checking errors
   - 80%+ test coverage
   - Complete documentation
   - Working CI/CD pipeline

2. Performance targets:
   - Cache hit time < 1ms
   - Image processing < 100ms
   - API response time < 200ms

3. Code quality metrics:
   - Cyclomatic complexity < 10
   - Method length < 50 lines
   - Class length < 300 lines

================
File: VERSION.txt
================
v2.6.1



================================================================
End of Codebase
================================================================
