Metadata-Version: 2.1
Name: nxbench
Version: 0.1.7
Summary: A centralized benchmarking suite to facilitate comparative profiling of tools across graph analytic libraries and datasets
Author-email: Derek Alexander <dpysalexander@gmail.com>
Keywords: networkx,graph,network,benchmarking,profiling
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26
Requires-Dist: scipy>=1.14.1
Requires-Dist: networkx>=3.3
Requires-Dist: psutil>=5.9
Requires-Dist: pandas>=2.2.2
Requires-Dist: pydantic>=2.8
Requires-Dist: setuptools>=75.1
Requires-Dist: nest-asyncio>=1.6
Requires-Dist: asv>=0.6.4
Requires-Dist: dash>=2.18.1
Requires-Dist: dash-bootstrap-components>=1.6.0
Requires-Dist: aiohttp>=3.9.5
Requires-Dist: aiofiles>=24.1.0
Requires-Dist: chardet>=4.0.0
Requires-Dist: beautifulsoup4>=4.12.3
Requires-Dist: multidict>=6.0.5
Requires-Dist: yarl>=1.9.4
Provides-Extra: test
Requires-Dist: pre-commit>=3.8.0; extra == "test"
Requires-Dist: pre-commit-hooks>=5.0.0; extra == "test"
Requires-Dist: black>=22.6.0; extra == "test"
Requires-Dist: ruff>=0.6.9; extra == "test"
Requires-Dist: pytest-sugar>=0.9.3; extra == "test"
Requires-Dist: pytest>=7.2; extra == "test"
Requires-Dist: pytest-cov>=2.7.1; extra == "test"
Requires-Dist: pytest-split>=0.7.0; extra == "test"
Requires-Dist: pytest-xdist>=1.32.0; extra == "test"
Requires-Dist: pytest-metadata>=2.0.2; extra == "test"
Requires-Dist: pytest-env>=1.1.5; extra == "test"
Requires-Dist: pytest-asyncio>=0.24.0; extra == "test"
Requires-Dist: coverage>=7.6.1; extra == "test"
Requires-Dist: build>=1.2.2; extra == "test"
Requires-Dist: isort>=5.13.2; extra == "test"
Provides-Extra: doc
Requires-Dist: setuptools>=45.0.0; extra == "doc"
Requires-Dist: pydata-sphinx-theme==0.15.4; extra == "doc"
Requires-Dist: furo==2024.8.6; extra == "doc"
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == "doc"
Requires-Dist: sphinx-autobuild==2024.10.3; extra == "doc"
Requires-Dist: sphinxcontrib-napoleon; extra == "doc"
Requires-Dist: myst_parser>=4.0.0; extra == "doc"
Requires-Dist: sphinx_autodoc_typehints>=2.4.4; extra == "doc"
Requires-Dist: sphinx<9.0.0,>=8.0.2; extra == "doc"

[![Python](https://img.shields.io/pypi/pyversions/nxbench.svg)](https://badge.fury.io/py/nxbench)
[![PyPI](https://badge.fury.io/py/nxbench.svg)](https://badge.fury.io/py/nxbench)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

# NxBench

<p align="center">
  <img src="doc/_static/assets/nxbench_logo.png" alt="NxBench Logo" width="150"/>
</p>

**nxbench** is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built on top of [Airspeed Velocity (ASV)](https://github.com/airspeed-velocity/asv), nxbench places an emphasis on extensible and granular performance analysis, enabling developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.

## Key Features

- **Cross-Backend Benchmarking**: Leverage NetworkX's backend system to profile algorithms across multiple implementations (NetworkX, nx-parallel, GraphBLAS, and CuGraph)
- **Configurable Suite**: YAML-based configuration for algorithms, datasets, and benchmarking parameters
- **Real-World Datasets**: Automated downloading and caching of networks and their metadata from NetworkRepository
- **Synthetic Graph Generation**: Support for generating benchmark graphs using any of NetworkX's built-in generators
- **Validation Framework**: Comprehensive result validation for correctness across implementations
- **Performance Monitoring**: Track execution time and memory usage with detailed metrics
- **Interactive Visualization**: Dynamic dashboard for exploring benchmark results using Plotly Dash
- **Flexible Storage**: SQLite-based result storage with pandas integration for analysis
- **CI Integration**: Support for automated benchmarking through ASV (Airspeed Velocity)

## Installation

PyPi:

```bash
pip install nxbench
```

From a local clone:

```bash
make install
```

Docker:

```bash
# CPU-only
docker-compose -f docker/docker-compose.cpu.yaml build

# With GPU
docker-compose -f docker/docker-compose.gpu.yaml build
```

## Quick Start

1. Configure your benchmarks in a yaml file (see `configs/example.yaml`):

```yaml
algorithms:
  - name: "pagerank"
    func: "networkx.pagerank"
    params:
      alpha: 0.85
    groups: ["centrality"]

datasets:
  - name: "karate"
    source: "networkrepository"
```

2. Run benchmarks based on the configuration:

```bash
nxbench --config 'nxbench/configs/example.yaml' benchmark run
```

3. Export results:

```bash
nxbench --config 'nxbench/configs/example.yaml' benchmark export 'results/results.csv' --output-format csv  # convert benchmarked results into csv format.
```

4. View results:

```bash
nxbench viz serve  # visualize results using parallel categories dashboard
```

<p align="center">
  <img src="doc/_static/assets/animation.gif" alt="Parallel Categories Animation" width="1000"/>
</p>


## Advanced Command Line Interface

The CLI provides comprehensive management of benchmarks, datasets, and visualization:

```bash
# Data Management
nxbench data download karate  # download specific dataset
nxbench data list --category social  # list available datasets

# Benchmarking
nxbench --config 'nxbench/configs/example.yaml' -vvv benchmark run  # debug benchmark runs
nxbench --config 'nxbench/configs/example.yaml' benchmark export 'results/benchmarks.sqlite' --output-format sql # export the results into a sql database
```

## Configuration

Benchmarks are configured through YAML files with the following structure:

```yaml
algorithms:
  - name: "algorithm_name"
    func: "fully.qualified.function.name"
    params: {}
    requires_directed: false
    groups: ["category"]
    validate_result: "validation.function"

datasets:
  - name: "dataset_name"
    source: "networkrepository"
    params: {}
```

## Supported Backends

- NetworkX (default)
- nx-CuGraph (requires separate CuGraph installation and supported GPU hardware)
- GraphBLAS Algorithms (optional)
- nx-parallel (optional)

## Reproducible benchmarking through containerization

```bash
# Run benchmarks with GPU
NUM_GPU=1 docker-compose -f docker/docker-compose.gpu.yaml up nxbench

# Run benchmarks CPU-only
docker-compose -f docker/docker-compose.cpu.yaml up nxbench

# Start visualization dashboard
docker-compose -f docker/docker-compose.cpu.yaml up dashboard

# Run specific backend
docker-compose -f docker/docker-compose.cpu.yaml run --rm nxbench --config 'nxbench/configs/example.yaml' benchmark run --backend networkx

# View results
docker-compose -f docker/docker-compose.cpu.yaml run --rm nxbench --config 'nxbench/configs/example.yaml' benchmark export results.csv
```

## Contributing

Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on:

- Code style guidelines
- Development setup
- Testing requirements
- Pull request process

## License

This project is licensed under the MIT License - see [LICENSE](LICENSE) for details.

## Acknowledgments

- NetworkX community for the core graph library and dispatching support
- NetworkRepository.com for harmonized dataset access
- ASV team for benchmark infrastructure

## Contact

For questions or suggestions:

- Open an issue on GitHub
- Email: <dpysalexander@gmail.com>
