Metadata-Version: 2.3
Name: zensvi
Version: 1.2.2
Summary: This package handles downloading, cleaning, analyzing street view imagery in a one-stop and zen manner.
License: MIT
Author: koito19960406
Requires-Python: >=3.9,<3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: Pillow (>=10.0.0)
Requires-Dist: Pygments (>=2.9.0,<3.0.0)
Requires-Dist: Shapely (>=1.8.1)
Requires-Dist: addict (>=2.4.0,<3.0.0)
Requires-Dist: aiohttp (>=3.9.5,<4.0.0)
Requires-Dist: astral (>=3.2,<4.0)
Requires-Dist: attrs (>=21.2.0,<22.0.0)
Requires-Dist: click (>=8.0.1,<9.0.0)
Requires-Dist: click-plugins (>=1.1.1,<2.0.0)
Requires-Dist: cligj (>=0.7.2,<0.8.0)
Requires-Dist: contextily (>=1.6.0,<2.0.0)
Requires-Dist: docutils (>=0.16)
Requires-Dist: faiss-cpu (>=1.9.0,<2.0.0)
Requires-Dist: geojson (>=2.5.0,<3.0.0)
Requires-Dist: geopandas (>=0.10.0)
Requires-Dist: groundingdino-py (>=0.4.0,<0.5.0)
Requires-Dist: h3 (>=3.7.7,<4.0.0)
Requires-Dist: haversine (>=2.3.1,<3.0.0)
Requires-Dist: img2vec-pytorch (>=1.0.1,<2.0.0)
Requires-Dist: lightning (>=2.2.3,<3.0.0)
Requires-Dist: mapbox-vector-tile (>=1.2.1,<2.0.0)
Requires-Dist: mercantile (>=1.2.1,<2.0.0)
Requires-Dist: munch (>=2.5.0,<3.0.0)
Requires-Dist: networkx (>=3.1,<4.0)
Requires-Dist: numpy (>=1.21.0,<2.0.0)
Requires-Dist: open3d (>=0.18.0,<0.19.0)
Requires-Dist: opencv-python (>=4.5.3)
Requires-Dist: osmnx (>=1.1.1,<2.0.0)
Requires-Dist: pandas (>=1.3.3,<2.0.0)
Requires-Dist: polars-lts-cpu (>=0.20.23,<0.21.0)
Requires-Dist: protobuf (>=3.17.3,<4.0.0)
Requires-Dist: psutil (>=5.8.0,<6.0.0)
Requires-Dist: pyarrow (>=16.0.0,<17.0.0)
Requires-Dist: python-dotenv (>=0.19.0,<0.20.0)
Requires-Dist: requests (>=2.26.0,<3.0.0)
Requires-Dist: requests-toolbelt (>=0.9.1)
Requires-Dist: scikit-learn (>=1.5.2,<2.0.0)
Requires-Dist: scipy (>=1.7,<2.0)
Requires-Dist: seaborn (>=0.13.2,<0.14.0)
Requires-Dist: setuptools (>=64.0.3)
Requires-Dist: streetlevel (>=0,<1)
Requires-Dist: tenacity (>=9.0.0,<10.0.0)
Requires-Dist: timezonefinder (>=6.5.0,<7.0.0)
Requires-Dist: torch (>=2.3,<3.0)
Requires-Dist: torchvision
Requires-Dist: tqdm (>=4.61.1,<5.0.0)
Requires-Dist: transformers (>=4.10.2,<5.0.0)
Requires-Dist: turfpy (>=0.0.7,<0.0.8)
Requires-Dist: vt2geojson (>=0.2.1,<0.3.0)
Description-Content-Type: text/markdown

[![PyPi version](https://img.shields.io/pypi/v/zensvi.svg)](https://pypi.org/project/zensvi/)
[![Python versions](https://img.shields.io/pypi/pyversions/zensvi.svg)](https://pypi.org/project/zensvi/)
[![License](https://img.shields.io/pypi/l/zensvi.svg)](https://pypi.org/project/zensvi/)
[![Downloads](https://pepy.tech/badge/zensvi)](https://pepy.tech/project/zensvi)
[![Downloads](https://pepy.tech/badge/zensvi/month)](https://pepy.tech/project/zensvi)
[![Downloads](https://pepy.tech/badge/zensvi/week)](https://pepy.tech/project/zensvi)
[![Documentation Status](https://readthedocs.org/projects/zensvi/badge/?version=latest)](https://zensvi.readthedocs.io/en/latest/?badge=latest)
[![codecov](https://codecov.io/gh/koito19960406/ZenSVI/graph/badge.svg?token=HAIMJIT9HQ)](https://codecov.io/gh/koito19960406/ZenSVI)

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/koito19960406/ZenSVI/main/docs/_static/logo_zensvi_white2.png">
    <img src="https://raw.githubusercontent.com/koito19960406/ZenSVI/main/docs/_static/logo_zensvi_fixed%202.png" alt="ZenSVI logo" width="400">
  </picture>
</p>

# ZenSVI

**Primary Author:** [Koichi Ito](https://koichiito.com/) (National University of Singapore)

Besides this documentation, we have published a [comprehensive paper](https://arxiv.org/abs/2412.18641) with detailed information and demonstration use cases. The paper provides in-depth insights into the package's architecture, features, and real-world applications.

ZenSVI is a comprehensive Python package for downloading, cleaning, and analyzing street view imagery. For more information about the package or to discuss potential collaborations, please visit my website at [koichiito.com](https://koichiito.com/). The source code is available on [GitHub](https://github.com/koito19960406/ZenSVI).

This package is a one-stop solution for downloading, cleaning, and analyzing street view imagery, with comprehensive API documentation available at [zensvi.readthedocs.io](https://zensvi.readthedocs.io/en/latest/autoapi/index.html).

## Table of Contents

- [ZenSVI](#zensvi)
  - [Table of Contents](#table-of-contents)
  - [Installation of `zensvi`](#installation-of-zensvi)
  - [Installation of `pytorch` and `torchvision`](#installation-of-pytorch-and-torchvision)
  - [Usage](#usage)
    - [Downloading Street View Imagery](#downloading-street-view-imagery)
    - [Analyzing Metadata of Mapillary Images](#analyzing-metadata-of-mapillary-images)
    - [Running Segmentation](#running-segmentation)
    - [Running Places365](#running-places365)
    - [Running PlacePulse 2.0 Prediction](#running-placepulse-20-prediction)
    - [Running Global Streetscapes Prediction](#running-global-streetscapes-prediction)
    - [Running Grounding Object Detection](#running-grounding-object-detection)
    - [Running Depth Estimation](#running-depth-estimation)
    - [Running Embeddings](#running-embeddings)
    - [Running Low-Level Feature Extraction](#running-low-level-feature-extraction)
    - [Transforming Images](#transforming-images)
    - [Creating Point Clouds from Images](#creating-point-clouds-from-images)
    - [Visualizing Results](#visualizing-results)
  - [Contributing](#contributing)
  - [License](#license)
  - [Credits](#credits)

## Installation of `zensvi`

```bash
$ pip install zensvi
```

## Installation of `pytorch` and `torchvision`

Since `zensvi` uses `pytorch` and `torchvision`, you may need to install them separately. Please refer to the [official website](https://pytorch.org/get-started/locally/) for installation instructions.

## Usage
### Downloading Street View Imagery
***Mapillary***

For downloading images from Mapillary, utilize the MLYDownloader. Ensure you have a Mapillary client ID:

```python
from zensvi.download import MLYDownloader

mly_api_key = "YOUR_OWN_MLY_API_KEY"  # Please register your own Mapillary API key
downloader = MLYDownloader(mly_api_key=mly_api_key)
# with lat and lon:
downloader.download_svi("path/to/output_directory", lat=1.290270, lon=103.851959)
# with a csv file with lat and lon:
downloader.download_svi("path/to/output_directory", input_csv_file="path/to/csv_file.csv")
# with a shapefile:
downloader.download_svi("path/to/output_directory", input_shp_file="path/to/shapefile.shp")
# with a place name that works on OpenStreetMap:
downloader.download_svi("path/to/output_directory", input_place_name="Singapore")
```

***KartaView***

For downloading images from KartaView, utilize the KVDownloader:

```python
from zensvi.download import KVDownloader

downloader = KVDownloader()
# with lat and lon:
downloader.download_svi("path/to/output_directory", lat=1.290270, lon=103.851959)
# with a csv file with lat and lon:
downloader.download_svi("path/to/output_directory", input_csv_file="path/to/csv_file.csv")
# with a shapefile:
downloader.download_svi("path/to/output_directory", input_shp_file="path/to/shapefile.shp")
# with a place name that works on OpenStreetMap:
downloader.download_svi("path/to/output_directory", input_place_name="Singapore")
```

***Amsterdam***

For downloading images from Amsterdam, utilize the AMSDownloader:

```python
from zensvi.download import AMSDownloader

downloader = AMSDownloader()
# with lat and lon:
downloader.download_svi("path/to/output_directory", lat=4.899431, lon=52.379189)
# with a csv file with lat and lon:
downloader.download_svi("path/to/output_directory", input_csv_file="path/to/csv_file.csv")
# with a shapefile:
downloader.download_svi("path/to/output_directory", input_shp_file="path/to/shapefile.shp")
# with a place name that works on OpenStreetMap:
downloader.download_svi("path/to/output_directory", input_place_name="Amsterdam")
```

***Global Streetscapes***

For downloading the NUS Global Streetscapes dataset, utilize the GSDownloader:

```python
from zensvi.download import GSDownloader

downloader = GSDownloader()
# Download all data
downloader.download_all_data(local_dir="data/")
# Or download specific subsets
downloader.download_manual_labels(local_dir="manual_labels/")
downloader.download_train(local_dir="manual_labels/train/")
downloader.download_test(local_dir="manual_labels/test/")
downloader.download_img_tar(local_dir="manual_labels/img/")
```

### Analyzing Metadata of Mapillary Images
To analyze the metadata of Mapillary images, use the `MLYMetadata`:

```python
from zensvi.metadata import MLYMetadata

path_input = "path/to/input"
mly_metadata = MLYMetadata(path_input)
mly_metadata.compute_metadata(
    unit="image", # unit of the metadata. Other options are "street" and "grid"
    indicator_list="all", # list of indicators to compute. You can specify a list of indicators in space-separated format, e.g., "year month day" or "all" to compute all indicators
    path_output="path/to/output" # path to the output file
)
```

### Running Segmentation
To perform image segmentation, use the `Segmenter`:

```python
from zensvi.cv import Segmenter

segmenter = Segmenter(dataset="cityscapes", # or "mapillary"
                      task="semantic" # or "panoptic"
                      )
segmenter.segment("path/to/input_directory", 
                  dir_image_output = "path/to/image_output_directory",
                  dir_summary_output = "path/to/segmentation_summary_output"
                  )
```

### Running Places365
To perform scene classification, use the `ClassifierPlaces365`:

```python
from zensvi.cv import ClassifierPlaces365

# initialize the classifier
classifier = ClassifierPlaces365(
    device="cpu",  # device to use (either "cpu" or "gpu")
)

# set arguments
classifier = ClassifierPlaces365()
classifier.classify(
    "path/to/input_directory",
    dir_image_output="path/to/image_output_directory",
    dir_summary_output="path/to/classification_summary_output"
)
```

### Running PlacePulse 2.0 Prediction
To predict the PlacePulse 2.0 score, use the `ClassifierPerception`:

```python
from zensvi.cv import ClassifierPerception

classifier = ClassifierPerception(
    perception_study="safer", # Other options are "livelier", "wealthier", "more beautiful", "more boring", "more depressing"
)
dir_input = "path/to/input"
dir_summary_output = "path/to/summary_output"
classifier.classify(
    dir_input,
    dir_summary_output=dir_summary_output
)
```

You can also use the ViT version for perception classification:

```python
from zensvi.cv import ClassifierPerceptionViT

classifier = ClassifierPerceptionViT(
    perception_study="safer", # Other options are "livelier", "wealthier", "more beautiful", "more boring", "more depressing"
)
dir_input = "path/to/input"
dir_summary_output = "path/to/summary_output"
classifier.classify(
    dir_input,
    dir_summary_output=dir_summary_output
)
```

### Running Global Streetscapes Prediction
To predict the Global Streetscapes indicators, use:
- `ClassifierGlare`: Whether the image contains glare
- `ClassifierLighting`: The lighting condition of the image
- `ClassifierPanorama`: Whether the image is a panorama
- `ClassifierPlatform`: Platform of the image
- `ClassifierQuality`: Quality of the image
- `ClassifierReflection`: Whether the image contains reflection
- `ClassifierViewDirection`: View direction of the image
- `ClassifierWeather`: Weather condition of the image

```python
from zensvi.cv import ClassifierGlare

classifier = ClassifierGlare()
dir_input = "path/to/input"
dir_summary_output = "path/to/summary_output"
classifier.classify(
    dir_input,
    dir_summary_output=dir_summary_output,
)
```


### Running Grounding Object Detection
To run grounding object detection on the images, use the `ObjectDetector`:

```python
from zensvi.cv import ObjectDetector

detector = ObjectDetector(
    text_prompt="tree",  # specify the object(s) (e.g., single type: "building", multi-type: "car . tree")
    box_threshold=0.35,  # confidence threshold for box detection
    text_threshold=0.25  # confidence threshold for text
)

detector.detect_objects(
    dir_input="path/to/image_input_directory",
    dir_output="path/to/image_output_directory",
    dir_summary_output="path/to/detection_summary_output",
    save_format="json" # or "csv"
)
```

### Running Depth Estimation
To estimate the depth of the images, use the `DepthEstimator`:

```python
from zensvi.cv import DepthEstimator

depth_estimator = DepthEstimator(
    task="relative" # task to perform (either "relative" or "absolute")
)

dir_input = "path/to/input"
dir_image_output = "path/to/image_output" # estimated depth map
depth_estimator.estimate_depth(
    dir_input,
    dir_image_output
)
```

### Running Embeddings
To generate embeddings and search for similar images, use the `Embeddings`:
```python
from zensvi.cv import Embeddings

emb = Embeddings(model_name="resnet-1", cuda=True)
emb.generate_embedding(
    "path/to/image_directory",
    "path/to/output_directory",
    batch_size=1000,
)
results = emb.search_similar_images("path/to/target_image_file", "path/to/embeddings_directory", 20)
```

### Running Low-Level Feature Extraction
To extract low-level features, use the `get_low_level_features`:

```python
from zensvi.cv import get_low_level_features

get_low_level_features(
    "path/to/input_directory",
    dir_image_output="path/to/image_output_directory",
    dir_summary_output="path/to/low_level_feature_summary_output"
)
```

### Transforming Images
Transform images from panoramic to perspective or fisheye views using the `ImageTransformer`:

```python
from zensvi.transform import ImageTransformer

dir_input = "path/to/input"
dir_output = "path/to/output"
image_transformer = ImageTransformer(
    dir_input="path/to/input", 
    dir_output="path/to/output"
)
image_transformer.transform_images(
    style_list="perspective equidistant_fisheye orthographic_fisheye stereographic_fisheye equisolid_fisheye",  # list of projection styles in the form of a string separated by a space
    FOV=90,  # field of view
    theta=120,  # angle of view (horizontal)
    phi=0,  # angle of view (vertical)
    aspects=(9, 16),  # aspect ratio
    show_size=100,  # size of the image to show (i.e. scale factor)
    use_upper_half=True,  # use the upper half of the image for sky view factor calculation
)
```

### Creating Point Clouds from Images
To create a point cloud from images with depth information, use the `PointCloudProcessor`:

```python
from zensvi.transform import PointCloudProcessor
import pandas as pd

processor = PointCloudProcessor(
    image_folder="path/to/image_directory",
    depth_folder="path/to/depth_maps_directory",
    output_coordinate_scale=45,  # scaling factor for output coordinates
    depth_max=255  # maximum depth value for normalization
)

# Create a DataFrame with image information
# The DataFrame should have columns similar to this structure:
data = pd.DataFrame({
    "id": ["Y2y7An1aRCeA5Y4nW7ITrg", "VSsVjWlr4orKerabFRy-dQ"],  # image identifiers
    "heading": [3.627108491916069, 5.209303414492613],           # heading in radians
    "lat": [40.77363963371641, 40.7757528007],                   # latitude
    "lon": [-73.95482278589579, -73.95668603003708],             # longitude
    "x_proj": [4979010.676803163, 4979321.30902424],             # projected x coordinate
    "y_proj": [-8232613.214232705, -8232820.629621736]           # projected y coordinate
})

# Process images and save point clouds
processor.process_multiple_images(
    data=data,
    output_dir="path/to/output_directory",
    save_format="ply"  # output format, can be "pcd", "ply", "npz", or "csv"
)
```

### Visualizing Results
To visualize the results, use the `plot_map`, `plot_image`, `plot_hist`, and `plot_kde` functions:

```python
from zensvi.visualization import plot_map, plot_image, plot_hist, plot_kde

# Plotting a map
plot_map(
    path_pid="path/to/pid_file.csv",  # path to the file containing latitudes and longitudes
    variable_name="vegetation", 
    plot_type="point"  # this can be either "point", "line", or "hexagon"
)

# Plotting images in a grid
plot_image(
    dir_image_input="path/to/image_directory", 
    n_row=4,  # number of rows
    n_col=5   # number of columns
)

# Plotting a histogram
plot_hist(
    dir_input="path/to/data.csv",
    columns=["vegetation"],  # list of column names to plot histograms for
    title="Vegetation Distribution by Neighborhood"
)

# Plotting a kernel density estimate
plot_kde(
    dir_input="path/to/data.csv",
    columns=["vegetation"],  # list of column names to plot KDEs for
    title="Vegetation Density by Neighborhood"
)
```

## Contributing

Interested in contributing? Check out the contributing guidelines. Please note that this project is released with a Code of Conduct. By contributing to this project, you agree to abide by its terms.

## License

`zensvi` was created by Koichi Ito. It is licensed under the terms of the MIT License.

Please cite the following paper if you use `zensvi` in a scientific publication:

```bibtex
@article{ito2024zensvi,
    title={ZenSVI: An Open-Source Software for the Integrated Acquisition, Processing and Analysis of Street View Imagery Towards Scalable Urban Science},
    author={Ito, Koichi and Zhu, Yihan and Abdelrahman, Mahmoud and Liang, Xiucheng and Fan, Zicheng and Hou, Yujun and Zhao, Tianhong and Ma, Rui and Fujiwara, Kunihiko and Ouyang, Jiani and Quintana, Matias and Biljecki, Filip},
    journal={arXiv preprint arXiv:2412.18641},
    year={2024}
}
```

## Credits
- Logo design by [Kunihiko Fujiwara](https://ual.sg/author/kunihiko-fujiwara/)
- All the packages used in this package: [requirements.txt](requirements.txt)
--------------------------------------------------------------------------------
<br>
<br>
<p align="center">
  <a href="https://ual.sg/">
    <img src="https://raw.githubusercontent.com/koito19960406/ZenSVI/main/docs/_static/ualsg.jpeg" width = 55% alt="Logo">
  </a>
</p>

