# Changelog

All notable changes to this project will be documented in this file.

## [v0.8.0] - 2026-02-10

### Added

-   **Google Earth Engine Handler (`GEEProfiler`)**
    -   **Complete GEE Integration**: New `gigaspatial/handlers/gee/` module with full Google Earth Engine support via `GEEProfiler` class.
    -   **Context & Impact: Multi-Sector GEE Intelligence**
        -   **Critical Data Access**: Built-in support for global infrastructure and environmental datasets (Nightlights, Population, Land Cover) provides instant, ready-to-use insights for emergency response, climate resilience, and urban development.  
        -   **Beyond Built-ins**: While the registry comes pre-configured for high-value datasets, `GEEProfiler` is fully flexible, unlocking the entire multi-petabyte Google Earth Engine public catalog (Landsat, Sentinel, MODIS, etc.) for custom analysis.
    -   **Intelligent Asset Handling**: Robust initialization automatically distinguishes `ee.Image` vs `ee.ImageCollection`, preventing backend loading errors during band validation.
    -   **Dataset Registry**: `GEEDatasetRegistry` and `GEEDatasetEntry` for managing built-in datasets (nightlights, population, etc.) with metadata (bands, scales, temporal coverage).
    -   **Inspection Utilities**: Comprehensive collection inspection including `display_collection_info()`, `get_band_names()`, `get_date_range()`, `display_band_names()`, `display_properties()`.
    -   **`map_to_points()`**: Extracts GEE raster values at point locations with optional circular buffers (metric CRS buffering, automatic UTM reprojection).
    -   **`map_to_zones()`**: Spatial aggregation of GEE rasters within polygon boundaries (admin zones, grids) using configurable reducers.
    -   **Automatic Chunking**: Handles GEE API limits with intelligent chunking (`chunk_size=1000` default), processing large feature sets without timeouts.
    -   **Unified Processing Pipeline**: Single `_reduce_regions_with_chunking()` core handles both points and zones with robust error recovery per chunk.
    -   **Temporal Filtering**: `start_date`/`end_date` filtering with automatic temporal reduction (`mean`, `median`, etc.) for ImageCollections.
    -   **NRT-Ready Logic**: Advanced date handling for Near Real-Time datasets, automatically defaulting to most recent data windows.
    -   **Config-Driven**: `GEEConfig` supports dataset lookup, authentication (service accounts), and defaults (band, reducer, scale, chunk_size).
    -   **Production-Ready**: Full validation (bands, dates, geometries), detailed logging, error handling, and config fallbacks.

-   **GEE Built-in Datasets**
    -   Pre-configured registry entries for popular datasets: `nightlights` (VIIRS), `population` (various sources), land cover, etc.
    -   Automatic collection loading via `dataset_id`: `profiler = GEEProfiler("nightlights")`.
    -   Standardized metadata: resolution, default bands, reducers, temporal cadence.

-   **DeltaSharingDataStore (formerly GigaDataAPI)**
    -   Refactored internal `GigaDataAPI` into public `DeltaSharingDataStore` as a reusable data-access layer for Delta Sharing tables.
    -   Replaced hardcoded global config with flexible initialization supporting global config (`API_PROFILE_FILE_PATH`, `API_SHARE_NAME`, `API_SCHEMA_NAME`) + selective per-instance overrides via keyword arguments.
    -   Added `DeltaSharingConfig` (Pydantic model) for validated configuration with profile file existence checks and `enable_cache` flag for in-memory table caching.
    -   Implemented lazy `SharingClient` initialization via `client` property and comprehensive cache management (`get_table_metadata()`, `clear_cache()`, `get_cached_tables()`, `cache_size_mb`).
    -   Maintained backward-compatible API: `list_tables()` → `get_country_list()`, `load_table()` → `load_country_data()`.

### Changed

-   **OvertureAmenityFetcher: Updated to latest Overture Places release and S3 layout**
    -   Updated default Overture release to `2026-01-21.0` for Places theme queries.
    -   Switched to the current S3 URL pattern for Places GeoParquet (`release/{release}/theme=places/...`) and now build the read path via a configurable `base_url` instead of hardcoding it in the SQL query.
    -   Fixed IO errors caused by outdated bucket patterns by reading directly from the official Overture S3 bucket with DuckDB’s `read_parquet()`.
    -   Enabled S3 access in DuckDB by installing and loading the `httpfs` extension and configuring the AWS region (`s3_region='us-west-2'`) during connection setup.

-   **DeltaSharingDataStore: Robust table discovery**
    -   Enhanced `list_tables()` to handle `SharingClient.list_all_tables()` returning empty lists (common in constrained shares) by falling back to explicit `list_shares()` → `list_schemas()` → `list_tables()` enumeration on configured share/schema.
    -   Added structured logging for debugging share/schema mismatches while preserving stable public API contract.

-   **DataStore: Cross-Platform Path Handling**
    -   **`ADLSDataStore` and `LocalDataStore`: Enhanced path type support**
        -   All path-accepting methods now support both `str` and `PathLike` objects (e.g., `pathlib.Path`) for improved type flexibility and cross-platform compatibility.
        -   Introduced `Pathish = Union[str, PathLike[str]]` type alias for consistent path parameter signatures across both data store implementations.
    
    -   **`ADLSDataStore`: Centralized blob key normalization**
        -   Added `_to_blob_key()` method as the single source of truth for converting input paths to Azure-compatible blob keys.
        -   Automatically handles Windows backslashes → forward slashes conversion via `PurePosixPath` for cross-platform compatibility.
        -   Strips leading slashes (Azure convention) and optionally ensures trailing slashes for directory operations.
        -   All methods now use `_to_blob_key()` internally, eliminating inconsistent path handling across different operations.
        -   `_normalize_path()` deprecated in favor of `_to_blob_key()` but kept for backward compatibility.
    
    -   **`LocalDataStore`: Improved path resolution**
        -   Enhanced `_resolve_path()` to accept `PathLike` objects alongside strings.
        -   Properly handles both absolute and relative paths with automatic resolution relative to `base_path`.
        -   All methods updated to use `Pathish` type signature for consistency with `ADLSDataStore`.
    
    -   **Benefits:**
        -   **Cross-platform safety**: Eliminates Windows vs. Linux path separator issues when handlers build paths using `pathlib` and pass to data stores.
        -   **Type flexibility**: Handlers can now pass `Path` objects, strings, or any `PathLike` without manual `str()` conversion.
        -   **Backward compatible**: Existing handler code with `str()` wrappers continues to work unchanged.
        -   **Maintainability**: Path normalization logic centralized in data store boundary, not scattered across handlers.


### Documentation

-   **OvertureAmenityFetcher: Amenity category documentation**
    -   Expanded the class docstring to explain that `amenity_types` should correspond to `categories.primary` values in the Overture Places schema.
    -   Linked to the authoritative Overture category list CSV on GitHub so users can discover all valid amenity categories without hardcoding them in giga-spatial:  
        `https://github.com/OvertureMaps/schema/blob/main/docs/schema/concepts/by-theme/places/overture_categories.csv`.

## [v0.7.6] - 2026-01-27

### Changed

-   **Maxar Imagery Handler: API Migration to GEGD Pro Platform**
    -   Updated `MaxarConfig` to support the new Maxar GEGD Pro API infrastructure:
        -   Replaced username/password/connection string authentication with API key-based authentication.
        -   Added support for OAuth bearer tokens as an alternative authentication method.
        -   Updated base URL from `evwhs.digitalglobe.com` to `pro.gegd.com/streaming/v1/ogc/wms`.
        -   Added `auth_method` parameter supporting three authentication modes: `api_key` (query parameter), `header` (custom header), and `bearer_token`.
        -   Updated layer names from `DigitalGlobe:Imagery`/`DigitalGlobe:ImageryFootprint` to `Maxar:Imagery`/`Maxar:FinishedFeature`.
        -   Replaced deprecated `featureProfile` parameter with optional `profile` parameter for stacking profiles.
        -   Renamed `coverage_cql_filter` to `cql_filter` for consistency with updated API specification.
        -   Added `styles` parameter supporting `raster` (imagery) and `footprints` (feature visualization) rendering modes.
        -   Updated to WMS version 1.3.0 as the default and recommended version.
    
    -   **Enhanced Initialization Flexibility:**
        -   `MaxarImageDownloader` now accepts configuration as `MaxarConfig` object, dictionary, or keyword arguments.
        -   Supports mixing configuration sources with kwargs taking precedence for convenient overrides.
        -   Added validation for API key or bearer token requirement via Pydantic field validators.
    
    -   **Updated Authentication Implementation:**
        -   Replaced basic authentication with header-based and query-parameter authentication methods.
        -   Implemented `_build_auth_headers()` for dynamic header construction based on auth method.
        -   Modified `_initialize_wms()` to handle API key injection and header configuration.
        -   Updated `_download_single_image()` to use `srs` parameter (OWSLib compatibility) while supporting WMS 1.3.0 `crs` in actual requests.

-   **Maxar Imagery Handler: WFS Metadata Integration**
    -   Added Web Feature Service (WFS) support for querying imagery metadata alongside image downloads:
        -   Implemented `_initialize_wfs()` for WFS 2.0.0 service initialization with authentication.
        -   Added `get_imagery_metadata()` method supporting bbox and CQL filter queries with configurable output formats.
        -   Implemented `get_metadata_for_bbox()` convenience method providing summary statistics (feature count, sensors, date ranges, cloud cover, resolution).
        -   Added `_download_single_image_with_metadata()` for atomic image download with associated feature metadata.
    
    -   **Metadata Features:**
        -   Retrieves comprehensive imagery attributes including: acquisition dates, sensor/source information, cloud cover percentage, ground sample distance, sun angles, off-nadir angle, product names, band descriptions, NIIRS quality ratings, and processing levels.
        -   Returns metadata as `GeoDataFrame` for seamless spatial analysis and filtering.
        -   Supports CQL filtering by date ranges, sensors, product types, and custom attribute queries.
        -   Includes BBOX-in-CQL support for combined spatial and attribute filtering (WFS limitation workaround).
    
    -   **Bulk Download Metadata Support:**
        -   Added `save_metadata` parameter to all bulk download methods (`download_images_by_tiles`, `download_images_by_bounds`, `download_images_by_coordinates`).
        -   When enabled, automatically saves JSON metadata files alongside downloaded images with matching filenames.
        -   Metadata JSON includes feature properties with proper datetime serialization (ISO 8601 format).
        -   Geometry column excluded from JSON output to minimize file size while preserving spatial reference in GeoDataFrame workflows.

-   **Maxar Imagery Handler: Date Filtering Convenience**
    -   Added `build_date_filter()` helper method for constructing CQL date range filters:
        -   Accepts string dates (`YYYY-MM-DD`), `datetime`, or `date` objects.
        -   Supports start-only, end-only, or date range filtering via `acquisitionDate` field.
        -   Uses `BETWEEN` syntax for cleaner queries when both dates provided.
        -   Configurable `date_field` parameter for filtering alternative temporal fields.
    
    -   **Integrated Date Filtering in Bulk Downloads:**
        -   Added `start_date` and `end_date` parameters to all bulk download methods.
        -   Date filters automatically combined with existing `cql_filter` configuration via logical AND.
        -   Original CQL filter state preserved and restored after download completion.
        -   Includes informative logging when date filters are applied.

-   **ADLSDataStore: Performance Optimizations for File and Directory Operations**
    -   **Optimized `list_files()` Method:**
        -   Replaced `list_blobs()` with `list_blob_names()` for performance improvement.
        -   Reduced memory usage by returning blob name strings instead of full `BlobProperties` objects.
        -   Added `_normalize_path()` helper method for consistent path handling (removes leading slashes, ensures trailing slashes for directories, converts backslashes to forward slashes).
        -   Maintains backward compatibility—still returns a list of file paths.
        -   Performance: ~2MB memory usage vs ~500MB for large directories.
    
    -   **New `list_files_iter()` Method:**
        -   Memory-efficient generator-based iteration over files in large directories.
        -   Enables early exit and lazy evaluation without loading entire file lists into memory.
        -   Returns iterator of blob name strings, supporting streaming workflows.
        -   Ideal for directories with 100K+ files where full materialization is unnecessary.
    
    -   **Optimized `walk()` Method:**
        -   Replaced `list_blobs()` with `list_files_iter()` for lazy evaluation.
        -   Eliminated full materialization of blob lists into memory.
        -   Performance: 2-3x faster and 100-500x less memory usage for large directory trees.
    
    -   **Optimized `list_directories()` Method:**
        -   Replaced `list_blobs()` iteration with Azure's `walk_blobs(delimiter='/')` for hierarchical listing.
        -   Azure now returns directory prefixes directly without scanning all files.
        -   Performance: 100-1000x faster for large directories (1-5 seconds vs 30-60 seconds for 1M files).
        -   No longer requires iterating through all files to identify subdirectories.
    
    -   **New Utility Methods:**
        -   `has_files_with_extension()`: Fast early-exit check for files with specific extensions without full directory scan.
        -   `count_files()`: Memory-efficient file counting using generator iteration.
        -   `count_files_with_extension()`: Memory-efficient counting of files by extension.

-   **SnowflakeDataStore: Multiprocessing Support**
    -   Implemented lazy connection initialization enabling safe pickling and multiprocessing usage.
    -   Added thread-safe connection creation via `_get_connection()` with double-check locking pattern.
    -   Implemented custom `__getstate__()` and `__setstate__()` for proper serialization, excluding non-picklable connection and lock objects.
    -   Each worker process creates its own database connection on first access.
    -   Maintained full backward compatibility via `connection` property accessor.
    -   Improved startup performance by deferring connection creation until first use.

### Performance

-   **Significant speedups for Azure Blob Storage operations:**
    -   File listing operations now faster due to `list_blob_names()` optimization.
    -   Directory listing operations now 100-1000x faster using Azure's hierarchical `walk_blobs()` with delimiter.
    -   Memory usage reduced by 100-500x for large directory operations (2-5MB vs 500MB-1GB).
    -   Directory tree walking now 2-3x faster with lazy evaluation via `list_files_iter()`.
    -   Benefits scale with directory size—most dramatic improvements for directories with 100K+ files.

-   **Improved startup time for SnowflakeDataStore:**
    -   Lazy connection initialization eliminates unnecessary connection overhead when data store is instantiated but not immediately used.

### Developer Notes

-   **Migration from Legacy Maxar API:**
    -   Existing code using `MAXAR_USERNAME`, `MAXAR_PASSWORD`, and `MAXAR_CONNECTION_STRING` environment variables must migrate to `MAXAR_API_KEY`.
    -   Old base URL and authentication methods are no longer supported by Maxar's API infrastructure.
    -   Layer names in existing configurations must be updated to new `Maxar:` namespace.
    -   CQL filter parameter names updated throughout codebase for consistency with new API specification.

-   **ADLSDataStore Improvements:**
    -   Existing `list_files()` calls remain fully compatible—no code changes required.
    -   For large directories (100K+ files), consider using `list_files_iter()` for memory efficiency.
    -   The `_normalize_path()` helper ensures consistent path handling across all methods.
    -   All optimizations leverage Azure SDK's native capabilities for maximum performance.

### Dependencies

-   Updated `owslib` usage to support WFS 2.0.0 specification for metadata retrieval.

## [v0.7.5] - 2026-01-20

### Added

-   **Google-Microsoft Combined Buildings Handler (VIDA)**
    -   Added `GoogleMSBuildingsHandler` (`gigaspatial/handlers/google_ms_combined_buildings.py`) to access the merged Google V3 Open Buildings (1.8B footprints) and Microsoft Global Building Footprints (1.24B footprints) dataset hosted by VIDA/source.coop.
    -   Supports multiple data formats: GeoParquet (default), FlatGeobuf, and PMTiles.
    -   Flexible partition strategies: country-level (single file per country) or S2 grid partitioning (tiled by S2 cells for large countries).
    -   Download strategies: S3 (default) or HTTPS for cloud-native access.
    -   Source filtering: filter buildings by data source (`google`, `microsoft`, or both).
    -   Integrated with `BaseHandler` architecture for consistent data lifecycle management (download, cache, read).
    -   Automatic partition discovery: resolves relevant S2 tiles or country files based on query geometry.
    -   Streaming support: can stream GeoParquet row groups directly from cloud storage without full download.

-   **Shared Building-Processing Engine**
    -   Added `GoogleMSBuildingsEngine` (`gigaspatial/processing/buildings_engine.py`) as a reusable, high-performance building workflow engine.
    -   Encapsulates common building-processing logic (S2 tile job creation, per-tile processing loops, and result accumulation) previously duplicated across view generators.
    -   Provides two main entrypoints:
        -   `count_buildings_in_zones()`: Efficiently counts buildings intersecting zones using `STRtree` spatial indexing.
        -   `nearest_buildings_to_pois()`: Computes nearest-building distances for POIs using KD-tree nearest-neighbor search with haversine distance calculation.
    -   Handles both single-file countries (processes entire dataset) and partitioned countries (loads only intersecting S2 tiles).
    -   Supports source filtering (`google`, `microsoft`, or both) at the engine level.
    -   Designed for reuse: both zonal and POI view generators now delegate to this engine, reducing code duplication and ensuring consistent performance optimizations.

-   **High-Performance Buildings Mapping for View Generators**
    -   Added `GeometryBasedZonalViewGenerator.map_buildings()` to efficiently count buildings per zone using the combined dataset.
        -   Uses S2 grid partitioning to load only intersecting building tiles for partitioned countries.
        -   Leverages `STRtree` spatial indexing for fast intersection queries between buildings and zones.
        -   Supports source filtering to count buildings from specific providers.
    -   Added `PoiViewGenerator.find_nearest_buildings()` to efficiently compute nearest-building distances for POIs.
        -   Uses S2 grid partitioning with configurable search radius to limit tile processing.
        -   Implements KD-tree nearest-neighbor search for fast candidate selection.
        -   Computes final distances using haversine (great-circle) distance in meters.
        -   Supports global nearest-building search with progressive radius expansion for partitioned countries.
        -   Returns distance metrics and boolean flags indicating buildings within specified search radius.

-   **Geo Processing Utilities**
    -   Added `estimate_utm_crs_with_fallback()` (`gigaspatial/processing/geo.py`) as a robust utility for UTM CRS estimation.
        -   Wraps `GeoDataFrame.estimate_utm_crs()` with comprehensive error handling and fallback logic.
        -   Automatically falls back to a configurable CRS (default: Web Mercator EPSG:3857) when UTM estimation fails or returns `None`.
        -   Handles edge cases: empty GeoDataFrames, estimation exceptions, and `None` return values.
        -   Centralizes the common UTM estimation pattern used across the codebase, reducing duplication.
        -   Provides optional logger parameter for warning messages when fallbacks occur.

### Fixed

-   **Spatial Matching Graph Construction (`build_distance_graph`)**
    -   Fixed critical bug where `exclude_same_index=True` returned fewer matches than requested when querying same dataframe against itself.
    -   Previously, when excluding self-matches, the function would query for `max_k` neighbors and then filter out self-matches, resulting in only `max_k - 1` actual matches returned.
    -   Most critically, `max_k=1` with `exclude_same_index=True` would always return zero matches (since the only neighbor found was the point itself).
    -   Now queries for `max_k + 1` neighbors when `exclude_same_index=True`, ensuring `max_k` valid matches are returned after self-match removal.
    -   Affects `build_distance_graph()` in spatial matching workflows where the same dataframe is matched against itself.

- **S2 cell polygon generation (`to_geoms`)**
    -   Now conversion of S2 cells to Shapely polygons enforces a consistent counter‑clockwise winding order using  shapely.geometry.polygon.orient() , avoiding accidental orientation flips or hole-like polygons when rendering or exporting.
    -   Added validation and automatic repair of invalid polygons via  buffer(0)  to handle rare projection-related self-intersections, logging the number of repaired cells and skipping any that remain invalid after repair.
    -   Improved logging for debugging by including the S2 token in warnings and errors when a cell fails conversion or cannot be repaired.

### Performance

-   **Significant speedups for buildings enrichment**
    -   Zonal building counts and POI nearest-building mapping are now substantially faster than mapping Google and Microsoft buildings separately, achieving performance gains through:
        -   **Single combined dataset**: Eliminates redundant I/O and repeated processing of overlapping building footprints from separate sources.
        -   **S2 tile filtering**: For partitioned countries, loads only intersecting S2 building tiles instead of scanning entire country files, dramatically reducing memory usage and processing time.
        -   **Spatial indexing**: Uses `STRtree` for O(log k) zone intersection queries and KD-tree for fast nearest-neighbor search, replacing slower sequential scans.
        -   **Shared engine architecture**: Centralized optimizations benefit both zonal and POI workflows, ensuring consistent performance improvements across use cases.

### Dependencies

-   Added `s3fs>=2024.12.0` as a new dependency

## [v0.7.4] - 2025-11-24

### Added

-   **TifProcessor: Raster Export Methods** 
    -   **`save_to_file()` method:** Comprehensive raster export functionality with flexible compression and optimization options.
        -   Supports multiple compression algorithms: LZW (default), DEFLATE, ZSTD, JPEG, WEBP, and NONE.
        -   Configurable compression parameters: `ZLEVEL` for DEFLATE (default: 6), `ZSTD_LEVEL` for ZSTD (default: 9), `JPEG_QUALITY` (default: 85), `WEBP_LEVEL` (default: 75).
        -   Predictor support for improved compression: predictor=2 for integer data (horizontal differencing), predictor=3 for floating-point data.
        -   Tiled output enabled by default (512×512 blocksize) for optimal random access performance.
        -   Cloud-Optimized GeoTIFF (COG) support via `cog=True` parameter with automatic overview generation.
        -   Customizable overview levels and resampling methods for COG creation.
        -   BigTIFF support for files >4GB via `bigtiff` parameter.
        -   Multi-threading support for compatible compression algorithms via `num_threads` parameter.
        -   Integrates with `self.open_dataset()` context manager, automatically handling merged, reprojected, and clipped rasters.
        -   Writes through `self.data_store` abstraction layer, supporting both local and remote storage (e.g., ADLS).
        -   Preserves all bands from source rasters without skipping.
    
    -   **`save_array_to_file()` method:** Export processed numpy arrays while preserving georeferencing metadata.
        -   Accepts 2D or 3D numpy arrays (with automatic dimension handling).
        -   Inherits CRS, transform, and nodata values from source raster or accepts custom values.
        -   Supports same compression options as `save_to_file()`.
        -   Enables saving modified/processed raster data while maintaining spatial reference.
        -   Writes through `self.data_store` for consistent storage abstraction.

-   **TifProcessor: Value-Based Filtering for DataFrame and GeoDataFrame Conversion**
    -   **`min_value` and `max_value` parameters:** Added optional filtering thresholds to `to_dataframe()` and `to_geodataframe()` methods.
        -   `min_value`: Filters out pixels with values ≤ threshold (exclusive).
        -   `max_value`: Filters out pixels with values ≥ threshold (exclusive).
        -   Filtering occurs **before** geometry creation in `to_geodataframe()`, significantly improving performance for sparse datasets.
        -   Supports both single-band and multi-band rasters with consistent behavior.
    
    -   **Enhanced `_build_data_mask()` method:** Extended to incorporate value threshold filtering alongside nodata filtering.
        -   Combines multiple mask conditions using logical AND for efficient filtering.
        -   Maintains backward compatibility when no thresholds are specified.
    
    -   **Enhanced `_build_multi_band_mask()` method:** Extended for multi-band value filtering.
        -   Drops pixels where ANY band has nodata or fails value thresholds.
        -   Ensures consistent filtering behavior across RGB, RGBA, and multi-band modes.

-   **TifProcessor: Raster Statistics in `get_raster_info()`**
    -   Added `include_statistics` and `approx_ok` flags to optionally return pixel statistics alongside metadata.
    -   New `_get_basic_statistics()` helper streams through raster blocks to compute per-band and overall min, max, mean, std, sum, and count with nodata-aware masking.
    -   Results are cached for reuse within the processor lifecycle to avoid repeated scans.

-   **BaseHandler: Tabular Load Progress**
    -   `_load_tabular_data()` now supports a `tqdm` progress bar, showing file-level load progress for large tabular batches.
    -   Added `show_progress` and `progress_desc` parameters so handlers can toggle or customize the indicator while keeping existing callers backward compatible.

-   **Improved developer usability by enabling easier access to primary components without deep module references:**
    - Exposed core handlers, view generators, and processing modules at the top-level `gigaspatial` package namespace for improved user experience and simplified imports.
    - Added convenient aliases for `gigaspatial.core.io` as `io` and `gigaspatial.processing.algorithms` as `algorithms` directly accessible from `gigaspatial`.
    - Declared explicit public API in `__init__.py` to clarify stable, supported components.

### Changed

-   **GHSLDataConfig: Improved SSL Certificate Handling for Tile Downloads**
    -   Replaced `ssl._create_unverified_context` approach with a robust two-tier fallback strategy for downloading GHSL tiles shapefile.
    
    -   **Primary method:** Attempts download via `gpd.read_file()` with unverified SSL context (fast, direct access).
    
    -   **Fallback method:** Uses `requests.get()` with `verify=False` for environments where `gpd.read_file()` fails (e.g., cloud compute instances with Anaconda certificate bundles).
    -   Downloads tiles to temporary local file before reading when fallback is triggered, ensuring compatibility across different Python environments.
    
    -   **Tile caching:** Implemented GeoJSON-based caching in `base_path/cache/` directory to minimize redundant downloads.
        -   Cache checked before any download attempts.
        -   Invalid cache automatically triggers re-download.
        -   Uses `write_dataset()` for consistent storage abstraction across local and remote data stores.
    
    -   **Enhanced error handling:** 
        -   Logs specific exception types (`type(e).__name__`) for better debugging.
        -   Graceful fallback with informative warning messages.
        -   Preserves exception chain for traceback analysis.
    -   Improved compatibility with Azure ML compute instances where multiple certificate stores (system, Anaconda, certifi) coexist.
    -   Temporary file cleanup guaranteed via `finally` block, preventing orphaned downloads.

### Fixed

-   **GHSLDataConfig: SSL certificate verification failures in cloud environments**
    -   Resolved `CERTIFICATE_VERIFY_FAILED` errors when downloading GHSL tiles shapefile on cloud compute instances.
  
### Performance

-   **Reduced network overhead for GHSL tile metadata**:
    -   Tiles shapefile downloaded only once per coordinate system (WGS84/Mollweide) and cached locally.
    -   Subsequent `GHSLDataConfig` instantiations load from cache, eliminating repeated ~MB shapefile downloads.
    -   Benefits scale with number of GHSL queries across application lifecycle.


### Documentation

- Improved `README` with clearer key workflows, core concepts, and updated overview text.

## [v0.7.3] - 2025-11-11

### Added

-   **SnowflakeDataStore Support**
    -   New `SnowflakeDataStore` class implementing the `DataStore` interface for Snowflake stages.
    -   Supports file operations (read, write, list, delete) on Snowflake internal stages.
    -   Integrated with `gigaspatial/config.py` for centralized configuration via environment variables.
    -   Provides directory-like operations (`mkdir`, `rmdir`, `walk`, `is_dir`, `is_file`) for conceptual directories in Snowflake stages.
    -   Includes context manager support and connection management.
    -   Full compatibility with existing `DataStore` abstraction.

-   **BaseHandler: Config-Level Data Unit Caching**
    -   `BaseHandlerConfig` now maintains internal `_unit_cache` for data unit and geometry caching.
    -   Cache stores tuples of `(units, search_geometry)` for efficient reuse across handler, downloader, and reader operations.
    -   New methods:
        -   `get_cached_search_geometry()`: Retrieve cached geometry for a source.
        -   `clear_unit_cache()`: Clear cached data for testing or manual refreshes.
        -   `_cache_key()`: Generate canonical cache keys from various source types.
    -   Benefits all components (handler, downloader, reader) regardless of entry point.

-   **Unified Geometry Extraction in BaseHandlerConfig**
    -   New `extract_search_geometry()` method providing standardized geometry extraction from various source types:
        -   Country codes (via `AdminBoundaries`)
        -   Shapely geometries (`BaseGeometry`)
        -   GeoDataFrames with automatic CRS handling
        -   Lists of points or coordinate tuples (converted to `MultiPoint`)
    -   Centralizes geometry conversion logic, eliminating duplication across handler methods.

-   **BaseHandler: Crop-to-Source Feature for Handlers**
    -   New `crop_to_source` parameter in `BaseHandlerReader.load()` and `BaseHandler.load_data()` methods.
    -   Allows users to load data clipped to exact source boundaries rather than full data units (e.g., tiles).
    -   Particularly useful for tile-based datasets (Google Open Buildings, GHSL) where tiles extend beyond requested regions.
    -   Implemented `crop_to_geometry()` method in `BaseHandlerReader` for spatial filtering:
        -   Supports `(Geo)DataFrame` clipping using geometry intersection.
        -   Supports raster clipping using `TifProcessor`'s `clip_to_geometry` method.
        -   Extensible for future cropping implementations.
    -   Search geometries are now cached alongside data units for efficient cropping operations.

-   **S2 Zonal View Generator (`S2ViewGenerator`)**
    -   New generator for producing zonal views using Google S2 cells (levels 0–30).
    -   Supports sources:
        -   Country name (`str`) via `CountryS2Cells.create(...)`
        -   Shapely geometry or `gpd.GeoDataFrame` via `S2Cells.from_spatial(...)`
        -   Points (`List[Point | (lon, lat)]`) via `S2Cells.from_points(...)`
        -   Explicit cells (`List[int | str]`, S2 IDs or tokens) via `S2Cells.from_cells(...)`
    -   Uses `cell_token` as the zone identifier.
    -   Includes `map_wp_pop()` convenience method (auto-uses stored country when available).

-   **H3 Zonal View Generator (`H3ViewGenerator`)**
    -   New generator for producing zonal views using H3 hexagons (resolutions 0–15).
    -   Supports sources:
        -   Country name (`str`) via `CountryH3Hexagons.create(...)`
        -   Shapely geometry or `gpd.GeoDataFrame` via `H3Hexagons.from_spatial(...)`
        -   Points (`List[Point | (lon, lat)]`) via `H3Hexagons.from_spatial(...)`
        -   Explicit H3 indexes (`List[str]`) via `H3Hexagons.from_hexagons(...)`
    -   Uses `h3` as the zone identifier.
    -   Includes `map_wp_pop()` convenience method (auto-uses stored country when available).

-   **TifProcessor: MultiPoint clipping support**
    -   `_prepare_geometry_for_clipping()` now accepts `MultiPoint` inputs and uses their bounding box for raster clipping.
    -   Enables passing collections of points as a `MultiPoint` to `clip_to_geometry()` without pre-converting to a polygon.

### Changed

-   **Configuration**
    -   Added Snowflake connection parameters to `gigaspatial/config.py`:
        -   `SNOWFLAKE_ACCOUNT`, `SNOWFLAKE_USER`, `SNOWFLAKE_PASSWORD`
        -   `SNOWFLAKE_WAREHOUSE`, `SNOWFLAKE_DATABASE`, `SNOWFLAKE_SCHEMA`
        -   `SNOWFLAKE_STAGE_NAME`
    -   Added Snowflake configuration variables to `.env_sample`

-   **BaseHandler**
    -   **Streamlined Data Unit Resolution**
        -   Consolidated `get_relevant_data_units_by_country()`, `get_relevant_data_units_by_points()`, and `get_relevant_data_units_by_geometry()` into a unified workflow.
        -   All source types now convert to geometry via `extract_search_geometry()` before unit resolution.
        -   Subclasses now only need to implement `get_relevant_data_units_by_geometry()` for custom logic.
        -   Significantly reduces code duplication in handler subclasses.
    
    -   **Optimized Handler Workflow**
        -   Eliminated redundant `get_relevant_data_units()` calls across handler, downloader, and reader operations.
        -   `ensure_data_available()` now uses cached units and paths, preventing multiple lookups per request.
        -   Data unit resolution occurs at most once per unique source query, improving performance for:
            -   Repeated `load_data()` calls with the same source.
            -   Operations involving both download and read steps.
            -   Direct usage of downloader or reader components.

-   **Enhanced BaseHandlerReader**
    -   `resolve_source_paths()` now primarily handles explicit file paths.
    -   Geometry/country/point conversion delegated to handler and config layers.
    -   `load()` method updated to support `crop_to_source` parameter with automatic geometry retrieval from cache.
    -   Fallback geometry computation if cache miss occurs (e.g., when reader used independently).


-   **BaseHandlerConfig Caching Logic**
    -   `get_relevant_data_units()` now checks cache before computing units.
    -   Added `force_recompute` parameter to bypass cache when needed (e.g., `force_download=True`).
    -   Cache operations include debug logging for transparency during development.


-   **TifProcessor temp-file handling**
    -   Simplified `_create_clipped_processor()` to mirror `_reproject_to_temp_file`: write clipped output to the new processor’s `_temp_dir`, set `_clipped_file_path`, update `dataset_path`, and reload metadata.
    -   `open_dataset()` now prioritizes `_merged_file_path`, `_reprojected_file_path`, then `_clipped_file_path`, and opens local files directly.
    -   Clipped processors consistently use `LocalDataStore()` for local temp files to avoid data-store path resolution issues.

## Fixed

-   **TifProcessor: clip_to_geometry() open failure after merge**
    -   Fixed a bug where `open_dataset()` failed for processors returned by `clip_to_geometry()` when the source was initialized with multiple paths and loaded via handlers with `merge_rasters=True`.
    -   The clipped raster is now saved directly into the new processor’s temp directory and tracked via `_clipped_file_path`, ensuring reliable access by `open_dataset()`.
    -   Absolute path checks in `__post_init__` now use `os.path.exists()` for absolute paths with `LocalDataStore`, preventing false negatives for temp files.

### Performance

-   **Significant reduction in redundant computations in handlers**:
    -   Single geometry extraction per source query (previously up to 3 times).
    -   Single data unit resolution per source query (previously 2-3 times).
    -   Cached geometry reuse for cropping operations.
    -   Benefits scale with:
        -   Number of repeated queries.
        -   Complexity of geometry extraction (especially country boundaries).
        -   Number of data units per query.

### Developer Notes

-   Subclass implementations should now:
    -   Only override `get_relevant_data_units_by_geometry()` for custom unit resolution.
    -   Use `extract_search_geometry()` for any geometry conversion needs.
    -   Optionally override `crop_to_geometry()` for dataset-specific cropping logic.


### Dependencies

-   Added `snowflake-connector-python>=3.0.0` as a new dependency

## [v0.7.2] - 2025-10-27

### Added

-   **Ookla Speedtest Handler Integration (`OoklaSpeedtestHandler`)**
    -   New classes `OoklaSpeedtestHandler`, `OoklaSpeedtestConfig`, `OoklaSpeedtestDownloader`, and `OoklaSpeedtestReader` for managing Ookla Speedtest data.
    -   `OoklaSpeedtestHandler.load_data` method supports Mercator tile filtering by country or spatial geometry and includes an optional `process_geospatial` parameter for WKT to GeoDataFrame conversion.
    -  In `OoklaSpeedtestConfig`, `year` and `quarter` fields are optional (defaulting to `None`) and `__post_init__` logs warnings if they are not explicitly provided, using the latest available data.
    -  In `OoklaSpeedtestReader`, `resolve_source_paths` method overridden to appropriately handle `None` or non-path sources by returning the `DATASET_URL`.
    -  `OoklaSpeedtestHandler`, the `__init__` method requires `type` as a mandatory argument, with `year` and `quarter` being optional.

-   **S2 Grid Generation Support (`S2Cells`)**
    -   Introduced `S2Cells` class for managing Google S2 cell grids using the `s2sphere` library.
    -   Supports S2 levels 0-30, providing finer granularity than H3 (30 levels vs 15).
    -   Provides multiple creation methods:
        -   `from_cells()`: Create from lists of S2 cell IDs (integers or tokens).
        -   `from_bounds()`: Create from geographic bounding box coordinates.
        -   `from_spatial()`: Create from various spatial sources (geometries, GeoDataFrames, points).
        -   `from_json()`: Load S2Cells from JSON files via DataStore.
    -   Includes methods for spatial operations:
        -   `get_neighbors()`: Get edge neighbors (4 per cell) with optional corner neighbors (8 total).
        -   `get_children()`: Navigate to higher resolution child cells.
        -   `get_parents()`: Navigate to lower resolution parent cells.
        -   `filter_cells()`: Filter cells by a given set of cell IDs.
    -   Provides conversion methods:
        -   `to_dataframe()`: Convert to pandas DataFrame with cell IDs, tokens, and centroid coordinates.
        -   `to_geoms()`: Convert cells to shapely Polygon geometries (square cells).
        -   `to_geodataframe()`: Convert to GeoPandas GeoDataFrame with geometry column.
    -   Supports saving to JSON, Parquet, or GeoJSON files via `save()` method.
    -   Includes `average_cell_area` property for approximate area calculation based on S2 level.
    
-   **Country-Specific S2 Cells (`CountryS2Cells`)**
    -   Extends `S2Cells` for generating S2 grids constrained by country boundaries.
    -   Integrates with `AdminBoundaries` to fetch country geometries for precise cell generation.
    -   Factory method `create()` enforces proper instantiation with country code validation via `pycountry`.

-   Expanded `write_dataset` to support generic JSON objects.
    -   The `write_dataset` function can now write any serializable Python object (like a dict or list) directly to a `.json` file by leveraging the dedicated write_json helper.

-   **NASA SRTM Elevation Data Handler (`NasaSRTMHandler`)**
    -   New handler classes for downloading and processing NASA SRTM elevation data (30m and 90m resolution).
    -   Supports Earthdata authentication via `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` environment variables.
    -   `NasaSRTMConfig` provides dynamic 1°x1° tile grid generation covering the global extent.
    -   `NasaSRTMDownloader` supports parallel downloads of SRTM .hgt.zip tiles using multiprocessing.
    -   `NasaSRTMReader` loads SRTM data with options to return as pandas DataFrame or list of `SRTMParser` objects.
    -   Integrated with `BaseHandler` architecture for consistent data lifecycle management.

-   **SRTM Parser (`SRTMParser`)**
    -   Efficient parser for NASA SRTM .hgt.zip files using memory mapping.
    -   Supports both SRTM-1 (3601x3601, 1 arc-second) and SRTM-3 (1201x1201, 3 arc-second) formats.
    -   Provides methods for:
        -   `get_elevation(latitude, longitude)`: Get interpolated elevation for specific coordinates.
        -   `get_elevation_batch(coordinates)`: Batch elevation queries with NumPy array support.
        -   `to_dataframe()`: Convert elevation data to pandas DataFrame with optional NaN filtering.
        -   Automatic tile coordinate extraction from filename (e.g., N37E023, S10W120).

-   **SRTM Manager (`SRTMManager`)**
    -   Manager class for accessing elevation data across multiple SRTM tiles with lazy loading.
    -   Implements LRU caching (default cache size: 10 tiles) for efficient memory usage.
    -   Methods include:
        -   `get_elevation(latitude, longitude)`: Get interpolated elevation for any coordinate.
        -   `get_elevation_batch(coordinates)`: Batch elevation queries across multiple tiles.
        -   `get_elevation_profile(latitudes, longitudes)`: Generate elevation profiles along paths.
        -   `check_coverage(latitude, longitude)`: Check if a coordinate has SRTM coverage.
        -   `get_available_tiles()`: List available SRTM tiles.
        -   `clear_cache()` and `get_cache_info()`: Cache management utilities.
    -   Automatically handles tile boundary crossings for elevation profiles.

-   **Earthdata Session (`EarthdataSession`)**
    -   Custom `requests.Session` subclass for NASA Earthdata authentication.
    -   Maintains Authorization headers through redirects to/from Earthdata hosts.
    -   Required for accessing NASA's SRTM data repository.

### Changed

-   **ADLSDataStore Enhancements**
    -   Modified `__init__` method to support initialization using either `ADLS_CONNECTION_STRING` or a combination of `ADLS_ACCOUNT_URL` and `ADLS_SAS_TOKEN`.
    -   Improved flexibility for authenticating with Azure Data Lake Storage.

-   **Configuration**
    -   Added `ADLS_ACCOUNT_URL` and `ADLS_SAS_TOKEN` to `gigaspatial/config.py` and `.env_sample` for alternative ADLS authentication.
    -   Added `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` to `gigaspatial/config.py` and `.env_sample` for NASA Earthdata authentication.

### Fixed

-   **WorldPop: `RuntimeError` during `school_age=True` data availability check:**
    -   Resolved a `RuntimeError: Could not ensure data availability for loading` that occurred when `school_age=True` and WorldPop data was not yet present in the data store.
    -   `WPPopulationConfig.get_data_unit_paths` now correctly returns the original `.zip` URLs to trigger the download/extraction process when filtered `.tif` files are missing.
    -   After successful download and extraction, it now accurately identifies and returns the paths to the local `.tif` files, allowing `BaseHandler` to confirm availability and proceed with loading.
-   **WorldPop: `list index out of range` when no datasets found**:
    -   Added a `RuntimeError` in `WPPopulationConfig.get_relevant_data_units_by_country` when `self.client.search_datasets` returns no results, providing a clearer error message with the search parameters.

-   **WorldPop: Incomplete downloads with `min_age`/`max_age` filters for non-school-age `age_structures`**:
    -   Fixed an issue where `load_data` with `min_age` or `max_age` filters (when `school_age=False`) resulted in incomplete downloads.
    -   `WPPopulationConfig.get_data_unit_paths` now returns all potential `.tif` URLs for non-school-age `age_structures` during the initial availability check, ensuring all necessary files are downloaded.
    -   Age/sex filtering is now deferred and applied by `WPPopulationReader.load_from_paths` using `WPPopulationConfig._filter_age_sex_paths` *after* download, guaranteeing data integrity.

-   **HealthSitesFetcher**
    -   Ensured correct Coordinate Reference System (CRS) assignment (`EPSG:4326`) when returning `GeoDataFrame` from fetched health facility data.

### Dependencies

-   Added `s2sphere` as a new dependency for S2 geometry operations

## [v0.7.1] - 2025-10-15

### Added

-   **Healthsites.io API Integration (`HealthSitesFetcher`):**
    -   New class `HealthSitesFetcher` to fetch and process health facility data from the Healthsites.io API.
    -   Supports filtering by country, bounding box extent, and date ranges (`from_date`, `to_date`).
    -   Provides methods for:
        -   `fetch_facilities()`: Retrieves health facility locations, returning a `pd.DataFrame` or `gpd.GeoDataFrame` based on output format.
        -   `fetch_statistics()`: Fetches aggregated statistics for health facilities based on provided filters.
        -   `fetch_facility_by_id()`: Retrieves details for a specific facility using its OSM type and ID.
    -   Includes robust handling for API pagination, different output formats (JSON, GeoJSON), and nested data structures.
    -   Integrates with `OSMLocationFetcher` and `pycountry` to standardize country names to OSM English names for consistent querying.
    -   Configurable parameters for API URL, API key, page size, flat properties, tag format, output format, and request sleep time.

-   **OSMLocationFetcher Enhancements:**
    -   **Historical Data Fetching (`fetch_locations_changed_between`):**
        -   New method `fetch_locations_changed_between()` to retrieve OSM objects that were created or modified within a specified date range. This enables historical analysis and change tracking.
        -   Defaults `include_metadata` to `True` for this method, as it's typically used for change tracking.
    -   **Comprehensive OSM Country Information (`get_osm_countries`):**
        -   New static method `get_osm_countries()` to fetch country-level administrative boundaries directly from the OSM database.
        -   Supports fetching all countries or a specific country by ISO 3166-1 alpha-3 code.
        -   Option to include various name variants (e.g., `name:en`, `official_name`) and ISO codes.
    -   **Metadata Inclusion in Fetched Locations:**
        -   Added `include_metadata` parameter to `fetch_locations()` to optionally retrieve change tracking metadata (timestamp, version, changeset, user, uid) for each fetched OSM element.
        -   This metadata is now extracted and included in the DataFrame for nodes, relations, and ways.
    -   **Flexible Date Filtering in Overpass Queries:**
        -   Introduced `date_filter_type` (`newer`, `changed`) and `start_date`/`end_date` parameters to `_build_queries()` for more granular control over time-based filtering in Overpass QL.
    -   **Date Normalization Utility:**
        -   Added `_normalize_date()` helper method to convert various date inputs (string, datetime object) into a standardized ISO 8601 format for Overpass API queries.

-   **TifProcessor**
    -   **Comprehensive Memory Management:**
        -   Introduced `_check_available_memory()`, `_estimate_memory_usage()`, and `_memory_guard()` methods for proactive memory assessment across various operations.
        -   Added warnings (`ResourceWarning`) for potentially high memory usage in batched operations, with suggestions for optimizing `n_workers`.
    -   **Chunked DataFrame Conversion:**
        -   Implemented `to_dataframe_chunked()` for memory-efficient processing of large rasters by converting them to DataFrames in manageable chunks.
        -   Automatic calculation of optimal `chunk_size` based on target memory usage via `_calculate_optimal_chunk_size()`.
        -   New helper methods: `_get_chunk_windows()`, `_get_chunk_coordinates()`.
    -   **Raster Clipping Functionality:**
        -   `clip_to_geometry()`: New method to clip rasters to arbitrary geometries (Shapely, GeoDataFrame, GeoSeries, GeoJSON-like dicts).
        -   `clip_to_bounds()`: New method to clip rasters to rectangular bounding boxes, supporting optional CRS transformation for the bounds.
        -   New helper methods for clipping: `_prepare_geometry_for_clipping()`, `_validate_geometry_crs()`, `_create_clipped_processor()`.

-   **WorldPopDownloader Zip Handling:**
    -   Modified `download_data_unit` in `WPPopulationDownloader` to correctly handle `.zip` files (e.g., school age datasets) by downloading them to a temporary location and extracting the contained `.tif` files.
    -   Updated `download_data_units` to correctly flatten the list of paths returned by `download_data_unit` when zip extraction results in multiple files.
    -   Adjusted `WPPopulationConfig.get_data_unit_paths` to correctly identify and return paths for extracted `.tif` files from zip resources. It is now intelligently resolves paths. For school-age datasets, it returns paths to extracted `.tif` files if available; otherwise, it returns the original `.zip` path(s) to trigger download and extraction.
    -   Added filter support to `WPPopulationConfig.get_data_unit_paths` hence to the `WPPopulationHandler` for:
        -   School-age datasets: supports `sex` (e.g., "F", "M", "F_M") and `education_level` (e.g., "PRIMARY", "SECONDARY") filters on extracted `.tif` filenames.
        -   Non-school-age age_structures: supports `sex`, `ages`, `min_age`, and `max_age` filters on `.tif` filenames.
-   **WorldPop: Filtered aggregation in `GeometryBasedZonalViewGenerator.map_wp_pop`**:
    -   `map_wp_pop` now enforces a single country input when `handler.config.project` is "age_structures".
    -   When `predicate` is "centroid_within" and the project is "age_structures", individual `TifProcessor` objects (representing age/sex combinations) are loaded, sampled with `map_rasters(stat="sum")`, and their results are summed per zone, preventing unintended merging.

-   **PoiViewGenerator: Filtered aggregation in `PoiViewGenerator.map_wp_pop`**:
    -   `map_wp_pop` now enforces a single country input when `handler.config.project` is "age_structures".
    -   When `predicate` is "centroid_within" and the project is "age_structures", individual `TifProcessor` objects (representing age/sex combinations) are loaded, sampled with `map_zonal_stats(stat="sum")`, and their results are summed per POI, preventing unintended merging.

-   **TifProcessor Multi-Raster Merging in Handlers and Generators:**
    -   Extended `_load_raster_data` in `BaseHandlerReader` to support an optional `merge_rasters` argument. When `True` and multiple raster paths are provided, `TifProcessor` now merges them into a single `TifProcessor` object during loading.
    -   Integrated `merge_rasters` argument into `GHSLDataReader` and `WPPopulationReader`'s `load_from_paths` and `load` methods, enabling control over raster merging at the reader level.
    -   Propagated `merge_rasters` to `GHSLDataHandler`'s `load_into_dataframe`, and `load_into_geodataframe` methods for consistent behavior across the handler interface.

### Changed

-   **TifProcessor**
    -   **Unified DataFrame Conversion:**
        -   Refactored `to_dataframe()` to act as a universal entry point, dynamically routing to internal, more efficient methods for single and multi-band processing.
        -   Deprecated the individual `_to_band_dataframe()`, `_to_rgb_dataframe()`, `_to_rgba_dataframe()`, and `_to_multi_band_dataframe()` methods in favor of the new unified `_to_dataframe()`.
        -   `to_dataframe()` now includes a `check_memory` parameter.
    -   **Optimized `open_dataset` Context Manager:**
        -   The `open_dataset` context manager now directly opens local files when `LocalDataStore` is used, avoiding unnecessary `rasterio.MemoryFile` creation for improved performance and reduced memory overhead.
    -   **Enhanced `to_geodataframe` and `to_graph`:**
        -   Added `check_memory` parameter to `to_geodataframe()` and `to_graph()` for memory pre-checks.
    -   **Refined `sample_by_polygons_batched`:**
        -   Included `check_memory` parameter for memory checks before batch processing.
        -   Implemented platform-specific warnings for potential multiprocessing issues on Windows/macOS.
    -   **Improved Multiprocessing Initialization:**
        -   The `_initializer_worker()` method now prioritizes merged, reprojected, or original local file paths for opening, ensuring workers access the most relevant data.
    -   **Modular Masking and Coordinate Extraction:**
        -   Introduced new private helper methods: `_extract_coordinates_with_mask()`, `_build_data_mask()`, `_build_multi_band_mask()`, and `_bands_to_dict()` to centralize and improve data masking and coordinate extraction logic.
    -   **Streamlined Band-Mode Validation:**
        -   Moved the logic for validating `mode` and band count compatibility into a dedicated `_validate_mode_band_compatibility()` method for better code organization.

-   **GigaSchoolLocationFetcher**
    -   `fetch_locations()` method:
        -   Added `process_geospatial` parameter (defaults to `False`) to optionally process geospatial data and return a `gpd.GeoDataFrame`.
    -   `_process_geospatial_data()` method:
        -   Modified to return a `gpd.GeoDataFrame` by converting the `pd.DataFrame` with a `geometry` column and `EPSG:4326` CRS.

-   **OSMLocationFetcher Refactoring:**
    -   **Unified Query Execution and Processing:** Refactored the core logic for executing Overpass queries and processing their results into a new private method `_execute_and_process_queries()`. This centralizes common steps and reduces code duplication between `fetch_locations()` and the new `fetch_locations_changed_between()`.
    -   **Enhanced `_build_queries`:** Modified `_build_queries` to accept `date_filter_type`, `start_date`, `end_date`, and `include_metadata` to construct more dynamic and feature-rich Overpass QL queries.
    -   **Updated `fetch_locations` Signature:**
        -   Replaced `since_year` parameter with `since_date` (which can be a `str` or `datetime` object) for more precise time-based filtering.
        -   Added `include_metadata` parameter.
    -   **Improved Logging of Category Distribution:**
        -   Modified the logging for category distribution to correctly handle cases where categories are combined into a list (when `handle_duplicates='combine'`).
    -   **`since_year` Parameter:** Removed `since_year` from `fetch_locations()` as its functionality is now superseded by the more flexible `since_date` parameter and the `_build_queries` enhancements.

-   **`PoiViewGenerator` Mapping Methods (`map_zonal_stats`, `map_nearest_points`, `map_google_buildings`, `map_ms_buildings`, `map_built_s`, `map_smod`):**
    -   Changed `map_zonal_stats` and `map_nearest_points` to return `pd.DataFrame` results (including `'poi_id'` and new mapped columns) instead of directly updating the internal view.
    -   Updated `map_google_buildings`, `map_ms_buildings`, `map_built_s`, and `map_smod` to capture the `pd.DataFrame` returned by their respective underlying mapping calls (`map_nearest_points` or `map_zonal_stats`) and then explicitly call `self._update_view()` with these results.
    -   This enhances modularity and allows for more flexible result handling and accumulation.

-   **`ZonalViewGenerator.map_rasters` Enhancements:**
    -   Modified `map_rasters` to accept `raster_data` as either a single `TifProcessor` or a `List[TifProcessor]`.
    -   Implemented internal merging of `List[TifProcessor]` into a single `TifProcessor` before performing zonal statistics.
    -   Replaced `sample_multiple_tifs_by_polygons` with the `TifProcessor.sample_by_polygons` method.

### Fixed

-   **TifProcessor**:
    -   **`to_graph()` Sparse Matrix Creation:**
        -   Corrected the sparse matrix creation logic in `to_graph()` to ensure proper symmetric graph representation when `graph_type="sparse"`.
    -   **Coordinate System Handling in `_initializer_worker`:**
        -   Ensured that `_initializer_worker` correctly handles different data storage scenarios to provide the correct dataset handle to worker processes, preventing `RuntimeError` due to uninitialized raster datasets.

### Removed

-   **OSMLocationFetcher**
    -   **Redundant Category Distribution Logging:** Removed the explicit category distribution logging for `handle_duplicates == "separate"` since the `value_counts()` method on the 'category' column already provides this.


## [v0.7.0] - 2025-09-17

### Added

- **TifProcessor Revamp**
    - **Explicit Reprojection Method:** Introduced `reproject_to()` method, allowing on-demand reprojection of rasters to a new CRS with customizable `resampling_method` and `resolution`.
    - **Reprojection Resolution Control:** Added `reprojection_resolution` parameter to `TifProcessor` for precise control over output pixel size during reprojection.
    - **Advanced Raster Information:** Added `get_raster_info()` method to retrieve a comprehensive dictionary of raster metadata.
    - **Graph Conversion Capabilities:** Implemented `to_graph()` method to convert raster data into a graph (NetworkX or sparse matrix) based on pixel adjacency (4- or 8-connectivity).
    - **Internal Refactoring: `_reproject_to_temp_file`:** Introduced `_reproject_to_temp_file` as a helper for reprojection into temporary files.

- **H3 Grid Generation**
    - **H3 Grid Generation Module (`gigaspatial/grid/h3.py`):**
        - Introduced `H3Hexagons` class for managing H3 cell IDs.
        - Supports creation from lists of hexagons, geographic bounds, spatial geometries, or points.
        - Provides methods to convert H3 hexagons to pandas DataFrames and GeoPandas GeoDataFrames.
        - Includes functionalities for filtering, getting k-ring neighbors, compacting hexagons, and getting children/parents at different resolutions.
        - Allows saving H3Hexagons to JSON, Parquet, or GeoJSON files.
    - **Country-Specific H3 Hexagons (`CountryH3Hexagons`):**
        - Extends `H3Hexagons` for generating H3 grids constrained by country boundaries.
        - Integrates with `AdminBoundaries` to fetch country geometries for precise H3 cell generation.

- **Documentation**
    - Improved `tif.md` example to showcase multi-raster initialization, explicit reprojection, and graph conversion.

### Changed

- **TifProcessor**
  - **Improved Temporary File Management:** Refactored temporary file handling for merging and reprojection using `tempfile.mkdtemp()` and `shutil.rmtree` for more robust and reliable cleanup. Integrated with context manager (`__enter__`, `__exit__`) and added a dedicated `cleanup()` method.
  - **Reprojection during Initialization:** Implemented automatic reprojection of single rasters to a specified `target_crs` during `TifProcessor` initialization.
  - **Enhanced `open_dataset` Context Manager:** The `open_dataset` context manager now intelligently opens the most up-to-date (merged or reprojected) version of the dataset.
  - **More Flexible Multi-Dataset Validation:** Modified `_validate_multiple_datasets` to issue a warning instead of raising an error for CRS mismatches when `target_crs` is not set.
  - **Optimized `_get_reprojection_profile`:** Dynamically calculates transform and dimensions based on `reprojection_resolution` and added LZW compression to reprojected TIFF files to reduce file size.

- **ADLSDataStore Enhancements**
    - **New `copy_file` method:** Implemented a new method for copying individual files within ADLS, with an option to overwrite existing files.
    - **New `rename` method:** Added a new method to rename (move) files in ADLS, which internally uses `copy_file` and then deletes the source, with options for overwrite, waiting for copy completion, and polling.
    - **Revamped `rmdir` method:** Modified `rmdir` to perform batch deletions of blobs, addressing the Azure Blob batch delete limit (256 sub-requests) and improving efficiency for large directories.

- **LocalDataStore Enhancements**
    - **New `copy_file` method:** Implemented a new method for copying individual files.

### Removed

- Removed deprecated `tabular` property and `get_zoned_geodataframe` method from `TifProcessor`. Users should now use `to_dataframe()` and `to_geodataframe()` respectively.

### Dependencies

- Added `networkx` and `h3` as new dependencies.

### Fixed

- Several small fixes and improvements to aggregation methods.

## [v0.6.9] - 2025-07-26

### Fixed

- Resolved a bug in the handler base class where non-hashable types (dicts) were incorrectly used as dictionary keys in `unit_to_path` mapping, preventing potential runtime errors during data availability checks.

## [v0.6.8] - 2025-07-26

### Added

- **OSMLocationFetcher Enhancements**
  - Support for querying OSM locations by arbitrary administrative levels (e.g., states, provinces, cities), in addition to country-level queries.
  - New optional parameters:
    - `admin_level`: Specify OSM administrative level (e.g., 4 for states, 6 for counties).
    - `admin_value`: Name of the administrative area to query (e.g., "California").
  - New static method `get_admin_names(admin_level, country=None)`:
    - Fetch all administrative area names for a given `admin_level`, optionally filtered by country.
    - Helps users discover valid admin area names for constructing precise queries.

- **Multi-Raster Merging Support in TifProcessor**
  - Added ability to initialize `TifProcessor` with **multiple raster datasets**.
  - Merges rasters on load with configurable strategies:
    - Supported `merge_method` options: `first`, `last`, `min`, `max`, `mean`.
  - Supports **on-the-fly reprojection** for rasters with differing coordinate reference systems via `target_crs`.
  - Handles **resampling** using `resampling_method` (default: `nearest`).
  - Comprehensive validation to ensure compatibility of input rasters (e.g., resolution, nodata, dtype).
  - Temporary file management for merged output with automatic cleanup.
  - Backward compatible with single-raster use cases.

  **New TifProcessor Parameters:**
  - `merge_method` (default: `first`) – How to combine pixel values across rasters.
  - `target_crs` (optional) – CRS to reproject rasters before merging.
  - `resampling_method` – Resampling method for reprojection.

  **New Properties:**
  - `is_merged`: Indicates whether the current instance represents merged rasters.
  - `source_count`: Number of raster datasets merged.

### Changed

- **OSMLocationFetcher Overpass Query Logic**
  - Refactored Overpass QL query builder to support **subnational queries** using `admin_level` and `admin_value`.
  - Improved flexibility and precision for spatial data collection across different administrative hierarchies.

### Breaking Changes

- None. All changes are fully backward compatible.

## [v0.6.7] - 2025-07-16

### Fixed

- Fixed a bug in WorldPopHandler/ADLSDataStore integration where a `Path` object was passed instead of a string, causing a `quote_from_bytes() expected bytes` error during download.

## [v0.6.6] - 2025-07-15

### Added

- **`AdminBoundaries.from_global_country_boundaries(scale="medium")`**
  - New class method to load global admin level 0 boundaries from Natural Earth.
  - Supports `"large"` (10m), `"medium"` (50m), and `"small"` (110m) scale options.

- **WorldPop Handler Refactor (API Integration)**
  - Introduced `WPPopulationHandler`, `WPPopulationConfig`, `WPPopulationDownloader`, and `WPPopulationReader`.
  - Uses new `WorldPopRestClient` to dynamically query the WorldPop REST API.
  - Replaces static metadata files and hardcoded logic with API-based discovery and download.
  - Country code lookup and dataset filtering now handled at runtime.
  - Improved validation, extensibility, logging, and error handling.

- **POI-Based WorldPop Mapping**
  - `PoiViewGenerator.map_wp_pop()` method:
    - Maps WorldPop population data around POIs using flexible spatial predicates:
      - `"centroid_within"`, `"intersects"`, `"fractional"` (1000m only), `"within"`
    - Supports configurable radius and resolution (100m or 1000m).
    - Aggregates population data and appends it to the view.

- **Geometry-Based Zonal WorldPop Mapping**
  - `GeometryBasedZonalViewGenerator.map_wp_pop()` method:
    - Maps WorldPop population data to polygons/zones using:
      - `"intersects"` or `"fractional"` predicate
    - Returns zonal population sums as a new view column.
    - Handles predicate-dependent data loading (raster vs. GeoDataFrame).

### Changed

- **Refactored `BaseHandler.ensure_data_available`**
  - More efficient data check and download logic.
  - Downloads only missing units unless `force_download=True`.
  - Cleaner structure and better reuse of `get_relevant_data_units()`.

- **Refactored WorldPop Module**
  - Complete handler redesign using API-based architecture.
  - Dataset paths and URLs are now dynamically constructed from API metadata.
  - Resolution/year validation is more robust and descriptive.
  - Removed static constants, gender/school_age toggles, and local CSV dependency.

### Fixed
- Several small fixes and improvements to zonal aggregation methods, especially around CRS consistency, missing values, and result alignment.

## [v0.6.5] - 2025-07-01

### Added

- **`MercatorTiles.get_quadkeys_from_points()`**  
  New static method for efficient 1:1 point-to-quadkey mapping using coordinate-based logic, improving performance over spatial joins.

- **`AdminBoundariesViewGenerator`**  
  New generator class for producing zonal views based on administrative boundaries (e.g., districts, provinces) with flexible source and admin level support.

- **Zonal View Generator Enhancements**  
  - `_view`: Internal attribute for accumulating mapped statistics.  
  - `view`: Exposes current state of zonal view.  
  - `add_variable_to_view()`: Adds mapped data from `map_points`, `map_polygons`, or `map_rasters` with robust validation and zone alignment.  
  - `to_dataframe()` and `to_geodataframe()` methods added for exporting current view in tabular or spatial formats.

- **`PoiViewGenerator` Enhancements**  
  - Consistent `_view` DataFrame for storing mapped results.  
  - `_update_view()`: Central method to update POI data.  
  - `save_view()`: Improved format handling (CSV, Parquet, GeoJSON, etc.) with geometry recovery.  
  - `to_dataframe()` and `to_geodataframe()` methods added for convenient export of enriched POI view.  
  - Robust duplicate ID detection and CRS validation in `map_zonal_stats`.

- **`TifProcessor` Enhancements**  
  - `sample_by_polygons_batched()`: Parallel polygon sampling.  
  - Enhanced `sample_by_polygons()` with nodata masking and multiple stats.  
  - `warn_on_error`: Flag to suppress sampling warnings.

- **GeoTIFF Multi-Band Support**  
  - `multi` mode added for multi-band raster support.  
  - Auto-detects band names via metadata.  
  - Strict validation of band count based on mode (`single`, `rgb`, `rgba`, `multi`).

- **Spatial Distance Graph Algorithm**  
  - `build_distance_graph()` added for fast KD-tree-based spatial matching.  
  - Supports both `DataFrame` and `GeoDataFrame` inputs.  
  - Outputs a `networkx.Graph` with optional DataFrame of matches.  
  - Handles projections, self-match exclusion, and includes verbose stats/logs.

- **Database Integration (Experimental)**  
  - Added `DBConnection` class in `core/io/database.py` for unified Trino and PostgreSQL access.  
  - Supports schema/table introspection, query execution, and reading into `pandas` or `dask`.  
  - Handles connection creation, credential management, and diagnostics.  
  - Utility methods for schema/view/table/column listings and parameterized queries.

- **GHSL Population Mapping**  
  - `map_ghsl_pop()` method added to `GeometryBasedZonalViewGenerator`.  
  - Aggregates GHSL population rasters to user-defined zones.  
  - Supports `intersects` and `fractional` predicates (latter for 1000m resolution only).  
  - Returns population statistics (e.g., `sum`) with customizable column prefix.

### Changed

- **`MercatorTiles.from_points()`** now internally uses `get_quadkeys_from_points()` for better performance.

- **`map_points()` and `map_rasters()`** now return `Dict[zone_id, value]` to support direct usage with `add_variable_to_view()`.

- **Refactored `aggregate_polygons_to_zones()`**  
  - `area_weighted` deprecated in favor of `predicate`.  
  - Supports flexible predicates like `"within"`, `"fractional"` for spatial aggregation.  
  - `map_polygons()` updated to reflect this change.

- **Optional Admin Boundaries Configuration**  
  - `ADMIN_BOUNDARIES_DATA_DIR` is now optional.  
  - `AdminBoundaries.create()` only attempts to load if explicitly configured or path is provided.  
  - Improved documentation and fallback behavior for missing configs.

### Fixed

- **GHSL Downloader**  
  - ZIP files are now downloaded into a temporary cache directory using `requests.get()`.  
  - Avoids unnecessary writes and ensures cleanup.

- **`TifProcessor`**  
  - Removed polygon sampling warnings unless explicitly enabled.

### Deprecated

- `TifProcessor.tabular` → use `to_dataframe()` instead.  
- `TifProcessor.get_zoned_geodataframe()` → use `to_geodataframe()` instead.  
- `area_weighted` → use `predicate` in aggregation methods instead.

## [v0.6.4] - 2025-06-19

### Added

- **GigaSchoolProfileFetcher**  
  - New class to fetch and process school profile data from the Giga School Profile API  
  - Supports paginated fetching, filtering by country and school ID  
  - Includes methods to generate connectivity summary statistics by region, connection type, and source

- **GigaSchoolMeasurementsFetcher**  
  - New class to fetch and process daily real-time connectivity measurements from the Giga API  
  - Supports filtering by date range and school  
  - Includes performance summary generation (download/upload speeds, latency, quality flags)

- **AdminBoundaries.from_geoboundaries**  
  - New class method to download and process geoBoundaries data by country and admin level  
  - Automatically handles HDX dataset discovery, downloading, and fallback logic

- **HDXConfig.search_datasets**  
  - Static method to search HDX datasets without full handler initialization  
  - Supports query string, sort order, result count, HDX site selection, and custom user agent

### Fixed

- Typo in `MaxarImageDownloader` causing runtime error

### Documentation

- **Improved Configuration Guide** (`docs/user-guide/configuration.md`)  
  - Added comprehensive table of environment variables with defaults and descriptions  
  - Synced `.env_sample` and `config.py` with docs  
  - Example `.env` file and guidance on path overrides using `config.set_path`  
  - New section on `config.ensure_directories_exist` and troubleshooting tips  
  - Clearer handling of credentials and security notes  
  - Improved formatting and structure for clarity

## [v0.6.3] - 2025-06-16

### Added
- Major refactor of `HDX` module to align with unified `BaseHandler` architecture:
  - `HDXConfig`: fully aligned with `BaseHandlerConfig` structure.
  - Added flexible pattern matching for resource filtering.
  - Improved data unit resolution by country, geometry, and points.
  - Enhanced resource filtering with exact and regex options.
- `HDXDownloader` fully aligned with `BaseHandlerDownloader`:
  - Simplified sequential download logic.
  - Improved error handling, validation, and logging.
- `HDXReader` fully aligned with `BaseHandlerReader`:
  - Added `resolve_source_paths` and `load_all_resources` methods.
  - Simplified source handling for single and multiple files.
  - Cleaned up redundant and dataset-specific logic.
- Introduced `HDXHandler` as unified orchestration layer using factory methods.

- Refactor of `RelativeWealthIndex (RWI)` module:
  - Added new `RWIHandler` class aligned with `HDXHandler` and `BaseHandler`.
  - Simplified class names: `RWIDownloader` and `RWIReader`.
  - Enhanced configuration with `latest_only` flag to select newest resources automatically.
  - Simplified resource filtering and country resolution logic.
  - Improved code maintainability, type hints, and error handling.


- **New raster multi-band support in TifProcessor:**
  - Added new `multi` mode for handling multi-band raster datasets.
  - Automatic band name detection from raster metadata.
  - Added strict mode validation (`single`, `rgb`, `rgba`, `multi`).
  - Enhanced error handling for invalid modes and band counts.

### Fixed
- Fixed GHSL tiles loading behavior for correct coordinate system handling:
  - Moved `TILES_URL` formatting and tile loading to `validate_configuration`.
  - Ensures proper tile loading after CRS validation.

### Documentation
- Updated and standardized API references across documentation.
- Standardized handler method names and usage examples.
- Added building enrichment examples for POI processing.
- Updated installation instructions.

### Deprecated
- Deprecated direct imports from individual handler modules.

## [v0.6.2] - 2025-06-11

### Added
- New `ROOT_DATA_DIR` configuration option to set a base directory for all data tiers
  - Can be configured via environment variable `ROOT_DATA_DIR` or `.env` file
  - Defaults to current directory (`.`) if not specified
  - All tier data paths (bronze, silver, gold, views) are now constructed relative to this root directory
  - Example: Setting `ROOT_DATA_DIR=/data/gigaspatial` will store all data under `/data/gigaspatial/bronze`, `/data/gigaspatial/silver`, etc.

### Fixed
- Fixed URL formatting in GHSL tiles by using Enum value instead of Enum member
  - Ensures consistent URL formatting with numeric values (4326) instead of Enum names (WGS84)
  - Fixes URL formatting issue across different Python environments

- Refactored GHSL downloader to follow DataStore abstraction
  - Directory creation is now handled by DataStore implementation
  - Removed redundant directory creation logic from download_data_unit method
  - Improves separation of concerns and makes the code more maintainable

## [v0.6.1] - 2025-06-09

### Fixed

- Gracefully handle missing or invalid GeoRepo API key in `AdminBoundaries.create()`:
  - Wrapped `GeoRepoClient` initialization in a `try-except` block
  - Added fallback to GADM if GeoRepo client fails
  - Improved logging for better debugging and transparency

## [v0.6.0] - 2025-06-09

### Added

#### POI View Generator
- **`map_zonal_stats`**: New method for enriched spatial mapping with support for:
  - Raster point sampling (value at POI location)
  - Raster zonal statistics (with buffer zone)
  - Polygon aggregation (with optional area-weighted averaging)
- **Auto-generated POI IDs** in `_init_points_gdf` for consistent point tracking.
- **Support for area-weighted aggregation** for polygon-based statistics.

#### BaseHandler Orchestration Layer
- New abstract `BaseHandler` class providing unified lifecycle orchestration for config, downloader, and reader.
- High-level interface methods:
  - `ensure_data_available()`
  - `load_data()`
  - `download_and_load()`
  - `get_available_data_info()`
- Integrated factory pattern for safe and standardized component creation.
- Built-in context manager support for resource cleanup.
- Fully backwards compatible with existing handler architecture.

#### Handlers Updated to Use BaseHandler
- `GoogleOpenBuildingsHandler`
- `MicrosoftBuildingsHandler`
- `GHSLDataHandler`
  - All now inherit from `BaseHandler`, supporting standardized behavior and cleaner APIs.

---

### Changed

#### POI View Generator
- `map_built_s` and `map_smod` now internally use the new `map_zonal_stats` method.
- `tif_processors` renamed to `data` to support both raster and polygon inputs.
- Removed parameters:
  - `id_column` (now handled internally)
  - `area_column` (now automatically calculated)

#### Internals and Usability
- Improved error handling with clearer validation messages.
- Enhanced logging for better visibility during enrichment.
- More consistent use of coordinate column naming.
- Refined type hints and parameter documentation across key methods.

---

### Notes

- Removed legacy POI generator classes and redundant `poi.py` file.
- Simplified imports and removed unused handler dependencies.
- All POI generator methods now include updated docstrings, parameter explanations, and usage examples.
- Added docs on the new `BaseHandler` interface and handler refactors.

## [v0.5.0] - 2025-06-02

### Changed

- **Refactored data loading architecture**:
  - Introduced **dedicated reader classes** for major datasets (Microsoft Global Buildings, Google Open Buildings, GHSL), each inheriting from a new `BaseHandlerReader`.
  - Centralized **file existence checks** and **raster/tabular loading** methods in `BaseHandlerReader`.
  - Improved maintainability by encapsulating dataset-specific logic inside each reader class.

- **Modularized source resolution**:
  - Each reader now supports resolving data **by country, geometry, or individual points**, improving code reuse and flexibility.

- **Unified POI enrichment**:
  - Merged all POI generators (Google Open Buildings, Microsoft Global Buildings, GHSL Built Surface, GHSL SMOD) into a single `PoiViewGenerator` class.
  - Supports flexible inputs: list of `(lat, lon)` tuples, list of dicts, DataFrame, or GeoDataFrame.
  - Maintains consistent internal state via `points_gdf`, updated after each mapping.
  - Enables **chained enrichment** of POI data using multiple datasets.

- **Modernized internal data access**:
  - All data loading now uses dedicated **handler/reader classes**, improving consistency and long-term maintainability.

### Fixed

- **Full DataStore integration**:
  - Fixed `OpenCellID` and `HDX` handlers to fully support the `DataStore` abstraction.
  - All file reads, writes, and checks now use the configured `DataStore` (local or cloud).
  - Temporary files are only used during downloads; final data is always stored and accessed via the DataStore interface.

### Removed

- Removed deprecated POI generator classes and the now-obsolete poi submodule. All enrichment is handled through the unified `PoiViewGenerator`.

### Notes

- This release finalizes the architectural refactors started in `v0.5.0`.
- While marked stable, please report any issues or regressions from the new modular structure.

## [v0.5.0b1] - 2025-05-27

### Added
- **New Handlers**:
  - `hdx.py`: Handler for downloading and managing Humanitarian Data Exchange datasets.
  - `rwi.py`: Handler for the Relative Wealth Index dataset.
  - `opencellid.py`: Handler for OpenCellID tower locations.
  - `unicef_georepo.py`: Integration with UNICEF’s GeoRepo asset repository.
- **Zonal Generators**:
  - Introduced the `generators/zonal/` module to support spatial aggregations of various data types (points, polygons, rasters)
    to zonal geometries such as grid tiles or catchment areas.
- **New Geo-Processing Methods**:
  - Added methods to compute centroids of (Multi)Polygon geometries.
  - Added methods to calculate area of (Multi)Polygon geometries in square meters.

### Changed
- **Refactored**:
  - `config.py`: Added support for new environment variables (OpenCellID and UNICEF GeoRepo keys).
  - `geo.py`: Enhanced spatial join functions for improved performance and clarity.
  - `handlers/`: 
    - Minor robustness improvements in `google_open_buildings` and `microsoft_global_buildings`.
    - Added a new class method in `boundaries` for initializing admin boundaries from UNICEF GeoRepo.
  - `core/io/`: 
    - Added `list_directories` method to both ADLS and local storage backends.
- **Documentation & Project Structure**:
  - Updated `.env_sample` and `.gitignore` to align with new environment variables and data handling practices.

### Dependencies
- Updated `requirements.txt` and `setup.py` to reflect new dependencies and ensure compatibility.

### Notes
- This is a **pre-release** (`v0.5.0b1`) and is intended for testing and feedback.
- Some new modules, especially in `handlers` and `generators`, are experimental and may be refined in upcoming releases.

## [v0.4.1] - 2025-04-17
### Added
- **Documentation**:
	- Added **API Reference** documentation for all modules, classes, and functions.
	- Added a **Configuration Guide** to explain how to set up paths, API keys, and other.
- **TifProcessor**: added new to_dataframe method.
- **config**: added set_path method for dynamic path management.

### Changed
- **Documentation**:
	- Restructured the `docs/` directory to improve organization and navigation.
	- Updated the `index.md` for the **User Guide** to provide a clear overview of available documentation.
	- Updated **Examples** for downloading, processing, and storing geospatial data - more to come.
- **README**:
	- Updated the README with a clear description of the package’s purpose and key features.
	- Added a section on **View Generators** to explain spatial context enrichment and mapping to grid or POI locations.
	- Included a **Supported Datasets** section with an image of dataset provider logos.

### Fixed
- Handled errors when processing nodes, relations, and ways in **OSMLocationFetcher**.
- Made `admin1` and `admin1_id_giga` optional in GigaEntity instances for countries with no admin level 1 divisions.

## [v0.4.0] - 2025-04-01
### Added
- **POI View Generators**: Introduced a new module, generators, containing a base class for POI view generation.
- **Expanded POI Support**: Added new classes for generating POI views from:
	- **Google Open Buildings**
	- **Microsoft Global Buildings**
	- **GHSL Settlement Model**
	- **GHSL Built Surface**
- **New Reader**: Added read_gzipped_json_or_csv to handle compressed JSON/CSV files.

### Changed
- **ADLSDataStore Enhancements**: Updated methods to match LocalDataStore for improved consistency.
- **Geo Processing Updates**:
	- Improved convert_to_dataframe for more efficient data conversion.
	- Enhanced annotate_with_admin_regions to improve spatial joins.
- **New TifProcessor Methods**:
	- sample_by_polygons for polygon-based raster sampling.
	- sample_multiple_tifs_by_coordinates & sample_multiple_tifs_by_polygons to manage multi-raster sampling.
- **Fixed Global Config Handling**: Resolved issues with handling configurations inside classes.

## [v0.3.2] - 2025-03-21
### Added
- Added a method to efficiently assign unique IDs to features.

### Changed
- Enhanced logging for better debugging and clarity.

### Fixed
- Minor bug fix in config.py

## [0.3.1] - 2025-03-20

### Added
- Enhanced AdminBoundaries handler with improved error handling for cases where administrative level data is unavailable for a country.
- Added pyproject.toml and setup.py, enabling pip install support for the package.
- Introduced a new method annotate_with_admin_regions in geo.py to perform spatial joins between input points and administrative boundaries (levels 1 and 2), handling conflicts where points intersect multiple admin regions.

### Removed
- Removed the utils module containing logger.py and integrated LOG_FORMAT and get_logger into config.py for a more streamlined logging approach.

## [0.3.0] - 2025-03-18  
### Added  
- Compression support in readers for improved efficiency  
- New GHSL data handler to manage GHSL dataset downloads  

### Fixed  
- Small fixes/improvements in Microsoft Buildings, Maxar, and Overture handlers  

## [v0.2.2] - 2025-03-12
- **Refactored Handlers**: Improved structure and performance of maxar_image.py, osm.py and overture.py to enhance geospatial data handling.

- **Documentation Improvements**:
	- Updated index.md, advanced.md, and use-cases.md for better clarity.
	- Added installation.md under docs/getting-started for setup guidance.
	- Refined API documentation in docs/api/index.md.

- **Configuration & Setup Enhancements**:
	•	Improved .gitignore to exclude unnecessary files.
	•	Updated mkdocs.yml for better documentation structuring.
- **Bug Fixes & Minor Optimizations**: Small fixes and improvements across the codebase for stability and maintainability.

## [v0.2.1] - 2025-02-28
### Added
- Introduced WorldPopDownloader feature to handlers
- Refactored TifProcessor class for better performance

### Fixed
- Minor bug fixes and performance improvements

## [v0.2.0] - MaxarImageDownloader & Bug Fixes - 2025-02-24
- **New Handler**: MaxarImageDownloader for downloading Maxar images.
- **Bug Fixes**: Various improvements and bug fixes.
- **Enhancements**: Minor optimizations in handlers.

## [v0.1.1] - 2025-02-24

### Added
- **Local Data Store**: Introduced a new local data store alongside ADLS to improve data storage and read/write functionality.
- **Boundaries Handler**: Added `boundaries.py`, a new handler that allows to read administrative boundaries from GADM.

### Changed
- **Handler Refactoring**: Refactored existing handlers to improve modularity and data handling.
- **Configuration Management**: Added `config.py` to manage paths, runtime settings, and environment variables.

### Removed
- **Administrative Schema**: Removed `administrative.py` since its functionality is now handled by the `boundaries` handler.
- **Globals Module**: Removed `globals.py` and replaced it with `config.py` for better configuration management.

### Updated Files
- `config.py`
- `boundaries.py`
- `google_open_buildings.py`
- `mapbox_image.py`
- `microsoft_global_buildings.py`
- `ookla_speedtest.py`
- `mercator_tiles.py`
- `adls_data_store.py`
- `data_store.py`
- `local_data_store.py`
- `readers.py`
- `writers.py`
- `entity.py`

## [v0.1.0] - 2025-02-07
### Added
- New data handlers: `google_open_buildings.py`, `microsoft_global_buildings.py`, `overture.py`, `mapbox_image.py`, `osm.py`
- Processing functions in `tif_processor.py`, `geo.py` and `transform.py`
- Grid generation modules: `h3_tiles.py`, `mercator_tiles.py`
- View managers: `grid_view.py` and `national_view.py`
- Schemas: `administrative.py`

### Changed
- Updated `requirements.txt` with new dependencies
- Improved logging and data storage mechanisms

### Removed
- Deprecated views: `h3_view.py`, `mercator_view.py`
