---
jupyter: python3
title: "Using `ez_zarr` to explore Fractal outputs"
author: Silvia Barbiero, Michael Stadler
---

# Goal
The aim of `ez_zarr` is to provide easy, high-level access to high content screening microscopy data, stored as OME-Zarr filesets according to the [NGFF](https://ngff.openmicroscopy.org/latest/) specifications with additional metadata fields, for example the ones generated by the [Fractal](https://fractal-analytics-platform.github.io/fractal-tasks-core/) platform.

The goal is that users can write simple scripts working with plates, wells and fields of view, without having to understand how these are represented within an OME-Zarr fileset.

# Installation
`ez_zarr` is hosted on GitHub at [github.com/fmicompbio/ez_zarr](https://github.com/fmicompbio/ez_zarr) and the documentation of can be found at [fmicompbio.github.io/ez_zarr](https://fmicompbio.github.io/ez_zarr/).

The release version of `ez_zarr` can be installed using:
```{.bash}
pip install ez-zarr
```

The current (development) `ez_zarr` can be installed from github.com using:
```{.bash}
pip install git+ssh://git@github.com/fmicompbio/ez_zarr.git
```

# Quickstart
Here are some examples of how you can use `ez_zarr`:

## Download example data
To run the code below, you will first need to download some Fractal output data.
You can do this for example by running the following from a terminal:
```{.bash}
# Download small Fractal output from Zenodo
curl -o temp.zip https://zenodo.org/records/10519143/files/20200812-CardiomyocyteDifferentiation14-Cycle1_mip.zarr.zip

# Unzip and remove the temporary zip file
unzip temp.zip
rm temp.zip
```

or directly from within a python notebook or session using:
```{python}
import requests
import zipfile
import os

# for info on data see: https://zenodo.org/records/10257149
url = 'https://zenodo.org/records/10519143/files/20200812-CardiomyocyteDifferentiation14-Cycle1_mip.zarr.zip'

# download
response = requests.get(url)
with open('temp.zip', 'wb') as temp_zip:
    temp_zip.write(response.content)

# unzip to current directory
with zipfile.ZipFile('temp.zip', 'r') as zip_ref:
    zip_ref.extractall('.')

# clean up
os.remove('temp.zip')
```

## Load packages
```{python}
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from matplotlib.colors import ListedColormap
import numpy as np
from skimage import color

from ez_zarr import hcs_wrappers, plotting
```

## Open OME-Zarr file set
We open a single OME-Zarr directory, typically representing a single plate:
```{python}
zarr_directory = '20200812-CardiomyocyteDifferentiation14-Cycle1_mip.zarr'
plate = hcs_wrappers.FractalZarr(zarr_directory, name = 'cardio-d14')
plate
```

`plate` represents a maximum intensity projection of an acquisition with 1 wells and 1 channel.

There are 5 `pyramid_levels` available (0 to 4), meaning that in addition to the full resolution data (level 0), we have four more levels that provide the data in 2-fold lower resolutions (see `pyramid_zyx_scalefactor`), for example for faster plotting.

This OME-Zarr fileset also contains a segmentation (`nuclei`), for which measurements ahve been extracted and saved in table `nuclei`. In addition, there are some further tables, such as `FOV_ROI_table`, which contains the coordinates of the fields of view, or `nuclei_ROI_table`, which contains the bounding box coordinates for the segmented nuclei.

## Get information from `plate`
You can obtain information on the content available in the fileset:
```{python}
# path to the OME-Zarr fileset
plate.get_path()
```
```{python}
# available channels
plate.get_channels()
```
```{python}
# available wells
plate.get_wells()
plate.get_wells(simplify=True)
```
```{python}
# available labels
plate.get_label_names()
```
```{python}
# available tables
plate.get_table_names()
```
```{python}
# zyx pixel spacing in micrometers for pyramid levels
# ... for images
plate.level_zyx_spacing_images
# ... for labels
plate.level_zyx_spacing_labels
```

## Extract a table from the OME-Zarr fileset
The tables are stored within the fileset as describe in the [Fractal documentation](https://fractal-analytics-platform.github.io/fractal-tasks-core/tables/).

As mentioned the goal of `ez_zarr` is to abstract the internal structure and make it simple to obtain these tables, if necessary accumulated over many wells, as a `pandas.DataFrame` :
```{python}
#| output: false
df = plate.get_table(table_name='FOV_ROI_table')
df
```
```{python}
#| echo: false
#| output: asis

# output table as md (instead of html to better play with mkdocs)
print(df.to_markdown())
```

or `anndata.AnnData` object:
```{python}
ann = plate.get_table(table_name='FOV_ROI_table', as_AnnData=True)
ann
```

## Get an overview of a full well
`get_image_ROI()` extracts a rectangular region of interest (ROI)
from a well image by coordinates. If no coordinates or
other arguments are given, it returns the whole well at the
lowest available resolution:
```{python}
my_well = plate.get_wells(simplify=True)[0]
print(my_well)

img = plate.get_image_ROI()
print(img.shape) # (ch, z, y, x)
```

Such an image array could be easily plotted using `matplotlib.pyplot.imshow`:
```{python}
#| eval: false
with plt.style.context('dark_background'):
    fig = plt.figure(figsize=(4, 4))
    fig.set_dpi(100)

    plt.imshow(img[0,0], cmap='gray', vmin=100, vmax=600)
    plt.title(plate.name + ' ' + my_well)
    plt.show()
    plt.close()
```

With `plot_well()`, `ez_zarr` provides a conventien function for this use case that
automatically extracts the well image and plots it:
```{python}
plate.plot_well(well=my_well,
                channels=[0],
                channel_colors=['white'],
                channel_ranges=[[100, 600]],
                title=plate.name + ' ' + my_well,
                scalebar_micrometer=100,
                scalebar_color='yellow',
                scalebar_position='topleft',
                fig_width_inch=5,
                fig_height_inch=4,
                fig_dpi=100)
```

`plot_well()` and also `plot_plate()` use internally `ez_zarr.plotting.plot_image()`
which is a convenient wrapper around `matplotlib.pyplot.imshow`.
`plot_image()` can also be used directly if you want to work directly with image arrays
(for examples, see section "Working with segmentation masks" below).

## Get salient regions at higher resolution
`get_image_grid_ROIs()` allows you to split the well image into a rectangular
grid of `num_x`-by-`num_y` regions, and automatically select the `num_select`
top regions based on a given property (`sample_method`).

Here, we pick three regions with the highest signal in channel zero (DAPI):
```{python}
coord_list, img_list = plate.get_image_grid_ROIs(well = my_well,
                                                 pyramid_level=1,
                                                 num_x=10, num_y=5, num_select=3,
                                                 sample_method='sum',
                                                 channel=0)
coord_list # (y_start, y_end, x_start, x_end)
len(img_list)

with plt.style.context('dark_background'):
    fig = plt.figure(figsize=(9,3))
    fig.set_dpi(100)
    for i in range(len(img_list)):
        plt.subplot(1, 3, i + 1)
        plt.imshow(img_list[i][0,0], cmap='gray', vmin=100, vmax=600)
        plt.axis('off')
        plt.title(my_well + ' ' + str(coord_list[i]))
    plt.show()
    plt.close()
```

## Zoom in
Alternatively, we can zoom in by coordinates:
```{python}
img = plate.get_image_ROI(well=my_well,
                          upper_left_yx=(50, 40),
                          lower_right_yx=(120, 90))
print(img.shape)

with plt.style.context('dark_background'):
    fig = plt.figure(figsize=(4, 4))
    fig.set_dpi(100)

    plt.imshow(img[0,0], cmap='gray', vmin=100, vmax=600)
    plt.title(plate.name + ' B03')
    plt.show()
    plt.close()
```

The above automatically uses the lowest resolution (highest pyramid level)
version of the data, and the pixel coordinates refer to that space.

If we want to get the same image at a higher resolution (`pyramid_level=1`),
we need to convert them using the `convert_pixel_to_pixel()` function:
```{python}
# convert coordinates from lowest-resolution (None, here: 4) to
#   highest-resolution (0) pyramid level
# remark: coordinates are (z,y,x)
new_upper_left = plate.convert_pixel_to_pixel(zyx = (0, 50, 40),
                                              pyramid_level_from = None,
                                              pyramid_level_to=1)
new_lower_right = plate.convert_pixel_to_pixel(zyx = (0, 120, 90),
                                               pyramid_level_from = None,
                                               pyramid_level_to=1)

# extract image
img = plate.get_image_ROI(well=my_well,
                          upper_left_yx=new_upper_left[1:],
                          lower_right_yx=new_lower_right[1:],
                          pyramid_level=1)
print(img.shape)

with plt.style.context('dark_background'):
    fig = plt.figure(figsize=(4, 4))
    fig.set_dpi(100)

    plt.imshow(img[0,0], cmap='gray', vmin=100, vmax=600)
    plt.title(plate.name + ' ' + my_well)
    plt.show()
    plt.close()
```

## Coordinate conversions
As mentioned the pixel coordinates depend on the `pyramid_level`, but as
the scaling factor between pyramid levels and also the pixel sizes in
micrometers are known, it's easy to convert between them:
```{python}
zyx = (1, 230, 180) # highest pyramid level (4)
print('starting coordinates (pyramid_level=4): ' + str(zyx))

# convert from pixel to micrometers
zyx_um = plate.convert_pixel_to_micrometer(zyx, pyramid_level=4)
print('in micrometer: ' + str(zyx_um))

# convert from micrometers to pixels
zyx_px = plate.convert_micrometer_to_pixel(zyx_um, pyramid_level=4)
print('back in pixel: ' + str(zyx_px))

# convert pixels between pyramid levels
zyx_px_0 = plate.convert_pixel_to_pixel(zyx_px, pyramid_level_from=4, pyramid_level_to=0)
print('in pixel (pyramid_level=0): ' + str(zyx_px_0))
```

This makes it quite easy to add a scalebar to an image:
```{python}
# calculate the pixel length of a scalebar
scalebar_micrometer = 30
scalebar_pixels = plate.convert_micrometer_to_pixel(zyx = (0, 0, scalebar_micrometer),
                                                    pyramid_level=1)[2]
print(str(scalebar_micrometer) + ' µm correspond to ' + str(scalebar_pixels) + ' pixel in x')

with plt.style.context('dark_background'):
    fig = plt.figure(figsize=(4, 4))
    fig.set_dpi(100)

    plt.imshow(img[0,0], cmap='gray', vmin=100, vmax=600)
    plt.title(plate.name + ' ' + my_well)
    # add scalebar
    # patches.Rectangle((x, y), width, height, edgecolor, facecolor)
    rect = patches.Rectangle((30, 30), scalebar_pixels, 6, edgecolor='yellow', facecolor='yellow')
    ax = plt.gca() # get current axes
    ax.add_patch(rect) # add the patch to the Axes
    # plt.text(x, y, 'Your text', color, ha='center')
    plt.text(30 + scalebar_pixels / 2, 45, str(scalebar_micrometer) + ' µm', color='yellow', ha='center', va='top')
    # show image
    plt.show()
    plt.close()
```

## Working with segmentation masks
When segmentation masks (labels) are available, we can also extract them.
For example, we can plot the labels with transparency on top of the image:
```{python}
# select which object to plot
label = 'nuclei'
img_pl = 2  # image pyramid level

# find corresponding mask pyramid level
img_name = plate.image_names[0]
pl_match = [plate.level_zyx_spacing_images[img_name][img_pl] == x for x in plate.level_zyx_spacing_labels[label]]
assert sum(pl_match) == 1
msk_pl = pl_match.index(True)

# load image and corresponding mask
img = plate.get_image_ROI(well=my_well, pyramid_level=img_pl,
                          upper_left_yx=(200, 160), lower_right_yx=(480, 360))
msk = plate.get_label_ROI(label_name=label, well=my_well, pyramid_level=msk_pl,
                          upper_left_yx=(200, 160), lower_right_yx=(480, 360))
assert img[0].shape == msk.shape

# plot image and mask
plotting.plot_image(im=img, msk=msk, msk_alpha=0.3,
                    channels=[0], channel_colors=['white'], channel_ranges=[[100,600]],
                    fig_width_inch=6, fig_height_inch=6, fig_dpi=100)
```

Using the tables, we can get the bounding box coordinates in micrometers from
`nuclei_ROI_table` (here just showing the rows 100 to 110):
```{python}
#| output: false
nucl = plate.get_table('nuclei_ROI_table')
nucl.iloc[100:111]
```
```{python}
#| echo: false
#| output: asis

# output table as md (instead of html to better play with mkdocs)
print(nucl.head(n=10).to_markdown())
```

We can plot the image region corresponding to a single segmented object,
for example the nucleus at index 106:
```{python}
# select which object to plot
idx = 106 # index of the object in `nucl`

img_nucl = plate.get_image_table_idx(table_name='nuclei_ROI_table',
                                     table_idx=idx,
                                     well=my_well,
                                     pyramid_level=img_pl,
                                     as_NumPy=True)
msk_nucl = plate.get_label_table_idx(label_name='nuclei',
                                     table_name='nuclei_ROI_table',
                                     table_idx=idx,
                                     well=my_well,
                                     pyramid_level=msk_pl,
                                     as_NumPy=True)
assert img_nucl.shape[1:] == msk_nucl.shape

# plot 
plotting.plot_image(im=img_nucl, msk=msk_nucl, msk_alpha=0.3,
                    channels=[0], channel_colors=['white'], channel_ranges=[[100,600]],
                    fig_width_inch=4, fig_height_inch=4, fig_dpi=100)
```


## Calculations on the image data
To save memory, images are stored on disk and only loaded when needed (for implementation details see the [`dask` Array documentation](https://docs.dask.org/en/stable/array.html)).

This can be demonstrated by `type(img)`. To force loading of the data, you can call `.compute()`:
```{python}
print(type(img)) # note that the image is an 'on-disk' array
print(type(img.compute())) # ... and can be loaded into memory using .compute()
```

In general, you can use `dask` arrays like `numpy` arrays.
For example, we can threshold the image: 
```{python}
# threshold
img_mask = img[0,0]>300
# ... and compute to numpy array
img_mask = img_mask.compute()

# plot 
plotting.plot_image(im=img[:,0], msk=img_mask, msk_alpha=0.5,
                    fig_width_inch=6, fig_height_inch=6, fig_dpi=100)
```

This is of course not a recommended way to segment this image, but rather meant
to demonstrate that the `dask` arrays can be used just like `numpy` arrays.

# Session info
```{python}
import session_info
session_info.show()
```