facetorch.analyzer.detector.post API documentation

# Module `facetorch.analyzer.detector.post`

## Classes

`class BaseDetPostProcessor (transform: torchvision.transforms.transforms.Compose, device: torch.device, optimize_transform: bool)`

  Base class for detector post processors.

  All detector post processors should subclass it. All subclass should overwrite:

  - Methods:`run`, used for running the processing

  ## Args

  `device` :`torch.device`
    Torch device cpu or cuda.
  `transform` :`transforms.Compose`
    Transform compose object to be applied to the image.
  `optimize_transform` :`bool`
    Whether to optimize the transform.
  Expand source code

      class BaseDetPostProcessor(BaseProcessor):
          @Timer(
              "BaseDetPostProcessor.__init__",
              "{name}: {milliseconds:.2f} ms",
              logger=logger.debug,
          )
          def __init__(
              self,
              transform: transforms.Compose,
              device: torch.device,
              optimize_transform: bool,
          ):
              """Base class for detector post processors.

              All detector post processors should subclass it.
              All subclass should overwrite:

              - Methods:``run``, used for running the processing

              Args:
                  device (torch.device): Torch device cpu or cuda.
                  transform (transforms.Compose): Transform compose object to be applied to the image.
                  optimize_transform (bool): Whether to optimize the transform.

              """
              super().__init__(transform, device, optimize_transform)

          @abstractmethod
          def run(
              self, data: ImageData, logits: Union[torch.Tensor, Tuple[torch.Tensor]]
          ) -> ImageData:
              """Abstract method that runs the detector post processing functionality
              and returns the data object.

              Args:
                  data (ImageData): ImageData object containing the image tensor.
                  logits (Union[torch.Tensor, Tuple[torch.Tensor]]): Output of the detector model.

              Returns:
                  ImageData: Image data object with Detection tensors and detected Face objects.

              """

  ### Ancestors

  - BaseProcessor

  ### Subclasses

  - PostRetFace

  ### Methods

  `def run(self, data: ImageData, logits: Union[torch.Tensor, Tuple[torch.Tensor]]) ‑> ImageData`

    Abstract method that runs the detector post processing functionality and returns the data object.

    ## Args

    `data` :`ImageData`
      ImageData object containing the image tensor.
    `logits` :`Union[torch.Tensor, Tuple[torch.Tensor]]`
      Output of the detector model.

    ## Returns

    `ImageData`
      Image data object with Detection tensors and detected Face objects.

  ### Inherited members

  - `BaseProcessor`:
    - `optimize`
`class PriorBox (min_sizes: List[List[int]], steps: List[int], clip: bool)`

  PriorBox class for generating prior boxes.

  ## Args

  `min_sizes` :`List[List[int]]`
    List of list of minimum sizes for each feature map.
  `steps` :`List[int]`
    List of steps for each feature map.
  `clip` :`bool`
    Whether to clip the prior boxes to the image boundaries.
  Expand source code

      class PriorBox:
          """
          PriorBox class for generating prior boxes.

          Args:
              min_sizes (List[List[int]]): List of list of minimum sizes for each feature map.
              steps (List[int]): List of steps for each feature map.
              clip (bool): Whether to clip the prior boxes to the image boundaries.
          """

          def __init__(self, min_sizes: List[List[int]], steps: List[int], clip: bool):
              self.min_sizes = [list(min_size) for min_size in min_sizes]
              self.steps = list(steps)
              self.clip = clip

          def forward(self, dims: Dimensions) -> torch.Tensor:
              """Generate prior boxes for each feature map.

              Args:
                  dims (Dimensions): Dimensions of the image.

              Returns:
                  torch.Tensor: Tensor of prior boxes.
              """
              feature_maps = [
                  [ceil(dims.height / step), ceil(dims.width / step)] for step in self.steps
              ]
              anchors = []
              for k, f in enumerate(feature_maps):
                  min_sizes = self.min_sizes[k]
                  for i, j in product(range(f[0]), range(f[1])):
                      for min_size in min_sizes:
                          s_kx = min_size / dims.width
                          s_ky = min_size / dims.height
                          dense_cx = [x * self.steps[k] / dims.width for x in [j + 0.5]]
                          dense_cy = [y * self.steps[k] / dims.height for y in [i + 0.5]]
                          for cy, cx in product(dense_cy, dense_cx):
                              anchors.append([cx, cy, s_kx, s_ky])

              output = torch.Tensor(anchors)
              if self.clip:
                  output.clamp_(min=0, max=1)
              return output

  ### Methods

  `def forward(self, dims: Dimensions) ‑> torch.Tensor`

    Generate prior boxes for each feature map.

    ## Args

    `dims` :`Dimensions`
      Dimensions of the image.

    ## Returns

    `torch.Tensor`
      Tensor of prior boxes.
`class PostRetFace (transform: torchvision.transforms.transforms.Compose, device: torch.device, optimize_transform: bool, confidence_threshold: float, top_k: int, nms_threshold: float, keep_top_k: int, score_threshold: float, prior_box: PriorBox, variance: List[float], reverse_colors: bool = False, expand_box_ratio: float = 0.0)`

  Initialize the detector postprocessor. Modified from https://github.com/biubug6/Pytorch_Retinaface.

  ## Args

  `transform` :`Compose`
    Composed Torch transform object.
  `device` :`torch.device`
    Torch device cpu or cuda.
  `optimize_transform` :`bool`
    Whether to optimize the transform.
  `confidence_threshold` :`float`
    Confidence threshold for face detection.
  `top_k` :`int`
    Top K faces to keep before NMS.
  `nms_threshold` :`float`
    NMS threshold.
  `keep_top_k` :`int`
    Keep top K faces after NMS.
  `score_threshold` :`float`
    Score threshold for face detection.
  `prior_box` :`PriorBox`
    PriorBox object.
  `variance` :`List[float]`
    Prior box variance.
  `reverse_colors` :`bool`
    Whether to reverse the colors of the image tensor from RGB to BGR or vice versa. If False, the colors remain unchanged. Default: False.
  `expand_box_ratio` :`float`
    Expand the box by this ratio. Default: 0.0.
  Expand source code

      class PostRetFace(BaseDetPostProcessor):
          @Timer("PostRetFace.__init__", "{name}: {milliseconds:.2f} ms", logger=logger.debug)
          def __init__(
              self,
              transform: transforms.Compose,
              device: torch.device,
              optimize_transform: bool,
              confidence_threshold: float,
              top_k: int,
              nms_threshold: float,
              keep_top_k: int,
              score_threshold: float,
              prior_box: PriorBox,
              variance: List[float],
              reverse_colors: bool = False,
              expand_box_ratio: float = 0.0,
          ):
              """Initialize the detector postprocessor. Modified from https://github.com/biubug6/Pytorch_Retinaface.

              Args:
                  transform (Compose): Composed Torch transform object.
                  device (torch.device): Torch device cpu or cuda.
                  optimize_transform (bool): Whether to optimize the transform.
                  confidence_threshold (float): Confidence threshold for face detection.
                  top_k (int): Top K faces to keep before NMS.
                  nms_threshold (float): NMS threshold.
                  keep_top_k (int): Keep top K faces after NMS.
                  score_threshold (float): Score threshold for face detection.
                  prior_box (PriorBox): PriorBox object.
                  variance (List[float]): Prior box variance.
                  reverse_colors (bool): Whether to reverse the colors of the image tensor from RGB to BGR or vice versa. If False, the colors remain unchanged. Default: False.
                  expand_box_ratio (float): Expand the box by this ratio. Default: 0.0.
              """
              super().__init__(transform, device, optimize_transform)
              self.confidence_threshold = confidence_threshold
              self.top_k = top_k
              self.nms_threshold = nms_threshold
              self.keep_top_k = keep_top_k
              self.score_threshold = score_threshold
              self.prior_box = prior_box
              self.variance = list(variance)
              self.reverse_colors = reverse_colors
              self.expand_box_ratio = expand_box_ratio

          @Timer("PostRetFace.run", "{name}: {milliseconds:.2f} ms", logger=logger.debug)
          def run(
              self,
              data: ImageData,
              logits: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor, torch.Tensor]],
          ) -> ImageData:
              """Run the detector postprocessor.

              Args:
                  data (ImageData): ImageData object containing the image tensor.
                  logits (Union[torch.Tensor, Tuple[torch.Tensor]]): Output of the detector model.

              Returns:
                  ImageData: Image data object with detection tensors and detected Face objects.
              """
              data.det = Detection(loc=logits[0], conf=logits[1], landmarks=logits[2])

              if self.reverse_colors:
                  data.tensor = rgb2bgr(data.tensor)

              data = self._process_dets(data)
              data = self._extract_faces(data)
              return data

          def _process_dets(self, data: ImageData) -> ImageData:
              """Compute the detections and add them to the data detector.

              Args:
                  data (ImageData): Image data with with locations and confidences from detector.

              Returns:
                  ImageData: Image data object with detections.
              """

              def _decode(
                  _loc: torch.Tensor, _priors: torch.Tensor, variances: List[float]
              ) -> torch.Tensor:
                  _boxes = torch.cat(
                      (
                          _priors[:, :2] + _loc[:, :2] * variances[0] * _priors[:, 2:],
                          _priors[:, 2:] * torch.exp(_loc[:, 2:] * variances[1]),
                      ),
                      1,
                  )
                  _boxes[:, :2] -= _boxes[:, 2:] / 2
                  _boxes[:, 2:] += _boxes[:, :2]
                  return _boxes

              def _extract_boxes(_loc: torch.Tensor) -> torch.Tensor:
                  priors = self.prior_box.forward(data.dims)
                  priors = priors.to(self.device)
                  prior_data = priors.data
                  _boxes = _decode(_loc.data.squeeze(0), prior_data, self.variance)
                  img_scale = torch.Tensor([data.dims.width, data.dims.height]).repeat(2)
                  _boxes = _boxes * img_scale.to(self.device)
                  return _boxes

              def _nms(dets: torch.Tensor, thresh: float) -> torch.Tensor:
                  """Non-maximum suppression."""
                  x1 = dets[:, 0]
                  y1 = dets[:, 1]
                  x2 = dets[:, 2]
                  y2 = dets[:, 3]

                  areas = (x2 - x1 + 1) * (y2 - y1 + 1)
                  order = torch.arange(dets.shape[0], device=self.device)

                  zero_tensor = torch.tensor(0.0).to(self.device)
                  keep = []
                  while order.size()[0] > 0:
                      i = order[0]
                      keep.append(i)
                      xx1 = torch.maximum(x1[i], x1[order[1:]])
                      yy1 = torch.maximum(y1[i], y1[order[1:]])
                      xx2 = torch.minimum(x2[i], x2[order[1:]])
                      yy2 = torch.minimum(y2[i], y2[order[1:]])

                      w = torch.maximum(zero_tensor, xx2 - xx1 + 1)
                      h = torch.maximum(zero_tensor, yy2 - yy1 + 1)
                      inter = torch.multiply(w, h)
                      ovr = inter / (areas[i] + areas[order[1:]] - inter)

                      inds = ovr <= thresh
                      order = order[1:][inds]

                  if len(keep) > 0:
                      keep = torch.stack(keep)
                  else:
                      keep = torch.tensor([])

                  return keep

              def _extract_dets(_conf: torch.Tensor, _boxes: torch.Tensor) -> torch.Tensor:
                  scores = _conf.squeeze(0).data[:, 1]
                  # ignore low scores
                  inds = scores > self.confidence_threshold
                  _boxes = _boxes[inds]
                  scores = scores[inds]
                  # keep top-K before NMS
                  order = torch.argsort(scores, descending=True)[: self.top_k]
                  _boxes = _boxes[order]
                  scores = scores[order]
                  # do NMS
                  _dets = torch.hstack((_boxes, scores.unsqueeze(1)))
                  keep = _nms(_dets, self.nms_threshold)

                  if not keep.shape[0] == 0:
                      _dets = _dets[keep, :]
                      # keep top-K after NMS
                      _dets = _dets[: self.keep_top_k, :]
                      # keep dets with score > score_threshold
                      _dets = _dets[_dets[:, 4] > self.score_threshold]

                  return _dets

              data.det.boxes = _extract_boxes(data.det.loc)
              data.det.dets = _extract_dets(data.det.conf, data.det.boxes)
              return data

          def _extract_faces(self, data: ImageData) -> ImageData:
              """Extracts the faces from the original image using the detections.

              Args:
                  data (ImageData): Image data with image tensor and detections.

              Returns:
                  ImageData: Image data object with extracted faces.

              """

              def _get_coordinates(_det: torch.Tensor) -> Location:
                  _det = torch.round(_det).int()
                  loc = Location(
                      x1=int(_det[0]),
                      y1=int(_det[1]),
                      x2=int(_det[2]),
                      y2=int(_det[3]),
                  )

                  loc.expand(amount=self.expand_box_ratio)
                  loc.form_square()

                  return loc

              for indx, det in enumerate(data.det.dets):
                  loc = _get_coordinates(det)
                  face_tensor = data.tensor[0, :, loc.y1 : loc.y2, loc.x1 : loc.x2]
                  dims = Dimensions(face_tensor.shape[-2], face_tensor.shape[-1])
                  size_img = data.tensor.shape[-2] * data.tensor.shape[-1]
                  size_ratio = (dims.height * dims.width) / size_img

                  if not any([dim == 0 for dim in face_tensor.shape]):
                      face = Face(
                          indx=indx, loc=loc, tensor=face_tensor, dims=dims, ratio=size_ratio
                      )
                      data.faces.append(face)

              return data

  ### Ancestors

  - BaseDetPostProcessor
  - BaseProcessor

  ### Methods

  `def run(self, data: ImageData, logits: Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor, torch.Tensor]]) ‑> ImageData`

    Run the detector postprocessor.

    ## Args

    `data` :`ImageData`
      ImageData object containing the image tensor.
    `logits` :`Union[torch.Tensor, Tuple[torch.Tensor]]`
      Output of the detector model.

    ## Returns

    `ImageData`
      Image data object with detection tensors and detected Face objects.

  ### Inherited members

  - `BaseDetPostProcessor`:
    - `optimize`