# Inline TODOs


# HACK

## [`.meta/scripts/make_docs.py`](/.meta/scripts/make_docs.py)

- this is kind of fragile  
  local link: [`/.meta/scripts/make_docs.py:152`](/.meta/scripts/make_docs.py#L152) 
  | view on GitHub: [.meta/scripts/make_docs.py#L152](https://github.com/mivanit/zanj/blob/main/.meta/scripts/make_docs.py#L152)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=this%20is%20kind%20of%20fragile&body=%23%20source%0A%0A%5B%60.meta%2Fscripts%2Fmake_docs.py%23L152%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2F.meta%2Fscripts%2Fmake_docs.py%23L152%29%0A%0A%23%20context%0A%60%60%60python%0A%09%09%22%22%22name%20of%20the%20module%2C%20which%20is%20the%20package%20name%20with%20%27-%27%20replaced%20by%20%27_%27%0A%0A%09%09HACK%3A%20this%20is%20kind%20of%20fragile%0A%09%09%22%22%22%0A%09%09return%20self.package_name.replace%28%22-%22%2C%20%22_%22%29%0A%60%60%60&labels=HACK)

  ```python
  """name of the module, which is the package name with '-' replaced by '_'

  HACK: this is kind of fragile
  """
  return self.package_name.replace("-", "_")
  ```





# TODO

## [`.meta/scripts/docs_clean.py`](/.meta/scripts/docs_clean.py)

- this is not recursive  
  local link: [`/.meta/scripts/docs_clean.py:71`](/.meta/scripts/docs_clean.py#L71) 
  | view on GitHub: [.meta/scripts/docs_clean.py#L71](https://github.com/mivanit/zanj/blob/main/.meta/scripts/docs_clean.py#L71)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=this%20is%20not%20recursive&body=%23%20source%0A%0A%5B%60.meta%2Fscripts%2Fdocs_clean.py%23L71%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2F.meta%2Fscripts%2Fdocs_clean.py%23L71%29%0A%0A%23%20context%0A%60%60%60python%0A%09%22%22%22delete%20files%20not%20in%20preserved%20set%0A%0A%09TODO%3A%20this%20is%20not%20recursive%0A%09%22%22%22%0A%09for%20path%20in%20docs_dir.iterdir%28%29%3A%0A%60%60%60&labels=enhancement)

  ```python
  """delete files not in preserved set

  TODO: this is not recursive
  """
  for path in docs_dir.iterdir():
  ```




## [`tests/unit/no_torch/test_load_item_recursive.py`](/tests/unit/no_torch/test_load_item_recursive.py)

- this doesn't raise any errors  
  local link: [`/tests/unit/no_torch/test_load_item_recursive.py:143`](/tests/unit/no_torch/test_load_item_recursive.py#L143) 
  | view on GitHub: [tests/unit/no_torch/test_load_item_recursive.py#L143](https://github.com/mivanit/zanj/blob/main/tests/unit/no_torch/test_load_item_recursive.py#L143)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=this%20doesn%27t%20raise%20any%20errors&body=%23%20source%0A%0A%5B%60tests%2Funit%2Fno_torch%2Ftest_load_item_recursive.py%23L143%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Ftests%2Funit%2Fno_torch%2Ftest_load_item_recursive.py%23L143%29%0A%0A%23%20context%0A%60%60%60python%0A%20%20%20%20assert%20result%20%3D%3D%20json_data%0A%0A%20%20%20%20%23%20TODO%3A%20this%20doesn%27t%20raise%20any%20errors%0A%20%20%20%20%23%20Test%20with%20allow_not_loading%3DFalse%20%28should%20raise%20an%20error%29%0A%20%20%20%20%23%20Create%20a%20ZANJ%20with%20EXCEPT%20error%20mode%20to%20ensure%20value%20errors%20are%20raised%0A%60%60%60&labels=enhancement)

  ```python
  assert result == json_data

  # TODO: this doesn't raise any errors
  # Test with allow_not_loading=False (should raise an error)
  # Create a ZANJ with EXCEPT error mode to ensure value errors are raised
  ```




## [`tests/unit/no_torch/test_zanj_serializable_dataclass.py`](/tests/unit/no_torch/test_zanj_serializable_dataclass.py)

- explicitly specifying the following does not work, since it gets automatically converted before we call load in `loading_fn`:  
  local link: [`/tests/unit/no_torch/test_zanj_serializable_dataclass.py:145`](/tests/unit/no_torch/test_zanj_serializable_dataclass.py#L145) 
  | view on GitHub: [tests/unit/no_torch/test_zanj_serializable_dataclass.py#L145](https://github.com/mivanit/zanj/blob/main/tests/unit/no_torch/test_zanj_serializable_dataclass.py#L145)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=explicitly%20specifying%20the%20following%20does%20not%20work%2C%20since%20it%20gets%20automatically%20converted%20before%20we%20call%20load%20in%20%60loading_fn%60%3A&body=%23%20source%0A%0A%5B%60tests%2Funit%2Fno_torch%2Ftest_zanj_serializable_dataclass.py%23L145%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Ftests%2Funit%2Fno_torch%2Ftest_zanj_serializable_dataclass.py%23L145%29%0A%0A%23%20context%0A%60%60%60python%0A%20%20%20%20%20%20%20%20%20%20%20%20Nested.load%28json.loads%28n%29%29%20for%20n%20in%20data%5B%22container%22%5D.split%28%22%5Cn%22%29%0A%20%20%20%20%20%20%20%20%5D%2C%0A%20%20%20%20%20%20%20%20%23%20TODO%3A%20explicitly%20specifying%20the%20following%20does%20not%20work%2C%20since%20it%20gets%20automatically%20converted%20before%20we%20call%20load%20in%20%60loading_fn%60%3A%0A%20%20%20%20%20%20%20%20%23%20serialization_fn%3Dlambda%20c%3A%20%5Bn.serialize%28%29%20for%20n%20in%20c%5D%2C%0A%20%20%20%20%20%20%20%20%23%20loading_fn%3Dlambda%20data%3A%20%5BNested.load%28n%29%20for%20n%20in%20data%5B%22container%22%5D%5D%2C%0A%60%60%60&labels=enhancement)

  ```python
      Nested.load(json.loads(n)) for n in data["container"].split("\n")
  ],
  # TODO: explicitly specifying the following does not work, since it gets automatically converted before we call load in `loading_fn`:
  # serialization_fn=lambda c: [n.serialize() for n in c],
  # loading_fn=lambda data: [Nested.load(n) for n in data["container"]],
  ```




## [`zanj/loading.py`](/zanj/loading.py)

- add a separate "asserts" function?  
  local link: [`/zanj/loading.py:98`](/zanj/loading.py#L98) 
  | view on GitHub: [zanj/loading.py#L98](https://github.com/mivanit/zanj/blob/main/zanj/loading.py#L98)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=add%20a%20separate%20%22asserts%22%20function%3F&body=%23%20source%0A%0A%5B%60zanj%2Floading.py%23L98%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Fzanj%2Floading.py%23L98%29%0A%0A%23%20context%0A%60%60%60python%0A%20%20%20%20%22%22%22handler%20for%20loading%20an%20object%20from%20a%20json%20file%20or%20a%20ZANJ%20archive%22%22%22%0A%0A%20%20%20%20%23%20TODO%3A%20add%20a%20separate%20%22asserts%22%20function%3F%0A%20%20%20%20%23%20right%20now%2C%20any%20asserts%20must%20happen%20in%20%60check%60%20or%20%60load%60%20which%20is%20annoying%20with%20lambdas%0A%60%60%60&labels=enhancement)

  ```python
  """handler for loading an object from a json file or a ZANJ archive"""

  # TODO: add a separate "asserts" function?
  # right now, any asserts must happen in `check` or `load` which is annoying with lambdas
  ```


- how can we type hint this without actually importing torch?  
  local link: [`/zanj/loading.py:156`](/zanj/loading.py#L156) 
  | view on GitHub: [zanj/loading.py#L156](https://github.com/mivanit/zanj/blob/main/zanj/loading.py#L156)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=how%20can%20we%20type%20hint%20this%20without%20actually%20importing%20torch%3F&body=%23%20source%0A%0A%5B%60zanj%2Floading.py%23L156%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Fzanj%2Floading.py%23L156%29%0A%0A%23%20context%0A%60%60%60python%0A%23%20TODO%3A%20how%20can%20we%20type%20hint%20this%20without%20actually%20importing%20torch%3F%0Adef%20_torch_loaderhandler_load%28%0A%20%20%20%20json_item%3A%20JSONitem%2C%0A%60%60%60&labels=enhancement)

  ```python
  # TODO: how can we type hint this without actually importing torch?
  def _torch_loaderhandler_load(
      json_item: JSONitem,
  ```




## [`zanj/serializing.py`](/zanj/serializing.py)

- Type `<module 'numpy.lib'>` has no attribute `format` --> zanj/serializing.py:54:5  
  local link: [`/zanj/serializing.py:52`](/zanj/serializing.py#L52) 
  | view on GitHub: [zanj/serializing.py#L52](https://github.com/mivanit/zanj/blob/main/zanj/serializing.py#L52)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=Type%20%60%3Cmodule%20%27numpy.lib%27%3E%60%20has%20no%20attribute%20%60format%60%20--%3E%20zanj%2Fserializing.py%3A54%3A5&body=%23%20source%0A%0A%5B%60zanj%2Fserializing.py%23L52%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Fzanj%2Fserializing.py%23L52%29%0A%0A%23%20context%0A%60%60%60python%0Adef%20store_npy%28self%3A%20_ZANJ_pre%2C%20fp%3A%20IO%5Bbytes%5D%2C%20data%3A%20np.ndarray%29%20-%3E%20None%3A%0A%20%20%20%20%22%22%22store%20numpy%20array%20to%20given%20file%20as%20.npy%22%22%22%0A%20%20%20%20%23%20TODO%3A%20Type%20%60%3Cmodule%20%27numpy.lib%27%3E%60%20has%20no%20attribute%20%60format%60%20--%3E%20zanj%2Fserializing.py%3A54%3A5%0A%20%20%20%20%23%20info%3A%20rule%20%60unresolved-attribute%60%20is%20enabled%20by%20default%0A%20%20%20%20np.lib.format.write_array%28%20%20%23%20ty%3A%20ignore%5Bunresolved-attribute%5D%0A%60%60%60&labels=enhancement)

  ```python
  def store_npy(self: _ZANJ_pre, fp: IO[bytes], data: np.ndarray) -> None:
      """store numpy array to given file as .npy"""
      # TODO: Type `<module 'numpy.lib'>` has no attribute `format` --> zanj/serializing.py:54:5
      # info: rule `unresolved-attribute` is enabled by default
      np.lib.format.write_array(  # ty: ignore[unresolved-attribute]
  ```


- somehow need to control whether a failure here causes a fallback to other handlers, or whether the except should propagate  
  local link: [`/zanj/serializing.py:122`](/zanj/serializing.py#L122) 
  | view on GitHub: [zanj/serializing.py#L122](https://github.com/mivanit/zanj/blob/main/zanj/serializing.py#L122)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=somehow%20need%20to%20control%20whether%20a%20failure%20here%20causes%20a%20fallback%20to%20other%20handlers%2C%20or%20whether%20the%20except%20should%20propagate&body=%23%20source%0A%0A%5B%60zanj%2Fserializing.py%23L122%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Fzanj%2Fserializing.py%23L122%29%0A%0A%23%20context%0A%60%60%60python%0A%20%20%20%20archive_path%3A%20str%20%3D%20f%22%7Bjoined_path%7D.%7Bitem_type%7D%22%0A%0A%20%20%20%20%23%20TODO%3A%20somehow%20need%20to%20control%20whether%20a%20failure%20here%20causes%20a%20fallback%20to%20other%20handlers%2C%20or%20whether%20the%20except%20should%20propagate%0A%20%20%20%20%23%20this%20will%20probably%20require%20changes%20to%20the%20upstream%20muutils.json_serialize%20code%0A%20%20%20%20if%20archive_path%20in%20jser._externals%3A%0A%60%60%60&labels=enhancement)

  ```python
  archive_path: str = f"{joined_path}.{item_type}"

  # TODO: somehow need to control whether a failure here causes a fallback to other handlers, or whether the except should propagate
  # this will probably require changes to the upstream muutils.json_serialize code
  if archive_path in jser._externals:
  ```




## [`zanj/zanj.py`](/zanj/zanj.py)

- calling self.json_serialize again here might be slow  
  local link: [`/zanj/zanj.py:185`](/zanj/zanj.py#L185) 
  | view on GitHub: [zanj/zanj.py#L185](https://github.com/mivanit/zanj/blob/main/zanj/zanj.py#L185)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=calling%20self.json_serialize%20again%20here%20might%20be%20slow&body=%23%20source%0A%0A%5B%60zanj%2Fzanj.py%23L185%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Fzanj%2Fzanj.py%23L185%29%0A%0A%23%20context%0A%60%60%60python%0A%20%20%20%20%20%20%20%20%23%20serialize%20the%20object%20--%20this%20will%20populate%20self._externals%0A%20%20%20%20%20%20%20%20%23%20TODO%3A%20calling%20self.json_serialize%20again%20here%20might%20be%20slow%0A%20%20%20%20%20%20%20%20json_data%3A%20JSONitem%20%3D%20self.json_serialize%28self.json_serialize%28obj%29%29%0A%60%60%60&labels=enhancement)

  ```python
  # serialize the object -- this will populate self._externals
  # TODO: calling self.json_serialize again here might be slow
  json_data: JSONitem = self.json_serialize(self.json_serialize(obj))
  ```


- load only some part of the zanj file by passing an ObjectPath  
  local link: [`/zanj/zanj.py:227`](/zanj/zanj.py#L227) 
  | view on GitHub: [zanj/zanj.py#L227](https://github.com/mivanit/zanj/blob/main/zanj/zanj.py#L227)
  | [Make Issue](https://github.com/mivanit/zanj/issues/new?title=load%20only%20some%20part%20of%20the%20zanj%20file%20by%20passing%20an%20ObjectPath&body=%23%20source%0A%0A%5B%60zanj%2Fzanj.py%23L227%60%5D%28https%3A%2F%2Fgithub.com%2Fmivanit%2Fzanj%2Fblob%2Fmain%2Fzanj%2Fzanj.py%23L227%29%0A%0A%23%20context%0A%60%60%60python%0A%20%20%20%20%29%20-%3E%20Any%3A%0A%20%20%20%20%20%20%20%20%22%22%22load%20the%20object%20from%20a%20ZANJ%20archive%0A%20%20%20%20%20%20%20%20%23%20TODO%3A%20load%20only%20some%20part%20of%20the%20zanj%20file%20by%20passing%20an%20ObjectPath%0A%20%20%20%20%20%20%20%20%22%22%22%0A%20%20%20%20%20%20%20%20file_path%20%3D%20Path%28file_path%29%0A%60%60%60&labels=enhancement)

  ```python
  ) -> Any:
      """load the object from a ZANJ archive
      # TODO: load only some part of the zanj file by passing an ObjectPath
      """
      file_path = Path(file_path)
  ```




