# Hierarchy Transformers (HiTs)

Code repository for the paper: "Language Models as Hierarchy Encoders".


## Main Dependencies

This repository follows a similar layout as the `sentence_transformers` library. It mainly depends on the following libraries:


- [Sentence Transformers](https://www.sbert.net/index.html) for language models.

- [DeepOnto](https://krr-oxford.github.io/DeepOnto/) for processing hierarchies and constructing datasets from hierarchies.

- [Geoopt](https://geoopt.readthedocs.io/en/latest/index.html) for arithmetic in hyperbolic space.


## Models on Huggingface Hub

Our HiT models will be released on the [Huggingface Hub](https://huggingface.co/Hierarchy-Transformers).

## Datasets

Datasets for HiTs are available at [Zenodo](https://zenodo.org/doi/10.5281/zenodo.10511042), including those constructed from:

- WordNet  
- SNOMED CT
- Schema.org 
- FoodOn
- DOID
