# fhir-query
Leveraging FHIR R5 GraphDefinition for Data Traversals and Local Analysis


---

## Overview  

This project leverages **FHIR R5 GraphDefinition** objects to define and execute graph-based traversals across multiple interconnected FHIR resource graphs. The data retrieved is written to a **local SQLite database** for persistence and later transformed into **analyst-friendly dataframes** for analysis using tools like Python’s pandas library.

---

## Motivation  

FHIR Search provides a robust querying framework but comes with significant limitations:  

1. **Deep Chaining Limits**:  
   Chaining searches (e.g., `Patient -> Observation -> Encounter -> Procedure`) often hits server depth limitations.  

2. **Inefficient Query Execution**:  
   Searching deeply related resources requires multiple chained requests, leading to performance issues and unnecessary round trips.  

3. **Lack of Explicit Traversals**:  
   Relationships in FHIR are implicit in references (e.g., `Observation.subject` pointing to `Patient`). This implicit structure requires manual composition of queries, which is prone to errors.  

By using **FHIR R5 GraphDefinition**, we declaratively define resource relationships and efficiently retrieve data. Once retrieved, the data is stored locally and can be transformed into dataframes for advanced analysis.

---

## Key Features  

- **GraphDefinition-Driven Traversals**: Use R5 GraphDefinition objects to define explicit relationships between resources and automate traversal logic.  
- **Local SQLite Storage**: Persist the retrieved FHIR data in a local SQLite database for querying and offline analysis.  
- **Analyst-Friendly Dataframes**: Convert stored FHIR resources into pandas dataframes for ease of use in analytical workflows.  
- **Reusable Graph Definitions**: Maintain a library of GraphDefinition YAML files that can be reused across different workflows and projects.  

---

## Architecture  

### Components  

1. **GraphDefinition Library**  
   - A collection of reusable [GraphDefinition](https://www.hl7.org/fhir/graphdefinition.html) objects in **JSON/YAML** format. A GraphDefinition defines a traversal path between resources.  
   - See [Example GraphDefinition](tests/fixtures/GraphDefinition.yaml), [FHIR Devdays 2021](https://www.devdays.com/wp-content/uploads/2021/12/Rene-Spronk-GraphDefinition-_-DevDays-2019-Amsterdam-1.pdf)

2. **Traversal Engine**  
   - Reads a **GraphDefinition** and iteratively queries the FHIR server using RESTful `_include` and `_revinclude` operations for efficiency.  
   - Stores the retrieved resources in a **SQLite database** in JSON format for flexibility.  

3. **SQLite Data Storage**  
   - Table Schema:  see fhir_query.ResourceDB
 
4. **Analyst-Friendly DataFrames**   **TODO** 
   - Transforms FHIR data from SQLite into pandas dataframes for easier analysis.  
   - Data can be filtered, aggregated, or visualized to meet analytical use cases.  

---

## Workflow  

1. **Load a GraphDefinition**  
   - Define a GraphDefinition object (e.g., `study-to-documents`) to specify the traversal path.  

2. **Execute Traversal**  
   - Use the `Traversal Engine` to query the FHIR server based on the GraphDefinition.  
   - Follow each link and include related resources efficiently using `_include` or `_revinclude`.  

3. **Store Data Locally**  
   - Write the retrieved resources to the SQLite database with their resource types and full JSON representation.  

4. **Transform to DataFrames**  **TODO**
   - Retrieve specific resource types or relationships from the SQLite database.  
   - Convert the JSON data into structured pandas dataframes for analysis.  

---

## Usage

To use the `fq` command, you need to provide the necessary options. Below is an example of how to use the command:

```sh
fq --fhir-base-url <FHIR_BASE_URL> \
   --graph-definition-id <GRAPH_DEFINITION_ID> \
   --path </Resource?params> \
   [--graph-definition-file-path <GRAPH_DEFINITION_FILE_PATH>] \
   [--db_path <DB_PATH>] \
   [--debug]

# example output
✔ research-study-graph is valid FHIR R5 GraphDefinition
✔ Running research-study-graph traversal
✔ Processing link: Patient/_has:ResearchSubject:subject:study={path}&_revinclude=Group:member&_count=1000&_total=accurate with 1 ResearchStudy(s)
✔ Processing link: Specimen/subject={path}&_revinclude=DocumentReference:subject&_revinclude=Group:member&_count=1000&_total=accurate with 537 Patient(s)
✔ Processing link: Group/member={path}&_count=1000&_total=accurate with 17121 Specimen(s)
✔ Processing link: DocumentReference/subject={path}&_count=1000&_total=accurate with 8169 Group(s)
✔ Processing link: Observation/subject={path}&_count=1000&_total=accurate with 537 Patient(s)
✔ Processing link: Procedure/subject={path}&_include=Procedure:encounter&_count=1000&_total=accurate with 537 Patient(s)
Aggregated Results: {'DocumentReference': 24452, 'Encounter': 20, 'Group': 8169, 'MedicationAdministration': 1074, 'Observation': 23676, 'Patient': 537, 'Procedure': 1616, 'Specimen': 17121}
   
```   
___
