# The Three-Pillar Taxonomy: A Compositional Architecture for Data Manipulation Systems

**Alexander Towell**
Southern Illinois University
Department of Computer Science
Edwardsville, IL, USA
lex@metafunctor.com, atowell@siue.edu

**Abstract**

We present a novel organizational taxonomy for data manipulation systems based on three fundamental pillars: *Depth* (addressing and extraction), *Truth* (boolean logic and validation), and *Shape* (transformation and modification). This taxonomy, derived from the design and implementation of the dotsuite ecosystem, provides a principled approach to decomposing complex data manipulation tasks into composable, mathematically-grounded operations. We demonstrate how this separation of concerns naturally leads to tools that follow the Unix philosophy while maintaining formal algebraic properties. The taxonomy lifts elegantly from single documents to collections through homomorphic mappings, preserving compositional semantics. Our approach offers both theoretical elegance and practical benefits: predictable composition, pedagogical clarity, and a natural progression from simple to sophisticated operations. We argue that this three-pillar taxonomy represents a general design pattern applicable beyond our specific implementation, offering a blueprint for future data manipulation systems that balance mathematical rigor with practical usability.

## 1. Introduction

Data manipulation systems face a fundamental tension between expressive power and conceptual clarity. SQL provides relational algebra but struggles with nested structures. JSONPath offers path-based addressing but lacks compositional logic. Functional programming languages provide mathematical rigor but often at the cost of accessibility. We propose that this tension arises from the conflation of orthogonal concerns within monolithic systems.

This paper presents the *three-pillar taxonomy*, a novel architectural pattern that decomposes data manipulation into three fundamental concerns: **Depth** (where is the data?), **Truth** (is this assertion valid?), and **Shape** (how should data be transformed?). This taxonomy emerged from the development of dotsuite, a Python ecosystem for manipulating nested data structures, but represents a general organizing principle applicable to any data manipulation system.

The key insight is that by cleanly separating these concerns, we achieve several desirable properties simultaneously:
- **Compositionality**: Operations compose predictably through well-defined interfaces
- **Mathematical soundness**: Each pillar maintains algebraic properties that ensure correctness
- **Pedagogical clarity**: Users can master concepts progressively without cognitive overload
- **Implementation simplicity**: Each tool can be "stolen" - simple enough to copy rather than import

## 2. The Three-Pillar Architecture

### 2.1 The Depth Pillar: Addressing and Extraction

The Depth pillar answers the fundamental question: "Where is the data?" It provides mechanisms for navigating and extracting values from nested structures. This pillar forms a *free algebra on selectors*, where paths compose through concatenation and support user-defined reduction operations.

**Mathematical Foundation**: Let $D$ be the space of documents and $P$ be the set of paths. The addressing function $addr: D \times P \rightarrow V \cup \{\bot\}$ maps document-path pairs to values or a distinguished "missing" element. The path algebra supports:
- Concatenation: $p_1 \cdot p_2$ forms compound paths
- Wildcards: $*$ generates sets of matching paths
- Predicates: $[?\phi]$ filters based on boolean conditions

**Implementation Spectrum**: The pillar provides tools ranging from simple (`dotget` for exact paths) to sophisticated (`dotpath` engine with JSONPath compatibility), demonstrating that complexity can be introduced gradually without invalidating simpler tools.

### 2.2 The Truth Pillar: Logic and Validation

The Truth pillar answers: "Is this assertion true?" It implements a boolean algebra that operates on documents and lifts homomorphically to collections, preserving logical structure across scales.

**Mathematical Foundation**: The pillar implements a boolean algebra $(B, \land, \lor, \neg, \top, \bot)$ where predicates form an abstract syntax tree. The crucial property is the homomorphic lifting:

$$filter(S, p_1 \land p_2) = filter(S, p_1) \cap filter(S, p_2)$$

This ensures that logical operations on predicates correspond directly to set operations on results, enabling both lazy evaluation and query optimization.

**Compositional Logic**: The `dotquery` tool provides a declarative DSL that compiles to the predicate algebra, supporting arbitrary boolean combinations of atomic predicates. The syntax deliberately mirrors natural language ("any equals role admin") to reduce cognitive load.

### 2.3 The Shape Pillar: Transformation and Modification

The Shape pillar answers: "How should the data be transformed?" It treats transformations as endofunctors on document spaces, with composition forming a monoid structure that ensures predictable chaining.

**Mathematical Foundation**: Transformations form the category $\mathcal{T}$ where:
- Objects are document spaces
- Morphisms are pure functions $f: D \rightarrow D'$
- Composition is function composition with identity

The monoid $(T, \circ, id)$ ensures that transformation pipelines can be optimized through algebraic rewriting while preserving semantics.

**Immutability Principle**: All transformations return modified copies, never mutating originals. This enables safe concurrent operations and predictable composition.

## 3. From Documents to Collections

The three-pillar taxonomy naturally lifts from single documents to collections through functorial mappings that preserve structure:

### 3.1 Homomorphic Lifting

Each pillar operation lifts to collections while preserving its essential properties:
- **Depth**: $map(extract_p, S)$ extracts path $p$ from each document
- **Truth**: $filter(\phi, S)$ selects documents satisfying predicate $\phi$
- **Shape**: $map(transform_f, S)$ applies transformation $f$ to each document

The homomorphism ensures that operations can be freely reordered for optimization without changing semantics.

### 3.2 Relational Algebra

The `dotrelate` tool demonstrates how the taxonomy extends to multi-collection operations. Joins are expressed as:

$$join(S_1, S_2, k_1, k_2) = \{merge(d_1, d_2) | d_1 \in S_1, d_2 \in S_2, get(d_1, k_1) = get(d_2, k_2)\}$$

This leverages the Depth pillar for key extraction and the Shape pillar for document merging, showing how pillars compose to enable complex operations.

## 4. The Unix Philosophy Applied to Data Structures

The three-pillar taxonomy embodies the Unix philosophy - "do one thing well" - applied to data manipulation:

### 4.1 Composable Tools

Each tool has a single, clear responsibility and composes through standard interfaces (JSON in, JSON out). This enables pipeline construction:

```bash
cat data.json | dotstar "users.*.email" | dotquery "contains gmail" | dotmod "set verified true"
```

### 4.2 The "Steal This Code" Principle

Many tools are deliberately simple enough to copy rather than import. For example, `dotget` is essentially:

```python
def get(data, path, default=None):
    for segment in path.split('.'):
        data = data.get(segment, default)
        if data is default: return default
    return data
```

This philosophy acknowledges that sometimes a dependency is more costly than duplication, especially for foundational operations.

## 5. Pedagogical Value: Progressive Enhancement

The taxonomy supports a natural learning progression:

1. **Novice**: Master single-pillar tools (`dotget`, `dotexists`, `dotmod`)
2. **Intermediate**: Combine pillars (`dotstar` + `dotquery`)
3. **Advanced**: Use compositional engines (`dotpath`, `dotpipe`)
4. **Expert**: Extend the system with custom predicates and reducers

This progression ensures that users are never overwhelmed, while experts retain full power. Crucially, simple tools never become obsolete - a `dotget` call remains the right choice for known paths.

## 6. Related Work

**JSONPath** [Goessner 2007] provides path-based addressing but lacks compositional logic and transformation capabilities. Our Depth pillar extends JSONPath while maintaining compatibility.

**jq** [Dolan 2013] offers powerful JSON manipulation but combines addressing, logic, and transformation in a single DSL, making it difficult to reason about individual operations.

**SQL** [Codd 1970] provides relational algebra but struggles with nested structures and lacks a clear separation between selection (WHERE) and transformation (SELECT).

**Functional Programming** languages like Haskell provide compositionality but often require sophisticated type systems that increase complexity. Our approach achieves similar compositional properties through simpler means.

## 7. Conclusion

The three-pillar taxonomy provides a principled decomposition of data manipulation concerns that balances mathematical rigor with practical usability. By clearly separating Depth (addressing), Truth (logic), and Shape (transformation), we achieve:

- **Theoretical elegance**: Each pillar maintains well-defined algebraic properties
- **Practical composability**: Tools combine predictably through standard interfaces
- **Pedagogical clarity**: Concepts can be learned progressively without prerequisites
- **Implementation flexibility**: From "steal this code" simplicity to extensible engines

We believe this taxonomy represents a general design pattern for data manipulation systems. Future work includes applying these principles to streaming data, extending the algebraic framework to handle uncertainty, and exploring optimizations enabled by the compositional structure.

The three-pillar taxonomy demonstrates that complex data manipulation need not require complex tools. By respecting the natural structure of the problem domain and maintaining clean separation of concerns, we can build systems that are simultaneously powerful, predictable, and pedagogical.

## References

[Codd 1970] Codd, E.F. "A Relational Model of Data for Large Shared Data Banks." *Communications of the ACM* 13.6 (1970): 377-387.

[Dolan 2013] Dolan, Stephen. "jq: Command-line JSON processor." https://stedolan.github.io/jq/ (2013).

[Goessner 2007] Goessner, Stefan. "JSONPath - XPath for JSON." https://goessner.net/articles/JsonPath/ (2007).

[Kernighan & Pike 1984] Kernighan, Brian W., and Rob Pike. *The Unix Programming Environment*. Prentice Hall (1984).

[McIlroy 1978] McIlroy, M.D., Pinson, E.N., and Tague, B.A. "Unix Time-Sharing System: Foreword." *The Bell System Technical Journal* 57.6 (1978): 1899-1904.

[Pierce 2002] Pierce, Benjamin C. *Types and Programming Languages*. MIT Press (2002).

[Wadler 1989] Wadler, Philip. "Theorems for free!" *Proceedings of the Fourth International Conference on Functional Programming Languages and Computer Architecture* (1989): 347-359.

---

*Acknowledgments*: We thank the Unix philosophy pioneers for demonstrating that simplicity and composability are not at odds with power, and the functional programming community for showing that practical tools can have solid mathematical foundations.