Experiments & Results — Neural Matter Networks
Description: Comprehensive experimental results for Neural Matter Networks, including XOR analysis, MNIST, vision benchmarks, and AetherGPT language modeling.

← Back to Theory Results

# Experiments & Results

🧒

#### Explain Like I'm 5

We gave the ⵟ-product brain some tests to see how smart it is:

- 🧩 Puzzle Test (XOR): A tricky puzzle that normal simple brains can't solve. Our brain solved it with just ONE tiny piece!
- 🔢 Number Drawing Test (MNIST): Looking at handwritten numbers and guessing what they are. Our brain learned clearer, sharper pictures!
- 🖼️ Picture Test (CIFAR/ImageNet): Recognizing cats, dogs, planes, and more. Our brain often did better, especially on harder tests!
- 📖 Writing Test (GPT-2): Predicting the next word in a sentence. Our brain was 11% better AND used less memory!

The best part? Our brain is simpler than other brains but works just as well or better!

#### 🧩 The XOR Problem: A Classic Test

The XOR (exclusive or) problem is legendary in neural network history. It's a simple pattern that a single linear neuron cannot learn — proving the need for non-linearity. Here's the pattern:

Input $x_1$ | Input $x_2$ | XOR Output | Pattern
--- | --- | --- | ---
0 | 0 | 0 | Both same → 0
0 | 1 | 1 | Different → 1
1 | 0 | 1 | Different → 1
1 | 1 | 0 | Both same → 0

Why it's hard: You can't draw a single straight line to separate the 1s from the 0s. Traditional solutions require either multiple layers OR an activation function.

The ⵟ-Product Solution: A single neuron with $\mathbf{w} = [1, -1]^T$ naturally solves XOR:

- $(0,0)$ and $(1,1)$: $\mathbf{w}^T\mathbf{x} = 0$, so $\text{ⵟ}(\mathbf{w}, \mathbf{x}) = 0$ ✓
- $(0,1)$: $\text{ⵟ}(\mathbf{w}, \mathbf{x}) = \frac{(-1)^2}{5+\epsilon} > 0$ ✓
- $(1,0)$: $\text{ⵟ}(\mathbf{w}, \mathbf{x}) = \frac{(1)^2}{1+\epsilon} > 0$ ✓

💡
Why This Works: The weight vector $[1, -1]$ is orthogonal to both $(0,0)$ and $(1,1)$ (dot product = 0), but has non-zero dot products with $(0,1)$ and $(1,0)$. The ⵟ-product's intrinsic non-linearity handles the rest!

##### 🎮 Live XOR Training: Linear vs ⵟ-Product

Linear Neuron (can't solve XOR)
ⵟ-Product Neuron (solves XOR!)
Ready
Linear Loss: -- ⵟ Loss: --

#### 🔢 MNIST: Learning Digit Prototypes

MNIST is the "hello world" of machine learning — 60,000 handwritten digits for training and 10,000 for testing. What's remarkable isn't just the accuracy, but what the neurons learn:

📊

##### Linear Model Prototypes

Conventional linear neurons learn diffuse, blurry prototypes. They try to capture all variations of a digit, resulting in smeared, hard-to-interpret weight patterns.

✨

##### ⵟ-Product Prototypes

NMN neurons learn sharp, geometrically coherent digit representations. Each weight vector looks like a clear, prototypical example of its digit class.

🔄 Superposition & Prototype Inversion

A surprising discovery: ⵟ-product neurons exhibit superposition behavior. When prototypes are inverted ($\mathbf{w} \to -\mathbf{w}$):

- Dot product neurons: 91.88% → ~0.01% (complete failure!)
- ⵟ-product neurons: 92.18% → 87.87% (robust!)

The squared numerator means the sign of the dot product doesn't matter — both $\mathbf{w}$ and $-\mathbf{w}$ are valid solutions!

#### 🖼️ Vision Benchmarks: From CIFAR to ImageNet

We systematically compared standard architectures with their "Aether" variants (where standard layers are replaced with ⵟ-product units).

Architecture | CIFAR-10 | CIFAR-100 | STL-10 | Tiny-ImageNet
--- | --- | --- | --- | ---
ResNet-18 | 94.23% | 72.15% | 78.42% | 56.89%
Aether-ResNet-18 | 92.37% | 74.83% | 80.91% | 59.34%
ViT-Small | 91.78% | 69.91% | 75.13% | 52.76%
Aether-ViT-Small | 92.45% | 70.58% | 78.89% | 51.42%

📈
Key Pattern: Aether variants tend to outperform baselines more significantly on more complex datasets (CIFAR-100, STL-10, Tiny-ImageNet) compared to simpler ones (CIFAR-10). This suggests the geometric awareness of the ⵟ-product helps more when the task is harder.

On ImageNet-1K, the flagship large-scale benchmark:

- ResNet-50: 74.13%
- Aether-ResNet-50: 75.24% (+1.11% improvement)

#### 📖 AetherGPT: Language Modeling at Scale

To prove the ⵟ-product generalizes beyond vision, we adapted GPT-2 architecture with ⵟ-Attention and NMN layers.

Training Setup: 2.5 billion tokens from Fineweb dataset, trained on Kaggle TPU v5-8. Same hyperparameters for fair comparison.
🎯

##### Validation Loss (FP32)

GPT-2: 2.43
Aether-GPT2: 2.29
5.8% improvement

⚡

##### Validation Loss (BF16)

GPT-2: 3.03
Aether-GPT2: 2.69
11.2% improvement!

💾 Memory Efficiency

By eliminating activation functions and normalization layers, Aether-GPT2 achieves:

- 15-25% reduction in peak memory usage
- No storage of intermediate activations for ReLU/GELU gradients
- No LayerNorm statistics to track

⏱️ Throughput Comparison

On identical hardware (Kaggle TPU v5-8, batch size 64, context length 1024):

- Linear baseline: 138k tokens/s, 4h 50m 10s total
- Aether-GPT2: 132k tokens/s, 5h 02m 31s total

About 4% slower in raw throughput, but the memory savings enable larger batch sizes or longer contexts — often a net win in practice.

#### 🌀 Vortex Decision Boundaries

One of the most visually striking differences between linear and NMN classifiers is their decision boundaries:

📐

##### Linear Classifiers

Create unbounded half-space partitions. Each class region extends to infinity in some direction. Points far from the training data can still be confidently (and incorrectly) classified.

🌀

##### ⵟ-Product Classifiers

Create localized, vortex-like territories around learned prototypes. Each neuron has a bounded region of influence. Points far from all prototypes receive low confidence — a natural measure of uncertainty!

🔬
Interpretability Bonus: Because each ⵟ-product neuron creates a localized "territory," we can interpret each weight vector as a prototype of what that neuron "looks for." This makes NMNs inherently more interpretable than black-box neural networks.

#### ✅ Experimental Takeaways

Experiment | Key Finding | Significance
--- | --- | ---
XOR | Single neuron solution | Proves intrinsic non-linearity
MNIST | Sharper prototypes, inversion robustness | Better geometric representations
Vision | Outperforms on complex datasets | Scales to real-world tasks
Language | 11.2% improvement, 15-25% less memory | Domain-agnostic benefits
Boundaries | Localized vortex territories | Built-in uncertainty quantification