A friend recently sent me a paper. He's part of the team behind it, and he thought I'd find it interesting given the AI agent work I've been doing. He was right.

The paper is NeMoCo: Self-supervised contrastive learning for ultrastructural 3D neuroscience morphologies, published in Elsevier's Graphical Models (August 2026). The lead author is Humaira Shaffique, supervised by Dr. Marco Agus at Hamad Bin Khalifa University, with co-authors Dr. Uzair Shah, Dr. Mahmood Alzubaidi, Dr. Jens Schneider, Dr. Pierre Julius Magistretti, Dr. Corrado Calì, and Prof. Mowafa H. The code is open-source on GitHub under MIT license.

I'm not a neuroscientist. I'm an AI engineer. I spend my days building Aura — a personal AI assistant that manages memory, routes tasks, retrieves context, and tries to be genuinely useful across sessions. But reading NeMoCo, I kept seeing the same structural problems I deal with in agent engineering, just dressed up in electron microscopy data and spectral geometry. So this is a technical writeup from that angle — what NeMoCo does, why it's interesting, and what it mirrors in the work of building agent systems.

The Problem: Labels Are Expensive, Biology Doesn't Use Them

Neuroscientists use volume electron microscopy (EM) to capture 3D images of brain tissue at nanometer resolution. From these images, they reconstruct 3D meshes of subcellular structures — neurites (axon and dendrite branches), organelles like mitochondria, nuclear envelopes. These meshes are the morphologies: the actual 3D shapes of neural structures.

To classify them — is this an axon, a dendrite, a spine? — you need expert neuroscientists to manually label each mesh. That's slow, expensive, and doesn't scale. You can't train a supervised classifier without labels, and you can't get labels without experts. Connectomics needs to process millions of these structures. Manual labeling is a bottleneck.

Here's the thing that caught me: the brain doesn't get labels either. A child doesn't need someone to say "this is a chair" for every chair they see. They learn the concept by observing chairs from different angles, in different lighting, at different distances — discovering the invariant properties that make a chair a chair. No labels. Just raw experience and the discovery of structure.

NeMoCo asks: can a machine do the same thing with neural structures?

What NeMoCo Does

NeMoCo is a self-supervised contrastive learning pipeline for 3D mesh classification. It learns to recognize neural structures without a single labeled example during pretraining. The architecture is a CLIP-style dual-encoder system applied to 3D meshes.

Here's how it works:

1. Take an unlabeled 3D mesh of a neurite or organelle.
2. Create two views of the same mesh: the original (or lightly augmented), and a transformed version — randomly rotated (±60°), scaled (0.6–1.4×), sheared, and optionally decimated (50–75% fewer faces).
3. Pass both views through a DiffusionNet encoder — a neural network that operates directly on 3D surfaces using spectral geometry (eigenvalues and eigenvectors of the mesh's Laplacian).
4. The encoder produces a 32-dimensional embedding vector for each view.
5. A CLIP-style contrastive loss trains the model to pull the two views of the same mesh together while pushing apart embeddings from different meshes.
6. One encoder branch updates via normal backpropagation; the other is a momentum-updated "teacher" (EMA, momentum=0.999) — the MoCo approach, which prevents representation collapse.

The key insight: by learning that a rotated, scaled, decimated mesh is "the same thing," the encoder discovers the intrinsic geometric features that define each neural structure type — without ever being told what the types are.

Why DiffusionNet Matters Here

DiffusionNet (Sharp et al., SIGGRAPH 2022) isn't something NeMoCo invented. It's a borrowed architecture for learning on 3D surfaces. But the choice is significant. Unlike CNNs (which need regular grids) or PointNet (which treats points independently), DiffusionNet operates on the spectral geometry of a surface — computing Laplacian eigenfunctions (think: vibration modes of the shape) and using Heat Kernel Signatures (HKS) as multi-scale descriptors that capture local shape at different spatial scales.

This matters because neurons are fundamentally 3D structures. The geometry IS the morphology. When you're classifying neural structures, the shape — its curvature, branching, thickness, surface texture — is the signal. DiffusionNet respects that. It doesn't flatten the mesh into an image or reduce it to a point cloud. It learns on the surface itself.

In NeMoCo, DiffusionNet takes 16-dimensional HKS features as input and outputs a 32-dimensional embedding, using 4 blocks with width=128. The architecture is borrowed, but the application is novel: this specific combination of DiffusionNet + MoCo + CLIP-style contrastive learning applied to ultrastructural neuroscience morphologies hasn't been done before.

The Brain-Construction Framing

This is where my AI engineer brain started lighting up.

NeMoCo is, in a real sense, constructing a piece of what a brain does — not biologically, but computationally. It's building a system that learns to categorize neural structures the way biological brains learn to categorize the world: from raw, unlabeled experience.

The parallel is at the level of principle, not mechanism. NeMoCo uses a specific mathematical loss function (CLIP-style cross-entropy). The brain uses spike-timing-dependent plasticity and neuromodulation — far more complex and not directly analogous. NeMoCo's "memory bank" is a kNN lookup table. Biological memory is associative, reconstructive, and constantly reorganized. The brain learns continuously from a stream; NeMoCo trains in discrete epochs on a fixed dataset.

But the principle holds: labels are not necessary for learning good representations. That's how biological brains operate, and NeMoCo is a computational demonstration of it.

The Elements of a Brain, Mapped

If you think about what a brain does — at a functional, not biological, level — you can break it into elements: perception, memory, representation, learning, reasoning. NeMoCo has analogues to each:

• Perception: Raw sensory input becomes structured representation. In NeMoCo, HKS features + DiffusionNet transform a raw mesh into a geometric embedding. In the brain, raw sensory signals become perceptual features.
• Memory: Stored representations, retrievable by similarity. NeMoCo's frozen kNN memory bank stores embeddings and retrieves by nearest-neighbor. The brain retrieves by associative similarity.
• Representation: Invariant features extracted from varied experience. NeMoCo's 32D embedding maps the same mesh under different transformations to the same vector. The brain maintains stable object representations across viewpoints.
• Learning: Updating based on experience. NeMoCo uses backpropagation + momentum EMA. The brain uses plasticity rules.
• Reasoning: Combining representations to make decisions. NeMoCo uses kNN voting and linear probing. The brain does... well, something far more complex, but the functional role is the same.

I'm not claiming NeMoCo is a brain. It's a mathematical pipeline. But it's a pipeline that constructs a functional piece of what brains do: building useful internal representations from raw, unlabeled experience.

Where It Mirrors Agent Engineering

This is the part that hit closest to home. I build Aura — an AI assistant that manages memory, routes tasks, retrieves context, and maintains continuity across sessions. The structural parallels to NeMoCo are concrete enough that I kept nodding while reading the paper.

Self-Supervised Pretraining ↔ Agent Memory

NeMoCo pretrains on 4,235 unlabeled mouse skull meshes (the MouseFinal dataset), building a general-purpose geometric representation. That pretrained encoder can then be frozen and reused for multiple downstream tasks — classification, retrieval, clustering — without retraining the core.

Agent memory works the same way. Aura's memory files — long-term memory, daily notes, tool notes — build a general-purpose understanding of the user, projects, and environment without task-specific labels. That memory persists across sessions and can be queried for any task. It's a "pretrained context" that doesn't need to be rebuilt each time.

The frozen kNN evaluation is the sharpest parallel. NeMoCo's encoder, pretrained without labels, achieves ~70–80% accuracy on a classification task it was never explicitly trained for — just by finding nearest neighbors in embedding space. A well-maintained agent memory allows an AI assistant to answer questions about projects it hasn't been explicitly briefed on in the current session — just by finding semantically similar entries. The mechanism is different (kNN in vector space vs. semantic search over text), but the principle is the same: a good representation, built from raw experience, transfers to tasks it was never designed for.

Representation Learning ↔ Agent Knowledge Structure

NeMoCo's 32-dimensional embedding is a compressed representation of a mesh's geometric identity. Embeddings that are close in vector space represent similar morphologies. The embedding space emerges from the data — no one designed which dimensions correspond to which features.

Agent memory is a compressed representation of facts, decisions, and context. The agent doesn't store raw transcripts; it stores distilled meaning. And the structure of that memory emerges from use, not from a predefined schema. I didn't design Aura's memory taxonomy upfront. I discovered what was important through interaction — which projects come up, which decisions matter, which preferences recur. The memory naturally clusters related concepts, the same way NeMoCo's embedding space naturally clusters similar neural structures.

The Contrastive Insight

This one's more speculative, but I think it's real. NeMoCo's contrastive loss works by pulling similar things together and pushing different things apart. The model learns what makes two views "the same mesh" by also learning what makes them "not some other mesh."

There's a lesson here for agent design. An agent that only stores facts — positive examples — has no way to know what's distinctive about those facts. I've been experimenting with tracking contrasts in Aura's memory: "this project uses SQLite, not Postgres"; "this user prefers direct answers, not explanations." The contrastive framing makes the representation richer. It's not enough to know what something is; you also need to know what it isn't.

The momentum encoder is another one. The slowly-updating teacher branch (EMA, τ=0.999) prevents the representation from oscillating wildly during training. Agent memory could use a similar "slow update" principle. I don't rewrite long-term memory on every new fact. I let important patterns accumulate weight over time before promoting them. Daily notes are raw logs; long-term memory is curated. The promotion from one to the other is a momentum-like update — slow, deliberate, resistant to noise.

The Results

NeMoCo evaluates the learned representations at three levels:

1. Frozen kNN (k=20): The encoder is completely frozen. Test meshes are classified by finding their 20 nearest neighbors in a memory bank of training embeddings and voting. Zero additional training. Result: ~70–80% accuracy on the six-class neural structure classification task, after pretraining on MouseFinal — a dataset the encoder never saw labels for.
2. Linear probing: A single linear layer is trained on top of the frozen encoder. This tests whether the learned features are linearly separable — do the classes form clean clusters in embedding space?
3. Fine-tuning: The entire encoder + classifier is trained end-to-end. This is the supervised ceiling: ~87% accuracy.

The headline finding: self-supervised and supervised approaches reach comparable accuracy. Not identical — the supervised ceiling is higher — but close enough that you can classify neural structures without large labeled datasets. The CLIP retrieval accuracy during pretraining is 200–400× better than random chance (random = 0.024% for 4,235 meshes), which tells you the encoder is learning real structure, not memorizing noise.

These numbers come from the project's Phase 4 README — expected ranges, not final published figures. The paper is paywalled on ScienceDirect, so I can't verify the exact final numbers. But the direction is clear: self-supervised pretraining closes most of the gap to fully supervised training, and it does so without a single label.

Why It Matters

Connectomics needs scale. The bottleneck isn't imaging — EM technology can capture terabytes of data. The bottleneck is annotation. If you need expert neuroscientists to manually label every mesh, you're limited by human throughput. NeMoCo's approach — pretrain on unlabeled data, transfer to labeled tasks with minimal fine-tuning — is a path to breaking that bottleneck.

The cross-domain transfer is particularly interesting. NeMoCo pretrains on mouse skull meshes and transfers to neural structure classification. Different domain, different shapes, same geometric primitives. The encoder learned general 3D shape features that transferred. That's the self-supervised promise: learn the structure of the world, then specialize cheaply.

The code is open-source. The datasets are described in the repo. The architecture is reproducible on an RTX 3090 (with reduced batch size) or an A100 for full-scale training. This is not a locked-down corporate system — it's a research project you can run yourself.

The Team

This came out of Hamad Bin Khalifa University in Qatar. Humaira Shaffique led the work as part of her Master's thesis. Dr. Marco Agus supervised. Dr. Uzair Shah (the repo owner and mentor), Dr. Mahmood Alzubaidi, Dr. Jens Schneider, Dr. Pierre Julius Magistretti, Dr. Corrado Calì, and Prof. Mowafa H. round out the co-authors. The paper is in Graphical Models (Elsevier), published around August 2026.

My friend is on the team. He sent it to me because he thought the parallels to agent engineering would click. They did.

What I'm Taking Away

Building Aura, I've been solving the same fundamental problem NeMoCo solves, just in a different domain: how do you build useful internal representations from raw, unlabeled experience?

NeMoCo's answer: contrastive learning on intrinsic geometry. Show the system the same thing from different angles, tell it "these are the same," and it discovers what matters. My answer for Aura: accumulate memory from interaction, let structure emerge from use, promote patterns slowly. Different mechanisms, same principle.

The brain-construction framing isn't a metaphor I'm forcing. It's the actual structure of the problem. When you build an agent — perception (input routing), memory (storage and retrieval), representation (compressed context), learning (memory updates), reasoning (tool selection and planning) — you're constructing something brain-like from components. Not biologically. Functionally. The same way NeMoCo constructs a system that categorizes neural structures the way brains categorize the world: without being told what the categories are.

That's the lesson. Labels are a shortcut. Structure is what matters. And you can learn structure from raw experience — whether you're a brain, a mesh classifier, or an AI agent trying to be useful.

NeMoCo on GitHub: github.com/Uzshah/NeMoCo

Paper: Elsevier, Graphical Models (~August 2026)

DiffusionNet: Sharp et al., SIGGRAPH 2022 — github.com/nmwsharp/diffusion-net

MoCo v2: He et al., CVPR 2020 — arxiv.org/abs/2003.04297

CLIP: Radford et al., ICML 2021 — arxiv.org/abs/2103.00020