The pg_bio Series This post is part of a 4-part series on building a bioinformatics engine natively in PostgreSQL.
- Part 1: Why Proteins are just High-Dimensional Vectors
- Part 2: Writing a Hybrid Database Operator for Biology (<~>)
- Part 3: Indexing 3D Space like a Video Game
- Part 4: Mining the Dark Proteome: Legacy Code with Missing Docs (You are here)
Imagine inheriting an enterprise codebase with billions of lines of code. It runs the most complex machine on earth. But there’s a catch: 30% of the functions are entirely undocumented. They are named things like Uncharacterized_Protein_A0A2Z2MSX3(). You can see the source code, but you have absolutely no idea what it does, what it interacts with, or why it exists.
Welcome to the Dark Proteome.
Across global biological databases (like UniProt), there are hundreds of millions of proteins that scientists have sequenced but never studied. Figuring out what a single protein does in a physical “wet lab” can take a PhD student five years of their life.
But what if we could just query the database to write the documentation for us?
Over the last three posts, we built a custom PostgreSQL extension called pg_bio. We added AI vector embeddings, a custom Rust <~> sequence operator, and 3D Z-Order spatial indexing. Today, we put it all together to mine the Dark Proteome live.
The Batch De-Orphanizer
To find out what these mysterious proteins do, we wrote a Python script called the batch_deorphanizer.py. It works like a massive reverse-dictionary.
We feed it the name of a well-documented family of proteins—let’s use Argonaute (the molecular engine behind gene silencing and CRISPR-adjacent technologies). The script takes all known Argonaute proteins and uses them as “bait” to go fishing in the Dark Proteome.
Behind the scenes, it executes a single, massive SQL query using everything we’ve built:
- HNSW Vector Search: It instantly scans 2.3 million proteins to find uncharacterized proteins that fold into the exact same 3D shape as Argonaute.
- The
<~>Hybrid Operator: It natively triggers a Smith-Waterman dynamic programming matrix in Rust to verify that the uncharacterized protein shares an evolutionary sequence history with our bait. - Z-Order Spatial Check: It verifies that the 3D binding pocket is physically identical.
The Live Output
When we hit Enter on our terminal, Postgres crunches the math in milliseconds:
$ uv run scripts/batch_deorphanizer.py Argonaute
🧬 pg_bio: Batch Dark Proteome De-Orphanizer
Targeting Family: Argonaute (Distance threshold < 0.35)
Found 77 known 'Argonaute' proteins to use as bait...
Mining Dark Proteome... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
✅ Batch Mining Complete in 3.42 seconds!
Novel Orphan Discoveries for 'Argonaute'
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Known Target (Bait) ┃ Orphan Discovery ┃ Vector Distance ┃ Hybrid Score ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ A0A0Q2M2Z1 │ A0A2Z2MSX3 │ 0.0000 │ 0.0128 │
│ Argonaute family │ Uncharacterized │ │ │
└──────────────────────┴──────────────────────┴─────────────────┴──────────────┘
We Just Wrote the Documentation
Look at that output. Out of millions of unknown proteins, the database successfully flagged A0A2Z2MSX3.
The Vector Distance is 0.0000, meaning our AI models are screaming that this unknown protein folds into the exact same bi-lobed shape as Argonaute.
The Hybrid Score is 0.0128, meaning our deterministic Rust operator verified that the underlying source code (the amino acid sequence) is a nearly flawless evolutionary match.
In 3.42 seconds, we just “wrote the documentation” for A0A2Z2MSX3. It is an Argonaute protein.
Call to Action: Become a Bioinformatics Hacker
Biology is in the middle of a computational renaissance. We no longer just need pipettes and petri dishes; we need developers who know how to optimize B-Trees, write Rust extensions, and traverse Neural Network attention maps.
If you are a software engineer, you already possess the skills required to cure diseases, engineer plastic-eating enzymes, and chart the Dark Proteome.
The pg_bio project is completely open source. You can spin it up locally using Docker, stream the UniProt database, and run the exact SQL queries we showed in this series.
Clone the repo, pick your favorite undocumented protein, and go write some documentation.
This concludes the pg_bio series! Check out Part 1 if you missed how it all started.
