The Sindri deep dive: architecture, trust, and lightweight delivery

Welcome to the deep dive.

Our mission today is to really unpack the architecture of a new cloud native software tool called Sindri.

Yeah, and we're looking at a pretty fascinating stack of raw engineering documents for this one.

We really are.

Yeah.

We've got Sindri's architecture decision records or ADRs,

along with their product pitch and their version 1 row map.

Whether you're a DevOps veteran or just someone curious about how modern software is built and secured,

these documents are essentially a masterclass.

Right. They really show how to build a complex system that manages to stay incredibly lightweight

and secure by design.

Exactly. And to really understand what they are doing to software delivery, I think you have to think

about what happens when you check a bag in an airport.

Oh, that's a good way to look at it.

Right. Like you hand it over to the agent, it goes down that little rubber conveyor belt,

and then it just disappears. And from that point on, your suitcase is basically moving from a cart

to a cargo hold to another cart to a carousel.

Yeah. And at every single hop, you're just sort of blindly trusting the system.

Yes. You're trusting they don't lose it or open it or accidentally swap it with someone else's

identical black suitcase. It is a chain of custody based entirely on where the bag is physically

located at any given moment.

Which is wild because, well, the software industry has historically managed its most precious cargo

in the exact same way.

Like with traditional artifact repositories.

Right. Traditional repos rely heavily on this location first tracking.

If a container image is, you know, sitting in a directory or a registry namespace labeled staging,

the system just assumes it belongs in staging.

Because it's physically sitting in that folder.

Exactly. And moving it to production means physically copying those bytes to a new location and

just hoping nothing corrupts them or intercepts them in transit. And Sindri fundamentally flips that

model on its head. Which is why these architecture documents are such a great read.

Because the problem Sindri tackles is one that a lot of DevOps engineers feel really acutely.

Oh, absolutely. The status quo is rough.

Yeah. I mean, existing artifact repositories like the massive enterprise tools like Nexus or

artifactory. They've just become incredibly heavy.

They really have. You basically need dedicated teams just to tune and operate the JVMs they run on.

Right. And then on the flip side, you have smaller teams who want to avoid that massive enterprise

footprint. So they end up patching together these fragmented single ecosystem tools.

Yeah. Like using harbor for their containers and then a Verdachio for their node packages.

Exactly. It's a mess. But Sindri's entire premise is that you can start with a single Docker

Compose file on a laptop and later scale that exact same system into a full enterprise Kubernetes

installation without ever switching to a different underlying product. Right. One single binary,

not two completely different products. And that capability, it stems directly from what they call

their operational lightness mandate. Okay. Let's get into that. What does that actually mean?

Well, if you look at architecture decision record number two, they explicitly outline why they

rejected languages that run on heavy virtual machines. Sindri is built entirely in go.

Like the latest stable version of go, right? Yeah. Exactly. And it's compiled into a single

static binary with absolutely zero runtime dependencies. Wow. Zero. Because requiring a user to

provision and monitor a Java virtual machine is precisely the operational bloat they are positioning

against. You know, it reminds me of like packing for a camping trip. Sindri is this meticulously

crafted Swiss Army knife, whereas the competitors are like rolling tool chests. That's a great analogy.

But I mean, they took this zero dependency constraint so seriously that they even applied it to

the user interface, which well, I had some questions about that. You're talking about the go templates.

Yeah. The UI is built using go templates and HTMLX simply to avoid introducing a no JS build step

into their pipeline, which would break the single binary promise. Right. But let me push back on

that for a second. I mean, skipping a modern react front end and relying on basic go templates.

Doesn't that make the product way less ergonomic for the end user? Like you're sacrificing a lot

of modern client side state management there. Well, the engineering team actually directly

acknowledges that trade off in the ADR. They admit that, you know, developing the UI with

go templates is definitely less ergonomic than spinning up a react application.

So why do it? Because it comes down to their ultimate priority, which is the deployment and

operational experience. In version one, the UI is purely for read only browsing.

Oh, like just inspecting packages and verifying metadata.

Exactly. Checking vulnerabilities, things like that. And server side templates handle that

perfectly well. Their absolute highest priority is that deployment means moving exactly one

file onto a server. And upgrading means replacing that one file. Right. Bringing in a massive

JavaScript ecosystem for the front end would introduce a separate build step and that fundamentally

violates the operational lightness mandate. Okay, that makes sense. It's like building a

deep space probe, right? If you sever its connection to the outside world, it has everything

it needs to survive packed into one solid, statically linked hull. No complex external life support

systems. Exactly. But for a binary that strips away so much operational overhead, it still has

to track the massive sprawling complexity of a CICD life cycle. Right. Which brings us back to the

whole airport luggage concept. Because traditional repositories are file first. You are constantly

moving bytes between environments. Moving a tarball from dev to qa to prod. Yeah. And you're just

asking this system to verify the file's location to determine its trust level, which feels so fragile.

It is fragile. And Sindri's alternative is what they call identity first tracking. This is

formalized as invariant i3. Promotion changes state, not location. Wait, so the artifact is never

actually moved. Never. An artifact is never copied as it matures through the pipeline. Instead,

they use an append only ledger called life cycle events. Okay. And how do they keep track of what's

what in the ledger? Every single event is keyed strictly by the artifacts unique Shaw 256 hash

digest. That is, well, that's just so much safer than shifting bytes around. It's like getting a

a visa stamp in your passport, right? You don't get issued a completely brand new blank passport

every single time you cross a border. Exactly. Your identity, the physical passport remains constant.

You just accumulate immutable stamps that grant you access to new areas. So the exact same compiled

code sits in one place. And the ledger just accumulates stamps saying, you know, it passed integration

testing. It passed a security scan. And it's finally approved for production. And that ledger

fundamentally alters the definition of auditing in a deployment pipeline. How so? Because rows in

that life cycle events table are only ever added. They are never updated. They are never deleted.

So auditing isn't this complex forensic exercise involving scraping a bunch of logs.

Oh, wow. So if an incident occurs and you need to know who promoted a specific image to production

or when they did it or what the vulnerability scan showed at that exact millisecond,

you just what run a database query? Yeah. You just run a standard postgres query against the ledger.

The immutable audit trail is just a free structural byproduct of the identity first architecture.

That is brilliant. But if they were appending metadata instead of copying files,

those physical bites still have to live somewhere permanent, right?

They do. Sindri manages this through a content addressable store or a CAS.

Right. I saw that in ADR3. They enforce invariant I1, which is one blob, many placements.

Exactly. A blob of data is stored exactly once per hash digest, regardless of how many repositories

or projects reference it. The storage savings there have to be massive. Oh, massive,

especially in a CICD environment. Think about a standard base Docker image like a really heavy

Ubuntu or Alpine layer. Right. Something used across hundreds of different microservices in an

enterprise. Yeah. In a location for a system, that thick base layer might be physically replicated

dozens of times across different project folders. But in a CAS architecture, Sindri writes those

bytes to the disk one single time. Exactly one time. Okay. But how do they handle the actual upload

process securely? ADR8 gets into the weeds a bit on this. It does. So when a client initiates an

upload, they claim they are sending a file with a specific hash, right? Sindri streams those incoming

bytes to a temporary staging key and it computes the hash on the fly as the bytes actually arrive.

Okay. It only commits the blob to its final permanent CAS location if the computed hash perfectly

matches what the client claimed. And if there's a mismatch, the temporary file is instantly trash.

It just guarantees the integrity of the storage layer from the very moment the bytes hit the server.

The file name doesn't matter. The folder doesn't matter. The hash is everything. Right. The hash is

the absolute cryptographic guarantee of the contents. Okay. But wait, wait, wait, wait. Implementing a shared

cast back end like that introduces a massive security risk, doesn't it? What do you mean?

A multi-tenant data leak. If I am running a proprietary enterprise application, my top secret code is

de-duplicated and physically sitting in the exact same backend bucket is code from other teams

or even completely different companies in a hosted environment. Okay. I see where you're going.

Right. Because the storage is just keyed by hashes. So if a malicious internal actor simply

computes or even guesses the shot 256 hash of my proprietary library, couldn't they just ask

the server for that hash and pull my secret files? That is a fantastic catch. And it is the exact

attack vector the architects anticipated. Oh, really? Yeah, it led directly to invariant i2,

placement gates every read. The system is mechanically hard coded so that it never,

ever serves bytes from the cast by the digest alone. Oh, okay. So what does it do instead?

Before the application ever touches the physical storage on the disk, every single read request

traverse a placement metadata table in the Postgres database.

So the database acts as this mandatory gatekeeper?

Precisely. It checks if the blob physically exists, sure, but then it crucially evaluates whether the

requester's context has a placement record, granting them access rights to that specific digest.

Wow. Okay, so ADR calls this out as a critical boundary?

It calls it the most critical hard boundary in the entire system.

It is implemented as a single shared software function.

Every single protocol adapter, whether it's handling Python packages or Docker containers,

they are all forced to call this one function.

Exactly. By passing this database check would result in a cross-tenant data leak,

so they enforce its usage with really rigorous automated invariant tests.

That is so smart. But you know, securing the internal data from leaking out is only one half of the equation.

The architecture also has to defend against malicious codes sneaking in from the outside,

which is a huge problem right now.

It is. The supply chain attacks we see constantly in the news, they usually exploit how internal build systems

pull external dependencies.

Right. And Sindri manages this by categorizing repositories into three distinct types.

You have hosted for internal code proxy for caching external registries and virtual.

And virtual is the one that multiplexes the two together into a single endpoint for the developer, right?

Exactly. But that multiplexing behavior is exactly where organizations usually expose themselves to dependency confusion attacks.

Okay. So how does Sindri lock that down with invariant i5?

It dictates that virtual repositories must resolve an explicit hosted first order.

Ah, the name claim rule.

Yes, the name claim rule. I love this concept.

If an internal team has ever published a private package named say internal dash authlib to the hosted repository,

that namespace is just permanently claimed.

Yeah, permanently.

It's basically like a nightclub VIP list operating at the namespace level.

If the bouncer, the system knows an internal component owns that name,

it completely ignores anyone from the outside trying to use the same name regardless of what credentials they present.

Or what version number they claim mechanically, this completely neutralizes dependency confusion without requiring any expensive external vulnerability scanning tools.

Walk me through a typical attack just to make it concrete for you listening.

Sure. So in a typical attack, a bad actor realizes your company uses an internal package called internal dash authlib.

So they go to a public registry like the global Python package index and upload a malicious payload with the exact same name.

But they tag it with a ridiculously high version number, right? Like version 99.0.0.

Exactly. So when your build server asks the virtual repository for the newest version, a poorly configured system will see version 99 on the public proxy,

assume it's a legitimate update and pull the malware right behind your firewall.

Yikes, but because Sindri's virtual repositories enforce that hosted first resolution.

The system sees the name internal dash auth dash lib in the internal postgres tables realizes the name space is claimed and flat out refuses to even query the public proxy for that package.

So the attack is just structurally impossible structurally impossible. That is amazing.

But managing all these guard reals, the placement gates, the name space claims, the virtual routing, it sounds like it would result in massive sprawling configuration files.

It usually does in other systems. Yeah, but ADR seven and ADR nine detail how Sindri scales this complexity for different team sizes using a config as code philosophy.

Right. They have one configuration loader with two different syntaxes.

There is a short form, which is just a highly simplified flat YAML file tailored for a solo developer or a small startup. Right.

And then there is the long form, which is your fully enveloped verbose Kubernetes style manifest for enterprise platform teams.

And the short form relies heavily on inference, like if a configuration block contains a list called members, the system automatically infers it must be a virtual repository.

So the user doesn't have to explicitly type that out. Exactly. And then the system performs a total one way deshugering process.

Yeah, it takes that simplified short form syntax and instantly converts it in memory into the verbose long form Kubernetes representation, the underlying engine only ever processes that long form data structure.

Okay, I have to push back here again because deshugering a flat YAML into a complex manifest tree is a classic software engineering trap. Oh, for sure.

Right. When you maintain two completely different configuration syntaxes, you're essentially maintaining two separate parsers. And over time, as developers add new features, those parsers inevitably drift apart.

Parser drift is a nightmare.

It is a field in the short form gets interpreted slightly differently than the same field in the long form. And suddenly you have this catastrophic configuration drift where the system behaves.

unpredictably depending on which syntax you used.

Well, the architects actually flag that exact risk.

They did.

Yeah, and to prevent parser drift, they implemented a mandatory architectural

faith guard called the Golden Test.

What's the Golden Test?

It is a strictly enforced automated test within their continuous integration pipeline.

The test takes a complex system setup written in the short form, and the exact same logical

setup written in the long form.

It parses both of them and statically asserts that they produce an identical bite-for-bite

post-cress database tree in memory.

Wait, really?

Bite-for-bite identical?

Yes.

If the resulting database states differ by even a single Boolean flag, the test fails,

and the code cannot be merged.

Wow.

So it mathematically guarantees that the short form is just discoverable syntactic sugar,

and not a drifting secondary configuration engine.

Exactly.

It's incredibly rigorous.

And speaking of that shared core architecture, looking at ADR6, they intentionally chose

the two most structurally divergent package formats to launch with Inversion 1.

Oh, right.

OCI, which is the standard for Docker containers, and PiPi, the standard for Python packages.

Yeah.

And those two protocols operate on completely different paradigms.

OCI is highly complex, right?

It relies on a graph of JSON manifests that reference individual, de-duplicated file system

layers.

Exactly.

Whereas PiPi is incredibly flat by comparison.

It is basically a simple XML or JSON index pointing to immutable, zipped tarballs and wheel

files.

So why launch with those two specifically?

Because by launching with two formats that are polar opposites, the engineering team proves

that there are one core many protocols abstraction is fundamentally sound.

Ah, I see.

They demonstrate that the underlying content addressable store and the postgres placement

logic can seamlessly handle both a graph of de-duplicated layers and flat immutable files.

That makes total sense.

Once you prove the abstraction holds for OCI and PiPi, bolting on a third protocol like

NPM or Mavenlater becomes a highly predictable low risk effort.

Precisely.

And the PiPi implementation specifically, which is detailed in ADR 13, it reveals a fascinating

stance on right time validation.

I read this part and it blew my mind.

When a developer's CI pipeline uploads a Python wheel file, they send along web form data

claiming certain metadata, right, like specifying the package is version 9.9.

Right.

However, inside the actual compiled wheel file, the internal metadata might say it is version

1.0.0, which happens a lot in messy pipelines.

Right.

And Sindri treats the uploaded artifact as the ultimate authority.

If the external claim doesn't perfectly match the internal bytes, Sindri immediately

rejects the upload with an HTTP 400 error.

Yes.

A lot of legacy systems try to be helpful in this scenario by silently auto correcting the

database record or modifying the file name to match the user's form input.

But mutating files server side destroys the integrity of the supply chain.

Exactly.

Because altering the file name or injecting new metadata actually changes the files physical

bytes.

And that immediately breaks the Shaw 256 hash digest that the cast relies on.

And the consumers downstream are trusting that exact cryptographic hash to guarantee

they are pulling safe, untampered code.

So a published file must be absolutely immutable.

Absolutely immutable.

By strictly rejecting the mismatched upload, Sindri forces the publishing team to go back

and fix the broken logic in their CICD pipeline.

It's basically trading immediate developer convenience for absolute deterministic trust

in the repository.

The file a consumer asks for is mathematically guaranteed to be the exact file the author

uploaded.

It is an uncompromising approach, but it creates a profoundly secure environment.

It really does.

And you know, whether you are operating a massive enterprise platform, trying to lock

down your name space against dependency, confusion attacks, or you are just a solo engineer

who wants to spin up a secure registry locally without tuning a JVM.

These documents prove that rigorous supply chain security and true operational lightness

can actually coexist in a single binary.

They really can.

You know, there is one final thought I think is worth considering here.

What's that?

The architectural reliance on the absolute immutability of an artifacts identity introduces a permanent

shift in software delivery.

Because every single byte's history, its test results, and its security profile are forever

traceable in an unerasable ledger, it completely removes the ambiguity of location-based deployments.

It's all permanent.

Right.

If this level of permanent cryptographic auditing becomes a standard across the entire software

industry, you have to wonder, how will that alter the psychology of how developers write,

test, and commit their code in the first place?

Knowing that every artifact's journey is permanently carved in stone.

Exactly.

It fundamentally changes the stakes of hitting commit.

When it comes to enterprise software, we can no longer afford to treat code like checked

luggage on a conveyor belt.

Thanks for joining us on The Steepdife.