(W.I.P) Local transactions and their tradeoffs

2026-04-07T00:00:00.000Z

What is a transaction?

The simplest definition is that it is a set of operations that must either succeed or fail as a complete unit, with no partial status to preserve the data integrity. When dealing with traditional relational databases, transaction semantics are usually defined in terms of ACID. However, as systems scale out, we have to trade these strict guarantees for BASE model.

The ACID principles (where consistency is more important than availability):

Atomicity

Ensures that all operations within a single database transaction are treated as a single, indivisible unit. They are either all fully completed or the database is reverted to its original state as if nothing happened.

Consistency

Guarantees that a transaction can only bring the database from one valid state to another, strictly adhering to all database schema constraints such as foreign keys, unique indexes, data types, and check constraints. Software engineers frequently mistake database consistency for application consistency. For a deeper dive into understanding application invariants I recommend this Milan Jovanovic blog post - what invariants are and why a domain model is the best place to enforce them

Isolation

Determines how visible the changes made by one ongoing transaction are to other concurrent transactions. It provides the illusion that transactions are executed serially (one after another), even though they run simultaneously. We'll dig deeper into isolation levels later on.

Durability

Guarantees that once a transaction is committed, its changes are permanent and will survive any subsequent system failure, such as a sudden power outage, an operating system crash, or a hardware reboot.

The BASE principles (where availability has a higher priority than availability):

Basically Available

The system guarantees availability in terms of the CAP theorem. Instead of locking down the entire system and returning errors during a network partition or a node failure, the system will choose to remain functional. Some parts of the data might be temporarily stale or inaccessible, but the database as a whole does not go down.

Soft State

In an ACID database, data is stable and deterministic at any given point. In a BASE system, the state of the data is "soft"—it can change over time, even without direct user interaction, as updates gradually trickle through different nodes in a distributed cluster.

Eventual Consistency

The system does not guarantee that everyone will see the same data at the exact same millisecond. Instead, it guarantees that if no new updates are made to a specific piece of data, all replicas across the network will eventually converge and become identical.

Isolation levels

SERIALIZABLE - a serial execution is viable but within certain constraints:

Every transaction must be fastly executed, because a slow one will stall the progress of next in line transactions.
Cross partition serializability can be implemented, but pose some challanges of their own (TODO: article on that?)

Phenomenas

Transactions in relational databases vs NoSQL

6 lessons I've learned from The Phoenix Project

2024-05-28T00:00:00.000Z

My take on the most important lessons I've learned from Gene Kim's book The Phoenix Project

reading: ~3m

1. Taking risks must be part of the company culture

Failing fast with new ideas, taking lessons, and improving is far more effective than following the standard way of working. With every technical task, think of ways to improve it next time. If you know how, try to find time to make those improvements. Delaying them for an "unknown later" will create technical debt, both for the project and yourself. Try to divide some Dev and IT time toward supporting tasks.

John is a chief information security officer who goes through the biggest change in work approach over the course of the book. His colleagues see him as a progress blocker. They blame his frequent IT security checks and following complex protocols for disturbing others. He changes after taking the initiative to teach and trust others. So, he goes from blocking progress to being an asset.

2. Documenting key project knowledge is required

The perfect solution would be to document every detail. Then, for a big project, like project Phoenix, we would know every step taken. In real-life projects, there's not enough time to document everything. So, focus on the complex, repeated processes (like onboarding, VPN setup, and vendor integrations). This will save time in the future.

3. Slack time is important

In the book, there's a guy called Brent who is the "know-it-all" type.

Brent is resolving different issues in many systems at a time and knows undocumented processes, so only he is able to solve them. Why is that? Brent is always busy, firefighting and working on projects, which means that until he's done, he can't take on any new work.

Like in the theory of constraints, some slack time is important. You can use it for new improvements, documentation, or high-priority work.

4. High trust and engineering spirit

A team does not achieve transparency, mutual trust, and vulnerability easily, but it needs them to function.

Mutual trust. It's better to say you can't finish the task on time due to a lack of knowledge than to lie about an unexpected issue. Next time the team takes on a similar task, they will know that it requires some nonstandard competencies and time to learn them.
Vulnerability - in the book, the CEO, Steve, encourages others to share personal information about themselves. By opening up first, he gives an example for others. After getting to know each other, we form stronger relationships, have improved self-acceptance, and less ego.
Transparency - Brent didn't share the details of his work with his colleagues, which prevented them from offering help in many ways. Documenting the setup, architecture, and visible CI/CD processes will help organize the work.

5. Priorities

Brent wasn't always doing the highest priority work. A lack of self-organization led him to try to pack in less important tasks, which prevented him from doing the priority work first. He often couldn't finish simple tasks on time as he neglected to set aside some slack time to automate them. Another issue with his work organization was a lack of trust in others. He took on work that only he could do, but someone else should have handled it.

I have a few strategies for handling tasks with efficiency. They are often high priority but boring, require neglected automation from long ago, or need rare skills. You can find them in this blog post.

6. Every company is now an IT company

At first, Steve, the company CEO, says, "IT is like keeping the toilets running. I should never have to think about it." As technology use grows, IT is now vital to the business. Companies must advance to keep up. So, he now sees IT workers as essential, not as janitors. Many companies still resist their IT departments and overdue change. They are at a competitive disadvantage.

I've been planning my kitchen lately. Before using Blender to model it, I checked various companies' design tools. IKEA offers an excellent, user-friendly kitchen design tool that works in the browser. In contrast, many other companies either had poor tools or none at all. I believe this can greatly influence customers looking to buy a new kitchen.

Pragmatic Engineering