tigerbeetle is kinda cool
Databases play a crucial role in any modern application, providing the backbone for data storage and management. Among the myriad of databases available, specialised ones like Tigerbeetle stand out for their unique capabilities.
Working in fintech, it quickly becomes obvious that storing financial information correctly is critical and difficult. Different companies have developed various solutions and methods for creating financial ledgers, often using general-purpose databases like PostgreSQL, MySQL, Cassandra, Google Cloud Spanner, and similar systems.
What is Tigerbeetle? #
Tigerbeetle is a new finanical database built from the ground up to handle finanical ledgers, accounts and transfers. One of the primary goals of the database is simplicity and performance. I’ve been following the development of Tigerbeetle for a while now and have come across some excellent videos and articles about it. This deep dive into Tigerbeetle’s ecosystem has involved watching talks, reading code, and experimenting with the database, and I find it truly impressive.
Tigerbeetle is built on a set of powerful primitives, including Two-Phase Transfers, Inflight Balances, Multi-Tenant Currencies, Connected Systems, Chart of Accounts, Net Balance Limits, Linked Events, and T-Accounts. These features make it an ideal choice for robust financial management and transaction processing.
The Role of Zig? #
The database has been written in a relatively new programming language called Zig. Zig is an interesting language that I’ve been experimenting with. You can read my post about rewriting a Bash utility script in Zig here. Zig is gaining attention for its simplicity and performance, making it a compelling choice for system-level programming. Zig was chosen for Tigerbeetle due to its simplicity, performance, and modern safety features, making it ideal for building a robust, high-performance Online Transaction Processing (OLTP) database.
Why Zig? #
The decision to write Tigerbeetle in a new programming language like Zig is an intriguing design choice by the developers. The reasons behind this choice contribute to what makes the database so fascinating and impressive. Matklad, the developer behind the rust-analyzer project, has started live streaming an excellent series of talks called IronBeetle with matklad. Throughout the series, Matklad answers the question “Why Tigerbeetle?” and covers topics such as reliability, safety, double-entry accounting, performance, distributed systems, simplicity, and many other subjects.
Technical Benefits of Zig #
Simplicity and Control: Zig provides low-level control over hardware and memory management without the complexity often associated with languages like C++, Rust, and D. This allows developers to write highly efficient and performant code, which is crucial for a database system like Tigerbeetle.
Safety Features: Zig includes modern safety features such as bounds checking, null safety, and memory safety. These features help prevent common programming errors that can lead to security vulnerabilities and crashes, ensuring the robustness of Tigerbeetle.
Performance: Zig is designed for performance, offering features like manual memory management and the ability to write code with minimal runtime overhead. Benchmarks have shown that Zig can achieve performance levels comparable to or exceeding those of C, making it an excellent choice for high-performance applications.
No hidden allocations: Zig takes an hands-off approach when it comes to heap allocations. There is new keyword or any other language feature that allocates memory. Instead developers must use initate a memory allocator ensuring all heap allocation is managed by library and application code.
My Learnings and Findings #
In this post, I aim to break down some of my learnings and interesting findings from researching the database. Additionally, I will discuss what I hope to achieve with it in the future.
Tigerbeetle is Not Your Normal Database #
Tigerbeetle is described as an OLTP database built for safety and performance. It is not a general-purpose database and limits what you can save against an Account or a Transfer. Therefore, to store metadata for these entities, you would need to use Tigerbeetle alongside a general-purpose database such as PostgreSQL, MySQL, or a similar system. Tigerbeetle is not designed to store the name of an account or user; that task should be left to a database better suited for such information.
This separation between your Tigerbeetle database and your chosen general-purpose database provides a “Division of Responsibilities” when handling transactions or other requests from your app or website that involve accounts and transfers. Tigerbeetle records transfers between accounts, tracks account balances, enforces balance limits, and ensures financial consistency through double-entry bookkeeping. Meanwhile, the general-purpose database stores metadata about ledgers, accounts, and transfers, maintaining mappings between Tigerbeetle and the internal representations used by the app, API, or website.
A particularly interesting design decision that I discovered through the Tigerbeetle blog, IronBeetle videos, and examining the code is that Tigerbeetle runs with a single-threaded event loop. By keeping all I/O code single-threaded, the design ensures simplicity and deterministic performance, making it easier to manage and optimise. Although Tigerbeetle employs a thread-per-core design for optimal performance, it restricts all I/O operations to a single thread to enforce a single-writer model.
Tigerbeetle replicates all incoming messages before they are processed and written by the primary node. This is done in a deterministic order to other nodes via the Viewstamped Replication (VR) consensus protocol. You might have heard of Paxos or Raft, the two most prominent consensus algorithms. VR uses replication to ensure reliability, views to manage system configurations with a primary and backups, and view numbers to track changes, ensuring consistency, fault tolerance, and continuous service through coordinated request handling and recovery from failures. A view in VR is a specific configuration of the system in which one server acts as the primary (leader) responsible for coordinating all client requests, while the other servers act as backups (followers) that replicate the operations. The primary ensures all backups apply the same sequence of operations, and the system can switch to a new view with a new primary if the current one fails or becomes unresponsive.
Use Cases and Applications #
Tigerbeetle can be particularly useful in scenarios requiring high reliability and performance in transaction processing, such as financial applications, payment gateways, and online marketplaces. Its ability to enforce financial consistency through double-entry bookkeeping makes it an ideal choice for applications needing robust financial management. Benchmarks suggest Tigerbeetle’s performance is very impressive, even when running on low-powered commodity hardware like a Raspberry Pi.
Comparisons #
Compared to other OLTP databases, Tigerbeetle’s use of Zig and its focus on transaction safety and performance set it apart. Traditional databases like PostgreSQL or MySQL, while versatile, may not offer the same level of specialised performance for transaction processing tasks that Tigerbeetle does.
My Next Steps #
We’ve covered a high-level overview of Tigerbeetle and roughly how it works. Now, I think my next steps will be to build something with it. First, I want to create a Tigerbeetle Dashboard that provides a user-friendly interface to browse ledgers, accounts, and transfers from Tigerbeetle, combined with a traditional database like PostgreSQL for related metadata.
Once I’ve got a working dashboard, I can link it to other systems, perhaps a cryptocurrency platform, Stripe, or a mock payment system.
Conclusion #
Tigerbeetle represents a promising approach to handling transactions with high safety and performance standards. If you’re looking for a specialised database for your financial or transaction-heavy applications, I encourage you to explore Tigerbeetle further. I am certainly going to figure out how and where I can use such a database in the future.