
# Trail of Bits Security Audit

Today, we’re thrilled to share the results of our first major audit!

We engaged the legendary software auditing firm Trail of Bits, challenging them to find issues in our suite of libraries: [Orga](https://github.com/turbofish-org/orga), [Merk](https://github.com/turbofish-org/merk), [Ed](https://github.com/turbofish-org/ed), and [abci2](https://github.com/turbofish-org/abci2). In parallel, they also audited the primary application of this software, [Nomic](https://github.com/nomic-io/nomic).  The full report of the audit findings is available for review on Trail of Bits Publications repository on GitHub:


**[Trail of Bits: Orga & Merk Security Assessment](https://github.com/trailofbits/publications/blob/master/reviews/2024-11-orgaandmerk-securityreview.pdf)**


In this post, I'll explore some of their findings and our learnings, with a focus on Turbofish's libraries -- if you'd like to read about the Nomic audit results specifically, head over [here!](https://blog.nomic.io/security-audit-trail-of-bits-8ae87ce19bf0)

## Background

**Orga** is our Tendermint-based state machine engine, the framework we use to build blockchain applications in Rust. Under the hood, Orga is powered by **Merk**, one of the fastest Merkle key-value stores on the planet. Check out [this post](https://turbofish.org/blog/orga) for more context.

**Trail of Bits** is a leading cybersecurity firm, with deep expertise in both blockchain and traditional software security. They showed their extensive expertise and experience with Rust, having previously audited Rust-based blockchain projects like Drift Protocol and Solana.

Two of Trail of Bits's best and brightest, Tjaden and Anish, set out to answer the following questions (in their own words):

- Can malicious parties cause the Orga state machine to misallocate funds?
- Can a minority of stakeholders gain undue control of the state machine?
- Are there DoS attack vectors that could halt the forward progress of the state machine or prevent node synchronization?
- Are third-party protocols such as IBC or Tendermint integrated correctly?
- Can a malicious attack spoof the existence of a node in the Merk tree?
- Can malicious inputs cause serialization/deserialization routines to trigger a panic?
- Do the arithmetic calculations in the system account for potential overflows, precision loss, and/or division by zero?
- Does the state machine manage accounts, balances, signatures, and fees correctly?
- Are the uses of cryptographic signatures, hashes, and commitments vulnerable to collisions or replays?

Spoiler-alert: no critical issues were discovered -- but the things they did find are still pretty interesting!

Let's jump into some of their findings.

## Findings

### Slashing of re-delegated stake is computed incorrectly
> When a validator is slashed due to double-signing, any stake delegated to that validator at the time of the misbehavior must also be slashed. The orga staking module allows delegators to transfer delegated coins from one validator to another; these coins are tracked so that the proper amount can be deducted if the original validator is slashed.
> 
> Due to a miscalculation in the staking module, coins that are slashed while in the process of being re-delegated will forfeit an incorrect percentage of funds. For example, if the slash_fraction_double_sign percentage is 1/20 , the re-delegated stake will be slashed by 95% rather than the intended 5%.
> 
> This miscalculation was not detected by the unit test suite because all unit testing is done with a slash_fraction_double_sign of ½, which is the unique value x for which 1 - x = x.

This is probably the most interesting issue, for two reasons:

1. It *technically*, *theoretically* could result in a loss of funds -- the main thing you're trying to prevent when you're working with cryptocurrency.
2. There's a generally-applicable lesson when testing software: be careful with 0.5!

For point 1, if this situation had ever occurred in a real application, only outbound redelegations would have been affected, resulting in an incorrect slashing penalty.

On point 2: if our unit tests had used *any* double-sign slashing rate other than 0.5, this error would have been screamingly obvious. The most intuitive value to reach for, in this case, was uniquely bad for detecting this bug. Think of something more creative the next time a unit test requires an arbitrary number from 0-1.

### State sync issues

> Malicious state sync peer can cause syncing nodes to crash


> Interrupted snapshots can lead to inconsistent state


> Malicious state sync peer can cause a stack overflow in Merk


State sync is a great feature of Tendermint and Orga apps, allowing a totally fresh node to catch up with the network in mere seconds!

The auditors found some issues which could be exploited by malicious nodes to cause a state syncing peer to crash. These issues were resolved, and malicious node operators are back to the drawing board.

### Stored IBC consensus states cannot be pruned

Orga includes an IBC module, built on [ibc-rs](https://github.com/cosmos/ibc-rs/), which enables communication with any other IBC-compatible chain (e.g. for token transfers to other IBC-compatible chains).

The auditors discovered an issue that prevented the pruning of consensus states -- the deletion attempted to remove from the string key "revision_height" instead of "revision_number-revision_height". Not dangerous, but still a good find, preventing state bloat in the long-term!

### Merk proofs can be forged to claim arbitrary key/value inclusions

> A malicious actor can construct a fraudulent Merk proof that demonstrates the inclusion of any arbitrary key-value pair in a target tree root. This could allow attackers to steal funds from bridges and other light clients by fraudulently claiming the existence of transactions that are not included in the genuine consensus state.

A core feature of a Merkle tree is the ability to construct partial proofs, proving the presence or absence of a node within the tree with a known root hash.

During a deep dive into Merk, the auditors discovered a way to forge existence proofs of absent values. Orga's usage of Merk isn't vulnerable to this attack, but it does violate a core promise of Merk as a standalone library and has been fixed!

## Conclusion

Overall, we're extremely pleased with the outcome of this audit. The auditors worked diligently, walking us through their processes throughout the duration of the audit. They discovered hard-to-find issues at different levels of the stack, and yet nothing critical was found. Thanks to them very much for their hard work!



Orga & Merk Trail of Bits Security Audit

Rust Foundation - Member Spotlight: Turbofish


# Orga: A Rust State Machine Engine

Hello, world!

We're Turbofish - a small team of Rust hackers in Austin, TX building next-gen decentralized systems like [Nomic](https://nomic.io/).

We're craftsmen. The joy of engineering things *right* motivates us as much as the finished product. So, in this blog post, I'm excited to share some detail about how we work, and one of our more behind-the-scenes projects, [Orga](https://github.com/nomic-io/orga).

You may already know this if you're on this blog, but we're founders and core contributors to the Nomic Bitcoin Sidechain. We believe **Bitcoin is the reserve currency of the internet**, and Nomic aims to bring Bitcoin to the rest of DeFi via Cosmos and the IBC protocol.

What you may not know, however, is that Nomic is built on a custom state machine engine that's been in development since 2018, Orga.

A priori, building a custom blockchain stack **and** a Bitcoin bridge seems a lot harder than just building the bridge. So, why build a custom stack instead of using existing tools?

### **Why we built Orga**

During our work in the early days of Cosmos, back before the Cosmos SDK had even been named, we developed [LotionJS](https://github.com/nomic-io/lotion): a JavaScript framework for writing blockchains with minimal friction. Our heuristic for Lotion: "It should feel like writing an Express app."

We formed strong opinions about how to write blockchain apps. The Cosmos SDK then matured into a solid piece of software, but it was too late for us; we'd been spoiled.

We wrote the first version of Nomic in JavaScript, but quickly felt the growing pains of the language and the framework, both in terms of performance and of managing complexity. We'd nailed "easy," without quite nailing "simple" or "composable".

## Merk

Our focus shifted to improving performance, so we began to rewrite just our custom Merkle KV store, [Merk](https://github.com/nomic-io/merk), in Rust. More specifically, we built a high-performance Merkle AVL tree on top of [RocksDB](https://github.com/facebook/rocksdb) (Facebook's fork of LevelDB).

Merk has now grown to offer the best performance in the industry, even offering from 2x to 20x better throughput than Jellyfish Merkle Tree (the storage engine used in Aptos, and currently thought of as the fastest Merkle tree available).

The storage layer is the most critical part of optimizing transaction throughput on a blockchain since every transaction creates reads and writes, meaning chains built on our stack have the potential for extremely high throughput.

You can learn more about Merk here. Keep an eye out for further posts on Merk and Merkle trees in more detail, and open-source some benchmark code for people to reproduce the results we teased here.

Cliche as it sounds by now: building Merk naturally became the start of our "[rewrite it all in Rust](https://github.com/ansuz/RIIR/issues)" journey.

## Orga

So, we developed Orga: a Tendermint-based state machine engine with a focus on composability, expressiveness, and performance. Our golden rule: "It should feel like vanilla Rust."

The result is very clean and succinct code, especially when compared to Cosmos SDK. Comparable functionality can be built with an average of 10x fewer lines of code, meaning less complexity, fewer bugs, and faster development cycles. All this without any tradeoffs since the result is a significant performance increase.

We value bottom-up design. We don't want to just build our programs down to the language, we also want to build our languages up to our problem domain.

To make this possible, we have created well-designed abstractions and we leverage the same construct as the seasoned LISP programmer: macros, which have been refined by dogfooding with Nomic for multiple years.

Here's a little taste:

<div className="not-prose">
```rust
use orga::prelude::*;

#[orga]
pub struct Counter {
    counts: Map<u8, u32>,
}

#[orga]
impl Counter {
    #[call]
    pub fn increment(&mut self, index: u8, n: u32) -> Result<()> {
        let count = self.counts.get_mut(index)??;
        count += n;
    }

    #[query]
    pub fn count(&self, index: u8) -> Result<Option<u32>> {
        self.counts.get(index)
    }

}
```
</div>

I'm not sure why you'd need a Globally-Distributed Instant-Finality Byzantine Fault-Tolerant Number Counting Application, but if you did, here's the whole thing. Just add validators.

Orga also features an advanced RPC system (calls and queries) which lets you build business/on-chain logic once, then reuse those types/methods on the client-side. So no extra code has to be written for clients, all data and logic are available to clients by default (also automatically generates and verifies proofs for all of this functionality, so that the data can be served by any node in a decentralized way).

In future blog posts, we'll talk more about the core ideas in Orga. It took a lot of exploration to find what we consider to be close to the global optimum in the design of the core concepts. We'll also share more thoughts on how we intend to push performance and scalability to the limit as we implement [concurrent transaction processing](https://github.com/nomic-io/orga/blob/ibc2/docs/concurrency.md) and improve our [store throughput](https://github.com/nomic-io/merk#benchmarks) (which is already extremely high).

Zooming out a bit: could we have built a Bitcoin bridge faster using existing tools? Probably. But it wouldn't be the kind of bridge we're building; it would've required compromises on complexity and scalability. We're inspired by Bitcoin to build something that'll stand the test of time.

We've invested a huge amount of effort into our tools and recently reached the inflection point where it's starting to pay off. I think you'll soon start to feel the love and craftsmanship leak out around the edges of everything we build, including Nomic and our future projects.

We’d love to hear your thoughts on Orga, as we continue to build. And if you’re good at Rust, we’re [hiring](https://boards.greenhouse.io/turbofish)!
