Best practices in fork handling?

If you are looking for an article telling you how to write bulletproof blockchain state handling given fork edge conditions, I am not sure this article is it. I am not a Bitcoin expert. If you are looking to explore the possible edge conditions, then this article may help with that.

Block headers and forks

Any block header has some well known properties of interest to a wallet:

• It links to the preceding block headers and naïvely represents represents the preferred fork for a given block height, if it has the most proof of work.
• It has a merkle root which can be used in combination with a merkle proof to prove that any transaction that was mined in the block, was actually mined in the block.

The only fork that matters (until it doesn’t)

There is no observably correct official Bitcoin SV blockchain. For most mined blocks, this is not a problem as there is only one block mined at a given height. But occasionally there will be competing blocks mined or even a dishonest attack on the blockchain. There will be times when the observer will not know which of the new blocks mined at the same height are the ones that will be further built on.

Forking incompatibilies

It is very possible for a wallet to follow the wrong fork for a period of time, and to receive merkle proofs for their transactions, for that wrong fork. To complicate this further, the wallet may be using a service for blockchain-related data which also chooses which fork to follow themselves. Unless the wallet can say for which fork they want data, or can explicitly follow the same fork as the service, there is no guarantee that the data they request will be for the right fork.

Services and forks

The wallet may ask the service for the merkle proof for a transaction, and if the transaction is in a block, they will get back the proof for that block. However, if the service is on a different fork the wallet now has two problems.

The proof against the other fork block cannot be used and it needs to obtain the proof for the block on it’s own fork. This is also not limited to merkle proofs, any other blockchain state that is related to a service processing a specific fork with the assumption it is the fork that will survive, will also produce results specific to that fork.

One source of blockchain state

In the simplest case, your application just uses one stateful blockchain service, a given miner’s MAPI service. You have no way of knowing what fork the service is following, outside of the headers provided in the merkle proofs. This means that if you get a proof notification, you can look at the header in the proof and identify whether they are on a different fork. And if you have already had a proof for the given transaction, it is possible they will send you a further proof if there is a reorg. Since you only have one source of state, you can just locally reorg to the anointed fork of the MAPI service.

How the Electrum wallet works for BTC and BCH is that it obtains the fork each server is using and picks one of the servers that is on the longest chain. This server is anointed the “main server”. Then it queries the full state of the wallet against the “main server”, and updates the wallet to reconcile with the fork the new server is on. All state updates now come from the “main server”. If it turns out the wallet is following the wrong fork, the user can go to the network dialog and force the wallet to connect to a specific server that is on the right fork.

This has long been a thorn in the side of ElectrumSV in that it is entirely possible to not correctly process reorgs and have a wallet with transactions in the wrong state — however, this does not matter so much for BCH or BTC where the proofs are discarded after transactions have been verified as having been mined in a block.

Multiple sources of blockchain state

But what happens if you use multiple services, each of which cannot be relied upon to be synchronised to the same fork at the same time? Now you cannot necessarily reorg when you see a service is on a different fork. The application will want to unregister all it’s usage of the service on the wrong fork, and reregister them with the service on the right fork. It would be a lot simpler from an application’s perspective if services followed all viable forks, and you could specify which one you want your state relative to. But this has other problems, like being more likely to result in you not receiving notifications about useful events.

Let’s say you are receiving notifications of blockchain state from a server for new blocks, but have no way to specify that you want it for a given fork. If the server provides you with the changes relative to another viable fork, then you can store it for processing in case you need it. If the server reorgs, it is responsible for sending you the changes above the reorg height to ensure it delivers you the state you asked for. In this case you did not miss any changes, even if the fork was different.

Let’s say you are receiving notifications of blockchain state from a server for new blocks, and can notify the server you are following a fixed fork that turns out to be the wrong one. The server will have processed the other fork in parallel, and will never reorg, so you will have missed out on the changes for the fork that lives. You may have to pay it additionally to reprocess those blocks to look for the things you are interested in. Alternatively, the server might follow all forks and notify of all changes, which might be a better model — but I do not believe any current services do this.

My suspicion is that the best way to do an application is to use some heuristic to choose the fork to follow. To take the state from services and reconcile it, storing it in case it is needed, and to be aware of inconsistencies. In case there it is not obvious which fork is the viable longest honest fork yet, notify the user that there are some constraints imposed.

Summing up

This article is about the problems with choosing one fork from multiple viable possibilities, when presenting the application and it’s spendable coins to the user. With the use of services that provide blockchain state to the application, when their choice of fork may differ from that of the applications.

There are no answers in this article, just some consideration of the possible problems.

The “early days” disclaimer

The future services, applications and usage of Bitcoin SV are still in development. The best ways to write a service, or features services need to provide for eventual ideal application usage, are not yet clear. Bitcoin SV is the only blockchain aiming to welcome usage, and this requires a lot of work to build out the infrastructure and standards.

It is interesting to consider that these things do not exist in BTC and BCH for us to wholesale adopt and build on. That for all the years BTC has been around, now ten or more, there has never been any serious usage to require these the time and money invested to build this out.

But here we are and this is happening now on Bitcoin SV. It will take us time and some iteration, to reach the solutions and best practices to be used in the longer term.

Make it someone else’s problem

All these problems go away if you just one of the up and coming wallet as a service solutions. But you get new problems, like their supported use cases may not be a clean union with your desired use cases.

Some possible examples of different ways to do wallet as a service are Handcash Connect and Relysia.

Best practices

It should be very possible to lay out the correct way to follow the right fork, and how to reconcile having been on the wrong fork.

That’s it. This article isn’t intended to provide any answers, just scratch an itch.