Reorgs, Electrum and P2P

Overview

As work on the second refactoring effort proceeds, we need to ensure that the model we use to respond to reorgs is correct.

Current approach

  1. The wallet is notified that a transaction is mined, by examination of the scripthash history for a key that is used in the transaction.
  2. The wallet knows when the blockchain is reorganised through processing the headers that it receives from any of the servers it is connected to, through a different chain of headers becoming the longest.
  3. Transactions above the fork height have their proof removed, and a new proof is fetched for each.

Flaws

Headers are synchronised independently of open wallets in ElectrumSV. If for instance a user sits on the wallet wizard for a while, it is possible that a reorg may be detected and processed before the wallet is loaded. There is no guarantee that the reorg will be detected. Similarly, if the user opens multiple consecutive wallets in the same ElectrumSV instance, it is very possible that one will be opened after the initial detection of any reorg. It is expected that the most common case is that most users do not keep ElectrumSV open all the time, and there is a good chance that any applicable wallet may not be open and listening when a reorg occurs.

ElectrumSV marks the block a transaction is mined in, with a block height. This is not enough to know if a reorg occurred while the wallet was not open.

For blockchains that use faux SPV, and do not plan to do real SPV, the merkle proof is not kept. That the transaction was once in a block is kind of enough, and the merkle proof is obtained to verify it was there, but discarded otherwise. That it can verify that it was in a reorged block, and reverify that it is in the new block, but not reverify if Electrum was not open at the time of the reorg is kind of a half arsed measure though.

Possible solutions

The networking code does obtain the latest headers on startup, and will detect any change in it’s last known longest chain, by doing so. Any loaded wallet would then have to process any reorg that happened while it was offline, in order to ensure their transactions had valid proofs. But it wouldn’t be enough to store this information in memory, it would have to be recorded indefinitely. There is no guarantee that any given wallet will be loaded at any point, so the information must be retained.

Worse than this, there is no guarantee that the wallet will be opened in the same ElectrumSV instance with it’s existing blockchain headers and reorg information. A user may upgrade computers and never become aware of a reorg. This renders the solution untenable.

Since around ElectrumSV 1.3.5, when a wallet was unloaded we started storing values for “stored_height” and “last_tip_hash”. These reflect the local state of the blockchain headers, at the time the wallet was unloaded. It should be possible to have a background process on wallet startup that reconciles these against the known block heights of the user’s mined transactions. To do so, we would want to add the “block_hash” field to Transaction records, and compare that against the current hash for the given height.

We could not do this as part of a wallet migration, because there is no guarantee that the applications block headers are synchronised, or even can be synchronised at that point.

ElectrumSV developer