On P2P, headers and reorgs

As part of ElectrumSV’s work on preparing for when the ElectrumX indexers die, I’ve been looking at what we need to do, and in the longer term how we combine the SPV-oriented LiteClient model introduced by Steve Shadders at CoinGeek NY 2021 with the blockchain monitoring we need to do .

I think ElectrumSV is in a unique position in that we are the closest to actually being a P2P application and this means that we need to do some things other applications do not, and we have different requirements from other applications.

I am not a Bitcoin expert. This article is intended to take a short step back, summarise thoughts on that subject and put them out there to get feedback.

P2P

What even is P2P? If a HandCash wallet talks to the HandCash servers and they send a transaction to the MoneyButton servers, is that P2P? Or is that client-server followed by server-server?

There is an inherent advantage in a business offering a client application which only interacts with their servers. It makes development easier and can make the product developed a lot more reliable and better quality. If an application is talking directly to other applications, and your server is not in the loop, then it becomes a lot more difficult to diagnose problems and introduces a lot of potential situations where things might go wrong.

What if CentBee’s mobile application in addition to routing through the value-added services of their own server, also made direct connections to other parties? The canonical case might be direct IP to IP connections between the user’s mobile application and another user’s mobile application.

The first problem is that IP to IP is not going to happen. It won’t work and it shouldn’t be used. People cannot expect to be in a situation where the network (public wifi, home, business, etc..) they are connected to allows them to receive external connections. Even if they are connected to a network which will allow it, you do not want to be giving out your IP address to other people. Cloudflare has a whole proxying service that they offer where you can avoid getting attacked by not giving someone your IP address and having the other party connect through their proxy instead.

This relates to Craig talking about IPv6. IPv6 allow the creation of temporary addresses that can be generated cryptographically (CGA). So you could create an address for a specific person, use it for the duration of communication and then dispose of it. This is also not going to happen. For this to be usable, you would need every network the user’s device is going to migrate between to support it and to ideally support the movement of that address. There is almost no support for these CGA addresses, and whether the addresses can be migrated between networks works or can even work is irrelevant at this point because of it.

This then brings us back to the client-server model that all the wallets use, RelayX, CentBee, HandCash and so on. It is the only way an application can have it’s users reliably interact with other users and whatever application they are using. It takes away the whole problem of the internet not supporting direct connectivity between user’s devices, because these application-specific servers proxy the application’s traffic and manage their usage.

ElectrumSV as a wallet does not run a server, in theory it should be the most P2P of the wallets. But even ElectrumSV relies on the free ElectrumX servers to proxy it’s traffic, and provide access to both data and services that cannot be provided on the user’s device by ElectrumSV itself. However, we can connect to any server the user wants that implements an open API, which means that we are probably as close as you can get to P2P realistically.

If a P2P application wants to interact directly with other parties, then the simplest way is to use a relaying service. This also solves the problem of giving out the user’s IP address, in that any connection established is indirect and both parties are connected to the relaying service.

If we look at Peer channels as a message relaying service, it is probably the shortest path to getting a direct relaying service. If the application is connected to a web socket on the relaying service, then they can bypass the message box and exchange messages directly.

It then offers the ability to reliably establish a proxied connection to another party, as long as you both are connected to the lowest common denominator of networks, ones with outgoing access to web sites. And it offers the ability to leave messages if the other party is offline.

What we are currently seeing is client-server centralisation. This is where all the services are offered by the businesses to the user’s of their own application. RelayX offers Paymail hosting to user’s of their applications, and similarly with MoneyButton and the other wallets. This makes sense because they are offering wallet services, and Paymail is currently primarily a way to send and receive funds into their wallets. Additionally time, money and attention are valuable resources and pouring them into one’s primary focus is the sensible thing to do.

Unfortunately, I suspect this is driving BSV development in the direction of the client-server model where use of remote services is by the server for a given client. The scarcity of applications that are closer to P2P, like ElectrumSV, means that the functionality needed to support them is not necessarily being focused on.

ElectrumSV needs LiteClient services our users can access and use. P2P applications in general, outside of client-server ones that have incentive to do their own, will also need these things.

To SPV or not to SPV

The SPV model is the path forward for the majority of what application developers will want to do. It removes the need for developers to run their own nodes, and in the form of the LiteClient services enables them to do the same things and more in a better way.

Monitoring the blockchain will not be going away, and ElectrumSV will still provide a way for user’s to receive legacy payments to an extent, and do things like restore wallets (at cost since it becomes more work with bigger blocks) and much more.

Headers

I have a strong suspicion that the requirements for how ElectrumSV as a P2P application needs to get headers, are pretty strict. I’ll go into that in a bit, but first let me describe the model the Electrum wallet has used for perhaps up to ten years.

ElectrumSV has a set of built-in servers whose domain name we know, and a way of hearing about new servers from those built-in servers. Once it has started up, it tries to stay connected to ten of these. It sends a message to each saying “I want to hear about new headers” and in return it gets a response containing the blockchain tip header the service is on and the height of that tip. It synchronises the headers it does not have up to that tip. At this point it knows about all the current forks, and picks the one with the most work as the valid fork.

There is one problem here, the servers are connected to by the JSON-RPC protocol and are often on non-standard ports like 50001. Much like we have to use relaying services to deal with the fact the network we are connected to may not allow incoming connections, there is no guarantee that the network we are using allows outgoing connections on non-standard ports. If we are to access services and their functionality in a way that always works in the widest variety of networks, then we probably want to do it by outgoing access to web sites.

The Bitcoin P2P protocol also uses a non-standard port which should have the same problem. If someone is running ElectrumSV in their home or business and the outgoing ports are open, then it could be used to obtain headers. However, I use a laptop and I travel with it, and I want ElectrumSV to work in all the places including the ones it didn’t before. And if we do add mobile versions of the wallet, then we want these to just work in every possible location. And also, this relates to a wider point, this will likely apply to all P2P applications that are not client-server. This isn’t just an ElectrumSV thing.

If this argument about reliable connectivity needs for P2P applications is correct, the future for all API usage is then likely to be REST APIs for requests and web sockets for streamed events.

This is pretty much copied from the ElectrumX server for a specific reason.

Let’s start with a web socket. The client connects and receives a packet containing both tip height and header. Then it receives any new tip height and header in the same packet format, every time the server moves to that new tip. It is important that if the service provides other functionality related to the blockchain and transactions, that they only broadcast a new tip when they have finished processing any state related to that block so that the processed results are available.

Next we need to handle the offline case. What if the user’s device was offline or disconnected for a while? Where does it get the headers it does not have. For this we add a REST API that gives a base height and number of headers to request.

This then gives us the following steps for a simple but not recommended model of header synchronisation:

  1. The application connects to the web socket and receives the tip for the server.
  2. The application requests the missing headers up to the tip.
  3. The application adds new tips from the web socket to it’s local blockchain headers.

The problem with the simple model of header synchronisation is that there may have been a reorg. This then leads to something like the following model:

  1. The application connects to the web socket and receives the tip for the server.
  2. The application requests the single header above their current height. If it does not connect there was a reorg to some part of their existing state. The application then requests headers (probably with a binary search) to locate the fork height and it then uses this as their starting point
  3. The application synchronises headers using the REST API from their starting point to fill in the missing headers.
  4. The application adds new tips from the web socket to it’s local blockchain headers.

Beyond this the application needs to reconcile it’s state with the reorg. The application may get MAPI callback messages for new proofs in their peer channel, or maybe they need to do more work.

If ElectrumSV and perhaps other P2P applications want access to APIs like the one above, in order to get headers, who will provide them? Ideally we would want multiple servers so that there are both redundancies, and a resilient infrastructure. Is there an incentive for services to provide this API?

A nuance to this is tip signaling. ElectrumSV actually judges the servers it uses by the tip the server has. It will only pick a server to use for functionality if it is on the longest chain. This has value in ensuring that the state it gets from these servers will be the correct state, and not incorrect. It might be that this is primarily valuable when an application uses a service for blockchain-first functionality, and not so much when it uses a service for SPV-first functionality.

I’ll throw in the most extreme case of two offline P2P applications synchronising headers between themselves over bluetooth, just to mix things up as well.

Final thoughts

This article is not intended to tell you how things should be done and it is by no means a full picture of the covered topics.

It is solely intended to share the sorts of things that me and my fellow developer on ElectrumSV, AustEcon [SV], are thinking about as we both move forward to supporting both the SPV-first LiteClient model and shoring up our blockchain-first support for when ElectrumX dies. And to share my own thoughts on P2P applications and their needs.

Keep in mind that the highest priority for ElectrumSV is to get everything ready to replace ElectrumX before it dies. The header API we propose is a short term “get it done” ElectrumX functionality replacement, and it benefits from being a web-based drop-in replacement for what we already do. It may not even be the longer term solution, where we move to a specialised form of the LiteClient solution.

As they say, sunlight is the best disinfectant. So feel free to point out any dumb ideas, or misconceptions. You can share those and any other thoughts on the above, either on the Metanet.ICU slack if you are a member or in our Github discussions.

ElectrumSV developer