Incomplete transactions, transaction templates and PSBT
This article is a way for me to write up my thoughts as I integrate the PSBT transaction encoding so that existing Bitcoin Core businesses can more easily integrate their tooling with ElectrumSV.
Let’s define some terminology:
- A fully signed transaction is considered “complete”.
- A transaction that is not fully signed is considered “incomplete”.
- “PSBT” or “Partially signed Bitcoin transaction” encoding was created for Bitcoin Core to store or communicate both incomplete and complete transactions with metadata about the contents.
- The “extended transaction” encoding was how the Electrum wallet exchanged incomplete transactions before PSBT was standardised.
The Bitcoin Core ecosystem has standardised on PSBT. It’s integrated into their node software, their Electrum wallet has integrated it and hardware wallets are phasing out their custom protocols of communicating transaction data with PSBT integration.
The PSBT format is also Bitcoin Core oriented, featuring their devtopian Segwit and Taproot additions as formally defined input and output fields. It is also based around parent transactions being provided for spends because of a perceived exploit where manipulation of the user may result in large fees being paid to miners. If you are spending several coins from the same parent transaction in your incomplete transaction, this means the parent transaction will be included several times.
The “exploit” and the repercussions
Users blindly sign things and they do not think things through. This is partly human nature and partly due to the extremely complicated nature of Bitcoin, and especially Bitcoin Core. In the example which was used to bring this whole notion of “the parent transaction must be provided” as a Bitcoin Core standard, the user had to be complicit in the act of their being defrauded. If a user signs things twice, or signs things blindly, then of course they are open to be cheated. And they will do this unless the technology is made more approachable and simpler, and will continue to do it past that point because we’re not robots.
To bake in the requirement for parent transactions to be provided because of a contrived example where the confused regular person is confronted with complicated devtopian devices and wallets and just wants to make a payment and then intentionally ignores the warning signs.. it’s odd. That this is blindly accepted and baked into their protocols and devices, is a sad reflection on the philosophy that underpins Bitcoin Core.
We do not need to include parent transactions for each spent coin for signing interoperability. It is redundant, and was added for no good reason. If a user ignores the warning signs, then they accept responsibility for doing so.
If the goal is to allow existing Bitcoin Core businesses with their own wallets, applications, libraries and tooling to easily integrate with ElectrumSV, then we must start by working with what they use. We can use their non-witness path where parent transactions will be included in each incomplete input. PSBT version 0 support is already been working.
As you can read above in the Ledger link, they are requiring PSBT version 2. The key difference in version 2 is that version 0 requires the incomplete transaction to be embedded as the base transaction to work from, where version 2 does not allow it to be included and specifies all relevant fields in PSBT section entries. We do have some simple support for version 2 and it is possible we will upgrade our hardware wallet libraries to use it, but it is a lot of work for little gain given that hardware wallets are based on the “standard transaction templates only” model that the Bitcoin Core controlling developers modified the protocol to be based around.
So we will support PSBT version 0 encoded transactions as long as they don’t contain incompatible fields like those defined for Segwit or Taproot. You will be able to read them from disk, write them to disk, and exchange them in QR codes.
Possible BitcoinSV adaptations
Now I think it is reasonable to say that the whole inclusion of parent transactions when the context is exchanging incomplete transactions with other parties, is odd. It’s a conceit made based on the extreme problems faced by the problematic hardware wallet use case — I’ve several other articles that go into how their paradigm does not work on Bitcoin SV, the latest being this one. And even then, the problem only occurs if the user is considered to be a mindless automaton.
The inclusion of the parent transaction and the blind acceptance that it is necessary, means that the PSBT format does not encode fields that contain the actual useful information it includes.
- Spent output value.
- Script template identifiers.
- Signing negotiation.
- Public keys.
- Non-BIP32 derivations.
Spent output value
We need to know the spent value of a given input for two reasons, the first is that it helps to display it as context for the signing user and the second is that it is part of the signature algorithm that the Bitcoin Cash developers included as part of their fork. You cannot sign a spend without including the spent value. It’s sad that I can just have the phrase “BIP 143” come to mind, google it and see the signature algorithm Bitcoin SV uses there.
Script template identifiers
Another thing we would benefit from having is having the actual script template declared. This could happen for both inputs and outputs, and would be a valuable addition. In the past you could look at the scripts you did have and infer the template based on the set of standard script types the Bitcoin Core managing developers dictated as valid. But with a restored protocol you shouldn’t have to do this, you should instead be able to work with declared templates. Anyone at this point enforcing a fixed script structure with script hashes is still working on unrestored protocol technology.
So what might a template declaration look like? Why are you asking me? I have no idea. Here’s a completely arbitrary suggestion. Let’s say we use mime-types.
- P2PKH might be “standard/p2pkh”.
- P2PK might be “standard/p2pk”.
- Bare multi-signature might be “standard/bare-multisignature”.
- P2SH multi-signature might be “standard/p2sh-multisignature”
- An older version of an sCrypt contract might be “sCrypt/zk-snark-thing.3.45” (which 3.45 is an updated version of that contract).
And so on. Then again you could do a binary format where there is a contract identifier segment and a contract version segment. Lots of different ways.
The PSBT and Electrum extended transaction formats define which public keys are required to sign by including the master public key and the derivation path. This is information that could perhaps be given to co-signers.. but will mainly be used by the owner of the corresponding master private key to sign their own transactions.
It is more likely that there will be different entries for different types of collaborative transaction construction and you will not reveal BIP32 extended keys and derivation paths to other parties.
One gap illustrated by the P2PK template is that there is no way to know what public key should be used to sign a P2PK spend. If the parent transaction is provided, sure it can be extracted from there, but it is easier just to specify it and not force inclusion of parent transactions. This is an easy addition of a new input key type to PSBT.
This is an indication of a more general problem to be considered when looking towards more types of non-standard locking scripts and what is needed to know how to sign their unlocking scripts.
Both Electrum extended transactions and PSBT enforce BIP32 derivations from a master public key. There are a lot of possible alternatives to this including nChain’s whitepaper 42, and we should support them.
That’s pretty much it. It often helps me to write these things down to give me another perspective so that I can see if there might be anything that feels like it is missing. And to put the information out there, and hopefully receive some feedback.
PSBT isn’t being integrated as the future path for coordinated transaction signing, but to provide an easy onramp for existing Bitcoin Core businesses to integrate with ElectrumSV. It is however there, and suitable to simple amendments to make it more suitable for Bitcoin SV. Maybe it will be the future path.
Some additional notes are provided here, but are not necessary to read unless you want to.
Electrum extended transactions
The Electrum extended transaction format is the Bitcoin transaction format but with some amendments that can be detected by compatible applications and processed.
When a transaction input is deserialised, the
scriptSig is parsed and metadata is extracted. Given that Bitcoin Core only allows standard transactions and that no key usage that varies from this are supported, it is possible to determine if the spend is P2PK, P2PKH, P2SH or bare multi-signature. Additionally the number of signatures that are required can be determined, and how many have already been provided.
Absent signatures have the placeholder byte
FF in their place in the script. Public keys are replaced by an extended public key, where the prefix
FF specifies a BIP32 xpub and includes both the xpub and the derivation path of the required signing key. Old Electrum style keys have the prefix
FE, include the “old Electrum” master public key followed by the derivation path.
This was how offline signing worked, where a user could have their main wallet be watch-only and not have signing keys and have a second offline air-gapped computer. When they go to create an outgoing payment they would save the file to disk, take the disk to the second computer, then load the extended transaction and sign it. The second computer needed nothing other than the signing keys, knowledge about BIP32 derivations and the transaction.
Similarly, you could pass an extended transaction around co-signers of a multi-signature transaction, and when the signature threshold had been passed the transaction would no longer be incomplete and would no longer be extended.