Author’s Note: The below post is meant primarily for a technical audience. While the end result of this optimization is reducing scenarios where customers would be affected by gap limit issues and making onchain payments with Zaprite simpler, the details themselves are very “inside baseball.” Most importantly, this does nothing to affect your day-to-day usage of Zaprite onchain payments, other than to make Zaprite work more seamlessly with standard bitcoin wallets, like Coldcard, Trezor, Ledger or Sparrow, etc.
By Will Cole
In what will almost certainly not be our last word on the subject, Zaprite has updated how we handle onchain address selection for connected wallets to improve the payment experience. The following only concerns onchain transactions, specifically to non-custodial wallets. If you’ve connected an Xpub from any hardware wallet, or connected your Unchained multisig wallet, this will be of particular interest. With that said, we’d like to take you behind the scenes on how we’re handling non-custodial onchain transactions to your preferred wallet.
When one of your customers initiates an on-chain payment on Zaprite, an address from your wallet is reserved for a week. This rather long reservation time is required to guarantee that we associate transactions detected on that address to the right Zaprite Order. If no payment is detected on an address after a week, the reservation expires and the address becomes available for a future payment. This means that many addresses could be reserved at the same time for multiple Orders. Our goal is to make sure we are always able to reserve a new address for a new payment.
While a bit esoteric, if you’ve accepted many onchain transactions to a single wallet, there is a good chance you have been exposed to a problem space referred to as “the gap limit.” However, more and more businesses are beginning to accept bitcoin payments and in the grand scheme, very few people have ever heard the term. In short, the gap limit refers to the number of addresses a wallet looks at to determine the wallet balance. Bitcoin wallets typically look ahead for a certain number of transactions, scanning a bitcoin node to look for deposits to those addresses. The number of addresses that a wallet scans ahead is the gap limit, and the policy can vary depending on the wallet.
If there are large gaps in the sequence of used addresses (addresses that have received transactions), some wallets may not detect an address with a balance, specifically if the gap in addresses exceeds the limit of the look ahead. In this case, the balance would appear lower than expected. Typically, addresses are selected in index order: starting at index 0 and incrementing one by one to index 1, 2, 3, etc., where at each index there is a new address corresponding to your wallet. Most wallets will only look for addresses 20 indexes ahead of the last used address.
If the wallet does not see used addresses within 20 indexes, the wallet will assume there are no balances in addresses outside that index limit. Imagine a scenario in which you run a popular eCommerce store and twenty customers click to checkout but have not yet completed an onchain payment, the next address would be the 21st address and if it were to be paid prior to the first twenty addresses, your wallet would likely not see the transaction or balance (though it would still ultimately be available and spendable).
This edge case is why we’ve optimized how we select addresses to maximize the number of addresses we can simultaneously reserve on your wallet, while ensuring that your wallet will see and recognize all incoming deposits–effectively expanding the gap limit. Our approach ensures flexibility and efficiency in how addresses are used and reserved, deviating from the traditional method of using addresses strictly in index order, which is more appropriate if a user is only using a wallet to deposit to cold storage (e.g. transfer from a bitcoin exchange).
Breaking Free from Index Order
Traditionally, bitcoin wallets use addresses in sequential order. This method, while straightforward, doesn’t always make the most efficient use of the addresses within the gap limit and particularly not for commerce applications given addresses often need to be reserved for transactions on checkout that may or may not be completed. Our solution, however, adopts a more dynamic approach to better account for a commerce use case:
Address Reservation
In a commerce application, Zaprite needs to reserve addresses for specific orders, both to tie individual payments with specific business transaction context AND to ensure onchain addresses are not reused, which is a best practice for privacy and security.
When a customer of a Zaprite user goes through an onchain checkout flow to pay for an order, Zaprite provides a quote for an amount of bitcoin and reserves a bitcoin address for that particular order. The quote is valid for 30 minutes and Zaprite reserves the address for 7 days (for the edge cases where a Zaprite user’s customer pays late or a batched transaction from an exchange is delayed). If a payment is made, Zaprite will recognize the incoming payment, associate it with the specific business order and the address displayed will only be used once.
If a payment is not actually made (e.g. if a customer decides not to complete an order), the address will be returned to the pool of usable addresses after the 7-day period expires. This is common user behavior in commerce checkouts, where a customer will fall out and not complete a payment order. And as a result, we need to account for such scenarios in address selection, particularly given the standard gap limit for which most bitcoin wallets scan for incoming deposits.
Dynamic Selection
Given most wallets look ahead for a minimum of 20 addresses from the last known address with a balance, we’re now utilizing the address at the index furthest out first that your wallet will scan for incoming deposits AND then work backwards, such that we can expand the universe of addresses your wallet will scan at any point, which is important if you are accepting a growing volume of payments.
Our address selection policy will always first use and reserve the address 20 indexes out from the last address that has actually received an incoming payment and then work from the next highest unused and unreserved index, incrementing down sequentially, until the next incoming payment is received AND until there are a minimum of 60 usable addresses that your wallet will scan at any point in time.
Example for a brand new wallet
Until we have 60 usable addresses, we choose the highest usable index that a standard wallet will see for the next address used by Zaprite. In this case, the first reserved address would be index 19 (or the 20th address, note that indexing starts at 0). If that address (index 19) does not receive a transaction, the next address reserved would be at index 18, the next lowest unused and unreserved address. If the address at index 19 did receive a transaction, your wallet will now be looking all the way to index 39, scanning for incoming deposits for the first 40 indexed addresses.
In this latter case, we would reserve the address at index 39 for the subsequent transaction facilitated by Zaprite. If a transaction is paid to the address at index 39, we would then utilize the address at index 59, and then go to index 79 if the address at index 59 were paid. In this scenario, we would then have reached a minimum of 60 usable/scannable address minimum and will start back at the beginning, utilizing addresses at index 0, 1, 2, 3, 4 etc…until enough payments have been made to bring us below our 60 usable address minimum.
Note in a practical application, addresses that are reserved for orders may or may not receive incoming payments (e.g., due to a customer not completing a payment) and Zaprite will always select an address within 20 indexes ahead from the last address that actually received a payment. Therefore, if the highest usable index is already reserved, Zaprite will select the next highest index that is unused and unreserved. Once there are 60 unused (but available) addresses, Zaprite will select the lowest possible index and increment up until there are fewer than 60 available addresses. Zaprite will continue to repeat this dynamic selection process as more transactions are completed.
Maximizing Usability
By operating this way, we maximize the chances that an address is available for your Zaprite checkout and that your customers can send you onchain payments that your wallet will be scanning for incoming payments. After that, by prioritizing addresses that are not reserved or whose reservations have expired, we ensure that each address has the opportunity to be used, maximizing your wallet’s efficiency. This method also considers addresses reserved for specific transactions, ensuring they’re used appropriately without disrupting the overall flow.
Confirming Addresses
In the rare instance that your wallet has a smaller gap limit than 20, or you can’t confirm the addresses being used on third party coordinators, Zaprite has added an address view showing your recently used addresses, and future addresses up to 100 addresses. This will help give you peace of mind that all received transactions are accounted for, and that address selection is working as intended.
Help and Support
If you run into any issues with gap limits or address selection, or have general questions about using Zaprite, feel free to reach out at https://help.zaprite.com.