FTSO Scaling
May. 25. 2024
What is FTSOv2 Scaling?
This blogpost will explain how to use the upgraded FTSOv2 data to access more pricing data at a much faster rate. The existing FTSO protocol deployed on both Flare and Songbird allows you to access 18 different price feeds with a refresh rate of 180 seconds. This is a great start but what if you want to access more price feeds? This is where FTSOv2 scaling comes in. FTSOv2 allows you to access more price feeds (currently 51 on Coston) at a much faster rate - the feeds refresh every 90 seconds. This is achieved by using a much more efficient method to reach a consensus on the price feeds that use less gas and decrease the time it takes to reach a consensus. It also allows for more price feeds to be added in the future without storage bloat, enabling the network to scale and meet the needs of dapps that require more price feeds. To learn more about FTSOv2, check out the FTSOv2 documentation.
In this demo, we will see, how you can easily combine FTSOv2 data with the Flare SDK to access price data and sell NFT on Flare with the price that is fixed in external assets like BTC and still be able to pay for the transaction in native FLR tokens.
FTSOv2 scaling
What does the price look like in the scaled version of the system? Previously, we had 18 price feeds that were deposited directly on the chain, but that was expensive and didn't scale well. With the newer version, the FTSO protocol does not deposit full data on the chain, as this produces a lot of unnecessary states and limits the full activity on the network. Instead, we use a much more efficient method, where the FTSO protocol deposits only a Merkle root of all prices gathered in the voting epoch. This allows us to have only a single storage slot on chain, that can efficiently represent the whole pricing data.
Why is this better? This allows us to add new pricing feeds without increasing the storage requirements on the chain and with the relay, this allows Flare to provide the pricing data to other chains and dApps in a much more gas-efficient way, which is also easier on the validators.
Does this make it less secure?
Absolutely not. The finalized Merkle root contains all the data and proving a false price would require enormous computational power and is equivalent to breaking the keccak256
hash function - so impossible for now.
Does it make it more difficult to use?
Not at all.
The Flare's SDKs provide a simple way to work with the new format of the pricing data, verify it against the Relay
and check that the provided price is correct and not manipulated.
External tooling is also provided to easily access the full pricing information together with proof, which can be used on the chain to verify the validity of the price and then act upon it.
IFTSOFeedVerifier
Let's take a look at the IFTSOFeedVerifier
interface that is used to verify the price data.
This interface is used to verify the price data that is provided by the FTSO protocol.
Don't worry, the implementation is already available on chain (coston link) and the interface is available in the updated periphery package, from where the code above is copied.
FTSO is now part of the FSP protocol and has a specific protocol feed id that distinguishes it from other feeds.
In our case, that id is 100
, but it's recommended to use the provided getter to get the correct id - so you don't need to remember it.
The most important thing to us is the FeedWithProof
struct that contains the price data and the Merkle proof that can be used to verify the price data.
The struct contains two fields, where Feed
is the price data and proof
is the Merkle proof that can be used to verify the price data.
The Feed
part should be easy to decipher.
Compared to the FTSOv1 it contains a lot more information.
Now, you can access the price and decimals easily in the same struct, while also getting the voting round id (when the price was selected), the turnout - how many data providers actively provided the price and the id of the feed.
If id of the field is a bit confusing, don't worry, we will come back to it later and explain exactly how to use it.
Importantly, the price can now be negative, as the protocol can now support assets that have a negative value (EURIBOR looking at you).
So we now have a struct from where we get the price data and the proof to verify it.
But how do we do that?
Well, we have to make sure, that the Merkle proof together with feed data hashes to exactly the root accepted in the FTSO protocol.
One can do this by himself, but you can also used the provided verifyPrice
function that does this for you.
As the feed already contains the voting round id, the method already knows which root it has to use and we don't need to provide any additional information.
For those that are interested in the implementation, it's available in the periphery package and can be found here, but it just calls openzeppelin
's MerkleProof
library to verify the proof and Relay
to get the root.
FeedId
Let's unpack the feed id
, this is the unique identifier of the FTSO feed to which the price corresponds to.
But how do we get it?
Well, you have to hex-encode the feed name as a string to get a representation that is easy to use in solidity.
But again, you don't need to do this by yourself.
The periphery package provides a library that does it for you.
It might look intimidating, but it's actually quite simple.
The first byte is the category of the feed and the rest is the name of the feed.
In our case, we have 4 categories - Crypto
, Forex
, Commodity
and Stock
and we will use the Crypto
category (01
) for our example.
To get the price of feed that provides FLR price in USD, one should just call FTSOFeedIdConverter.getCryptoFeedId("FLR/USD")
and the result would be 01464c522f55534400000000000000000000000000
(zero padded to 21 bytes).
Similarly, the BTC/USD
price feed would be 014254432f55534400000000000000000000000000
.
If you want to easily check what feeds are available, you can check the contracts yourself, or just hop on the system explorer where you can also see the feed ids and the prices.
Show me solidity code
Ok, let's put it all together.
The main goal is simple - the user should be able to buy NFT on Flare, which is priced at 1 ETH and pay for it in FLR.
To do this, our contract will need to convert from ETH
price, to $
price and then express it in FLR
(in wei).
We will use the FTSOv2 data to get the price of ETH/USD
and FLR/USD
and then calculate the price of ETH
in FLR
and then check if the user has paid enough to buy the NFT.
The code seems quite long, but most of it is boilerplate and the autoformatter makes a bit too many lines. Still, we create a mapping and list that contains information about available NFTs, their base price and the price feed that we will use to price them (together with metadata URI - the most important part).
The most important part is the getPriceInFlare
, which receives two prices - the quote currency price and the base currency price and calculates the price of the NFT in FLR.
Pay attention to the first two checks - the user is supplying us with the price data and we need to make sure that the data is correct.
We do not check the price if there is no proof - just to make it simpler for users to get the price in FLR without the need to provide the proof, but we most definitely check the correctness of the proof if it is provided and require the proof when buying.
Checking is simple - we just delegate it to the library that we have mentioned before.
But checking is also mandatory - this is the security we are after.
Anyone can provide (possibly incorrect) pricing data, but we require that the proof matches the Merkle root found on the chain and that one cannot be tampered with.
Why is this important?
Even if we get a malicious user or get data from a malicious source, the on-chain checking will always revert it and there is no incentive to provide false data.
The rest of the function contains some simple checks - that the provided price is for FLR, that the quote currency is the same as the one provided in the function and that the voting round is not too old. Then we calculate the price of the NFT in FLR with some gimmicks to handle a larger range of prices and decimals.
Buying NFT
Buying NFT is simple as seen in the following code:
The user has to provide us with two prices - from which we calculate the final price of the NFT in FLR. We then just check that the prices indeed come with proofs and that the user has paid enough. Then we just mint the NFT and set the metadata uri.
Relay
The Relay
is a contract that is used to store the Merkle roots of all the FSP protocols - in our example, we use it to query FTSO protocol root.
The most important method for us is function getConfirmedMerkleRoot(uint256 _protocolId, uint256 _votingRoundId)
, which returns the Merkle root for the given protocol id and voting round id - exactly the root we need to verify the price data.
It is exactly the one that is internally used by the IFTSOFeedVerifier
to verify the price data.
Another one that we use is function getVotingRoundId(uint256 _timestamp)
, which gives us the voting round id for the given timestamp.
Why do we use it?
To get the most fresh price we can get.
Remember, in the old FTSo system, we had to wait for 90s after each price epoch for the price to get finalized, but now, the finalization can be much faster, as the data providers will finalize the price as soon as all the data is available, which can be even faster than in 30s.
To do this (and to allow fronted to interact with the contract easier), we have implemented function getSafeVotingRoundId() public view returns (uint256)
, which gets current voting round id and then goes back in time to get the most recent finalized price.
This is important, as we want to make sure that the price we are using is the most recent one - and get is as soon as it is finalized.
To be a bit more safe, we have MAX_HISTORICAL_VOTING_EPOCHS
that limits how far back we can go to get the finalized price - if there is no price in the last few rounds, we will rather wait to get a bit more fresh price.
How to get the price and proof?
To get the price and proof struct off-chain, you need to do some calculations or get to the API that does it for you. Fortunately, every data provider will have this information already available, so if they run an open API, you can try and get it there. Another possibility is to use Flare's provided API that is available on TODO.
Once you have the API, it is simple - the endpoint /specific-feed/SYMBOL/EPOCH
returns exactly the needed proof, that can be directly fed into the contract.
A simple CURL call then looks like this:
and returns