Author Topic: [Witness Proposal] 1.6.129 - zapata42-witness  (Read 16191 times)

0 Members and 1 Guest are viewing this topic.

Offline ioex

  • Sr. Member
  • ****
  • Posts: 201
    • View Profile
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).

Witness ioex-test has been running steadily on the test network for three weeks. Can I get your vote on mainnet?
Mainnet witness account ioex.

Offline matle85

  • Full Member
  • ***
  • Posts: 148
    • View Profile
I'm not aware of any agreed weighting, but abit has publicly expressed his opinions, and this topic has been discussed multiple times here and on telegram.
I agree that some volumes (and trade history) seems faked, so I'll take some time to forge an updated opinion and scale my volumes accordingly.
I'll keep you informed.

Thanks Zapata and just to say, reading through all your posts I'm really impressed by the professional approach you have to this work. You're a great asset to the community.

It would be really interesting to see how the prices at different exchanges vary over time. That will show if some are skewing the price and causing some of the odd effects that Bitcrab & others are concerned about around discounts and force settlements etc.

It may be we consider some more reliable / relevant than others for bitCNY specifically. If the weighting is transparent and justified then it would be fine in my view to give some greater weight and others a reduced weight.
« Last Edit: November 17, 2018, 12:06:24 pm by matle85 »

Offline zapata42

  • Full Member
  • ***
  • Posts: 62
    • View Profile
  • BitShares: zapata42
Have we got agreed weighting for the different price feed sources? I.e. if we are concerned one exchange has faked volume it could be having an unjustified impact on the feed.

Currently my weighting is volume based average.
I currently don't apply any adjustment by market, however the script has a `scaleVolumeBy` feature that let you adjust each source volume according to an arbitrary ratio.
I'm not aware of any agreed weighting, but abit has publicly expressed his opinions, and this topic has been discussed multiple times here and on telegram.
I agree that some volumes (and trade history) seems faked, so I'll take some time to forge an updated opinion and scale my volumes accordingly.
I'll keep you informed.

Offline matle85

  • Full Member
  • ***
  • Posts: 148
    • View Profile
That's great - thank you!

Have we got agreed weighting for the different price feed sources? I.e. if we are concerned one exchange has faked volume it could be having an unjustified impact on the feed - I can see various discussions on the reliability of different sources but not sure if we maintain a weighting somewhere?
« Last Edit: November 16, 2018, 05:52:57 pm by matle85 »

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12922
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
+5% for transparency and your work on the price feed script!

Offline zapata42

  • Full Member
  • ***
  • Posts: 62
    • View Profile
  • BitShares: zapata42
In order to give more visibility on what I currently use, and summarize the changes since my initial setup, here is an update of the my algorithm:

CNY:
   
  • Compute CNY/BTS price from a weighted mean of those routes:
           
    • 'aex - CNY:BTS'
    • 'magicwallet - CNY:BITCNY', 'bitshares - BITCNY:BTS'
    • 'qc_fees - CNY:QC', 'zb - QC:BITCNY', 'bitshares - BITCNY:BTS'
    • 'qc_fees - CNY:QC', 'zb - QC:BTS'
    • 'magicwallet - CNY:BITCNY', 'zb - BITCNY:QC', 'zb - QC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'zb - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'binance - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'poloniex - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'openledger - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'gdex - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'coinegg - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'cointiger - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'cointiger - ETH:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'huobi - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'huobi - ETH:BTS'

          Notes:
         
    • 'magicwallet - CNY:BITCNY' is computed using 1h deposit/withdrawal ratio, if less than 10 operation where made the 24h volume is used.
    • 'qc_fees - CNY:QC' is manually set using ZB OTC market, currently is 0.997
       
  • Get BITCNY/BTS price from Bitshares DEX
  • Get current feed price from Bitshares DEX
  • Compute premium or discount: CNY/BTS / BITCNY/BTS
  • Compute adjusted feed price: current_feed_price * (1 + premium_or_discount * adjustment_scale)
            adjustment_scale is manually adjusted depending my appreciation on how fast the market react. Current value is 1.
  • Publish the adjusted feed price (this means BSIP-42 is activated on BitCNY)


USD:
   
  • Compute USD/BTS price from a weighted mean of those routes:
           
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'binance - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'openledger - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'gdex - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'coinegg - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'cointiger - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'huobi - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'poloniex - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'zb - BTC:BTS'
    • 'weighted_mean(bittrex, kraken) - USD:USDT', 'huobi - USDT:BTS'
    • 'weighted_mean(bittrex, kraken) - USD:USDT', 'zb - USDT:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'cointiger - ETH:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'huobi - ETH:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'cointiger - BTC:BITCNY', 'bitshares - BITCNY:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'cointiger - ETH:BITCNY', 'bitshares - BITCNY:BTS'

            Notes:
           
    • USD:BTC is computed only from CEX that have a FIAT ramp
         
  • Get BITUSD/BTS price from Bitshares DEX
  • Get current feed price from Bitshares DEX
  • Compute premium or discount: USD/BTS / BITUSD/BTS
  • Compute adjusted feed price: current_feed_price * (1 + premium_or_discount * adjustment_scale)
            adjustment_scale is manually adjusted depending my appreciation on how fast the market react. Current value is 1.2.
         
  • Publish the adjusted feed price (this means BSIP-42 is activated on BitUSD)


BTC:
   
  • Compute BTC/BTS price from a weighted mean of those routes:
           
    • 'binance - BTC:BTS'
    • 'openledger - BTC:BTS'
    • 'gdex - BTC:BTS'
    • 'coinegg - BTC:BTS'
    • 'cointiger - BTC:BTS'
    • 'huobi - BTC:BTS'
    • 'poloniex - BTC:BTS'
    • 'zb - BTC:BTS'
    • 'cointiger - BTC:BITCNY', 'bitshares - BITCNY:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(bittrex, kraken) - USD:USDT', 'huobi - USDT:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(bittrex, kraken) - USD:USDT', 'zb - USDT:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'cointiger - ETH:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'huobi - ETH:BTS'
    • 'cointiger - BTC:BITCNY', 'cointiger - BITCNY:ETH', 'cointiger - ETH:BTS'
    • 'cointiger - BTC:BITCNY', 'cointiger - BITCNY:ETH', 'huobi - ETH:BTS'
    • 'cointiger - BTC:BITCNY', 'zb - BITCNY:QC', 'zb - QC:BTS'
       
  • Publish BTC/BTS unadjusted (no BSIP-42)

RUBLE/EUR/AUD/JPY/CHF/MXN/GBP/CAD/ARS/HKD/SEK/SGD/NZD/KRW:
  • Get USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)
  • Get XXX/USD from the mean of CurrencyLayer and OpenExchangeRates
  • Then apply XXX/USD * USD/BTS to publish XXX/BTS (no BSIP-42)

GOLD/SILVER:
  • Get USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)
  • Get XXX/USD from Quadl LMBA source
  • Then apply XXX/USD * USD/BTS to publish XXX/BTS  (no BSIP-42)

HERTZ:
  • Apply the formula to USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)

AMAZONCOM:
  • Get USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)
  • Get AMAZONCOM/USD from IEX source
  • Then apply XXX/USD * USD/BTS to publish XXX/BTS  (no BSIP-42)

The up-to-date version of the code used is available there.
It contains implementations for many BSIP 42 algorithms, a large variety of price sources, and is largely configurable.

You can use the pricefeed tracker to check the published prices over time and against other witnesses, and/or to compare feed price to CEX and DEX price and track premium/discount.

I plan to update my setup to:
  • implement the "protect discount limit" as disccussed here.
  • use a lower scale at discount and a higher scale at premium as suggested by abit here.
  • publish all my intermediary values to the pricefeed tracker website so anyone would be able to analyze where my price comes from.


Edit:
  - 20181116 17:55 UTC : Added forgotten AEX source for BitCNY (api is unstable, so sometime it's not used).
« Last Edit: November 16, 2018, 05:56:29 pm by zapata42 »

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
Thanks again.

As discussed in Telegram:

I think it's better to use highest bid price (in terms of buying BTS with bitcny) than to use latest fill price to judge if bitcny is at discount. Likewise, use lowest ask to judge if bitcny is at premium. Perhaps using volume weighted moving average price is best.

Recently I feel the scripts made quite some false positives. The adjustment is too sensitive when bitcny is at discount, causing unnecessary margin calls. When someone bought up some BTS with little volume, the script thinks bitcny is at discount, thus lowers the price feed and triggers margin calls, which usually cause the price swing more than expected. Also the 110% MSSR is a big penalty to debt positions and big pressure on buy support.

Since we're able to "control" existing debt positions (via margin calls) to counter discount, but can only "incentivize" new debt positions and buy support to counter premium, that means we have more "power" on the former, so it's wise to save our power as much as possible. So it's better to adjust slower/softer/ with smaller steps when at discount, and adjust quicker/harder/with bigger steps when at premium.

For same reason, for my "adjusted_price" algorithm, it's wise to apply larger scale when at premium, but smaller scale when at discount.

Wish this helps.
« Last Edit: October 24, 2018, 09:38:29 am by abit »
BitShares committee member: abit
BitShares witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 62
    • View Profile
  • BitShares: zapata42
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
As discussed in Telegram channel, CMC's price is not reliable thus should be avoided.

Agree, I have removed it since, my configuration now use a weighted mean of exchanges with FIAT deposit (Coinbase, Kraken and Bitstamp).
Here is the config:

Code: [Select]
    btc_and_eth_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            coinbase:
                klass: Coinbase
                quotes:
                    - BTC
                    - ETH
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - XXBT
                    - XETH
                bases:
                    - ZUSD
                aliases:
                    XXBT: BTC
                    XETH: ETH
                    ZUSD: USD
            bitstamp:
                klass: Bitstamp
                quotes:
                    - BTC
                    - ETH
                bases:
                    - USD

Note that Coinbase source has been added for that.

http://www.untether.space/ has some interesting data about USDT, perhaps we can use it or the way it calculates USDT price.

Unfortunately the website does not provide any API currently. We can easely recompute the same data (see the code), but I don't see big differences with current approach (weighted average of USD/USDT of Bittrex and Kraken). Market arbitrage seems efficient.

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.

It was already possible to configure the script to retrieve USDT price from Bittrex.
However, the Kraken source was not implemented, so I just added it.

As an example witnesses can add this configuration to the script in order to use Bittrex+Kraken sources for USDT:
Code: [Select]
    usdt_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            bittrex:
                klass: Bittrex
                quotes:
                    - USDT
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - USDT
                bases:
                    - ZUSD
                aliases:
                    ZUSD: USD

I have updated my script to use those sources.

I would also like to take this opportunity to say that the PID formula implementation is also available if poeple want to practice with it.

I am also working on a tool to visualize witness pricefeeds over time: http://pricefeed-tracker.dex.trading/
(code is here)
http://www.untether.space/ has some interesting data about USDT, perhaps we can use it or the way it calculates USDT price.
BitShares committee member: abit
BitShares witness: in.abit

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
As discussed in Telegram channel, CMC's price is not reliable thus should be avoided.
BitShares committee member: abit
BitShares witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 62
    • View Profile
  • BitShares: zapata42
Another thing mentioned in Telegram group by @pc:
> please make sure that you publish a CER that is not higher than the settlement price for assets in black swan state. "higher" as in more BTS per bitXYZ

If CER is too low, it's cheaper for people to pay fees in that asset than in BTS if there is some BTS in the fee pool, although not a big deal because there should not be much BTS in the fee pool.

I have updated the script to handle the case, see commit 1, 2.
The applyed formula is: If CER < asset global settlement price, then published CER is the asset global settlement price.

My feed for those assets will be impacted: BTC, SILVER, GOLD, SGD, CHF, AUD.

« Last Edit: October 07, 2018, 10:08:48 am by zapata42 »

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
Thanks!

Another thing mentioned in Telegram group by @pc:
> please make sure that you publish a CER that is not higher than the settlement price for assets in black swan state. "higher" as in more BTS per bitXYZ

If CER is too low, it's cheaper for people to pay fees in that asset than in BTS if there is some BTS in the fee pool, although not a big deal because there should not be much BTS in the fee pool.
BitShares committee member: abit
BitShares witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 62
    • View Profile
  • BitShares: zapata42
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.

It was already possible to configure the script to retrieve USDT price from Bittrex.
However, the Kraken source was not implemented, so I just added it.

As an example witnesses can add this configuration to the script in order to use Bittrex+Kraken sources for USDT:
Code: [Select]
    usdt_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            bittrex:
                klass: Bittrex
                quotes:
                    - USDT
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - USDT
                bases:
                    - ZUSD
                aliases:
                    ZUSD: USD

I have updated my script to use those sources.

I would also like to take this opportunity to say that the PID formula implementation is also available if poeple want to practice with it.

I am also working on a tool to visualize witness pricefeeds over time: http://pricefeed-tracker.dex.trading/
(code is here)

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.
BitShares committee member: abit
BitShares witness: in.abit

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.

Do you know when "the highest one" price feed will be implemented?
It's up to the witnesses to decide. Personally I don't think "the highest one" is correct.
BitShares committee member: abit
BitShares witness: in.abit