Author Topic: [Witness Proposal] 1.6.129 - zapata42-witness  (Read 2104 times)

0 Members and 1 Guest are viewing this topic.

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
[Witness Proposal] 1.6.129 - zapata42-witness
« on: July 30, 2018, 08:21:09 pm »
Hello,

dear Bitshares community, I want to propose my services as a Bitshares Witness.

Please vote for zapata42-witness.

Who am I?

I am a developer with 13 years of experience in finance sector, based in Paris (France). I have worked for BNP Paribas Corporate Investment Banking in the equity derivatives business for the past 10 years in projects as varied as price publication, position keeping or request for quotes, and as a developer, technical lead or architect. Java is my main programming language but I have good knowledge of Python, Javascript, and Ruby. See below for my full resume.

I came to know Bitshares about one year ago, and since then I closely followed it as I was impressed by the technology, the balanced economical model and the distributed governance. I started to contribute on my spare time as I could:

Now I want to be more involved in Bitshares, and as I am not a C++ nor a React programmer (yet?), I feel like witness role currently suit me best to help the community. You will found below my current setup, and what I plan to do once voted in.


My current setup:

    Hardware: 3 x VPS 4 cores CPU at 2.4ghz, 24GB RAM, 100GB SSD hard drive, 250Mpbs internet bandwith, Location: France.
    (I consider this a minimum, see below for my target plan)
   
    I run 2 witness mainnet nodes (account zapata42-witness) and 2 witness testnet node (witness-zapata).
    They are monitored through an improved version of @clockwork's telegram monitoring bot that handle automatic key switch and price feed monitoring.

    The price feed is ready, I already publish @nasdaq assets (see AMAZONCOM) and I use a stabilized version of @xeroc's pricefeed.

    Additionnaly I host some services:
        Mainnet API:         https://api.dex.trading  (max-ops-per-account = 100)
        Mainnet Seed:       seed.dex.trading:1776
        Testnet API:          https://testnet.dex.trading
        Wallet:                  https://wallet.dex.trading or https://bitshares.dex.trading (patched version that use my referral link as default)

    Services are served through multiple proxies (currently NGINX) that do rate limiting and SSL termination.
    For SSL certificate is issued by Let's encrypt and renewal is automated.
    Everything run on a private multi datacenter Kubernetes cluster (so fully Dockerized and resilient) and monitored through Prometheus/Grafana.
    The setup is automated and ready to scale, see my plan below.

My plan:

If I get voted as an active witness my focus will be on:
  • Providing a stable block producer
  • Providing a stable, accurate and auditable price feed of most Market Pegged Assets
  • Grow Bitshares infrastructure, community and software ecosystem
This means:
  • Upgrade the infrastructure. With the current witness pay (~30 witness / bts=0.2$ / 1bts per block) I aim to add:
                        6 x Dedicated 4c/8t 3Ghz CPU, 64go RAM, 1To SSD hard drive, 250Mpbs internet bandwith, Locations: france, germany, uk, canada, singapore, australia
  • Continuous improvements of the setup like: geo optimizations for latency (GeoDNS + regional vlan), premium wildcard SSL certificate, premium api acess, ...
  • Additional to classic API nodes, I will install and serve community tools like: Open-Explorer, Python API Backend, Elastic Search database with a data visualization tool like Kibana/Graphana, Bitshares HUG, and any future interesting project of the community;
  • Active participation (code contributions, code/issue/bsip review, testing) to community initiatives (core, ui, tools, documentation) as a developer;
  • Participate to meetups and hackathons to promote Bitshares;
  • Regularly run third party professional penetration testing on the setup;
  • if BTS price allow me to do it, hire other people to help me.

So why voting for zapata42-witness ?
  • Experienced software developer with professional experience in developing and running financial highly critical software;
  • Professional experience in price contribution in a main financial company;
  • Proven will and capacity to contribute on Bitshares related projects;
  • Full setup (witness, pricefeed + api) in place, and wide range of services in target;
  • Active investment in Bitshares ecosystem (infrastructure & time) planned;
  • Knowledgeable in blockchain internals;
  • Relations in French fintech and financial institutions;
  • Fresh blood.


Contact:
  Telegram: @Zapata_42
  LinkedIn profile

* Edit 20180901: Update seed port.
« Last Edit: September 01, 2018, 05:25:26 pm by zapata42 »

Online Bangzi

  • Full Member
  • ***
  • Posts: 164
    • View Profile
    • Steemit: Bangzi
  • BitShares: bangzi
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #1 on: July 30, 2018, 10:34:13 pm »
Voted! +5% +5% +5%
Please Vote for My Witness: Bangzi
My Blog: https://steemit.com/@bangzi
请投票支持比特股见证人: Bangzi

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12694
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #2 on: July 31, 2018, 08:55:02 am »
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #3 on: July 31, 2018, 10:34:10 am »
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).

My testnet block producer is running since early may and active since 12/05/2018. The account is: witness-zapata.

I also consider that running a testnet block producer is a mandatory task for a witness. I always test my changes on testnet before applying them on mainnet. It was also required for some developments. I am trying to keep it as stable as the main one.

On testnet, I missed 579 blocks so far:
  • the first ~540 was because initially I didn't put the correct private/public key pair, and didn't notice that I was activated. To prevent the issue on mainnet I monitor the activation (see this commit)
  • then I missed ~20 to update the version and at that time I was not running a backup node;
  • 1 block missed for a non investigated issue (short network loss);
  • then I missed ~15 block during my chaos testing to validate the backup procedures: I switch signing key after 3 blocks, and I tested muliple cases.
Note that I have heavely updated the underlying servers and kubernetes cluster while running the witness without issues.
« Last Edit: July 31, 2018, 01:02:35 pm by zapata42 »

Offline Digital Lucifer

  • Global Moderator
  • Full Member
  • *****
  • Posts: 123
  • 13 years of being Slackware abUser
    • View Profile
    • Bitshares DEX Thailand
  • BitShares: steem-not
  • GitHub: dls-cipher
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #4 on: July 31, 2018, 11:01:12 am »
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).

My testnet block producer is running since early may and active since 12/05/2018. The account is: witness-zapata.

I also consider that running a testnet block producer is a mandatory task for a witness. I always test my changes on testnet before applying them on mainnet. It was also required for some developments. I am trying to keep it as stable as the main one.

On testnet, I missed 579 blocks so far:
  • the first ~540 was because initially I didn't put the correct private/public key pair, and didn't notice that I was activated. To prevent the issue on mainnet I monitor the activation (see this commit)
  • then I missed ~20 to update the version and at that time I was not running a backup node;
  • 1 block missed for a non investigated issue (short network loss);
  • then I missed ~15 block during my chaos testing to validate the backup procedures: I switch signing key after 3 blocks, and I tested muliple cases.
Note that I have heavely updated the underlying servers and kubernetes cluster while running the witness without issues.

Voted (just humble ~200k BTS). I have a testnet witness, so I can co-relate with issues you had and highly appreciate your precision in reporting.

Milos (Mike) Preocanin - General Manager @ Syntek Solutions
TOANDI Co., LTD. (BOI Approved) - TAX ID: 0205549016913 - 95/5 Moo 4 Siam Country Club Rd.
Nong Prue, Bang Lamung, Chonburi 20250, Thailand.

Offline fav

  • Administrator
  • Hero Member
  • *****
  • Posts: 4244
  • No Pain, No Gain
    • View Profile
  • BitShares: fav
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #5 on: July 31, 2018, 02:24:31 pm »
voted
► How-to Buy BitShares: https://goo.gl/i9j2YZ

Online clockwork

  • Committee member
  • Full Member
  • *
  • Posts: 159
    • View Profile
  • BitShares: clockwork
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #6 on: July 31, 2018, 09:18:35 pm »
Will vote for this. Zapata is a huge asset to the community :)

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #7 on: July 31, 2018, 11:27:21 pm »
Thanks. Voted.

Please submit your mainnet API node to https://github.com/bitshares/bitshares-ui/ .
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12694
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #8 on: August 02, 2018, 10:38:22 am »
voted
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #9 on: August 05, 2018, 02:45:18 pm »
Thanks for your votes so far.

Please submit your mainnet API node to https://github.com/bitshares/bitshares-ui/ .

My nodes will be available in the next UI release (see PR). You can already find them in the staging wallet.
Any feedback on the latency is welcome.

My testnet witness (witness.zapata) and API node (wss://testnet.dex.trading) have been updated (see release thread).
I missed one block in the deployment process as I switched my signing key back to my main witness node while it was my turn.
I will take care to avoid this case in the future.
(Note that I had a typo in my previous post, you should have read 597, instead of 579. I'm now at 598).

This week I also added Bitshares integration to EOSIO/demux-js. You can find out more information on Demux here.

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12694
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #10 on: August 08, 2018, 08:24:58 am »
This week I also added Bitshares integration to EOSIO/demux-js. You can find out more information on Demux here.
Extraordinary!
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline apasia.tech

Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #11 on: August 08, 2018, 04:08:57 pm »
Just wanted to say, I've read the full proposal with enormous interest, in a good way and with the background and resume - this is a welcome addition to BitShares. Furthermore, I've heard more good feedback from others who welcome this witness proposal. Some exceptional contributions we didn't see coming, thank you.
Ross Walker - Founder @ apasia.tech
AP Asia Tech Co., LTD.  TAX ID: 0205549016913 - 14/11 Floraville Complex 2, Pattanakarn Soi 51, Suan Luang, Bangkok 10250, Thailand.

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #12 on: August 17, 2018, 10:02:26 pm »
Following current various discussions on price feeds I spend a couple of days tuning mine.


Here is the algorithm I use:
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
  • Get Fiat forex USD/CNY, USD/EUR, ... from CurrencyLayer (fallback to OpenExchangeRates)
  • Get BITCNY/CNY from MagicWallet (use last 1h data, and if not enough trades use last 24h data)
  • Get last price and daily volume of BTS/USDT, BTS/BTC, BTS/ETH if available from Centralized Exchanges: Aex, Binance, CoinEgg, CoinTiger, Huobi, Lbank, Poloniex, Zb
  • Get last price and daily volume from BTS/OPEN.BTC, BTS/GDEX.BTC on BitShares DEX (and consider xxx.BTC = BTC)
  • Get last price for BITCNY/BTC, BITCNY/ETH from CoinTiger
  • Get last price for BTS/BITCNY from BitShares DEX

With that I compute all possible routes using BTC, ETH, USDT or BITCNY as intermediate asset.
Then I use a weighted average using the BTS/{BTC,ETH,USDT,BITCNY} daily volume.

Here is a sample result of all the routes for USD/BTS and CNY/BTS at 17/08/2018 21:58 UTC:

Code: [Select]
+------+-------+--------------+-------+--------------------+------------------------------------------------------------------------------------+
| base | quote | price        | diff  | quote volume       | source                                                                             |
+------+-------+--------------+-------+--------------------+------------------------------------------------------------------------------------+
| CNY  | BTS   | 0.8008708826 | +1.58 | 31726149.140697803 | ['magicwallet - CNY:BITCNY', 'bitshares - BITCNY:BTS']                             |
| CNY  | BTS   | 0.7775048363 | -1.38 | 275814.57983       | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'gdex - BTC:BTS']       |
| CNY  | BTS   | 0.7767051983 | -1.48 | 6099695.41644055   | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'poloniex - BTC:BTS']   |
| CNY  | BTS   | 0.7776046832 | -1.37 | 6823092.1          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'zb - BTC:BTS']         |
| CNY  | BTS   | 0.7826822718 | -0.73 | 1944911.60043      | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'openledger - BTC:BTS'] |
| CNY  | BTS   | 0.7897477292 | +0.17 | 8090672.0          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'coinegg - BTC:BTS']    |
| CNY  | BTS   | 0.7821021076 | -0.80 | 719268.12          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'cointiger - BTC:BTS']  |
| CNY  | BTS   | 0.7758057134 | -1.60 | 3068685.9901371617 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'huobi - BTC:BTS']      |
| CNY  | BTS   | 0.7816523652 | -0.86 | 1648254.1922999984 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'lbank - BTC:BTS']      |
| CNY  | BTS   | 0.7798533954 | -1.08 | 5457868.0          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'binance - BTC:BTS']    |
| CNY  | BTS   | 0.7968423519 | +1.07 | 275814.57983       | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'gdex - BTC:BTS']           |
| CNY  | BTS   | 0.7960228259 | +0.97 | 6099695.41644055   | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'poloniex - BTC:BTS']       |
| CNY  | BTS   | 0.7969446821 | +1.08 | 6823092.1          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'zb - BTC:BTS']             |
| CNY  | BTS   | 0.8021485567 | +1.74 | 1944911.60043      | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'openledger - BTC:BTS']     |
| CNY  | BTS   | 0.8093897408 | +2.66 | 8090672.0          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'coinegg - BTC:BTS']        |
| CNY  | BTS   | 0.8015539631 | +1.67 | 719268.12          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'cointiger - BTC:BTS']      |
| CNY  | BTS   | 0.7951009697 | +0.85 | 3068685.9901371617 | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'huobi - BTC:BTS']          |
| CNY  | BTS   | 0.8010930350 | +1.61 | 1648254.1922999984 | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'lbank - BTC:BTS']          |
| CNY  | BTS   | 0.7992493226 | +1.38 | 5457868.0          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'binance - BTC:BTS']        |
| CNY  | BTS   | 0.7739653886 | -1.83 | 17591765.91406     | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:USDT', 'aex - USDT:BTS']      |
| CNY  | BTS   | 0.7767270849 | -1.48 | 15976040.0         | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:USDT', 'zb - USDT:BTS']       |
| CNY  | BTS   | 0.7794887812 | -1.13 | 10213236.688898135 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:USDT', 'huobi - USDT:BTS']    |
| CNY  | BTS   | 0.7760030565 | -1.57 | 188647.0           | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:ETH', 'cointiger - ETH:BTS']  |
| CNY  | BTS   | 0.7760030565 | -1.57 | 1040797.2504070019 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:ETH', 'huobi - ETH:BTS']      |
| CNY  | BTS   | 0.7778900585 | -1.33 | 4698376.790000004  | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:ETH', 'lbank - ETH:BTS']      |
| CNY  | BTS   | 0.7930701250 | +0.59 | 188647.0           | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'cointiger - ETH:BTS']      |
| CNY  | BTS   | 0.7930701250 | +0.59 | 1040797.2504070019 | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'huobi - ETH:BTS']          |
| CNY  | BTS   | 0.7949986288 | +0.84 | 4698376.790000004  | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'lbank - ETH:BTS']          |
| USD  | BTS   | 0.1130179495 | -0.27 | 275814.57983       | ['coinmarketcap - USD:BTC', 'gdex - BTC:BTS']                                      |
| USD  | BTS   | 0.1129017143 | -0.37 | 6099695.41644055   | ['coinmarketcap - USD:BTC', 'poloniex - BTC:BTS']                                  |
| USD  | BTS   | 0.1130324632 | -0.26 | 6823092.1          | ['coinmarketcap - USD:BTC', 'zb - BTC:BTS']                                        |
| USD  | BTS   | 0.1137705405 | +0.39 | 1944911.60043      | ['coinmarketcap - USD:BTC', 'openledger - BTC:BTS']                                |
| USD  | BTS   | 0.1147975740 | +1.30 | 8090672.0          | ['coinmarketcap - USD:BTC', 'coinegg - BTC:BTS']                                   |
| USD  | BTS   | 0.1136862080 | +0.32 | 719268.12          | ['coinmarketcap - USD:BTC', 'cointiger - BTC:BTS']                                 |
| USD  | BTS   | 0.1127709653 | -0.49 | 3068685.9901371617 | ['coinmarketcap - USD:BTC', 'huobi - BTC:BTS']                                     |
| USD  | BTS   | 0.1136208335 | +0.26 | 1648254.1922999984 | ['coinmarketcap - USD:BTC', 'lbank - BTC:BTS']                                     |
| USD  | BTS   | 0.1133593356 | +0.03 | 5457868.0          | ['coinmarketcap - USD:BTC', 'binance - BTC:BTS']                                   |
| USD  | BTS   | 0.1125034561 | -0.72 | 17591765.91406     | ['coinmarketcap - USD:USDT', 'aex - USDT:BTS']                                     |
| USD  | BTS   | 0.1129048957 | -0.37 | 15976040.0         | ['coinmarketcap - USD:USDT', 'zb - USDT:BTS']                                      |
| USD  | BTS   | 0.1133063354 | -0.02 | 10213236.688898135 | ['coinmarketcap - USD:USDT', 'huobi - USDT:BTS']                                   |
| USD  | BTS   | 0.1127996511 | -0.46 | 188647.0           | ['coinmarketcap - USD:ETH', 'cointiger - ETH:BTS']                                 |
| USD  | BTS   | 0.1127996511 | -0.46 | 1040797.2504070019 | ['coinmarketcap - USD:ETH', 'huobi - ETH:BTS']                                     |
| USD  | BTS   | 0.1130739453 | -0.22 | 4698376.790000004  | ['coinmarketcap - USD:ETH', 'lbank - ETH:BTS']                                     |
| USD  | BTS   | 0.1128235134 | -0.44 | 275814.57983       | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'gdex - BTC:BTS']         |
| USD  | BTS   | 0.1127074782 | -0.54 | 6099695.41644055   | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'poloniex - BTC:BTS']     |
| USD  | BTS   | 0.1128380022 | -0.43 | 6823092.1          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'zb - BTC:BTS']           |
| USD  | BTS   | 0.1135748097 | +0.22 | 1944911.60043      | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'openledger - BTC:BTS']   |
| USD  | BTS   | 0.1146000762 | +1.13 | 8090672.0          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'coinegg - BTC:BTS']      |
| USD  | BTS   | 0.1134906222 | +0.15 | 719268.12          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'cointiger - BTC:BTS']    |
| USD  | BTS   | 0.1125769542 | -0.66 | 3068685.9901371617 | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'huobi - BTC:BTS']        |
| USD  | BTS   | 0.1134253602 | +0.09 | 1648254.1922999984 | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'lbank - BTC:BTS']        |
| USD  | BTS   | 0.1131643122 | -0.14 | 5457868.0          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'binance - BTC:BTS']      |
| USD  | BTS   | 0.1128514039 | -0.42 | 275814.57983       | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'gdex - BTC:BTS']           |
| USD  | BTS   | 0.1127353400 | -0.52 | 6099695.41644055   | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'poloniex - BTC:BTS']       |
| USD  | BTS   | 0.1128658963 | -0.40 | 6823092.1          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'zb - BTC:BTS']             |
| USD  | BTS   | 0.1136028859 | +0.25 | 1944911.60043      | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'openledger - BTC:BTS']     |
| USD  | BTS   | 0.1146284059 | +1.15 | 8090672.0          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'coinegg - BTC:BTS']        |
| USD  | BTS   | 0.1135186776 | +0.17 | 719268.12          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'cointiger - BTC:BTS']      |
| USD  | BTS   | 0.1126047837 | -0.63 | 3068685.9901371617 | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'huobi - BTC:BTS']          |
| USD  | BTS   | 0.1134533995 | +0.11 | 1648254.1922999984 | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'lbank - BTC:BTS']          |
| USD  | BTS   | 0.1131922869 | -0.12 | 5457868.0          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'binance - BTC:BTS']        |
| USD  | BTS   | 0.1126973406 | -0.55 | 17591765.91406     | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:USDT', 'aex - USDT:BTS']          |
| USD  | BTS   | 0.1130994721 | -0.20 | 15976040.0         | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:USDT', 'zb - USDT:BTS']           |
| USD  | BTS   | 0.1135016035 | +0.16 | 10213236.688898135 | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:USDT', 'huobi - USDT:BTS']        |
| USD  | BTS   | 0.1135893251 | +0.23 | 31726149.140697803 | ['coinmarketcap - USD:BTC', 'cointiger - BTC:BITCNY', 'bitshares - BITCNY:BTS']    |
| USD  | BTS   | 0.1139091655 | +0.52 | 31726149.140697803 | ['coinmarketcap - USD:ETH', 'cointiger - ETH:BITCNY', 'bitshares - BITCNY:BTS']    |
| USD  | BTS   | 0.1129661203 | -0.32 | 188647.0           | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:ETH', 'cointiger - ETH:BTS']      |
| USD  | BTS   | 0.1129661203 | -0.32 | 1040797.2504070019 | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:ETH', 'huobi - ETH:BTS']          |
| USD  | BTS   | 0.1132408193 | -0.07 | 4698376.790000004  | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:ETH', 'lbank - ETH:BTS']          |
+------+-------+--------------+-------+--------------------+------------------------------------------------------------------------------------+

And the final results I get:

Code: [Select]
+--------+------------+--------------+--------------+---------------------+---------------------+---------------------+--------------+---------------------+--------+--------+-------------------------+-------------+
| symbol | collateral |  new price   |     cer      |         mean        |        median       |      wgt. avg.      | wgt. std (#) |      blockchain     |  mssr  |  mcr   |      my last price      | last update |
+--------+------------+--------------+--------------+---------------------+---------------------+---------------------+--------------+---------------------+--------+--------+-------------------------+-------------+
|  CNY   |    BTS     | 0.7884066494 | 0.9460879792 | 0.7880032477 (0.05) | 0.7862150005 (0.28) | 0.7884066494 (0.00) |  1.17 (28)   | 0.7728333333 (2.02) | 110.0% | 175.0% | -1.0000000000 (-178.84) | unknown ago |
|  USD   |    BTS     | 0.1133240311 | 0.1189902327 | 0.1132431633 (0.07) | 0.1130994721 (0.20) | 0.1133240311 (0.00) |  0.06 (41)   | 0.1123699660 (0.85) | 110.0% | 175.0% | -1.0000000000 (-111.33) | unknown ago |
+--------+------------+--------------+--------------+---------------------+---------------------+---------------------+--------------+---------------------+--------+--------+-------------------------+-------------+

I use xeroc's pricefeed script that I modified again to add:
  • Composite source to group multiple sources as one (to handle things like fallbacks);
  • Add CoinTiger, CoinEgg, Huobi and MagicWallet sources;
  • Change the way derived routes volumes are computed;
  • Better tracking of source for debug/audit purpose;
  • Various code refactoring and small enhancements.
All code is accessible at Zapata/bitshares-pricefeed.

Your comments are more than welcome, this is intended to be adjusted over time.

Currently my ideas to further improve are:
  • use the volume weighted average of the last X minutes trades instead of the last prices.
  • use the order book depth to weight prices instead of daily volume.
  • plot and track current active feeds and backup/test feeds over time. This will help shareholders to vote accurately.
  • make all the data used for all my feeds publicly available for auditing

I would be able to progress on that once voted. Since then I'm helping Alfredo on the open-explorer (see last PR).
« Last Edit: August 17, 2018, 10:51:03 pm by zapata42 »

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #13 on: August 17, 2018, 11:00:18 pm »
Good job about the price feeding thing. I'd say you're getting quite accurate price data.

You can add a few more markets:

* Aex: BTS/CNC (CNC is IOU CNY)

* Zb: BTS/QC (QC is IOU CNY as well)


BTW, IMHO, Zb's volume is faked, Coinegg's volume is faked, Lbank's volume is faked. Huobi, Binance and Aex? I don't know. Poloniex? I don't trust them because they left a bad history. If adjust their weight to lower, likely we'll get a slightly higher BTS price in the end.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline kimchi-king

Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #14 on: August 22, 2018, 09:21:37 am »
Voted.

Keep up the great work.

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #15 on: August 31, 2018, 11:45:33 pm »
Thanks for your votes, I am an active witness since the begining of the week.

Some updates from the last two weeks:
  • my nodes have been updated to 2.0.180823.
  • my pricefeed configuration has been updated following Abit's comments.
  • I have published code to experiment with BSIP42. It currently implements all the algorithms publicly discussed.  I have not activated any of them in my feed as I wait for the end of the BSIP42 voting period. This should allow anyone to experiment their formulas easely (just add your algorithm there).
  • a first Dedicated 4c/8t 3Ghz CPU, 64go RAM, 1To SSD hard drive, 250Mpbs internet bandwith has been ordered. I will use it to setup ElasticSearch + OpenExplorer + Explorer API in the coming weeks.
  • I am helping Alfredo on the backend of open-explorer.io to improve performances and make it easier to install and maintain (see this, this, this and this PR.. and more to come)

I will be at BitFest, Friday and Saturday, ping me here or on Telegram if you want to have a chat there.

Online gghi

  • Sr. Member
  • ****
  • Posts: 281
    • View Profile
  • BitShares: ttt888
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #16 on: September 01, 2018, 08:27:54 am »
Identity, we must increase feedback so that we can quickly rectify the deviation. It is suggested that the ideal price = BTS market price * (bitcny/cny price +M) ^N, M indicates the loose index, N is the acceleration factor. For example, when the premium is zero, the easing target is negative 2%, the acceleration is 1.8, then the ideal feed = BTS market price * 1 + 0.02 ^ 1.8, for example, when the premium is 0.001, the easing target is negative 1%, the acceleration is 1.5, then the ideal feed = BTS market price * 1.001 + 0.01 ^ 1.5.
For example, the formula is proved to be effective, and the loose target is minus 1%.
When the premium is negative 0.01 (discount 1%), the acceleration factor is 2. Ideal feed price = BTS market price * 1-0.01 + 0.01) ^ 2 = BTS market price * 1 ^ 2 = BTS market price * 1. Conclusion: The target is achieved, very ideal.
When the premium is 0, the acceleration factor is 2. Ideal feeding price = BTS market price * 1 + 0.01) ^ 2 = BTS market price * 1.01 ^ 2 = BTS market price * 1.02, conclusion: the market price increase by 2%, very satisfactory, very safe.
When the premium is 0.01, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.01 + 0.01) ^ 2 = BTS market price * 1.02 ^ 2 = BTS market price * 1.0404. Conclusion: The increase of market price by 4% is very satisfactory and safe.
When the premium is 0.02, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.02 + 0.01) ^ 2 = BTS market price * 1.03 ^ 2 = BTS market price * 1.0609. Conclusion: The increase of the market price by 6% is very satisfactory and safe.
When the premium is 0.03, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.03 + 0.01) ^ 2 = BTS market price * 1.04 ^ 2 = BTS market price * 1.0816. Conclusion: The higher the market price is, the more satisfied and safe it is.

When the premium is 0.04, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.04 + 0.01) ^ 2 = BTS market price * 1.05 ^ 2 = BTS market price * 1.1025. Conclusion: The market price increases by about 10%, satisfactory and safe.
When the premium is 0.05, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.05 + 0.01) ^ 2 = BTS market price * 1.06 ^ 2 = BTS market price * 1.1236. Conclusion: The market price increases about 11.2%, satisfies, speeds up the correction.
When the premium is 0.06, the acceleration factor is 2. Ideal Feed = BTS Market Price * 1.06+0.01) ^ 2 = BTS Market Price * 1.07 ^ 2 = BTS Market Price * 1.1449. Conclusion: The market price increases about 11.4%, satisfactory, accelerate the correction of deviation.
When the premium is 0.07, the acceleration factor is 2. Ideal Feed = BTS Market Price * 1.07+0.01) ^ 2 = BTS Market Price * 1.08 ^ 2 = BTS Market Price * 1.1664. Conclusion: Market Price increases by about 11.66%, satisfies, speeds up the correction.
When the premium is 0.08, the acceleration factor is 2. Ideal Feed = BTS Market Price * 1.08+0.01) ^ 2 = BTS Market Price * 1.09 ^ 2 = BTS Market Price * 1.1881, Conclusion: Market Price increased by 11.88%, satisfactory, accelerate correction.
When the premium is 0.09, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.09 + 0.01) ^ 2 = BTS market price * 1.1 ^ 2 = BTS market price * 1.21. Conclusion: The market price increases about 12%, satisfies, accelerates the correction, and the risk is still controllable.

  认同,必须加大反馈力度才可以快速纠偏。建议理想喂价 = BTS市场价 * (bitcny/cny价+M)^N,M表示宽松指标,N为加速因子。比如溢价为零时,宽松目标负2%,加速度1.8,那么,理想喂价 = BTS市场价 * (1+0.02)^1.8,比如溢价为0.001时,宽松目标负1%,加速度1.5,那么,理想喂价 = BTS市场价 * (1.001+0.01)^1.5。
       举例论证公式效果,宽松目标负1%。
当溢价负0.01时(折扣1%),加速因子喂2。理想喂价 = BTS市场价 * (1-0.01+0.01)^2=BTS市场价 *1^2=BTS市场价 *1,结论:目标达到,非常理想。
当溢价0时,加速因子喂2。理想喂价 = BTS市场价 * (1+0.01)^2=BTS市场价 *1.01^2=BTS市场价 *1.02,结论:市场价提高2%,很满意,也很安全。
当溢价0.01时,加速因子喂2。理想喂价 = BTS市场价 * (1.01+0.01)^2=BTS市场价 *1.02^2=BTS市场价 *1.0404,结论:市场价提高约4%,很满意,也很安全。
当溢价0.02时,加速因子喂2。理想喂价 = BTS市场价 * (1.02+0.01)^2=BTS市场价 *1.03^2=BTS市场价 *1.0609,结论:市场价提高约6%,很满意,也很安全。
当溢价0.03时,加速因子喂2。理想喂价 = BTS市场价 * (1.03+0.01)^2=BTS市场价 *1.04^2=BTS市场价 *1.0816,结论:市场价提高越8%,很满意,也很安全。

当溢价0.04时,加速因子喂2。理想喂价 = BTS市场价 * (1.04+0.01)^2=BTS市场价 *1.05^2=BTS市场价 *1.1025,结论:市场价提高约10%,满意,也很安全。
     当溢价0.05时,加速因子喂2。理想喂价 = BTS市场价 * (1.05+0.01)^2=BTS市场价 *1.06^2=BTS市场价 *1.1236,结论:市场价提高约11.2%,满意,加速纠偏。
     当溢价0.06时,加速因子喂2。理想喂价 = BTS市场价 * (1.06+0.01)^2=BTS市场价 *1.07^2=BTS市场价 *1.1449,结论:市场价提高约11.4%,满意,加速纠偏
当溢价0.07时,加速因子喂2。理想喂价 = BTS市场价 * (1.07+0.01)^2=BTS市场价 *1.08^2=BTS市场价 *1.1664,结论:市场价提高约11.66%,满意,加速纠偏
当溢价0.08时,加速因子喂2。理想喂价 = BTS市场价 * (1.08+0.01)^2=BTS市场价 *1.09^2=BTS市场价 *1.1881,结论:市场价提高约11.88%,满意,加速纠偏
当溢价0.09时,加速因子喂2。理想喂价 = BTS市场价 * (1.09+0.01)^2=BTS市场价 *1.1^2=BTS市场价 *1.21,结论:市场价提高约12%,满意,加速纠偏,风险依然可控。

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #17 on: September 07, 2018, 12:03:13 am »
As BSIP42 has been voted in, my feed price has been adjusted accordingly.

The current adjustment is the use the same formula as gdex-witness.

Online gghi

  • Sr. Member
  • ****
  • Posts: 281
    • View Profile
  • BitShares: ttt888
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #18 on: September 07, 2018, 12:35:37 am »
As BSIP42 has been voted in, my feed price has been adjusted accordingly.

The current adjustment is the use the same formula as gdex-witness.

     GOOD,voted

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #19 on: September 08, 2018, 10:17:13 am »
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline armin

  • Full Member
  • ***
  • Posts: 90
    • View Profile
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #20 on: September 08, 2018, 12:06:58 pm »
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.

Do you know when "the highest one" price feed will be implemented?

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #21 on: September 09, 2018, 08:59:10 am »
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.

Do you know when "the highest one" price feed will be implemented?
It's up to the witnesses to decide. Personally I don't think "the highest one" is correct.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #22 on: October 03, 2018, 11:31:59 pm »
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #23 on: October 04, 2018, 10:16:52 pm »
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.

It was already possible to configure the script to retrieve USDT price from Bittrex.
However, the Kraken source was not implemented, so I just added it.

As an example witnesses can add this configuration to the script in order to use Bittrex+Kraken sources for USDT:
Code: [Select]
    usdt_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            bittrex:
                klass: Bittrex
                quotes:
                    - USDT
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - USDT
                bases:
                    - ZUSD
                aliases:
                    ZUSD: USD

I have updated my script to use those sources.

I would also like to take this opportunity to say that the PID formula implementation is also available if poeple want to practice with it.

I am also working on a tool to visualize witness pricefeeds over time: http://pricefeed-tracker.dex.trading/
(code is here)

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #24 on: October 05, 2018, 07:18:52 pm »
Thanks!

Another thing mentioned in Telegram group by @pc:
> please make sure that you publish a CER that is not higher than the settlement price for assets in black swan state. "higher" as in more BTS per bitXYZ

If CER is too low, it's cheaper for people to pay fees in that asset than in BTS if there is some BTS in the fee pool, although not a big deal because there should not be much BTS in the fee pool.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #25 on: October 07, 2018, 12:15:46 am »
Another thing mentioned in Telegram group by @pc:
> please make sure that you publish a CER that is not higher than the settlement price for assets in black swan state. "higher" as in more BTS per bitXYZ

If CER is too low, it's cheaper for people to pay fees in that asset than in BTS if there is some BTS in the fee pool, although not a big deal because there should not be much BTS in the fee pool.

I have updated the script to handle the case, see commit 1, 2.
The applyed formula is: If CER < asset global settlement price, then published CER is the asset global settlement price.

My feed for those assets will be impacted: BTC, SILVER, GOLD, SGD, CHF, AUD.

« Last Edit: October 07, 2018, 10:08:48 am by zapata42 »

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #26 on: October 16, 2018, 10:43:40 pm »
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
As discussed in Telegram channel, CMC's price is not reliable thus should be avoided.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #27 on: October 20, 2018, 10:08:04 am »
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.

It was already possible to configure the script to retrieve USDT price from Bittrex.
However, the Kraken source was not implemented, so I just added it.

As an example witnesses can add this configuration to the script in order to use Bittrex+Kraken sources for USDT:
Code: [Select]
    usdt_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            bittrex:
                klass: Bittrex
                quotes:
                    - USDT
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - USDT
                bases:
                    - ZUSD
                aliases:
                    ZUSD: USD

I have updated my script to use those sources.

I would also like to take this opportunity to say that the PID formula implementation is also available if poeple want to practice with it.

I am also working on a tool to visualize witness pricefeeds over time: http://pricefeed-tracker.dex.trading/
(code is here)
http://www.untether.space/ has some interesting data about USDT, perhaps we can use it or the way it calculates USDT price.
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #28 on: October 21, 2018, 05:34:41 pm »
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
As discussed in Telegram channel, CMC's price is not reliable thus should be avoided.

Agree, I have removed it since, my configuration now use a weighted mean of exchanges with FIAT deposit (Coinbase, Kraken and Bitstamp).
Here is the config:

Code: [Select]
    btc_and_eth_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            coinbase:
                klass: Coinbase
                quotes:
                    - BTC
                    - ETH
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - XXBT
                    - XETH
                bases:
                    - ZUSD
                aliases:
                    XXBT: BTC
                    XETH: ETH
                    ZUSD: USD
            bitstamp:
                klass: Bitstamp
                quotes:
                    - BTC
                    - ETH
                bases:
                    - USD

Note that Coinbase source has been added for that.

http://www.untether.space/ has some interesting data about USDT, perhaps we can use it or the way it calculates USDT price.

Unfortunately the website does not provide any API currently. We can easely recompute the same data (see the code), but I don't see big differences with current approach (weighted average of USD/USDT of Bittrex and Kraken). Market arbitrage seems efficient.

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3429
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #29 on: October 24, 2018, 09:12:36 am »
Thanks again.

As discussed in Telegram:

I think it's better to use highest bid price (in terms of buying BTS with bitcny) than to use latest fill price to judge if bitcny is at discount. Likewise, use lowest ask to judge if bitcny is at premium. Perhaps using volume weighted moving average price is best.

Recently I feel the scripts made quite some false positives. The adjustment is too sensitive when bitcny is at discount, causing unnecessary margin calls. When someone bought up some BTS with little volume, the script thinks bitcny is at discount, thus lowers the price feed and triggers margin calls, which usually cause the price swing more than expected. Also the 110% MSSR is a big penalty to debt positions and big pressure on buy support.

Since we're able to "control" existing debt positions (via margin calls) to counter discount, but can only "incentivize" new debt positions and buy support to counter premium, that means we have more "power" on the former, so it's wise to save our power as much as possible. So it's better to adjust slower/softer/ with smaller steps when at discount, and adjust quicker/harder/with bigger steps when at premium.

For same reason, for my "adjusted_price" algorithm, it's wise to apply larger scale when at premium, but smaller scale when at discount.

Wish this helps.
« Last Edit: October 24, 2018, 09:38:29 am by abit »
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #30 on: November 16, 2018, 03:38:35 pm »
In order to give more visibility on what I currently use, and summarize the changes since my initial setup, here is an update of the my algorithm:

CNY:
   
  • Compute CNY/BTS price from a weighted mean of those routes:
           
    • 'aex - CNY:BTS'
    • 'magicwallet - CNY:BITCNY', 'bitshares - BITCNY:BTS'
    • 'qc_fees - CNY:QC', 'zb - QC:BITCNY', 'bitshares - BITCNY:BTS'
    • 'qc_fees - CNY:QC', 'zb - QC:BTS'
    • 'magicwallet - CNY:BITCNY', 'zb - BITCNY:QC', 'zb - QC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'zb - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'binance - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'poloniex - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'openledger - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'gdex - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'coinegg - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'cointiger - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'cointiger - ETH:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'huobi - BTC:BTS'
    • 'magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'huobi - ETH:BTS'

          Notes:
         
    • 'magicwallet - CNY:BITCNY' is computed using 1h deposit/withdrawal ratio, if less than 10 operation where made the 24h volume is used.
    • 'qc_fees - CNY:QC' is manually set using ZB OTC market, currently is 0.997
       
  • Get BITCNY/BTS price from Bitshares DEX
  • Get current feed price from Bitshares DEX
  • Compute premium or discount: CNY/BTS / BITCNY/BTS
  • Compute adjusted feed price: current_feed_price * (1 + premium_or_discount * adjustment_scale)
            adjustment_scale is manually adjusted depending my appreciation on how fast the market react. Current value is 1.
  • Publish the adjusted feed price (this means BSIP-42 is activated on BitCNY)


USD:
   
  • Compute USD/BTS price from a weighted mean of those routes:
           
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'binance - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'openledger - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'gdex - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'coinegg - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'cointiger - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'huobi - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'poloniex - BTC:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'zb - BTC:BTS'
    • 'weighted_mean(bittrex, kraken) - USD:USDT', 'huobi - USDT:BTS'
    • 'weighted_mean(bittrex, kraken) - USD:USDT', 'zb - USDT:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'cointiger - ETH:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'huobi - ETH:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:BTC', 'cointiger - BTC:BITCNY', 'bitshares - BITCNY:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'cointiger - ETH:BITCNY', 'bitshares - BITCNY:BTS'

            Notes:
           
    • USD:BTC is computed only from CEX that have a FIAT ramp
         
  • Get BITUSD/BTS price from Bitshares DEX
  • Get current feed price from Bitshares DEX
  • Compute premium or discount: USD/BTS / BITUSD/BTS
  • Compute adjusted feed price: current_feed_price * (1 + premium_or_discount * adjustment_scale)
            adjustment_scale is manually adjusted depending my appreciation on how fast the market react. Current value is 1.2.
         
  • Publish the adjusted feed price (this means BSIP-42 is activated on BitUSD)


BTC:
   
  • Compute BTC/BTS price from a weighted mean of those routes:
           
    • 'binance - BTC:BTS'
    • 'openledger - BTC:BTS'
    • 'gdex - BTC:BTS'
    • 'coinegg - BTC:BTS'
    • 'cointiger - BTC:BTS'
    • 'huobi - BTC:BTS'
    • 'poloniex - BTC:BTS'
    • 'zb - BTC:BTS'
    • 'cointiger - BTC:BITCNY', 'bitshares - BITCNY:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(bittrex, kraken) - USD:USDT', 'huobi - USDT:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(bittrex, kraken) - USD:USDT', 'zb - USDT:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'cointiger - ETH:BTS'
    • 'weighted_mean(coinbase, kraken, bitstamp) - BTC:USD', 'weighted_mean(coinbase, kraken, bitstamp) - USD:ETH', 'huobi - ETH:BTS'
    • 'cointiger - BTC:BITCNY', 'cointiger - BITCNY:ETH', 'cointiger - ETH:BTS'
    • 'cointiger - BTC:BITCNY', 'cointiger - BITCNY:ETH', 'huobi - ETH:BTS'
    • 'cointiger - BTC:BITCNY', 'zb - BITCNY:QC', 'zb - QC:BTS'
       
  • Publish BTC/BTS unadjusted (no BSIP-42)

RUBLE/EUR/AUD/JPY/CHF/MXN/GBP/CAD/ARS/HKD/SEK/SGD/NZD/KRW:
  • Get USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)
  • Get XXX/USD from the mean of CurrencyLayer and OpenExchangeRates
  • Then apply XXX/USD * USD/BTS to publish XXX/BTS (no BSIP-42)

GOLD/SILVER:
  • Get USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)
  • Get XXX/USD from Quadl LMBA source
  • Then apply XXX/USD * USD/BTS to publish XXX/BTS  (no BSIP-42)

HERTZ:
  • Apply the formula to USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)

AMAZONCOM:
  • Get USD/BTS price from Bitshares DEX (I do not use the feed price since BSIP-42 is applied to USD)
  • Get AMAZONCOM/USD from IEX source
  • Then apply XXX/USD * USD/BTS to publish XXX/BTS  (no BSIP-42)

The up-to-date version of the code used is available there.
It contains implementations for many BSIP 42 algorithms, a large variety of price sources, and is largely configurable.

You can use the pricefeed tracker to check the published prices over time and against other witnesses, and/or to compare feed price to CEX and DEX price and track premium/discount.

I plan to update my setup to:
  • implement the "protect discount limit" as disccussed here.
  • use a lower scale at discount and a higher scale at premium as suggested by abit here.
  • publish all my intermediary values to the pricefeed tracker website so anyone would be able to analyze where my price comes from.


Edit:
  - 20181116 17:55 UTC : Added forgotten AEX source for BitCNY (api is unstable, so sometime it's not used).
« Last Edit: November 16, 2018, 05:56:29 pm by zapata42 »

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12694
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #31 on: November 16, 2018, 04:26:21 pm »
+5% for transparency and your work on the price feed script!
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline matle85

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #32 on: November 16, 2018, 05:10:06 pm »
That's great - thank you!

Have we got agreed weighting for the different price feed sources? I.e. if we are concerned one exchange has faked volume it could be having an unjustified impact on the feed - I can see various discussions on the reliability of different sources but not sure if we maintain a weighting somewhere?
« Last Edit: November 16, 2018, 05:52:57 pm by matle85 »

Offline zapata42

  • Full Member
  • ***
  • Posts: 57
    • View Profile
  • BitShares: zapata42
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #33 on: November 16, 2018, 06:11:30 pm »
Have we got agreed weighting for the different price feed sources? I.e. if we are concerned one exchange has faked volume it could be having an unjustified impact on the feed.

Currently my weighting is volume based average.
I currently don't apply any adjustment by market, however the script has a `scaleVolumeBy` feature that let you adjust each source volume according to an arbitrary ratio.
I'm not aware of any agreed weighting, but abit has publicly expressed his opinions, and this topic has been discussed multiple times here and on telegram.
I agree that some volumes (and trade history) seems faked, so I'll take some time to forge an updated opinion and scale my volumes accordingly.
I'll keep you informed.

Offline matle85

  • Jr. Member
  • **
  • Posts: 47
    • View Profile
Re: [Witness Proposal] 1.6.129 - zapata42-witness
« Reply #34 on: November 17, 2018, 11:34:18 am »
I'm not aware of any agreed weighting, but abit has publicly expressed his opinions, and this topic has been discussed multiple times here and on telegram.
I agree that some volumes (and trade history) seems faked, so I'll take some time to forge an updated opinion and scale my volumes accordingly.
I'll keep you informed.

Thanks Zapata and just to say, reading through all your posts I'm really impressed by the professional approach you have to this work. You're a great asset to the community.

It would be really interesting to see how the prices at different exchanges vary over time. That will show if some are skewing the price and causing some of the odd effects that Bitcrab & others are concerned about around discounts and force settlements etc.

It may be we consider some more reliable / relevant than others for bitCNY specifically. If the weighting is transparent and justified then it would be fine in my view to give some greater weight and others a reduced weight.
« Last Edit: November 17, 2018, 12:06:24 pm by matle85 »