BitShares Forum

Main => Stakeholder Proposals => Topic started by: zapata42 on July 30, 2018, 08:21:09 pm

Title: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on July 30, 2018, 08:21:09 pm
Hello,

dear Bitshares community, I want to propose my services as a Bitshares Witness.

Please vote for zapata42-witness.

Who am I?

I am a developer with 13 years of experience in finance sector, based in Paris (France). I have worked for BNP Paribas Corporate Investment Banking in the equity derivatives business for the past 10 years in projects as varied as price publication, position keeping or request for quotes, and as a developer, technical lead or architect. Java is my main programming language but I have good knowledge of Python, Javascript, and Ruby. See below for my full resume.

I came to know Bitshares about one year ago, and since then I closely followed it as I was impressed by the technology, the balanced economical model and the distributed governance. I started to contribute on my spare time as I could:

Now I want to be more involved in Bitshares, and as I am not a C++ nor a React programmer (yet?), I feel like witness role currently suit me best to help the community. You will found below my current setup, and what I plan to do once voted in.


My current setup:

    Hardware: 3 x VPS 4 cores CPU at 2.4ghz, 24GB RAM, 100GB SSD hard drive, 250Mpbs internet bandwith, Location: France.
    (I consider this a minimum, see below for my target plan)
   
    I run 2 witness mainnet nodes (account zapata42-witness) and 2 witness testnet node (witness-zapata).
    They are monitored through an improved version (https://github.com/Zapata/bitshares-witness-monitor) of @clockwork's telegram monitoring bot that handle automatic key switch and price feed monitoring.

    The price feed is ready, I already publish @nasdaq assets (see AMAZONCOM (https://wallet.bitshares.org/#/asset/AMAZONCOM)) and I use a stabilized version (https://github.com/Zapata/bitshares-pricefeed) of @xeroc's pricefeed.

    Additionnaly I host some services:
        Mainnet API:         https://api.dex.trading  (max-ops-per-account = 100)
        Mainnet Seed:       seed.dex.trading:1776
        Testnet API:          https://testnet.dex.trading
        Wallet:                  https://wallet.dex.trading or https://bitshares.dex.trading (patched version (https://github.com/Zapata/bitshares-ui) that use my referral link as default (https://github.com/Zapata/bitshares-ui/blob/master/app/branding.js#L30))

    Services are served through multiple proxies (currently NGINX) that do rate limiting and SSL termination.
    For SSL certificate is issued by Let's encrypt and renewal is automated.
    Everything run on a private multi datacenter Kubernetes cluster (so fully Dockerized and resilient) and monitored through Prometheus/Grafana.
    The setup is automated and ready to scale, see my plan below.

My plan:

If I get voted as an active witness my focus will be on:
This means:

So why voting for zapata42-witness ?


Contact:
  Telegram: @Zapata_42
  LinkedIn profile (http://www.linkedin.com/in/marco-tessari)

* Edit 20180901: Update seed port.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: Bangzi on July 30, 2018, 10:34:13 pm
Voted! +5% +5% +5%
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: xeroc on July 31, 2018, 08:55:02 am
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on July 31, 2018, 10:34:10 am
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).

My testnet block producer is running since early may and active since 12/05/2018. The account is: witness-zapata.

I also consider that running a testnet block producer is a mandatory task for a witness. I always test my changes on testnet before applying them on mainnet. It was also required for some developments. I am trying to keep it as stable as the main one.

On testnet, I missed 579 blocks so far:
Note that I have heavely updated the underlying servers and kubernetes cluster while running the witness without issues.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: Digital Lucifer on July 31, 2018, 11:01:12 am
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).

My testnet block producer is running since early may and active since 12/05/2018. The account is: witness-zapata.

I also consider that running a testnet block producer is a mandatory task for a witness. I always test my changes on testnet before applying them on mainnet. It was also required for some developments. I am trying to keep it as stable as the main one.

On testnet, I missed 579 blocks so far:
  • the first ~540 was because initially I didn't put the correct private/public key pair, and didn't notice that I was activated. To prevent the issue on mainnet I monitor the activation (see this commit (https://github.com/Zapata/bitshares-witness-monitor/commit/9a3227cfa03db2e6e567f697470bb8c5bf338378))
  • then I missed ~20 to update the version and at that time I was not running a backup node;
  • 1 block missed for a non investigated issue (short network loss);
  • then I missed ~15 block during my chaos testing to validate the backup procedures: I switch signing key after 3 blocks, and I tested muliple cases.
Note that I have heavely updated the underlying servers and kubernetes cluster while running the witness without issues.

Voted (just humble ~200k BTS). I have a testnet witness, so I can co-relate with issues you had and highly appreciate your precision in reporting.

Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: fav on July 31, 2018, 02:24:31 pm
voted
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: clockwork on July 31, 2018, 09:18:35 pm
Will vote for this. Zapata is a huge asset to the community :)
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on July 31, 2018, 11:27:21 pm
Thanks. Voted.

Please submit your mainnet API node to https://github.com/bitshares/bitshares-ui/ .
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: xeroc on August 02, 2018, 10:38:22 am
voted
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on August 05, 2018, 02:45:18 pm
Thanks for your votes so far.

Please submit your mainnet API node to https://github.com/bitshares/bitshares-ui/ .

My nodes will be available in the next UI release (see PR (https://github.com/bitshares/bitshares-ui/pull/1742)). You can already find them in the staging wallet (https://staging.bitshares.org).
Any feedback on the latency is welcome.

My testnet witness (witness.zapata) and API node (wss://testnet.dex.trading) have been updated (see release thread (https://bitsharestalk.org/index.php?topic=26902.msg320546#msg320546)).
I missed one block in the deployment process as I switched my signing key back to my main witness node while it was my turn.
I will take care to avoid this case in the future.
(Note that I had a typo in my previous post, you should have read 597, instead of 579. I'm now at 598).

This week I also added Bitshares integration (https://github.com/EOSIO/demux-js/pull/23) to EOSIO/demux-js (https://github.com/EOSIO/demux-js). You can find out more information on Demux here (https://medium.com/eosio/introducing-demux-deterministic-databases-off-chain-verified-by-the-eosio-blockchain-bd860c49b017).
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: xeroc on August 08, 2018, 08:24:58 am
This week I also added Bitshares integration (https://github.com/EOSIO/demux-js/pull/23) to EOSIO/demux-js (https://github.com/EOSIO/demux-js). You can find out more information on Demux here (https://medium.com/eosio/introducing-demux-deterministic-databases-off-chain-verified-by-the-eosio-blockchain-bd860c49b017).
Extraordinary!
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: apasia.tech on August 08, 2018, 04:08:57 pm
Just wanted to say, I've read the full proposal with enormous interest, in a good way and with the background and resume - this is a welcome addition to BitShares. Furthermore, I've heard more good feedback from others who welcome this witness proposal. Some exceptional contributions we didn't see coming, thank you.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on August 17, 2018, 10:02:26 pm
Following current various discussions on price feeds I spend a couple of days tuning mine.


Here is the algorithm I use:

With that I compute all possible routes using BTC, ETH, USDT or BITCNY as intermediate asset.
Then I use a weighted average using the BTS/{BTC,ETH,USDT,BITCNY} daily volume.

Here is a sample result of all the routes for USD/BTS and CNY/BTS at 17/08/2018 21:58 UTC:

Code: [Select]
+------+-------+--------------+-------+--------------------+------------------------------------------------------------------------------------+
| base | quote | price        | diff  | quote volume       | source                                                                             |
+------+-------+--------------+-------+--------------------+------------------------------------------------------------------------------------+
| CNY  | BTS   | 0.8008708826 | +1.58 | 31726149.140697803 | ['magicwallet - CNY:BITCNY', 'bitshares - BITCNY:BTS']                             |
| CNY  | BTS   | 0.7775048363 | -1.38 | 275814.57983       | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'gdex - BTC:BTS']       |
| CNY  | BTS   | 0.7767051983 | -1.48 | 6099695.41644055   | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'poloniex - BTC:BTS']   |
| CNY  | BTS   | 0.7776046832 | -1.37 | 6823092.1          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'zb - BTC:BTS']         |
| CNY  | BTS   | 0.7826822718 | -0.73 | 1944911.60043      | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'openledger - BTC:BTS'] |
| CNY  | BTS   | 0.7897477292 | +0.17 | 8090672.0          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'coinegg - BTC:BTS']    |
| CNY  | BTS   | 0.7821021076 | -0.80 | 719268.12          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'cointiger - BTC:BTS']  |
| CNY  | BTS   | 0.7758057134 | -1.60 | 3068685.9901371617 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'huobi - BTC:BTS']      |
| CNY  | BTS   | 0.7816523652 | -0.86 | 1648254.1922999984 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'lbank - BTC:BTS']      |
| CNY  | BTS   | 0.7798533954 | -1.08 | 5457868.0          | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:BTC', 'binance - BTC:BTS']    |
| CNY  | BTS   | 0.7968423519 | +1.07 | 275814.57983       | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'gdex - BTC:BTS']           |
| CNY  | BTS   | 0.7960228259 | +0.97 | 6099695.41644055   | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'poloniex - BTC:BTS']       |
| CNY  | BTS   | 0.7969446821 | +1.08 | 6823092.1          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'zb - BTC:BTS']             |
| CNY  | BTS   | 0.8021485567 | +1.74 | 1944911.60043      | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'openledger - BTC:BTS']     |
| CNY  | BTS   | 0.8093897408 | +2.66 | 8090672.0          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'coinegg - BTC:BTS']        |
| CNY  | BTS   | 0.8015539631 | +1.67 | 719268.12          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'cointiger - BTC:BTS']      |
| CNY  | BTS   | 0.7951009697 | +0.85 | 3068685.9901371617 | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'huobi - BTC:BTS']          |
| CNY  | BTS   | 0.8010930350 | +1.61 | 1648254.1922999984 | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'lbank - BTC:BTS']          |
| CNY  | BTS   | 0.7992493226 | +1.38 | 5457868.0          | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:BTC', 'binance - BTC:BTS']        |
| CNY  | BTS   | 0.7739653886 | -1.83 | 17591765.91406     | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:USDT', 'aex - USDT:BTS']      |
| CNY  | BTS   | 0.7767270849 | -1.48 | 15976040.0         | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:USDT', 'zb - USDT:BTS']       |
| CNY  | BTS   | 0.7794887812 | -1.13 | 10213236.688898135 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:USDT', 'huobi - USDT:BTS']    |
| CNY  | BTS   | 0.7760030565 | -1.57 | 188647.0           | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:ETH', 'cointiger - ETH:BTS']  |
| CNY  | BTS   | 0.7760030565 | -1.57 | 1040797.2504070019 | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:ETH', 'huobi - ETH:BTS']      |
| CNY  | BTS   | 0.7778900585 | -1.33 | 4698376.790000004  | ['openexchangerates - CNY:USD', 'coinmarketcap - USD:ETH', 'lbank - ETH:BTS']      |
| CNY  | BTS   | 0.7930701250 | +0.59 | 188647.0           | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'cointiger - ETH:BTS']      |
| CNY  | BTS   | 0.7930701250 | +0.59 | 1040797.2504070019 | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'huobi - ETH:BTS']          |
| CNY  | BTS   | 0.7949986288 | +0.84 | 4698376.790000004  | ['magicwallet - CNY:BITCNY', 'cointiger - BITCNY:ETH', 'lbank - ETH:BTS']          |
| USD  | BTS   | 0.1130179495 | -0.27 | 275814.57983       | ['coinmarketcap - USD:BTC', 'gdex - BTC:BTS']                                      |
| USD  | BTS   | 0.1129017143 | -0.37 | 6099695.41644055   | ['coinmarketcap - USD:BTC', 'poloniex - BTC:BTS']                                  |
| USD  | BTS   | 0.1130324632 | -0.26 | 6823092.1          | ['coinmarketcap - USD:BTC', 'zb - BTC:BTS']                                        |
| USD  | BTS   | 0.1137705405 | +0.39 | 1944911.60043      | ['coinmarketcap - USD:BTC', 'openledger - BTC:BTS']                                |
| USD  | BTS   | 0.1147975740 | +1.30 | 8090672.0          | ['coinmarketcap - USD:BTC', 'coinegg - BTC:BTS']                                   |
| USD  | BTS   | 0.1136862080 | +0.32 | 719268.12          | ['coinmarketcap - USD:BTC', 'cointiger - BTC:BTS']                                 |
| USD  | BTS   | 0.1127709653 | -0.49 | 3068685.9901371617 | ['coinmarketcap - USD:BTC', 'huobi - BTC:BTS']                                     |
| USD  | BTS   | 0.1136208335 | +0.26 | 1648254.1922999984 | ['coinmarketcap - USD:BTC', 'lbank - BTC:BTS']                                     |
| USD  | BTS   | 0.1133593356 | +0.03 | 5457868.0          | ['coinmarketcap - USD:BTC', 'binance - BTC:BTS']                                   |
| USD  | BTS   | 0.1125034561 | -0.72 | 17591765.91406     | ['coinmarketcap - USD:USDT', 'aex - USDT:BTS']                                     |
| USD  | BTS   | 0.1129048957 | -0.37 | 15976040.0         | ['coinmarketcap - USD:USDT', 'zb - USDT:BTS']                                      |
| USD  | BTS   | 0.1133063354 | -0.02 | 10213236.688898135 | ['coinmarketcap - USD:USDT', 'huobi - USDT:BTS']                                   |
| USD  | BTS   | 0.1127996511 | -0.46 | 188647.0           | ['coinmarketcap - USD:ETH', 'cointiger - ETH:BTS']                                 |
| USD  | BTS   | 0.1127996511 | -0.46 | 1040797.2504070019 | ['coinmarketcap - USD:ETH', 'huobi - ETH:BTS']                                     |
| USD  | BTS   | 0.1130739453 | -0.22 | 4698376.790000004  | ['coinmarketcap - USD:ETH', 'lbank - ETH:BTS']                                     |
| USD  | BTS   | 0.1128235134 | -0.44 | 275814.57983       | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'gdex - BTC:BTS']         |
| USD  | BTS   | 0.1127074782 | -0.54 | 6099695.41644055   | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'poloniex - BTC:BTS']     |
| USD  | BTS   | 0.1128380022 | -0.43 | 6823092.1          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'zb - BTC:BTS']           |
| USD  | BTS   | 0.1135748097 | +0.22 | 1944911.60043      | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'openledger - BTC:BTS']   |
| USD  | BTS   | 0.1146000762 | +1.13 | 8090672.0          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'coinegg - BTC:BTS']      |
| USD  | BTS   | 0.1134906222 | +0.15 | 719268.12          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'cointiger - BTC:BTS']    |
| USD  | BTS   | 0.1125769542 | -0.66 | 3068685.9901371617 | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'huobi - BTC:BTS']        |
| USD  | BTS   | 0.1134253602 | +0.09 | 1648254.1922999984 | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'lbank - BTC:BTS']        |
| USD  | BTS   | 0.1131643122 | -0.14 | 5457868.0          | ['coinmarketcap - USD:USDT', 'coinmarketcap - USDT:BTC', 'binance - BTC:BTS']      |
| USD  | BTS   | 0.1128514039 | -0.42 | 275814.57983       | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'gdex - BTC:BTS']           |
| USD  | BTS   | 0.1127353400 | -0.52 | 6099695.41644055   | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'poloniex - BTC:BTS']       |
| USD  | BTS   | 0.1128658963 | -0.40 | 6823092.1          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'zb - BTC:BTS']             |
| USD  | BTS   | 0.1136028859 | +0.25 | 1944911.60043      | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'openledger - BTC:BTS']     |
| USD  | BTS   | 0.1146284059 | +1.15 | 8090672.0          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'coinegg - BTC:BTS']        |
| USD  | BTS   | 0.1135186776 | +0.17 | 719268.12          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'cointiger - BTC:BTS']      |
| USD  | BTS   | 0.1126047837 | -0.63 | 3068685.9901371617 | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'huobi - BTC:BTS']          |
| USD  | BTS   | 0.1134533995 | +0.11 | 1648254.1922999984 | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'lbank - BTC:BTS']          |
| USD  | BTS   | 0.1131922869 | -0.12 | 5457868.0          | ['coinmarketcap - USD:ETH', 'coinmarketcap - ETH:BTC', 'binance - BTC:BTS']        |
| USD  | BTS   | 0.1126973406 | -0.55 | 17591765.91406     | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:USDT', 'aex - USDT:BTS']          |
| USD  | BTS   | 0.1130994721 | -0.20 | 15976040.0         | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:USDT', 'zb - USDT:BTS']           |
| USD  | BTS   | 0.1135016035 | +0.16 | 10213236.688898135 | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:USDT', 'huobi - USDT:BTS']        |
| USD  | BTS   | 0.1135893251 | +0.23 | 31726149.140697803 | ['coinmarketcap - USD:BTC', 'cointiger - BTC:BITCNY', 'bitshares - BITCNY:BTS']    |
| USD  | BTS   | 0.1139091655 | +0.52 | 31726149.140697803 | ['coinmarketcap - USD:ETH', 'cointiger - ETH:BITCNY', 'bitshares - BITCNY:BTS']    |
| USD  | BTS   | 0.1129661203 | -0.32 | 188647.0           | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:ETH', 'cointiger - ETH:BTS']      |
| USD  | BTS   | 0.1129661203 | -0.32 | 1040797.2504070019 | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:ETH', 'huobi - ETH:BTS']          |
| USD  | BTS   | 0.1132408193 | -0.07 | 4698376.790000004  | ['coinmarketcap - USD:BTC', 'coinmarketcap - BTC:ETH', 'lbank - ETH:BTS']          |
+------+-------+--------------+-------+--------------------+------------------------------------------------------------------------------------+

And the final results I get:

Code: [Select]
+--------+------------+--------------+--------------+---------------------+---------------------+---------------------+--------------+---------------------+--------+--------+-------------------------+-------------+
| symbol | collateral |  new price   |     cer      |         mean        |        median       |      wgt. avg.      | wgt. std (#) |      blockchain     |  mssr  |  mcr   |      my last price      | last update |
+--------+------------+--------------+--------------+---------------------+---------------------+---------------------+--------------+---------------------+--------+--------+-------------------------+-------------+
|  CNY   |    BTS     | 0.7884066494 | 0.9460879792 | 0.7880032477 (0.05) | 0.7862150005 (0.28) | 0.7884066494 (0.00) |  1.17 (28)   | 0.7728333333 (2.02) | 110.0% | 175.0% | -1.0000000000 (-178.84) | unknown ago |
|  USD   |    BTS     | 0.1133240311 | 0.1189902327 | 0.1132431633 (0.07) | 0.1130994721 (0.20) | 0.1133240311 (0.00) |  0.06 (41)   | 0.1123699660 (0.85) | 110.0% | 175.0% | -1.0000000000 (-111.33) | unknown ago |
+--------+------------+--------------+--------------+---------------------+---------------------+---------------------+--------------+---------------------+--------+--------+-------------------------+-------------+

I use xeroc's pricefeed script that I modified again to add:
All code is accessible at Zapata/bitshares-pricefeed (https://github.com/Zapata/bitshares-pricefeed).

Your comments are more than welcome, this is intended to be adjusted over time.

Currently my ideas to further improve are:

I would be able to progress on that once voted. Since then I'm helping Alfredo on the open-explorer (see last PR (https://github.com/oxarbitrage/explorer-api/pull/25)).
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on August 17, 2018, 11:00:18 pm
Good job about the price feeding thing. I'd say you're getting quite accurate price data.

You can add a few more markets:

* Aex: BTS/CNC (CNC is IOU CNY)

* Zb: BTS/QC (QC is IOU CNY as well)


BTW, IMHO, Zb's volume is faked, Coinegg's volume is faked, Lbank's volume is faked. Huobi, Binance and Aex? I don't know. Poloniex? I don't trust them because they left a bad history. If adjust their weight to lower, likely we'll get a slightly higher BTS price in the end.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: kimchi-king on August 22, 2018, 09:21:37 am
Voted.

Keep up the great work.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on August 31, 2018, 11:45:33 pm
Thanks for your votes, I am an active witness since the begining of the week.

Some updates from the last two weeks:

I will be at BitFest, Friday and Saturday, ping me here or on Telegram if you want to have a chat there.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: gghi on September 01, 2018, 08:27:54 am
Identity, we must increase feedback so that we can quickly rectify the deviation. It is suggested that the ideal price = BTS market price * (bitcny/cny price +M) ^N, M indicates the loose index, N is the acceleration factor. For example, when the premium is zero, the easing target is negative 2%, the acceleration is 1.8, then the ideal feed = BTS market price * 1 + 0.02 ^ 1.8, for example, when the premium is 0.001, the easing target is negative 1%, the acceleration is 1.5, then the ideal feed = BTS market price * 1.001 + 0.01 ^ 1.5.
For example, the formula is proved to be effective, and the loose target is minus 1%.
When the premium is negative 0.01 (discount 1%), the acceleration factor is 2. Ideal feed price = BTS market price * 1-0.01 + 0.01) ^ 2 = BTS market price * 1 ^ 2 = BTS market price * 1. Conclusion: The target is achieved, very ideal.
When the premium is 0, the acceleration factor is 2. Ideal feeding price = BTS market price * 1 + 0.01) ^ 2 = BTS market price * 1.01 ^ 2 = BTS market price * 1.02, conclusion: the market price increase by 2%, very satisfactory, very safe.
When the premium is 0.01, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.01 + 0.01) ^ 2 = BTS market price * 1.02 ^ 2 = BTS market price * 1.0404. Conclusion: The increase of market price by 4% is very satisfactory and safe.
When the premium is 0.02, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.02 + 0.01) ^ 2 = BTS market price * 1.03 ^ 2 = BTS market price * 1.0609. Conclusion: The increase of the market price by 6% is very satisfactory and safe.
When the premium is 0.03, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.03 + 0.01) ^ 2 = BTS market price * 1.04 ^ 2 = BTS market price * 1.0816. Conclusion: The higher the market price is, the more satisfied and safe it is.

When the premium is 0.04, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.04 + 0.01) ^ 2 = BTS market price * 1.05 ^ 2 = BTS market price * 1.1025. Conclusion: The market price increases by about 10%, satisfactory and safe.
When the premium is 0.05, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.05 + 0.01) ^ 2 = BTS market price * 1.06 ^ 2 = BTS market price * 1.1236. Conclusion: The market price increases about 11.2%, satisfies, speeds up the correction.
When the premium is 0.06, the acceleration factor is 2. Ideal Feed = BTS Market Price * 1.06+0.01) ^ 2 = BTS Market Price * 1.07 ^ 2 = BTS Market Price * 1.1449. Conclusion: The market price increases about 11.4%, satisfactory, accelerate the correction of deviation.
When the premium is 0.07, the acceleration factor is 2. Ideal Feed = BTS Market Price * 1.07+0.01) ^ 2 = BTS Market Price * 1.08 ^ 2 = BTS Market Price * 1.1664. Conclusion: Market Price increases by about 11.66%, satisfies, speeds up the correction.
When the premium is 0.08, the acceleration factor is 2. Ideal Feed = BTS Market Price * 1.08+0.01) ^ 2 = BTS Market Price * 1.09 ^ 2 = BTS Market Price * 1.1881, Conclusion: Market Price increased by 11.88%, satisfactory, accelerate correction.
When the premium is 0.09, the acceleration factor is 2. The ideal feeding price = BTS market price * 1.09 + 0.01) ^ 2 = BTS market price * 1.1 ^ 2 = BTS market price * 1.21. Conclusion: The market price increases about 12%, satisfies, accelerates the correction, and the risk is still controllable.

  认同,必须加大反馈力度才可以快速纠偏。建议理想喂价 = BTS市场价 * (bitcny/cny价+M)^N,M表示宽松指标,N为加速因子。比如溢价为零时,宽松目标负2%,加速度1.8,那么,理想喂价 = BTS市场价 * (1+0.02)^1.8,比如溢价为0.001时,宽松目标负1%,加速度1.5,那么,理想喂价 = BTS市场价 * (1.001+0.01)^1.5。
       举例论证公式效果,宽松目标负1%。
当溢价负0.01时(折扣1%),加速因子喂2。理想喂价 = BTS市场价 * (1-0.01+0.01)^2=BTS市场价 *1^2=BTS市场价 *1,结论:目标达到,非常理想。
当溢价0时,加速因子喂2。理想喂价 = BTS市场价 * (1+0.01)^2=BTS市场价 *1.01^2=BTS市场价 *1.02,结论:市场价提高2%,很满意,也很安全。
当溢价0.01时,加速因子喂2。理想喂价 = BTS市场价 * (1.01+0.01)^2=BTS市场价 *1.02^2=BTS市场价 *1.0404,结论:市场价提高约4%,很满意,也很安全。
当溢价0.02时,加速因子喂2。理想喂价 = BTS市场价 * (1.02+0.01)^2=BTS市场价 *1.03^2=BTS市场价 *1.0609,结论:市场价提高约6%,很满意,也很安全。
当溢价0.03时,加速因子喂2。理想喂价 = BTS市场价 * (1.03+0.01)^2=BTS市场价 *1.04^2=BTS市场价 *1.0816,结论:市场价提高越8%,很满意,也很安全。

当溢价0.04时,加速因子喂2。理想喂价 = BTS市场价 * (1.04+0.01)^2=BTS市场价 *1.05^2=BTS市场价 *1.1025,结论:市场价提高约10%,满意,也很安全。
     当溢价0.05时,加速因子喂2。理想喂价 = BTS市场价 * (1.05+0.01)^2=BTS市场价 *1.06^2=BTS市场价 *1.1236,结论:市场价提高约11.2%,满意,加速纠偏。
     当溢价0.06时,加速因子喂2。理想喂价 = BTS市场价 * (1.06+0.01)^2=BTS市场价 *1.07^2=BTS市场价 *1.1449,结论:市场价提高约11.4%,满意,加速纠偏
当溢价0.07时,加速因子喂2。理想喂价 = BTS市场价 * (1.07+0.01)^2=BTS市场价 *1.08^2=BTS市场价 *1.1664,结论:市场价提高约11.66%,满意,加速纠偏
当溢价0.08时,加速因子喂2。理想喂价 = BTS市场价 * (1.08+0.01)^2=BTS市场价 *1.09^2=BTS市场价 *1.1881,结论:市场价提高约11.88%,满意,加速纠偏
当溢价0.09时,加速因子喂2。理想喂价 = BTS市场价 * (1.09+0.01)^2=BTS市场价 *1.1^2=BTS市场价 *1.21,结论:市场价提高约12%,满意,加速纠偏,风险依然可控。
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on September 07, 2018, 12:03:13 am
As BSIP42 has been voted in, my feed price has been adjusted accordingly.

The current adjustment is the use the same  formula as gdex-witness (https://bitsharestalk.org/index.php?topic=26315.msg321931#msg321931).
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: gghi on September 07, 2018, 12:35:37 am
As BSIP42 has been voted in, my feed price has been adjusted accordingly.

The current adjustment is the use the same  formula as gdex-witness (https://bitsharestalk.org/index.php?topic=26315.msg321931#msg321931).

     GOOD,voted
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on September 08, 2018, 10:17:13 am
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: armin on September 08, 2018, 12:06:58 pm
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.

Do you know when "the highest one" price feed will be implemented?
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on September 09, 2018, 08:59:10 am
Hi Zapata, we're discussing new algorithm: https://bitsharestalk.org/index.php?topic=26315.msg322068#msg322068, please check.

Do you know when "the highest one" price feed will be implemented?
It's up to the witnesses to decide. Personally I don't think "the highest one" is correct.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on October 03, 2018, 11:31:59 pm
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on October 04, 2018, 10:16:52 pm
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.

It was already possible to configure the script to retrieve USDT price from Bittrex.
However, the Kraken source was not implemented, so I just added it (https://github.com/Zapata/bitshares-pricefeed/commit/111f623cb27e4d3afb9671b17b8cdb4b44de316b).

As an example witnesses can add this configuration to the script in order to use Bittrex+Kraken sources for USDT:
Code: [Select]
    usdt_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            bittrex:
                klass: Bittrex
                quotes:
                    - USDT
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - USDT
                bases:
                    - ZUSD
                aliases:
                    ZUSD: USD

I have updated my script to use those sources.

I would also like to take this opportunity to say that the PID formula implementation (https://github.com/Zapata/bitshares-pricefeed/commit/2993671147664a7fe822c7d361b3cb7041e7af5f) is also available if poeple want to practice with it.

I am also working on a tool to visualize witness pricefeeds over time: http://pricefeed-tracker.dex.trading/
(code is here (https://github.com/Zapata/bitshares-pricefeed-tracker))
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on October 05, 2018, 07:18:52 pm
Thanks!

Another thing mentioned in Telegram group by @pc:
> please make sure that you publish a CER that is not higher than the settlement price for assets in black swan state. "higher" as in more BTS per bitXYZ

If CER is too low, it's cheaper for people to pay fees in that asset than in BTS if there is some BTS in the fee pool, although not a big deal because there should not be much BTS in the fee pool.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on October 07, 2018, 12:15:46 am
Another thing mentioned in Telegram group by @pc:
> please make sure that you publish a CER that is not higher than the settlement price for assets in black swan state. "higher" as in more BTS per bitXYZ

If CER is too low, it's cheaper for people to pay fees in that asset than in BTS if there is some BTS in the fee pool, although not a big deal because there should not be much BTS in the fee pool.

I have updated the script to handle the case, see commit 1 (https://github.com/Zapata/bitshares-pricefeed/commit/b5a141822f844990093a38c20db9501bed6a8f8b), 2 (https://github.com/Zapata/bitshares-pricefeed/commit/196d4f3b43cf623c54f97cb26c2ab7a82aa4291d).
The applyed formula is: If CER < asset global settlement price, then published CER is the asset global settlement price.

My feed for those assets will be impacted: BTC, SILVER, GOLD, SGD, CHF, AUD.

Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on October 16, 2018, 10:43:40 pm
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
As discussed in Telegram channel, CMC's price is not reliable thus should be avoided.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on October 20, 2018, 10:08:04 am
@zapata42 USDT is now trading at around 0.991 USD. Data sources (direct trading pairs):
* https://www.kraken.com/charts
* https://bittrex.com/Market/Index?MarketName=USD-USDT

CoinMarketCap is reporting 0.9968 which seems a bit off. https://coinmarketcap.com/currencies/tether/

Perhaps your script need to be updated.

It was already possible to configure the script to retrieve USDT price from Bittrex.
However, the Kraken source was not implemented, so I just added it (https://github.com/Zapata/bitshares-pricefeed/commit/111f623cb27e4d3afb9671b17b8cdb4b44de316b).

As an example witnesses can add this configuration to the script in order to use Bittrex+Kraken sources for USDT:
Code: [Select]
    usdt_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            bittrex:
                klass: Bittrex
                quotes:
                    - USDT
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - USDT
                bases:
                    - ZUSD
                aliases:
                    ZUSD: USD

I have updated my script to use those sources.

I would also like to take this opportunity to say that the PID formula implementation (https://github.com/Zapata/bitshares-pricefeed/commit/2993671147664a7fe822c7d361b3cb7041e7af5f) is also available if poeple want to practice with it.

I am also working on a tool to visualize witness pricefeeds over time: http://pricefeed-tracker.dex.trading/
(code is here (https://github.com/Zapata/bitshares-pricefeed-tracker))
http://www.untether.space/ has some interesting data about USDT, perhaps we can use it or the way it calculates USDT price.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on October 21, 2018, 05:34:41 pm
  • Get BTC/USD, ETH/USD, USDT/USD from CoinMarketCap.com (fallback to worldcoinindex.com)
As discussed in Telegram channel, CMC's price is not reliable thus should be avoided.

Agree, I have removed it since, my configuration now use a weighted mean of exchanges with FIAT deposit (Coinbase, Kraken and Bitstamp).
Here is the config:

Code: [Select]
    btc_and_eth_price:
        klass: Composite
        aggregation_type: 'weighted_mean'
        exchanges:
            coinbase:
                klass: Coinbase
                quotes:
                    - BTC
                    - ETH
                bases:
                    - USD
            kraken:
                klass: Kraken
                quotes:
                    - XXBT
                    - XETH
                bases:
                    - ZUSD
                aliases:
                    XXBT: BTC
                    XETH: ETH
                    ZUSD: USD
            bitstamp:
                klass: Bitstamp
                quotes:
                    - BTC
                    - ETH
                bases:
                    - USD

Note that Coinbase source has been added (https://github.com/Zapata/bitshares-pricefeed/commit/5a78a42983eb4b7d78afeb63279b906c6886a614) for that.

http://www.untether.space/ has some interesting data about USDT, perhaps we can use it or the way it calculates USDT price.

Unfortunately the website does not provide any API currently. We can easely recompute the same data (see the code (https://github.com/jpatokal/untether/blob/master/public/js/untether.js#L118)), but I don't see big differences with current approach (weighted average of USD/USDT of Bittrex and Kraken). Market arbitrage seems efficient.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: abit on October 24, 2018, 09:12:36 am
Thanks again.

As discussed in Telegram:

I think it's better to use highest bid price (in terms of buying BTS with bitcny) than to use latest fill price to judge if bitcny is at discount. Likewise, use lowest ask to judge if bitcny is at premium. Perhaps using volume weighted moving average price is best.

Recently I feel the scripts made quite some false positives. The adjustment is too sensitive when bitcny is at discount, causing unnecessary margin calls. When someone bought up some BTS with little volume, the script thinks bitcny is at discount, thus lowers the price feed and triggers margin calls, which usually cause the price swing more than expected. Also the 110% MSSR is a big penalty to debt positions and big pressure on buy support.

Since we're able to "control" existing debt positions (via margin calls) to counter discount, but can only "incentivize" new debt positions and buy support to counter premium, that means we have more "power" on the former, so it's wise to save our power as much as possible. So it's better to adjust slower/softer/ with smaller steps when at discount, and adjust quicker/harder/with bigger steps when at premium.

For same reason, for my "adjusted_price" algorithm, it's wise to apply larger scale when at premium, but smaller scale when at discount.

Wish this helps.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on November 16, 2018, 03:38:35 pm
In order to give more visibility on what I currently use, and summarize the changes since my initial setup (https://bitsharestalk.org/index.php?topic=26881.msg321170#msg321170), here is an update of the my algorithm:

CNY:
   

USD:
   

BTC:
   
RUBLE/EUR/AUD/JPY/CHF/MXN/GBP/CAD/ARS/HKD/SEK/SGD/NZD/KRW:

GOLD/SILVER:

HERTZ:

AMAZONCOM:

The up-to-date version of the code used is available there (https://github.com/Zapata/bitshares-pricefeed).
It contains implementations for many BSIP 42 algorithms (https://github.com/Zapata/bitshares-pricefeed/blob/develop/bitshares_pricefeed/examples/bsip42.yaml#L194-L233), a large variety of price sources (https://github.com/Zapata/bitshares-pricefeed#sources), and is largely configurable (https://github.com/Zapata/bitshares-pricefeed/tree/develop/bitshares_pricefeed/examples).

You can use the pricefeed tracker (http://pricefeed-tracker.dex.trading/) to check the published prices over time and against other witnesses, and/or to compare feed price to CEX and DEX price and track premium/discount.

I plan to update my setup to:


Edit:
  - 20181116 17:55 UTC : Added forgotten AEX source for BitCNY (api is unstable, so sometime it's not used).
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: xeroc on November 16, 2018, 04:26:21 pm
+5% for transparency and your work on the price feed script!
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: matle85 on November 16, 2018, 05:10:06 pm
That's great - thank you!

Have we got agreed weighting for the different price feed sources? I.e. if we are concerned one exchange has faked volume it could be having an unjustified impact on the feed - I can see various discussions on the reliability of different sources but not sure if we maintain a weighting somewhere?
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: zapata42 on November 16, 2018, 06:11:30 pm
Have we got agreed weighting for the different price feed sources? I.e. if we are concerned one exchange has faked volume it could be having an unjustified impact on the feed.

Currently my weighting is volume based average.
I currently don't apply any adjustment by market, however the script has a `scaleVolumeBy` feature that let you adjust each source volume according to an arbitrary ratio.
I'm not aware of any agreed weighting, but abit has publicly expressed his opinions (https://bitsharestalk.org/index.php?topic=26881.msg321171#msg321171), and this topic has been discussed multiple times here and on telegram.
I agree that some volumes (and trade history) seems faked, so I'll take some time to forge an updated opinion and scale my volumes accordingly.
I'll keep you informed.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: matle85 on November 17, 2018, 11:34:18 am
I'm not aware of any agreed weighting, but abit has publicly expressed his opinions (https://bitsharestalk.org/index.php?topic=26881.msg321171#msg321171), and this topic has been discussed multiple times here and on telegram.
I agree that some volumes (and trade history) seems faked, so I'll take some time to forge an updated opinion and scale my volumes accordingly.
I'll keep you informed.

Thanks Zapata and just to say, reading through all your posts I'm really impressed by the professional approach you have to this work. You're a great asset to the community.

It would be really interesting to see how the prices at different exchanges vary over time. That will show if some are skewing the price and causing some of the odd effects that Bitcrab & others are concerned about around discounts and force settlements etc.

It may be we consider some more reliable / relevant than others for bitCNY specifically. If the weighting is transparent and justified then it would be fine in my view to give some greater weight and others a reduced weight.
Title: Re: [Witness Proposal] 1.6.129 - zapata42-witness
Post by: ioex on June 05, 2019, 02:57:15 pm
Can you please provide a block producer for the testnet as well?
To get my votes, I usually look how well they perform on the testnet (block production only, no price feeds).

Witness ioex-test has been running steadily on the test network for three weeks. Can I get your vote on mainnet?
Mainnet witness account ioex.