Professional Price Feeds for DEX

[sahkan]

(use /slots in Telegram to list which minute is assigned to what witness. There are 3 groups that bot works in: BitsharesDEX, Bitshares_Witnesses and BTS Witness Alerts)

The slots are assigned to 59 witnesses and we have only 21 active.

Here is the real price feed feed:

datasecuritynode   9 minutes ago   94.2   89.5   1750   1100
xman   11 minutes ago   93.4   88.7   1750   1100
xn-delegate   11 minutes ago   87.3   86.4   1750   1100
verbaltech2   12 minutes ago   88.6   84.2   1750   1100
sahkan-bitshares   15 minutes ago   88.5   84   1750   1100
delegate.freedom   15 minutes ago   87.8   87   1750   1100
bhuz   18 minutes ago   91.9   87.3   1750   1100
witness.yao   19 minutes ago   87.3   86.5   1750   1100
witness.still   20 minutes ago   87.3   86.5   1750   1100
jerryliu   20 minutes ago   87.2   86.4   1750   1100
delegate.ihashfury   21 minutes ago   89.1   84.7   1750   1100
abc123   24 minutes ago   89.1   84.7   1750   1100
harvey-xts   25 minutes ago   87   86.2   1750   1100
in.abit   26 minutes ago   94.1   89.4   1750   1100
delegate.baozi   26 minutes ago   86.9   86   1750   1100
xeldal   27 minutes ago   91.4   90.5   1750   1100
fox   28 minutes ago   88   83.6   1750   1100
rnglab   29 minutes ago   92.7   88   1750   1100
wackou   52 minutes ago   87.3   83   1750   1100
roadscape   1 hour ago   87.3   83   1750   1100
delegate-clayop   8 days ago   116.3   116.3   1750   1100

We have discussed a worker proposal to get professional feed sources, but they are quite expensive from what I gather. There is also the issue of centralization. That's one reason why no single feed script should be relied upon.

This should be our endstate where we can provide real time ticker from a professional feed to DEX. But I think if we can get witnesses to cooperate what I proposed might work. Single script is ok, as long as witnesses can add/use multiple sources, as an example we could use yahoo, google and 1 professional feed.

I would like to hear more about your offer to buy a subscription to a professional price feed source. Your offer is generous but I don't think we should continue to rely on altruism to support necessary blockchain operations to publish accurate price feeds.

I looked at feasibility of running a professional feed only through my witness and publishing it every 60 seconds but came to a conclusion that if we don't get a buy in from everyone it will not make sense. If everyone upgrades their script and publishes feeds so we have only 1-2 intervals in between (now we have sometimes 9-10 minutes) I will pay for the yearly subscription and provide the API key in the witness channel. Running a witness should not be considered a business, but rather a service to the community and responsibility for providing capable infrastructure for DEX. You should have your main business stream on DEX (for example building referrals) or somewhere else, but unfortunately I get an impression that some witness scale back on publishing feeds to save on fees, and their equipment is being maxed out on performance and not future ready so they can make money out of it. So if we can reach a consensus on this I will absorb the cost as long as they are willing to implement and fix their feed frequency. Of course if everyone thinks that current feeds are acceptable we can stay in status quo.

[Thom]

Glad to see your desire to improve the quality of the witness role.

These are not new topics, but it seems to be very difficult to get traction. I believe it's gonna require coming up with a BSIP and then getting community to recognize it.

There don't seem to be many witnesses willing to discuss such details, let alone take the initiative to write a BSIP to formalize the conventions. I also don't recall if there is a formal approval process BSIPs must go through by committee or whoever. I think they're more a matter of formalizing proposals in a document. BSIPs are good standards to have.

Some reform / evolution of the witness role will require coding, for example to implement standards that can be measured and automatically removing or disabling witnesses when those standards are violated, such as when X number of blocks are missed, or fails to publish feeds at a minimum frequency, or the feed accuracy slips outside an certain deviation from other witness feeds.

We have discussed a worker proposal to get professional feed sources, but they are quite expensive from what I gather. There is also the issue of centralization. That's one reason why no single feed script should be relied upon. To my knowledge there are 3 primary scripts most witnesses use, however the parameters each witness uses for them can be different based on that witness' influence. There are few standards on setting those parameters.

There is a Telegram bot created by witness spartaco to notify of missed blocks, set time slots for when during the hour a witness should publish their feed (use /slots in Telegram to list which minute is assigned to what witness. There are 3 groups that bot works in: BitsharesDEX, Bitshares_Witnesses and BTS Witness Alerts). Here are the commands the bot responds to:

/help
/price [ASSET]
/missed WITNESS
/monitor WITNESS
/listmonitor WITNESS
/stopmonitor WITNESS
/status
/feed [ASSET]
/slots
/setslot WITNESS MIN
/rmslot WITNESS
/locations
/setlocation WITNESS LOCATION
/rmlocation WITNESS

Please use the BTS Witness Alerts channel for urgent matters only. It's kindof like the "Hotline" to all witnesses. Witness alt recently posted in the forum saying several witnesses were broadcasting double blocks back to back. If true that would be a major issue and could cause forking of the blockchain. It turned out to be a false claim however the best place to communicate such things is via this real-time Telegram channel.

I would like to hear more about your offer to buy a subscription to a professional price feed source. Your offer is generous but I don't think we should continue to rely on altruism to support necessary blockchain operations to publish accurate price feeds.

Price feeds and block production require 2 different skill sets and I suggested these should be separate roles. The general attitude I see expressed however is that "producing a price feed isn't very complicated" and thus need not be split out. I have a different perspective which I've expressed in more detail elsewhere but it seems I'm alone in it. There does seem to be some agreement on the idea of automating some type of witness monitoring, but when it comes to saying what corrective action to take mostly I hear crickets.

It's also important to note these issues are of interest primarily to witnesses, and many of them are not interested in them enough to contribute to the discussion.

[sahkan]

Team,

First off, Happy Easter everyone!

My witness got voted in to the active witnesses list recently and I started to pay more attention to the price feeds.
Clearly we should put some governance in place to get some consistency. For example:
1. What is the list of required asset feeds for the witnesses? (Some do 6 some 22, some in between)
2. What is the minimum frequency for posting the feeds (1 hour?)
3. What are the required sources? (If any)

I hope we can get the witnesses involved (both active and standby) and work this out with community's input.

I also see a need for a professional feed, I think it would be a positive thing for DEX. Since the price feed script was not originally developed with Graphene, there are few scripts out there that do the job.
I would be willing to pay for an Enterprise level subscription Plan from a provider like https://currencylayer.com/ and provide the feed to all active witnesses if everyone was willing to update their scripts and come up with a price feed schedule that would provide consistent price updates on DEX.

As far as the implementation timeline, early May would be a good target date as I know some of the guys are busy at work.

Also, since we have an international community here and some witnesses are not fluent in English, please have this translated for them if they are your friends. Let me know your thoughts!