About rising witness quality standards (with emphasis on accurate pricefeeds), I like some of the ideas that have been discussed in the last months, but most of them would need heavy development, after a long way to reach consensus. 

At this stage I think our best trade off between priorities and resources would be to fund development for an open source block explorer, with an initial focus on witness monitoring tools (like historical pricefeed charts).

I like this approach from @lafona: https://lafona.shinyapps.io/Feed_History/
Just adding a highlighted line with the median price published by the blockchain would make a good enough reference to start monitoring feeds accuracy. Maybe another line with asset prices from a professional paid source, as an off chain reference.

The lack of both an open source block explorer codebase, and a simple way to monitor witnesses, are top prioity issues that will probably result more expensive if we wait for someone to solve them for free some day, than if we just fund its development as a priority.
@svk @svk @svk   

I wouldn't care about the /slots feature from spartako-bot. It was added by spartako almost instantly when Dan asked us to provide price feeds and xeroc's script was released.  As we started being just a few witnesses publishing feeds until different timezones and avaiability allowed everyone to update nodes, making the time span as distributed as possible was one of the first actions.

If that information were up to date and leaked, it would be easy for every feedsource API provider to know most witnesses IP addresses.

The blockchain takes prices from the majority of witnesses that "agrees" on a median, withing the feed_lifetime parameter  (if someone can add a more accurate description would be apreciated).
It has prooved to be very resilent against wrong feeds, and less dependant on publishing frequency.

this are the time parameters from xeroc's pricefeeds script:
################################################################################
# Publishing Criteria
################################################################################
#
# Publish price if any of these are valid:
#
#  1.) price feed is older than maxAgeFeedInSeconds
#  2.) price has moved from my last published price by more than change_min
#
# Do NOT publish if
#
#  * Price has moved more than 'change_max'
#     AND
#  * Manual confirmation is 'false'
#
# A feed becomes invalid if it is older than 24h. Hence, it should be force
# published once a day (e.g. every 12h) Note that the script should be run more
# frequently and publishes only those prices that fit the publishing criteria
maxAgeFeedInSeconds          = 12 * 60 * 60  # max age of 12h
change_min                   = 0.5       # Percentage of price change to force an update
change_max                   = 5       # Percentage of price change to cause a warning

a low change_max helps to prevent market manipulation, specially black swan events. It will stop publishing until manual confirmation, if the value is exceeded in the time span in which you run the script.
with high volatility is where you need to fin tune this parameters. A low change_max without frequently running your script will ask for manual confirmation as false alarm too often to handle.

On the other hand, most if not all Chinese feed sources has a DDOS protection that blacklists your ip with *many* less API calls than the specified. Sometimes you can make reverse DNS loockups to find direct connection to an API, sometimes they provide a private IP if you mail them, but in both cases the addresses change often and you'll have to find out what to put in your hosts file very often if blacklisted by Baidu.

Feeds can be published from a non producing witness with the active key.
If  someone has troubles reaching btc38 at this moment,  producing feeds from a node in China may be a solution.

Hey good to know the RUB black swan event didn't stop you from starting your business.
I will provide RUBLE price feeds as a witness.

I'd just ask not to vote me or other witnesses just because of that, but also taking into consideration  other behavior parameters that you consider important for the network.

OT: are you related to blockchained.com?

Forgot to comment back okcoin and btcc from CNY feed sources after running some tests.

abit , data, xman, witness.still, abc123: try removing the chinese exchanges with  BTS:BTC market from your CNY feed sources.

I agree to let the competition machanisim solve the problem: obviously shareholders appreciate and would like to vote the witnesses that provide more and better service.

and AFAIK, to provide good price feed is also possible via free/open source script, for example, alt has published his script at https://pypi.python.org/pypi/btsprice/0.2.21, it's a good reference.

on the other side, I wonder whether  we need to consider one question: suppose publishing price feed become what witness must do,  shall we increase the block reward? and to how much?

now being a witness can get some reward, but the reward is not very attractive, we need to balance the responsibility and the reward, maybe we need to reestimate what a reward is more reasonable? maybe a more reasonable reward can lead to more active ecosystem?

kindly give your input on this topic.

@Bhuz sorry if I did things carelessly, I' recheck and take actions accordingly.

 @alt  runs a bussines which depends on precise and resilient feeds. His code skills along with a real incentive, brings us a good chance to improve the way we fetch and derive external prices.
Besides that I think it would be more productive to fork and improve @xeroc script, as a way to sum the scarce resources we have on this topic. 

Some conditions have to be given for diversity to be enriching, and for fair competition to exist at all. In the meanwhile, I think that cooperation is the way to go. Competition will favor aptitude and commitment only when we become able to incentivize it.


 Some numbers:

  - Witness reward averaged something around $200/month this year.

 - Let's say a regular VPS with 8Gb RAM costs 80 USD per month, and 50 USD for 4Gb.
 - That is a 150 USD reward for witnesses running just one node with only 4Gb  (at its very memory limit).
 - There's a 120 USD incentive to have 8Gb on just one node, with no failover at all.
 - Even running two 4Gb nodes with a proper failover strategy (for ~100 USD), both  nodes could easily crash if RAM becomes not enough.

Automatic fork resolution is working great, but it has not been prooven beyond our current TPS.
With checkpoints and delayed nodes, we have revived the network in less than an hour with no major consequences when forks turned into lost of consensus.
But I wouldn't like to see the network crashing just when the real action starts.

 - It seems to me that having two 8Gb nodes per witness, with failover able to identify and move away from minority forks (when possible with just two nodes), are the minimum technical requirements to handle the upcoming network growth. That means a reward of 40 USD per month at most.
Add seed nodes and the time required to constantly check and fix feeds, having everything up to date and the need to remain informed in a daily basis.

That means good withesses working at a loss, while icentivizing poor performance for a proffit.



This also brings another concern to me: as many good neutral witnesses lack of incentive or resources/time despite they good will, it drives business to run it's own witnesses given their need to have reliable feeds.
I see a flaw here:  business tend to have more resources, voting stake, reward incentives and thus good performance , this raises their motivation and chances to have many witnesses voted in.
As business models may disagree about market parameters that are defined by feed producers, this could lead to unfair competition or other undesirable behavors, specially now that new/unknown players are steadily arriving.

This is one of the reasons why witnesses, committee and proxies were planed as independent powers.



I'm not pushing to rise block rewards here, nor discouraging our most valuable witnesses who also runs business over bitshares.
 I still enjoy being reliable and helpful as a witness, rewards aside, and  I'm glad to see this debate going on in a constructive way, finally.
Just want to sumarize some facts I see.

At this point we really need a good balance between competition and collaborative work to make network and market more stable and to reduce the entry barriers.

As expressed before, I see no reason to promote bitshares scalability before prooving the yet outstanding throughput we already have, over a sound patform.

There are features being developed that will require a hardfork.
We have @vikram back willing to work for bitshares.
Steem's graphene implementation is free software and actively developed.
If the p2p protocol or something else needs optimizations, this seems to me like the right moment to start working on it.

This would need funding of course. As I get it, dilution not always means devaluation, and even if price drops it can become a good investment if done right.

I can run segregated witness nodes and spamming peers.

And I'd be glad if we can organize this towards an official stable release.

 

Quote from: rnglab on April 25, 2016, 08:48:37 am

hey xeroc, I started writing before you posted . Could you review my answers when you can? We can complement good answers to make all this very clear.

looks great!

As for brain key entropy: The entropy of brain keys is exactly 256 bit .. so the same as private keys.

Side remark: Considering Security, BitShares has a slight advantage over most satoshi/bitcoin based blockchains because it does NOT use addresses but only public keys.
Since addresses are only 150bits, there are multiple (2^(256-150)) public keys that theoretically derive to the same address and if you never spend from your bitcoin address, they all could do so.
Because BitShares does NOT use addresses but the full length public key, you could consider it slightly more secure. Cheers

Thanks for the clarification! Cheers