BitShares Forum
Main => Stakeholder Proposals => Topic started by: jsidhu on March 11, 2015, 06:35:26 pm
-
Seems most ppl use bitsuperlab's price feed scripts because it comes first in the how to guide so most ppl use it.
Xeroc's feeds seem to track better and are off by 1% on average... that is a big difference. Please fix the price feed or everyone should switch to xeroc's.
-
The issue with the feed scripts currently is that too few BTS pairs are traded on exchanges .. if there were more BTS/CNY, BTS/USD, BTS/BTC and BTS/EUR .. taking the 'average price' for gold silver etc. would smooth out more ..
also the volatility of BTC makes it difficult IMHO .. atm we should run the feed script more often to track the price better
-
The issue with the feed scripts currently is that too few BTS pairs are traded on exchanges .. if there were more BTS/CNY, BTS/USD, BTS/BTC and BTS/EUR .. taking the 'average price' for gold silver etc. would smooth out more ..
also the volatility of BTC makes it difficult IMHO .. atm we should run the feed script more often to track the price better
How often would you suggest now?
-
I am running the script every 9 minutes now ..
coin@minecraft:~$ /home/coin/bitshares-pytools/delegate-feed/main.py ALL
Loading data: yahoo, Yunbi, BTC38, BTer, Poloniex, bittrex -- done. Calculating bts feeds prices and checking publish rules.
+--------+-------------+-------------+-------------+-------------------+---------------+-----------------------+-----------------+--------------------+----------------+
| asset | my price | my mean | my median | blockchain median | % change (my) | % change (blockchain) | % std exchanges | % spread exchanges | last update |
+--------+-------------+-------------+-------------+-------------------+---------------+-----------------------+-----------------+--------------------+----------------+
| BTC | 0.00003208 | 0.00003424 | 0.00003190 | 0.00003138 | -0.00002 | 0.00007 | -0.55994 | 34.87154 | 0:05:28.358102 |
| SILVER | 0.00061402 | 0.00062073 | 0.00061585 | 0.00060565 | -0.00051 | 0.00084 | 0.29749 | 1.88576 | 0:05:28.358277 |
| GOLD | 0.00000817 | 0.00000834 | 0.00000820 | 0.00000807 | -0.00001 | 0.00001 | 0.29749 | 3.85298 | 0:05:28.358474 |
| TRY | 0.02461504 | 0.02488039 | 0.02468827 | 0.02428019 | -0.01995 | 0.03348 | 0.29749 | 1.85849 | 0:05:28.358681 |
| SGD | 0.01307933 | 0.01322029 | 0.01311824 | 0.01290005 | -0.01107 | 0.01793 | 0.29749 | 1.85801 | 0:05:28.358855 |
| HKD | 0.07323128 | 0.07402157 | 0.07344913 | 0.07224605 | -0.06119 | 0.09852 | 0.29749 | 1.86086 | 0:05:28.359011 |
| RUB | 0.58037768 | 0.58668129 | 0.58210423 | 0.57722316 | -0.41293 | 0.31545 | 0.29749 | 1.87476 | 0:05:28.359184 |
| SEK | 0.08134566 | 0.08222397 | 0.08158765 | 0.08032429 | -0.06028 | 0.10214 | 0.29749 | 1.86197 | 0:05:28.359351 |
| NZD | 0.01309006 | 0.01323130 | 0.01312901 | 0.01289887 | -0.01081 | 0.01912 | 0.29749 | 1.86034 | 0:05:28.359521 |
| CNY | 0.05902441 | 0.05966100 | 0.05920000 | 0.05822609 | -0.04784 | 0.07983 | 0.29749 | 1.85955 | 0:05:28.359710 |
| MXN | 0.14570331 | 0.14727040 | 0.14613676 | 0.14368973 | -0.11085 | 0.20136 | 0.29749 | 1.85360 | 0:05:28.363558 |
| CAD | 0.01202396 | 0.01215356 | 0.01205973 | 0.01186628 | -0.00960 | 0.01577 | 0.29749 | 1.85826 | 0:05:28.363727 |
| CHF | 0.00950887 | 0.00961143 | 0.00953716 | 0.00938189 | -0.00896 | 0.01270 | 0.29749 | 1.85961 | 0:05:28.364509 |
| AUD | 0.01243169 | 0.01256575 | 0.01246867 | 0.01226248 | -0.01029 | 0.01692 | 0.29749 | 1.85938 | 0:05:28.364710 |
| GBP | 0.00630906 | 0.00637712 | 0.00632783 | 0.00623020 | -0.00517 | 0.00789 | 0.29749 | 1.85992 | 0:05:28.364915 |
| JPY | 1.14388392 | 1.15645820 | 1.14728685 | 1.12845345 | -1.15104 | 1.54305 | 0.29749 | 1.90103 | 0:05:28.365074 |
| EUR | 0.00893483 | 0.00903126 | 0.00896141 | 0.00882361 | -0.00868 | 0.01112 | 0.29749 | 1.86095 | 0:05:28.365246 |
| USD | 0.00942220 | 0.00952382 | 0.00945023 | 0.00929769 | -0.00776 | 0.01245 | 0.29749 | 1.85955 | 0:05:28.365432 |
| KRW | 10.73171087 | 10.82715000 | 10.76363650 | 10.58601807 | -8.69817 | 14.56928 | 0.29749 | 1.48115 | 0:05:28.365597 |
+--------+-------------+-------------+-------------+-------------------+---------------+-----------------------+-----------------+--------------------+----------------+
I guess I am calculating some numbers wrongly that decide whether to push a new price or not .. (see KRW).
Gonna have to go through the math again .. Won't happen today I fear ... Will keep you posted!
-
I think the problem maybe because of USD market is out of control yesterday.
delegate should adjust the weight for every market, change scale_xxx at config.json
"market_weight": {
"common": "weight depenth on (depth * scale)",
"common1": "include depth between price*(1-depth_change) and price*(1+depth_change)",
"depth_change": 0.03,
"scale_bts_usd": 0.5,
"scale_bts_cny": 1,
"scale_btc38": 1,
"scale_yunbi": 1,
"scale_bter": 0
},
Seems most ppl use bitsuperlab's price feed scripts because it comes first in the how to guide so most ppl use it.
Xeroc's feeds seem to track better and are off by 1% on average... that is a big difference. Please fix the price feed or everyone should switch to xeroc's.
-
I think the problem maybe because of USD market is out of control yesterday.
delegate should adjust the weight for every market, change scale_xxx at config.json
"market_weight": {
"common": "weight depenth on (depth * scale)",
"common1": "include depth between price*(1-depth_change) and price*(1+depth_change)",
"depth_change": 0.03,
"scale_bts_usd": 0.5,
"scale_bts_cny": 1,
"scale_btc38": 1,
"scale_yunbi": 1,
"scale_bter": 0
},
Seems most ppl use bitsuperlab's price feed scripts because it comes first in the how to guide so most ppl use it.
Xeroc's feeds seem to track better and are off by 1% on average... that is a big difference. Please fix the price feed or everyone should switch to xeroc's.
But gold was updating.. silver wasn't.. gold was down by 1% while silver was the same.. it took some time for silver to finally start catching up..
-
the publish rule is come from bytemaster's advice, here is it
5 #When attempting to write a market maker the slow movement of the feed can be difficult.
76 #I would recommend the following:
77 #if REAL_PRICE < MEDIAN and YOUR_PRICE > MEDIAN publish price
78 #if you haven't published a price in the past 20 minutes
79 # if REAL_PRICE > MEDIAN and YOUR_PRICE < MEDIAN and abs( YOUR_PRICE - REAL_PRICE ) / REAL_PRICE > 0.005 publish price
80 #The goal is to force the price down rapidly and allow it to creep up slowly.
81 #By publishing prices more often it helps market makers maintain the peg and minimizes opportunity for shorts to sell USD below the peg that the market makers then have to absorb.
82 #If we can get updates flowing smoothly then we can gradually reduce the spread in the market maker bots.
83 #*note: all prices in USD per BTS
-
One more fairly obvious item you should add to the list is:
Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
-
One more fairly obvious item you should add to the list is:
Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
Really? And this totally screws with ppl dependant on accurate feeds
-
One more fairly obvious item you should add to the list is:
Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
Really? And this totally screws with ppl dependant on accurate feeds
the error introduced is averaged out over time ... I think it's a good idea .. if you chose a spread of .. say 0.1% or so ..
-
One more fairly obvious item you should add to the list is:
Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
Really? And this totally screws with ppl dependant on accurate feeds
the error introduced is averaged out over time ... I think it's a good idea .. if you chose a spread of .. say 0.1% or so ..
As a business that will accept bit assets this is a awful idea.
I think we would be better served adding a 100% pay delegate and paying for feeds. Their are entire businesses who only have one product and that's providing feeds. It will give us a more accurate, dependable feed for every
market that we currently have running.
-
Thinking about it I cannot come up with an attack vector that can exploit a predictable feed price .. unless you can manipulate exchanges with high volume ..
so maybe @gentso is right about this one ..
what's everyone else's opinion? Do we want a (predictable) outcome of a feed script?
//edit: a huge argument to have a predictable outcome would be reproducibility and accountability of delegate feeds .. so that's another big minus for adding randomness
-
@Gentso- do you believe there is any such thing as a perfectly accurate price? What's the spread at btc38? 1%? 0.5%? Which price is the price you think is "correct?" The bid or ask? The price of the last trade? I think you'll agree with me that none of those are quite right; you'd almost always be more interested in something between the bid and ask. So what do you pick? The average? A recent moving average of market prices? Those sound better, but they're still only estimates. Maybe I'm being long-winded, but I'm trying to make the point that there isn't ever 1 price and the best you can ever do is estimate it. So adding in 0.2% noise will still give an accurate price feed, especially if everybody is doing it and all the noise averages out.
@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?
-
@Gentso- do you believe there is any such thing as a perfectly accurate price? What's the spread at btc38? 1%? 0.5%? Which price is the price you think is "correct?" The bid or ask? The price of the last trade? I think you'll agree with me that none of those are quite right; you'd almost always be more interested in something between the bid and ask. So what do you pick? The average? A recent moving average of market prices? Those sound better, but they're still only estimates. Maybe I'm being long-winded, but I'm trying to make the point that there isn't ever 1 price and the best you can ever do is estimate it. So adding in 0.2% noise will still give an accurate price feed, especially if everybody is doing it and all the noise averages out.
@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?
maybe we can find a compromise and have a randomness in-line with the lowest spread of your exchanges .. if btc38 says 1% you may add randomness with a spread of less than 1% with confidence >90% or so ..
-
@Gentso- do you believe there is any such thing as a perfectly accurate price? What's the spread at btc38? 1%? 0.5%? Which price is the price you think is "correct?" The bid or ask? The price of the last trade? I think you'll agree with me that none of those are quite right; you'd almost always be more interested in something between the bid and ask. So what do you pick? The average? A recent moving average of market prices? Those sound better, but they're still only estimates. Maybe I'm being long-winded, but I'm trying to make the point that there isn't ever 1 price and the best you can ever do is estimate it. So adding in 0.2% noise will still give an accurate price feed, especially if everybody is doing it and all the noise averages out.
@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?
maybe we can find a compromise and have a randomness in-line with the lowest spread of your exchanges .. if btc38 says 1% you may add randomness with a spread of less than 1% with confidence >90% or so ..
Yeah, that's more or less what I had in mind.
Another way to do it halfway is to randomize update times. Update every 20 minutes plus or minus 5.
Also there should be no hard triggers that push a new price with 100% certainty, because those could be pretty easily exploitable. Again, based on the principle that a deterministic price feed is exploitable.
-
@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?
It isn't obvious to me. I don't see how a deterministic price feed opens up any new attack vectors.
Let's say you could predict what the price feed would be within the next hour with perfect precision and greater than 99% confidence. What could you do with that information over someone who knows nothing about the price feed other than what it currently is? It doesn't allow you to predict how the bids and asks on the DEX will change. It allows you to know how short sells with a price limit below the feed price will change (assuming they do not get cancelled). So if you can predict that the price feed will shrink (in BTS/BitUSD) in the next 30 minutes, then maybe you decide to sell BitUSD right below the price feed and then wait for 30 minutes and then buy back more BitUSD from the short wall at the lower price. I just view this as similar to profiting from arbitrage and find it to be totally legitimate market behavior. Of course it isn't without risk since the short sell at the price feed might be cancelled after you sell the BitUSD but before the 30 minutes has passed. It also is even less of a serious issue if the delegates incorporate the most recent outside exchange info into their price feed just prior to publishing. In that case, you cannot predict the price feed very far into the future (less than 8 minutes if delegates update their price feeds each time they produce a block) since you can't predict how bids and asks will change on outside exchanges.
-
I guess I should just go design a feed-prediction attack and make money myself, since people seem so hard to convince.
At the very least, feed scripts should randomize update times. I'll leave it at that until I can give a better argument.
-
I guess I should just go design a feed-prediction attack and make money myself, since people seem so hard to convince.
Please do, and then after you make some money to compensate you for the time spent designing the attack, please post the attack details on the forums. :)
-
I guess I should just go design a feed-prediction attack and make money myself, since people seem so hard to convince.
Please do, and then after you make some money to compensate you for the time spent designing the attack, please post the attack details on the forums. :)
+5% +5%
-
Perhaps its just over my head but I don't understand the attack vector for feed prediction myself. I just don't see how you can predict the price feed without a crystal ball.
As to answering your question, no I don't believe their is ever a 100% accurate price. I think instead of adding .2% of random filler we should focus on where we get our feeds from.
Through some of my own research I have seen that quality price feeds that are updated from many sources/markets and guarantee 99% up time and are updated more often.
Right now it looks like we are dependent upon many free sources.
Cryptosmith accepts payment in bitgold/bitsilver so if the silver feed isn't keeping up with the live market I am sure you can see what a issue this will be, not just for myself but any merchant that wants to accept bitassets.
Many sources/greater reliability is what I am advocating here
-
Many sources/greater reliability is what I am advocating here
+1
-
Right now it looks like we are dependent upon many free sources.
Cryptosmith accepts payment in bitgold/bitsilver so if the silver feed isn't keeping up with the live market I am sure you can see what a issue this will be, not just for myself but any merchant that wants to accept bitassets.
Many sources/greater reliability is what I am advocating here
Will there be a problem to Cryptosmith when the price feeds are depending on free sources?
-
Right now it looks like we are dependent upon many free sources.
Cryptosmith accepts payment in bitgold/bitsilver so if the silver feed isn't keeping up with the live market I am sure you can see what a issue this will be, not just for myself but any merchant that wants to accept bitassets.
Many sources/greater reliability is what I am advocating here
Will there be a problem to Cryptosmith when the price feeds are depending on free sources?
Only if those sources go off line.....even if for a short time, the market would keep moving and the feed would not be updated. The longer the period the feed would be off line well the greater the problem.
I believe any decent paid service also will take a average of many markets for any commodity thus giving you a market average.
-
I guess I should just go design a feed-prediction attack and make money myself, since people seem so hard to convince.
At the very least, feed scripts should randomize update times. I'll leave it at that until I can give a better argument.
Feed-prediction? if you can predict the feed you would be a trillionaire by now... do you mean feed arbitrage or some other form of manipulation?
-
I guess I should just go design a feed-prediction attack and make money myself, since people seem so hard to convince.
At the very least, feed scripts should randomize update times. I'll leave it at that until I can give a better argument.
Feed-prediction? if you can predict the feed you would be a trillionaire by now... do you mean feed arbitrage or some other form of manipulation?
I think feed prediction is entirely possible because the feed is trailing the actual market. By closely monitoring the delegates' individual feeds it should be possible to predict when a given delegate updates his feed, from what sources he derives the feed price and what his next published price will be. Such prediction will likely be possible a couple of blocks before the publishing, and the accuracy of the predicted price will likely be high compared to the difference between the new price and the previous price published by that delegate.
But since a delegate does not control the price directly I would assume that the predictable price *movement* is too small to be exploitable.
-
I guess I should just go design a feed-prediction attack and make money myself, since people seem so hard to convince.
At the very least, feed scripts should randomize update times. I'll leave it at that until I can give a better argument.
Feed-prediction? if you can predict the feed you would be a trillionaire by now... do you mean feed arbitrage or some other form of manipulation?
I think feed prediction is entirely possible because the feed is trailing the actual market. By closely monitoring the delegates' individual feeds it should be possible to predict when a given delegate updates his feed, from what sources he derives the feed price and what his next published price will be. Such prediction will likely be possible a couple of blocks before the publishing, and the accuracy of the predicted price will likely be high compared to the difference between the new price and the previous price published by that delegate.
But since a delegate does not control the price directly I would assume that the predictable price *movement* is too small to be exploitable.
Aah i see but its averaged out amongst all delegates providing feeds right?
-
One more fairly obvious item you should add to the list is:
Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
It is not necessary in my mind since the noise we need comes already from delegates that publish less often price feeds :)
Sent from my ALCATEL ONE TOUCH 997D
-
Aah i see but its averaged out amongst all delegates providing feeds right?
Not quite. If it was averaged a delegate could manipulate the feed by publishing a price that is extremely off. The market price feed is the median (i. e. middle) value of all published feed prices. So if a delegate changes his published price from below the market feed to above the market feed (or vice versa) the market feed will move to the next published price (which should be very close to its neighbour).
So IMO published feeds should be as accurate as possible, but the time of publishing should be unpredictable.
-
Just wanted to post up to bring attention to this
current silver feed price median is showing as 15.7845021684$
current market price from kitco(ny spot) bid SILVER bid-16.86 ask-16.96
anyone else seeing a problem here?
-
blockchain_get_feeds_for_asset 6
returns quite the range -
"price": 0.00051223015243678999 for a bunch of folks
"price": 0.0011302211302211 for others
Not sure how many places to move the decimal over, but its still a hell of a gap
-
I am using bitsharesblocks median feeds on cryptosmith.info ...
BTS median price: 124.20 BTS/bitUSD
bitSilver median price: 2,003.83 BTS/bitSILVER
2,003.83 / 124.20 = $16.13
Price of real silver: $16.88
Difference = $0.75
its slowly catching up but its been off for 10-15 mins.
-
Just wanted to post up to bring attention to this
current silver feed price median is showing as 15.7845021684$
current market price from kitco(ny spot) bid SILVER bid-16.86 ask-16.96
anyone else seeing a problem here?
welcome in our world :D
on metaexchange.info (http://metaexchange.info) we have the same problems on the bitBTC pair. We should discuss if we can improve the feed prices or if we can find a better solution for it. If we want to create businesses around the BitShares ecosystem we need better feedprices.
-
Just wanted to post up to bring attention to this
current silver feed price median is showing as 15.7845021684$
current market price from kitco(ny spot) bid SILVER bid-16.86 ask-16.96
anyone else seeing a problem here?
welcome in our world :D
on metaexchange.info (http://metaexchange.info) we have the same problems on the bitBTC pair. We should discuss if we can improve the feed prices or if we can find a better solution for it. If we want to create businesses around the BitShares ecosystem we need better feedprices.
The first step in solving a problem is that we all acknowledge that their is one.
Hopefully others will join the discussion and we can begin to work toward a common goal.
-
Just wanted to post up to bring attention to this
current silver feed price median is showing as 15.7845021684$
current market price from kitco(ny spot) bid SILVER bid-16.86 ask-16.96
anyone else seeing a problem here?
welcome in our world :D
on metaexchange.info (http://metaexchange.info) we have the same problems on the bitBTC pair. We should discuss if we can improve the feed prices or if we can find a better solution for it. If we want to create businesses around the BitShares ecosystem we need better feedprices.
Your problem is not the inaccurate feed price Shentist (and monsterer) [it is a problem but not the most significant one]...your main problem is due to the inability (impossibility) to hedge (or actually take) the bitBTC positions when you are forced to take such position in the current BTS system.
-
Its still better than using bticoin even though vendors like overstock accept bitcoin they will still price based on BTSUSD price I believe... however its not as good as it can be and a 1% feed difference can make a big difference for people relying on commisions, which is what most businesses will tend to do to start off with this system.
-
I reported this issue here some days ago. Feed tool is no longer working after latest git pull
https://github.com/bitsuperlab/operation_tools/issues/6
Warning: unknown error, can't fetch price
Expecting value: line 1 column 1 (char 0)
-
I have rewrite the feed price script, can you try the new version?
https://bitsharestalk.org/index.php/topic,16170.new.html#new