Author [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] Topic: BitSuperLab please fix your feeds  (Read 1246 times)

Offline jsidhu

  • Hero Member
  • *****
  • Posts: 1337
    • View Profile
BitSuperLab please fix your feeds
« on: March 11, 2015, 06:35:26 PM »

Seems most ppl use bitsuperlab's price feed scripts because it comes first in the how to guide so most ppl use it.

Xeroc's feeds seem to track better and are off by 1% on average... that is a big difference. Please fix the price feed or everyone should switch to xeroc's.
Hired by blockchain | Developer
delegate: dev.sidhujag

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 11959
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BTS: xeroc
  • GitHub: xeroc
Re: BitSuperLab please fix your feeds
« Reply #1 on: March 11, 2015, 07:04:24 PM »
The issue with the feed scripts currently is that too few BTS pairs are traded on exchanges .. if there were more BTS/CNY, BTS/USD, BTS/BTC and BTS/EUR .. taking the 'average price' for gold silver etc. would smooth out more ..

also the volatility of BTC makes it difficult IMHO .. atm we should run the feed script more often to track the price better
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline BunkerChain Labs

Re: BitSuperLab please fix your feeds
« Reply #2 on: March 11, 2015, 07:09:00 PM »
The issue with the feed scripts currently is that too few BTS pairs are traded on exchanges .. if there were more BTS/CNY, BTS/USD, BTS/BTC and BTS/EUR .. taking the 'average price' for gold silver etc. would smooth out more ..

also the volatility of BTC makes it difficult IMHO .. atm we should run the feed script more often to track the price better

How often would you suggest now?
+-+-+-+-+-+-+-+-+-+-+
www.Peerplays.com | World's First Decentralized Tournament Platform Built Entirely on the Blockchain!
+-+-+-+-+-+-+-+-+-+-+

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 11959
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BTS: xeroc
  • GitHub: xeroc
Re: BitSuperLab please fix your feeds
« Reply #3 on: March 11, 2015, 07:11:27 PM »
I am running the script every 9 minutes now ..
Code: [Select]
[email protected]:~$ /home/coin/bitshares-pytools/delegate-feed/main.py ALL
Loading data: yahoo, Yunbi, BTC38, BTer, Poloniex, bittrex -- done. Calculating bts feeds prices and checking publish rules.
+--------+-------------+-------------+-------------+-------------------+---------------+-----------------------+-----------------+--------------------+----------------+
|  asset |    my price |     my mean |   my median | blockchain median | % change (my) | % change (blockchain) | % std exchanges | % spread exchanges |    last update |
+--------+-------------+-------------+-------------+-------------------+---------------+-----------------------+-----------------+--------------------+----------------+
|    BTC |  0.00003208 |  0.00003424 |  0.00003190 |        0.00003138 |      -0.00002 |               0.00007 |        -0.55994 |           34.87154 | 0:05:28.358102 |
| SILVER |  0.00061402 |  0.00062073 |  0.00061585 |        0.00060565 |      -0.00051 |               0.00084 |         0.29749 |            1.88576 | 0:05:28.358277 |
|   GOLD |  0.00000817 |  0.00000834 |  0.00000820 |        0.00000807 |      -0.00001 |               0.00001 |         0.29749 |            3.85298 | 0:05:28.358474 |
|    TRY |  0.02461504 |  0.02488039 |  0.02468827 |        0.02428019 |      -0.01995 |               0.03348 |         0.29749 |            1.85849 | 0:05:28.358681 |
|    SGD |  0.01307933 |  0.01322029 |  0.01311824 |        0.01290005 |      -0.01107 |               0.01793 |         0.29749 |            1.85801 | 0:05:28.358855 |
|    HKD |  0.07323128 |  0.07402157 |  0.07344913 |        0.07224605 |      -0.06119 |               0.09852 |         0.29749 |            1.86086 | 0:05:28.359011 |
|    RUB |  0.58037768 |  0.58668129 |  0.58210423 |        0.57722316 |      -0.41293 |               0.31545 |         0.29749 |            1.87476 | 0:05:28.359184 |
|    SEK |  0.08134566 |  0.08222397 |  0.08158765 |        0.08032429 |      -0.06028 |               0.10214 |         0.29749 |            1.86197 | 0:05:28.359351 |
|    NZD |  0.01309006 |  0.01323130 |  0.01312901 |        0.01289887 |      -0.01081 |               0.01912 |         0.29749 |            1.86034 | 0:05:28.359521 |
|    CNY |  0.05902441 |  0.05966100 |  0.05920000 |        0.05822609 |      -0.04784 |               0.07983 |         0.29749 |            1.85955 | 0:05:28.359710 |
|    MXN |  0.14570331 |  0.14727040 |  0.14613676 |        0.14368973 |      -0.11085 |               0.20136 |         0.29749 |            1.85360 | 0:05:28.363558 |
|    CAD |  0.01202396 |  0.01215356 |  0.01205973 |        0.01186628 |      -0.00960 |               0.01577 |         0.29749 |            1.85826 | 0:05:28.363727 |
|    CHF |  0.00950887 |  0.00961143 |  0.00953716 |        0.00938189 |      -0.00896 |               0.01270 |         0.29749 |            1.85961 | 0:05:28.364509 |
|    AUD |  0.01243169 |  0.01256575 |  0.01246867 |        0.01226248 |      -0.01029 |               0.01692 |         0.29749 |            1.85938 | 0:05:28.364710 |
|    GBP |  0.00630906 |  0.00637712 |  0.00632783 |        0.00623020 |      -0.00517 |               0.00789 |         0.29749 |            1.85992 | 0:05:28.364915 |
|    JPY |  1.14388392 |  1.15645820 |  1.14728685 |        1.12845345 |      -1.15104 |               1.54305 |         0.29749 |            1.90103 | 0:05:28.365074 |
|    EUR |  0.00893483 |  0.00903126 |  0.00896141 |        0.00882361 |      -0.00868 |               0.01112 |         0.29749 |            1.86095 | 0:05:28.365246 |
|    USD |  0.00942220 |  0.00952382 |  0.00945023 |        0.00929769 |      -0.00776 |               0.01245 |         0.29749 |            1.85955 | 0:05:28.365432 |
|    KRW | 10.73171087 | 10.82715000 | 10.76363650 |       10.58601807 |      -8.69817 |              14.56928 |         0.29749 |            1.48115 | 0:05:28.365597 |
+--------+-------------+-------------+-------------+-------------------+---------------+-----------------------+-----------------+--------------------+----------------+
I guess I am calculating some numbers wrongly that decide whether to push a new price or not .. (see KRW).

Gonna have to go through the math again .. Won't happen today I fear ... Will keep you posted!
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline alt

Re: BitSuperLab please fix your feeds
« Reply #4 on: March 11, 2015, 11:48:58 PM »
I think the problem maybe because of USD market is out of control yesterday.
delegate should adjust the weight for every market, change scale_xxx at   config.json
Code: [Select]
  "market_weight": {
    "common": "weight depenth on (depth * scale)",
    "common1": "include depth between price*(1-depth_change) and price*(1+depth_change)",
    "depth_change": 0.03,
    "scale_bts_usd": 0.5,
    "scale_bts_cny": 1,
    "scale_btc38": 1,
    "scale_yunbi": 1,
    "scale_bter": 0
  },

Seems most ppl use bitsuperlab's price feed scripts because it comes first in the how to guide so most ppl use it.

Xeroc's feeds seem to track better and are off by 1% on average... that is a big difference. Please fix the price feed or everyone should switch to xeroc's.

Offline jsidhu

  • Hero Member
  • *****
  • Posts: 1337
    • View Profile
Re: BitSuperLab please fix your feeds
« Reply #5 on: March 12, 2015, 12:01:49 AM »
I think the problem maybe because of USD market is out of control yesterday.
delegate should adjust the weight for every market, change scale_xxx at   config.json
Code: [Select]
  "market_weight": {
    "common": "weight depenth on (depth * scale)",
    "common1": "include depth between price*(1-depth_change) and price*(1+depth_change)",
    "depth_change": 0.03,
    "scale_bts_usd": 0.5,
    "scale_bts_cny": 1,
    "scale_btc38": 1,
    "scale_yunbi": 1,
    "scale_bter": 0
  },

Seems most ppl use bitsuperlab's price feed scripts because it comes first in the how to guide so most ppl use it.

Xeroc's feeds seem to track better and are off by 1% on average... that is a big difference. Please fix the price feed or everyone should switch to xeroc's.

But gold was updating.. silver wasn't.. gold was down by 1% while silver was the same.. it took some time for silver to finally start catching up..
Hired by blockchain | Developer
delegate: dev.sidhujag

Offline alt

Re: BitSuperLab please fix your feeds
« Reply #6 on: March 12, 2015, 12:09:22 AM »
the publish rule is come from bytemaster's advice, here is it
Quote
   5   #When attempting to write a market maker the slow movement of the feed can be difficult.
 76   #I would recommend the following:
 77   #if  REAL_PRICE < MEDIAN and YOUR_PRICE > MEDIAN publish price
 78   #if  you haven't published a price in the past 20 minutes
 79   #   if  REAL_PRICE > MEDIAN  and  YOUR_PRICE < MEDIAN and abs( YOUR_PRICE - REAL_PRICE ) / REAL_PRICE  > 0.005 publish price
 80   #The goal is to force the price down rapidly and allow it to creep up slowly.
 81   #By publishing prices more often it helps market makers maintain the peg and minimizes opportunity for shorts to sell USD below the peg that the market makers then have to absorb.
 82   #If we can get updates flowing smoothly then we can gradually reduce the spread in the market maker bots.
 83   #*note: all prices in USD per BTS

Offline biophil

  • Hero Member
  • *****
  • Posts: 774
  • Incentives run the world
    • View Profile
  • BTS: zebulon
Re: BitSuperLab please fix your feeds
« Reply #7 on: March 16, 2015, 03:15:00 AM »
One more fairly obvious item you should add to the list is:

Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.

Offline jsidhu

  • Hero Member
  • *****
  • Posts: 1337
    • View Profile
Re: BitSuperLab please fix your feeds
« Reply #8 on: March 16, 2015, 06:41:16 AM »
One more fairly obvious item you should add to the list is:

Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
Really? And this totally screws with ppl dependant on accurate feeds
Hired by blockchain | Developer
delegate: dev.sidhujag

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 11959
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BTS: xeroc
  • GitHub: xeroc
Re: BitSuperLab please fix your feeds
« Reply #9 on: March 16, 2015, 07:02:28 AM »
One more fairly obvious item you should add to the list is:

Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
Really? And this totally screws with ppl dependant on accurate feeds
the error introduced is averaged out over time ... I think it's a good idea .. if you chose a spread of .. say 0.1% or so ..
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline Gentso1

Re: BitSuperLab please fix your feeds
« Reply #10 on: March 16, 2015, 01:20:08 PM »
One more fairly obvious item you should add to the list is:

Always multiply your published price by a random number that is close to 1; maybe within a quarter percent of one. It's an easy step that makes the price feed that much less predictable and improves security against precision market manipulation.
Really? And this totally screws with ppl dependant on accurate feeds
the error introduced is averaged out over time ... I think it's a good idea .. if you chose a spread of .. say 0.1% or so ..

As a business that will accept bit assets this is a awful idea.

I think we would be better served adding a 100% pay delegate and paying for feeds. Their are entire businesses who only have one product and that's providing feeds. It will give us a more accurate, dependable  feed for every

market that we currently have running.

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 11959
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BTS: xeroc
  • GitHub: xeroc
Re: BitSuperLab please fix your feeds
« Reply #11 on: March 16, 2015, 01:29:31 PM »
Thinking about it I cannot come up with an attack vector that can exploit a predictable feed price .. unless you can manipulate exchanges with high volume ..
so maybe @gentso is right about this one ..

what's everyone else's opinion? Do we want a (predictable) outcome of a feed script?

//edit: a huge argument to have a predictable outcome would be reproducibility and accountability of delegate feeds .. so that's another big minus for adding randomness
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline biophil

  • Hero Member
  • *****
  • Posts: 774
  • Incentives run the world
    • View Profile
  • BTS: zebulon
Re: BitSuperLab please fix your feeds
« Reply #12 on: March 16, 2015, 02:21:32 PM »
@Gentso- do you believe there is any such thing as a perfectly accurate price? What's the spread at btc38? 1%? 0.5%? Which price is the price you think is "correct?" The bid or ask? The price of the last trade? I think you'll agree with me that none of those are quite right; you'd almost always be more interested in something between the bid and ask. So what do you pick? The average? A recent moving average of market prices? Those sound better, but they're still only estimates. Maybe I'm being long-winded, but I'm trying to make the point that there isn't ever 1 price and the best you can ever do is estimate it. So adding in 0.2% noise will still give an accurate price feed, especially if everybody is doing it and all the noise averages out.

@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 11959
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BTS: xeroc
  • GitHub: xeroc
Re: BitSuperLab please fix your feeds
« Reply #13 on: March 16, 2015, 02:58:11 PM »
@Gentso- do you believe there is any such thing as a perfectly accurate price? What's the spread at btc38? 1%? 0.5%? Which price is the price you think is "correct?" The bid or ask? The price of the last trade? I think you'll agree with me that none of those are quite right; you'd almost always be more interested in something between the bid and ask. So what do you pick? The average? A recent moving average of market prices? Those sound better, but they're still only estimates. Maybe I'm being long-winded, but I'm trying to make the point that there isn't ever 1 price and the best you can ever do is estimate it. So adding in 0.2% noise will still give an accurate price feed, especially if everybody is doing it and all the noise averages out.

@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?

maybe we can find a compromise and have a randomness in-line with the lowest spread of your exchanges .. if btc38 says 1% you may add randomness with a spread of less than 1% with confidence >90% or so ..
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline biophil

  • Hero Member
  • *****
  • Posts: 774
  • Incentives run the world
    • View Profile
  • BTS: zebulon
Re: BitSuperLab please fix your feeds
« Reply #14 on: March 16, 2015, 03:03:27 PM »
@Gentso- do you believe there is any such thing as a perfectly accurate price? What's the spread at btc38? 1%? 0.5%? Which price is the price you think is "correct?" The bid or ask? The price of the last trade? I think you'll agree with me that none of those are quite right; you'd almost always be more interested in something between the bid and ask. So what do you pick? The average? A recent moving average of market prices? Those sound better, but they're still only estimates. Maybe I'm being long-winded, but I'm trying to make the point that there isn't ever 1 price and the best you can ever do is estimate it. So adding in 0.2% noise will still give an accurate price feed, especially if everybody is doing it and all the noise averages out.

@Xeroc, I haven't actually designed a manipulation algorithm to exploit feed price prediction, but isn't it obvious that a noisy feed would be better than a predictable one? I'm working from this principle: if I can execute trades on btc38 and predict exactly what that will do to the price feed, then I can make money moving the price feed around. Probably a fraction of a percent per trade, but the fact is I'll be taking money from the system without providing any benefit. Doesn't that sound like something we should try to prevent?

maybe we can find a compromise and have a randomness in-line with the lowest spread of your exchanges .. if btc38 says 1% you may add randomness with a spread of less than 1% with confidence >90% or so ..
Yeah, that's more or less what I had in mind.

Another way to do it halfway is to randomize update times. Update every 20 minutes plus or minus 5.

Also there should be no hard triggers that push a new price with 100% certainty, because those could be pretty easily exploitable. Again, based on the principle that a deterministic price feed is exploitable.

 

Google+