BitShares Forum

Main => General Discussion => Topic started by: wasthatawolf on December 21, 2014, 10:36:07 pm

Title: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on December 21, 2014, 10:36:07 pm
I setup a site to analyze the USD and BTC price pegs by comparing the asset feed, bid and ask prices to the market price of BTS on Coinmarketcap.  The price variance mean and standard deviation are calculated to produce Probability Density Curves.  Data is recorded every fifteen minutes.

http://delegate.ninja/assets/?t=bitbtc (http://delegate.ninja/assets/?t=bitbtc)

http://delegate.ninja/assets/?t=bitusd (http://delegate.ninja/assets/?t=bitusd)
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on December 21, 2014, 11:32:25 pm
Wow, this is great stuff.

I find it interesting that the mean of the feed price curve for both BitAssets is shifted to the right. I'm speculating this has to do with the asymmetry (https://bitsharestalk.org/index.php?topic=9348.msg121036#msg121036) in how quickly delegates update feed prices depending on whether it is going up vs going down. If cmc_bts value drops (which means the price of BTS relative to the BitAsset, such as BitUSD, goes up) the bts_feed value can lag a bit without any worry. While if cmc_bts value increases, the lag until the bts_feed value updates should be much shorter so that the correct price information can be quickly updated in the blockchain to properly handle margin calls when the price is falling quickly. Anyone see a mistake with this interpretation or perhaps a different explanation for the data?

Having the bid and ask curves peak to the left of the feed mean in the case of BitUSD also makes sense to me. People selling actual BitUSD to get back into BTS need to compete with the superbulls short sellling BitUSD at the price feed by offering better asks. But the bid and ask curves for BitBTC are pretty surprising, although it is consistent with the order book I usually see in the BTS/BitBTC market (barely anyone shorting, and the actual BitBTC holders only willing to sell for an overvalued price). Perhaps that market is just too small currently to make any meaningful interpretations from the data.

And finally, the variance difference between the various curve peaks is still so high for the BitUSD market. It shows how bad the liquidity is currently in even our most liquid BitAsset. Can't wait to see those curves squeeze sharper, the peaks get closer together, and the peaks all move closer to zero.


Edit: By the way, do you think you could calculate these statistics in a moving window of the data and use it to generate an animated plot (or a live generated plot using Javascript that can adjust the plots using a slider representing time)? It would be interesting to see how these curves evolve with time.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: davidpbrown on December 21, 2014, 11:41:53 pm
Very interesting.. it would be useful to see CNY and perhaps Gold.. and later any other markets of consequence.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on December 22, 2014, 02:12:21 am
Edit: By the way, do you think you could calculate these statistics in a moving window of the data and use it to generate an animated plot (or a live generated plot using Javascript that can adjust the plots using a slider representing time)? It would be interesting to see how these curves evolve with time.

Interesting idea, an animation or slider would be a great way to visualize how the data changes over time.  An easier way might be to create line charts of each variance vs time and use the standard deviation to create confidence bands.

Once I've collected enough data, my plan is to generate these plots with the most recent 672 data points (one week's worth of data).  This will help to keep the probability functions relevant when evaluating the current market.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on December 23, 2014, 04:18:54 am
I built the visualization tools I was talking about. Check it out here: http://jsfiddle.net/wb45otcd/3.

It scrapes the data off your website by the way, hope you don't mind.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on December 23, 2014, 04:45:29 am
I built the visualization tools I was talking about. Check it out here: http://jsfiddle.net/wb45otcd/.

It scrapes the data off your website by the way, hope you don't mind.

Awesome!  This is great.  No need for a fixed window. 

Feel free to scrape, I'll put together a json API to make the data more accessible.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on December 23, 2014, 03:38:06 pm
@wasthatawolf, there is a problem with item 563 in the data (for both BitUSD and BitBTC). The feed/bid/ask prices are given as 0. This throws off the calculation of the mean and standard deviation.

I fixed the problem on my end by filtering out the erroneous data. You can compare the change by comparing http://jsfiddle.net/wb45otcd/2/ to http://jsfiddle.net/wb45otcd/3/.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on December 23, 2014, 04:12:20 pm
I just removed the bad row of data.  Modifying the script to try and prevent that from happening again.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on December 23, 2014, 04:42:04 pm
@arhag

Full data set in JSON

http://delegate.ninja/assets/functions/alldata.php?t=bitusd (http://delegate.ninja/assets/functions/alldata.php?t=bitusd)
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on December 23, 2014, 05:04:19 pm
@arhag

Full data set in JSON

http://delegate.ninja/assets/functions/alldata.php?t=bitusd (http://delegate.ninja/assets/functions/alldata.php?t=bitusd)

Nice. Could you also enable callbacks so that people could use JSONP directly from a static page rather than needing a proxy?
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: clayop on December 23, 2014, 05:16:08 pm
Here is paired t test results

Code: [Select]

> t.test(data$feed, data$bid, paired=TRUE)

        Paired t-test

data:  data$feed and data$bid
t = 39.863, df = 588, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 2.752576 3.037864
sample estimates:
mean of the differences
                2.89522

> t.test(data$feed, data$ask, paired=TRUE)

        Paired t-test

data:  data$feed and data$ask
t = 1.6808, df = 588, p-value = 0.09333
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.03797046  0.48868061
sample estimates:
mean of the differences
              0.2253551


According to the results, feed price and ask price shows no statistical difference, that is pegging greatly works for bitUSD buyers.
However, between bid and feed price, there is a significant difference (2.895) at 1% level. This implies we need improvements for bitUSD sellers.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on December 23, 2014, 05:52:50 pm
However, between bid and feed price, there is a significant difference (2.895) at 1% level. This implies we need improvements for bitUSD sellers.

I wonder how much this has to do with grandfathered shorts who aren't forced to cover in 30 days. Anyone know what fraction of the BitUSD supply is owed by grandfathered short positions?

I would also love to see statistics on the grandfathered short positions such as:
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on December 23, 2014, 06:16:38 pm
@arhag

Full data set in JSON

http://delegate.ninja/assets/functions/alldata.php?t=bitusd (http://delegate.ninja/assets/functions/alldata.php?t=bitusd)

Nice. Could you also enable callbacks so that people could use JSONP directly from a static page rather than needing a proxy?

Try this:  http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd (http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd)

I used the example here:  https://gist.github.com/mathiasbynens/5547352 (https://gist.github.com/mathiasbynens/5547352)
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on December 23, 2014, 09:38:24 pm
Try this:  http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd (http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd)

Thanks. It is working well: http://jsfiddle.net/wb45otcd/5/.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: vikram on December 23, 2014, 10:40:04 pm
 +5% This is awesome. Keep up the good work guys.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on January 05, 2015, 01:54:17 pm
Try this:  http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd (http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd)

Thanks. It is working well: http://jsfiddle.net/wb45otcd/5/.

I made some very minor modifications to your code and I'm hosting it here:  http://delegate.ninja/assets/stats.html (http://delegate.ninja/assets/stats.html)
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: xeroc on January 05, 2015, 02:56:41 pm
+5% .. awesome work
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: cass on January 05, 2015, 03:01:14 pm
 +5% awesome
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on January 05, 2015, 07:55:30 pm
Try this:  http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd (http://delegate.ninja/assets/functions/alldata_callback.php?t=bitusd)

Thanks. It is working well: http://jsfiddle.net/wb45otcd/5/.

I made some very minor modifications to your code and I'm hosting it here:  http://delegate.ninja/assets/stats.html (http://delegate.ninja/assets/stats.html)

+5% Great
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on February 03, 2015, 07:00:09 am
@wasthatawolf, do you have any idea why the bid price for BitUSD is reported higher than the ask price over the period of time roughly spanning Jan 22 to Jan 26? That is something that I thought would be impossible. It is resulting in a negative spread!

Edit: Also, the last time the price data was updated was Feb 01 2015 01:45:03 GMT.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: wasthatawolf on February 04, 2015, 03:39:27 am
@wasthatawolf, do you have any idea why the bid price for BitUSD is reported higher than the ask price over the period of time roughly spanning Jan 22 to Jan 26? That is something that I thought would be impossible. It is resulting in a negative spread!

Edit: Also, the last time the price data was updated was Feb 01 2015 01:45:03 GMT.

Doesn't look like the bitsharesblocks API from @svk is working... http://www.bitsharesblocks.com:2086/api/feedstats

My guess is there was an issue with the data from that API for those 4 days in January.
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: arhag on February 04, 2015, 04:07:20 am
@wasthatawolf, do you have any idea why the bid price for BitUSD is reported higher than the ask price over the period of time roughly spanning Jan 22 to Jan 26? That is something that I thought would be impossible. It is resulting in a negative spread!

Edit: Also, the last time the price data was updated was Feb 01 2015 01:45:03 GMT.

Doesn't look like the bitsharesblocks API from @svk is working... http://www.bitsharesblocks.com:2086/api/feedstats

My guess is there was an issue with the data from that API for those 4 days in January.

Try this new URL: http://api.bitsharesblocks.com/v1/feedstats/USD?callback=angular.callbacks._1
Title: Re: Probability Density Curves for BitAsset feed, bid and ask vs. market price
Post by: vikram on June 18, 2015, 08:54:54 pm
Any recent work on this or similar projects?