BitShares Forum

Main => General Discussion => Topic started by: bytemaster on July 27, 2014, 08:51:33 pm

Title: USD, BTC, GLD price feeds....
Post by: bytemaster on July 27, 2014, 08:51:33 pm
Can someone put together a python script that will fetch the BTC, USD, and GLD price of BTSX and then generate an RPC call to publish that feed. 

Ideally the data would be from as many different sources as possible.

The RPC call that you will want to call is:

wallet_publish_price_feed ${DELEGATE} ${PRICE} USD
wallet_publish_price_feed ${DELEGATE} ${PRICE} GLD
wallet_publish_price_feed ${DELEGATE} ${PRICE} BTC

Where ${PRICE} is USD PER BTSX


Title: Re: USD, BTC, GLD price feeds....
Post by: bitmeat on July 28, 2014, 04:37:10 am
As it turns out I have already written some code very close to your ask. Only have BTC/USD (sourced from coinbase/bitstamp API) and BTC/BTSX from BTER for the moment.

If you point me to free public feeds with GLD I can incorporate that as well.

Also, already have done integration to the wallet with the RPCJSON (I posted it in another thread)

---

Any terms on the bounty itself?

Title: Re: USD, BTC, GLD price feeds....
Post by: bitmeat on July 28, 2014, 05:02:03 am
This claims to have free anonymous calls up to 50 times/day.

Not sure it's worth the effort but for documenting purposes:

http://www.quandl.com/help/api


UPDATE: more specifically Historical Bundesbank here:

http://www.quandl.com/api/v1/datasets/BUNDESBANK/BBK01_WT5511.json?trim_start=1968-04-01&trim_end=2014-07-25

Title: Re: USD, BTC, GLD price feeds....
Post by: bytemaster on July 28, 2014, 12:12:48 pm
As it turns out I have already written some code very close to your ask. Only have BTC/USD (sourced from coinbase/bitstamp API) and BTC/BTSX from BTER for the moment.

If you point me to free public feeds with GLD I can incorporate that as well.

Also, already have done integration to the wallet with the RPCJSON (I posted it in another thread)

---

Any terms on the bounty itself?

I am accepting proposals on what people think is fair.  Just need something opensource and easy to use.
Title: Re: USD, BTC, GLD price feeds....
Post by: bitmeat on July 28, 2014, 02:22:36 pm
I'm offering to fully develop and maintain this important component until launch for 500K BTSX total (~7BTC).
Naturally others will offer to do it for way less, but I offer top quality and maintenance. It is also not worth my time for less than that I wish it was.

I would also recommend that various other pieces are included such as liquidity and spread at volume X.

e.g. if buying 50K BTSX for 0.7BTC is possible as per the order book, buying 1M BTSX for 14BTC may not be, this may actually cost 20BTC.
Also the median price for the order book at that volume may be different than the center of the spread.

UPDATE: e.g. look at bitcoinwisdom orderbook chart:

(http://i.imgur.com/M8RS477.png)

Notice how the median price is going up if you take into account the volume in the order book.
Title: Re: USD, BTC, GLD price feeds....
Post by: luckybit on July 28, 2014, 07:45:06 pm
I'm offering to fully develop and maintain this important component until launch for 500K BTSX total (~7BTC).
Naturally others will offer to do it for way less, but I offer top quality and maintenance. It is also not worth my time for less than that I wish it was.

I would also recommend that various other pieces are included such as liquidity and spread at volume X.

e.g. if buying 50K BTSX for 0.7BTC is possible as per the order book, buying 1M BTSX for 14BTC may not be, this may actually cost 20BTC.
Also the median price for the order book at that volume may be different than the center of the spread.

UPDATE: e.g. look at bitcoinwisdom orderbook chart:

(http://i.imgur.com/M8RS477.png)

Notice how the median price is going up if you take into account the volume in the order book.

Can you post some of your code somewhere so we can have a look at it? I've been thinking along the same direction recently.
Title: Re: USD, BTC, GLD price feeds....
Post by: AEcharts on August 12, 2014, 10:47:07 pm
Hello, I am new here. I have just discovered BitShares a few hours ago, read a few of the wiki pages and BitShares seems to have some interesting ideas. Because I am soo new I may not be understanding the request completely, but have described what I could provide below.

What I could provide is a data visualization of the correlation and/or cointegration between two different time series over an n period moving window. The data visualization could be in either PNG, SVG or PDF format. Just the data could also be provided in JSON or CSV format to be used for constructing trade logic. This could be done using a simple HTTP/HTTPS RPC or API call. The caller would supply the trade pair, n period moving window and what time resolution to use (eg. 5 minute, 1 hour, etc). The service would respond with either an image format or data depending on the suffix of the requested file (eg. .png, .svg, .json, etc).

I would like to implement this service as a way to get involved in the BitShares platform.

I just need to know how we could get started with this?
Title: Re: USD, BTC, GLD price feeds....
Post by: maqifrnswa on August 12, 2014, 11:54:01 pm
Hello, I am new here. I have just discovered BitShares a few hours ago, read a few of the wiki pages and BitShares seems to have some interesting ideas. Because I am soo new I may not be understanding the request completely, but have described what I could provide below.

What I could provide is a data visualization of the correlation and/or cointegration between two different time series over an n period moving window. The data visualization could be in either PNG, SVG or PDF format. Just the data could also be provided in JSON or CSV format to be used for constructing trade logic. This could be done using a simple HTTP/HTTPS RPC or API call. The caller would supply the trade pair, n period moving window and what time resolution to use (eg. 5 minute, 1 hour, etc). The service would respond with either an image format or data depending on the suffix of the requested file (eg. .png, .svg, .json, etc).

I would like to implement this service as a way to get involved in the BitShares platform.

I just need to know how we could get started with this?

that sounds great but a little overkill for what bytemaster is asking. Basically the market can be bootstrapped if delegates inform the market as to what a "fair" price is. One could go to different exchanges and type the command in themselves based on what they saw on the website, or someone can use a python script to scrape the web (like you're doing) and then the script can submit the following commands to the bitshares client:

wallet_publish_price_feed ${DELEGATE} ${PRICE} USD
wallet_publish_price_feed ${DELEGATE} ${PRICE} GLD
wallet_publish_price_feed ${DELEGATE} ${PRICE} BTC

It's pretty simple, just will take some time to get it working right. You can then set up a cron job to keep it up to date.

so, he's looking for a simple python script:
1) scrape websites/grab json feeds/do whatever you need to do to get an accurate price quote at this moment (no history required)
2) send an RPC command with that information to a running bitshares client
Title: Re: USD, BTC, GLD price feeds....
Post by: bytemaster on August 13, 2014, 12:25:18 am
Hello, I am new here. I have just discovered BitShares a few hours ago, read a few of the wiki pages and BitShares seems to have some interesting ideas. Because I am soo new I may not be understanding the request completely, but have described what I could provide below.

What I could provide is a data visualization of the correlation and/or cointegration between two different time series over an n period moving window. The data visualization could be in either PNG, SVG or PDF format. Just the data could also be provided in JSON or CSV format to be used for constructing trade logic. This could be done using a simple HTTP/HTTPS RPC or API call. The caller would supply the trade pair, n period moving window and what time resolution to use (eg. 5 minute, 1 hour, etc). The service would respond with either an image format or data depending on the suffix of the requested file (eg. .png, .svg, .json, etc).

I would like to implement this service as a way to get involved in the BitShares platform.

I just need to know how we could get started with this?

that sounds great but a little overkill for what bytemaster is asking. Basically the market can be bootstrapped if delegates inform the market as to what a "fair" price is. One could go to different exchanges and type the command in themselves based on what they saw on the website, or someone can use a python script to scrape the web (like you're doing) and then the script can submit the following commands to the bitshares client:

wallet_publish_price_feed ${DELEGATE} ${PRICE} USD
wallet_publish_price_feed ${DELEGATE} ${PRICE} GLD
wallet_publish_price_feed ${DELEGATE} ${PRICE} BTC

It's pretty simple, just will take some time to get it working right. You can then set up a cron job to keep it up to date.

so, he's looking for a simple python script:
1) scrape websites/grab json feeds/do whatever you need to do to get an accurate price quote at this moment (no history required)
2) send an RPC command with that information to a running bitshares client

Other requirements:   It should only run once per day or when the price changes by more than 10%.  It should never publish bogus data, so should probably have some redundant checks.   
Title: Re: USD, BTC, GLD price feeds....
Post by: bitmeat on August 13, 2014, 12:37:31 am
Have you decided on what the reward for this bounty is?
Title: Re: USD, BTC, GLD price feeds....
Post by: bytemaster on August 13, 2014, 01:43:01 am
Have you decided on what the reward for this bounty is?

I'll judge based upon the robustness of the solution as well as the variety of assets supported.   Any delegates want to pledge any support for this?    I would suggest the bounty on this should be 25% of the delegate pay earned by DACSunLimited while they use your solution.
Title: USD, BTC, GLD price feeds....
Post by: bitmeat on August 13, 2014, 01:49:10 am
I think having multiple independent solutions would strengthen the network.
Title: Re: USD, BTC, GLD price feeds....
Post by: bytemaster on August 13, 2014, 01:58:28 am
I think having multiple independent solutions would strengthen the network.

Right...
Title: Re: USD, BTC, GLD price feeds....
Post by: fuzzy on August 13, 2014, 04:29:29 am
Have you decided on what the reward for this bounty is?

I'll judge based upon the robustness of the solution as well as the variety of assets supported.   Any delegates want to pledge any support for this?    I would suggest the bounty on this should be 25% of the delegate pay earned by DACSunLimited while they use your solution.

If we get delegates wanting to help pay the bounty requirements, would you be opposed to giving them time after your Dev Hangout to introduce themselves to the community and let investors know how to vote for them?
Title: Re: USD, BTC, GLD price feeds....
Post by: alt on August 13, 2014, 05:54:49 am
solution from BitSuperLab
https://github.com/Bitsuperlab/operation_tools/tree/master/btsxfeed (https://github.com/Bitsuperlab/operation_tools/tree/master/btsxfeed)

fetch btc/cny price from btc38 and bter,  fetch rate from yahoo finance.
for now, you must confirm before update the feed.
Title: Re: USD, BTC, GLD price feeds....
Post by: alt on August 13, 2014, 07:37:34 am
update:
btsx_feed.py need confirm
btsx_feed_auto.py don't need confirm, check ever 1 minute, only publish when price change more than 5%

solution from BitSuperLab
https://github.com/Bitsuperlab/operation_tools/tree/master/btsxfeed (https://github.com/Bitsuperlab/operation_tools/tree/master/btsxfeed)

fetch btc/cny price from btc38 and bter,  fetch rate from yahoo finance.
for now, you must confirm before update the feed.
Title: Re: USD, BTC, GLD price feeds....
Post by: AEcharts on August 13, 2014, 07:20:39 pm
Hello, I am new here. I have just discovered BitShares a few hours ago, read a few of the wiki pages and BitShares seems to have some interesting ideas. Because I am soo new I may not be understanding the request completely, but have described what I could provide below.

What I could provide is a data visualization of the correlation and/or cointegration between two different time series over an n period moving window. The data visualization could be in either PNG, SVG or PDF format. Just the data could also be provided in JSON or CSV format to be used for constructing trade logic. This could be done using a simple HTTP/HTTPS RPC or API call. The caller would supply the trade pair, n period moving window and what time resolution to use (eg. 5 minute, 1 hour, etc). The service would respond with either an image format or data depending on the suffix of the requested file (eg. .png, .svg, .json, etc).

I would like to implement this service as a way to get involved in the BitShares platform.

I just need to know how we could get started with this?

that sounds great but a little overkill for what bytemaster is asking. Basically the market can be bootstrapped if delegates inform the market as to what a "fair" price is. One could go to different exchanges and type the command in themselves based on what they saw on the website, or someone can use a python script to scrape the web (like you're doing) and then the script can submit the following commands to the bitshares client:

wallet_publish_price_feed ${DELEGATE} ${PRICE} USD
wallet_publish_price_feed ${DELEGATE} ${PRICE} GLD
wallet_publish_price_feed ${DELEGATE} ${PRICE} BTC

It's pretty simple, just will take some time to get it working right. You can then set up a cron job to keep it up to date.

so, he's looking for a simple python script:
1) scrape websites/grab json feeds/do whatever you need to do to get an accurate price quote at this moment (no history required)
2) send an RPC command with that information to a running bitshares client

To make things clear, I never said that I would be doing any kind of web scraping for the data. That could possibly be used to circumvent the ad revenue that a data provider is rightfully due, depending on how that data is used. I would be using a data provider to get the higher resoluton data. That would come at a cost, that is why I was talking about using a service type model for what I described.

I could possibly provide an implementation for the request that has been described, if I could ascertain that I would be compensated and how much.
Title: Re: USD, BTC, GLD price feeds....
Post by: bytemaster on August 13, 2014, 07:21:52 pm
I don't think we need real time feeds... only feeds that are accurate within ~5%
Title: Re: USD, BTC, GLD price feeds....
Post by: bytemaster on August 13, 2014, 07:22:18 pm
I don't think we need real time feeds... only feeds that are accurate within ~5%

Remember... it costs a transaction fee to publish a feed.
Title: Re: USD, BTC, GLD price feeds....
Post by: AEcharts on August 13, 2014, 07:39:32 pm
If a service like I described in my first post is something I could get funding to implement, how and where would I present this? The provided services would be much broader in scope than the specific solution I provided in my first post.

I am quite new here, so any suggestions or advice any has would be much appreciated.
Title: Re: USD, BTC, GLD price feeds....
Post by: monsterer on October 14, 2014, 06:29:21 pm
There are a lot of forex brokers providing price feeds for metatrader. Has anyone considered a solution using those feeds?
Title: Re: USD, BTC, GLD price feeds....
Post by: xeroc on October 14, 2014, 07:11:12 pm
There are a lot of forex brokers providing price feeds for metatrader. Has anyone considered a solution using those feeds?
If theses are public APIs why not use them .. sure .. however I cannot run metatrader on linux here so I cannot check ...
is there documentation of the apis? sources? used
Title: Re: USD, BTC, GLD price feeds....
Post by: jsidhu on October 14, 2014, 07:48:53 pm
There are a lot of forex brokers providing price feeds for metatrader. Has anyone considered a solution using those feeds?

If you find a good broker which does not filter their feed then you can write a simply EA or indicator that will read the tickchart and write out to a file. Its fairly easy. The issue is that brokers freeze pause the feed during news so feed dependability becomes an issue. Also feeds are not available during weekends.
Title: Re: USD, BTC, GLD price feeds....
Post by: xeroc on October 14, 2014, 07:51:59 pm
... Also feeds are not available during weekends.
lol woot?

I never participated in 'regular' markets ... but not being able to trade 24/7 sometimes still amazed me  :D
Title: Re: USD, BTC, GLD price feeds....
Post by: jsidhu on October 14, 2014, 07:54:39 pm
... Also feeds are not available during weekends.
lol woot?

I never participated in 'regular' markets ... but not being able to trade 24/7 sometimes still amazed me  :D

Heh well forex is 24/7 just no volume... so brokers figure its not worth the CODB to be online then.