100 BitUSD to First Person to Publish a working Price Feed Script for Graphene

[tonyk]

FYI this bounty has been paid.

That's worth mentioning for sure ... not paid in timely manner but for sure better than

 half ass excuses citing unseen yet but upcoming best practices.

[bytemaster]

FYI this bounty has been paid.

[abit]

Just a note: alt made another price feed script, see https://bitsharestalk.org/index.php/topic,20529.msg265069.html

[abit]

Bounty is all yours, where do you want the BitUSD sent?

you can send it all to "xeroc" .. I'll share a cut with @abit

Thanks  my account is "abit"

[xeroc]

Bounty is all yours, where do you want the BitUSD sent?

you can send it all to "xeroc" .. I'll share a cut with @abit

[Thom]

That payout took awhile. Luck for you BM many of us here are a patient lot. Some of us grumble more than others but we're still here.

Xeroc certainly deserves the bounty, and perhaps some BROWNIES for the patience.

[bytemaster]

Bounty is all yours, where do you want the BitUSD sent?

[xeroc]

We are looking for testers who will publish a BitUSD price feed in graphene.  You will need to be an elected witness to publish the feed.

You can either provide the script to an existing witness tester or become a tester yourself. 

Unfortunately there is little documentation on HOW to do this right now, hence the bounty.   Xeroc may be able to help.

Can I request the bounty now?

[BunkerChainLabs-DataSecurityNode]

I have updated the code:

+ reneabled bter
+ added another parameter "core_exchange_factor"
+ readded publishing rules which now allows to run it in a cron job on only publish every X seconds or on larger price changes (config parameter)
+ reenabled all other currencies that have been available in BTS1 already (INDICES still not complete)

TODO:
 + indices
 + bundle prices into a single transaction

 

[xeroc]

I have updated the code:

+ reneabled bter
+ added another parameter "core_exchange_factor"
+ readded publishing rules which now allows to run it in a cron job on only publish every X seconds or on larger price changes (config parameter)
+ reenabled all other currencies that have been available in BTS1 already (INDICES still not complete)

TODO:
 + indices
 + bundle prices into a single transaction

[xeroc]

You script should attempt to include all feed update operations in a single transaction to minimize load on the network.

Sounds like a good idea .. but for that I need to use the transaction_builder ..
I can do this but I may not be able to finish this up until after Shanghai

Beside that, could you please give you thoughts on this topic?
https://bitsharestalk.org/index.php/topic,18831.msg242310.html#msg242310

[bytemaster]

You script should attempt to include all feed update operations in a single transaction to minimize load on the network.

[xeroc]

Can we get a summary of what data sources are being used by this script?

Code: [Select]

mythreads["yahoo"]    = threading.Thread(target = fetch_from_yahoo)
 mythreads["yunbi"]    = threading.Thread(target = fetch_from_yunbi)
 mythreads["btc38"]    = threading.Thread(target = fetch_from_btc38)
 #mythreads["bter"]     = threading.Thread(target = fetch_from_bter)
 mythreads["poloniex"] = threading.Thread(target = fetch_from_poloniex)
 mythreads["bittrex"]  = threading.Thread(target = fetch_from_bittrex)
 mythreads["btcavg"]   = threading.Thread(target = fetch_bitcoinaverage)

Yahoo for traditional assets like GOLD, Silver.
Yunbi, btc38, bter (disabled atm), poloniex and bittrex for BTS/BTC pair
btcavg for USD/EUR to BTC pair ..


I am not sure how my script was even possible to submit prices for the wrong
paris.

Code: [Select]

  price_feed = {
                      "settlement_price": {
                        "quote": {
                          "asset_id": "1.3.0",
                          "amount": denominator
                        },
                        "base": {
                          "asset_id": assets[asset]["id"],
                          "amount": numerator
                        }
                      },
                      "core_exchange_rate": {
                        "quote": {
                          "asset_id": assets[asset]["id"],
                          "amount": numerator
                        },
                        "base": {
                          "asset_id": "1.3.0",
                          "amount": int(denominator * 1.05) # 5% extra
                        }
                      }
                    }
I load all publishing assets into ``assets`` and read the id JIT.
I'll investigate

[bytemaster]

Can we get a summary of what data sources are being used by this script?

[Riverhead]

So @bytemaster, what else needs to be done for the price feed script to qualify for the bounty?

I think you have qualified.

Grats Xeroc!

[bytemaster]

So @bytemaster, what else needs to be done for the price feed script to qualify for the bounty?

I think you have qualified.   In the test net your script published some prices with the wrong asset id.     We tightened the validation to catch those errors, but it appears Ben has not committed them yet.

[xeroc]

I have now added a quick "howto": http://python-graphenelib.readthedocs.org/en/latest/pricefeed.html

[xeroc]

So @bytemaster, what else needs to be done for the price feed script to qualify for the bounty?

[abit]

I know see what you mean.

This patch (https://github.com/xeroc/python-graphenelib/commit/421d9c04785b5ce3d70f77376485109f808133cd) will result in the following output:

Code: [Select]

            "core_exchange_rate": {
                "quote": {
                    "amount": 52,
                    "asset_id": "1.3.626"
                },
                "base": {
                    "amount": 92170,
                    "asset_id": "1.3.0"
                }
            }
would the numbers now fit you too?

Yes they are right.

BTW, I am using "fractions" to get a numerator and denominator instead of just using 10^x .. but in the end, they all represent the same ratio ..

Yes I saw it. But personally I like 10^x more because I don't need to take out a calculator to check the numbers

Then you need to publish a feed.  If you did it today on the testnet, you'd use the publish_asset_feed command.  But soon (as in I'm working on it right now) there will be a much better command for this (publish_asset_feed requires you to specify prices in satoshis which is...inconvenient).

Good news. Easier to use is better.

[xeroc]

Then you need to publish a feed.  If you did it today on the testnet, you'd use the publish_asset_feed command.  But soon (as in I'm working on it right now) there will be a much better command for this (publish_asset_feed requires you to specify prices in satoshis which is...inconvenient).

[xeroc]

I know see what you mean.

This patch (https://github.com/xeroc/python-graphenelib/commit/421d9c04785b5ce3d70f77376485109f808133cd) will result in the following output:

Code: [Select]

            "core_exchange_rate": {
                "quote": {
                    "amount": 52,
                    "asset_id": "1.3.626"
                },
                "base": {
                    "amount": 92170,
                    "asset_id": "1.3.0"
                }
            }
would the numbers now fit you too?

BTW, I am using "fractions" to get a numerator and denominator instead of just using 10^x .. but in the end, they all represent the same ratio ..

[abit]

The numbers are wrong..

How so?

The output says:

Code: [Select]

   "settlement_price": {
       "base": {
           "asset_id": "1.3.626",
           "amount": 85427
       },
       "quote": {
           "asset_id": "1.3.0",
           "amount": 55
       }
   },

which is:
55/85427 BTS/USD = 0.000644 * 10 (precision) BTS/USD
That has been the price yesterday.

For the core_exchange rate (if you look at the code), I added a 5% premium.

Don't see how that is 'wrong'

The result says 85427/10^4 USD == 55/10^5 CORE. (it can be checked with https://graphene.bitshares.org/ after pushed a price feed.)
The correct result should be like 0.00644 USD == 1 CORE, or 0.0644 USD = 10 CORE. Bring in the precision, it should be 644/10^4 USD == 1000000/10^5 CORE, so the json should be

Code: [Select]

   "settlement_price": {
       "base": {
           "asset_id": "1.3.626",
           "amount": 644
       },
       "quote": {
           "asset_id": "1.3.0",
           "amount": 1000000
       }
   },
By the way, I think
1, it's more human-readable if one of the amount is 10^n
2, a little more precision could be better

[xeroc]

From what I understa.d I shouldn really matter since you have the asset-id side by side with the amount ... 10 x/y is identical to 0.1 y/x. ... isnt it?

[svk]

The numbers are wrong..

How so?

The output says:

Code: [Select]

   "settlement_price": {
       "base": {
           "asset_id": "1.3.626",
           "amount": 85427
       },
       "quote": {
           "asset_id": "1.3.0",
           "amount": 55
       }
   },

which is:
55/85427 BTS/USD = 0.000644 * 10 (precision) BTS/USD
That has been the price yesterday.

For the core_exchange rate (if you look at the code), I added a 5% premium.

Don't see how that is 'wrong'

Why are the base and quote assets inverted for settlement_price and core_exchange_rate?

[xeroc]

The numbers are wrong..

How so?

The output says:

Code: [Select]

   "settlement_price": {
       "base": {
           "asset_id": "1.3.626",
           "amount": 85427
       },
       "quote": {
           "asset_id": "1.3.0",
           "amount": 55
       }
   },

which is:
55/85427 BTS/USD = 0.000644 * 10 (precision) BTS/USD
That has been the price yesterday.

For the core_exchange rate (if you look at the code), I added a 5% premium.

Don't see how that is 'wrong'

[abit]

@abit: thanks for your inputs .. I took some of your patch but sticked to my philisophy of [base][qu ote] ..
anyway .. I also added another btc price feed and modified the price feed structure .. it should now do what it is supposed to do ..

also, the exchange_rate is for transactions fees that are paid in bitasset .. in my script .. paying your transactions in bitUSD (or any other bitasset) is 5% more expensive (read: supposed to be)

If you find the time, please go through the script .. I appreciate your input!!!

Result of your latest script is here:

Code: [Select]

    [
        "USD",
        {
            "settlement_price": {
                "base": {
                    "asset_id": "1.3.626",
                    "amount": 85427
                },
                "quote": {
                    "asset_id": "1.3.0",
                    "amount": 55
                }
            },
            "core_exchange_rate": {
                "base": {
                    "asset_id": "1.3.0",
                    "amount": 55
                },
                "quote": {
                    "asset_id": "1.3.626",
                    "amount": 89698
                }
            }
        }
    ]
The numbers are wrong..

[xeroc]

@abit: thanks for your inputs .. I took some of your patch but sticked to my philisophy of [base][qu ote] ..
anyway .. I also added another btc price feed and modified the price feed structure .. it should now do what it is supposed to do ..

also, the exchange_rate is for transactions fees that are paid in bitasset .. in my script .. paying your transactions in bitUSD (or any other bitasset) is 5% more expensive (read: supposed to be)

If you find the time, please go through the script .. I appreciate your input!!!

[abit]

@bytemaster: should "core_exchange_rate" equal to external exchanges, and "settlement_price" be a bit higher?

[abit]

It lacks the ratio between precisions .. i will fix this tomorrow ..

Not only that.. I'll submit a pull request soon.
//Update:
pull request requested. https://github.com/xeroc/python-graphenelib/pull/1
New source can be found here:
 https://github.com/abitmore/python-graphenelib/
 https://github.com/abitmore/python-graphenelib/blob/master/scripts/pricefeeds.py

[xeroc]

It lacks the ratio between precisions .. i will fix this tomorrow ..

[abit]

I've managed to get the script run, but the result seems a bit wrong.

Code: [Select]

[
    [
        "CNY",
        {
            "core_exchange_rate": {
                "quote": {
                    "amount": 6670,
                    "asset_id": "1.3.379"
                },
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }
            },
            "settlement_price": {
                "quote": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                },
                "base": {
                    "amount": 6353,
                    "asset_id": "1.3.379"
                }
            }
        }
    ],
...
...
    [
        "USD",
        {
            "core_exchange_rate": {
                "quote": {
                    "amount": 1044,
                    "asset_id": "1.3.626"
                },
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }
            },
            "settlement_price": {
                "quote": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                },
                "base": {
                    "amount": 995,
                    "asset_id": "1.3.626"
                }
            }
        }
    ]
]


[bytemaster]

So ... assuming the following to be correct:

Code: [Select]

[
    [   
        "USD",
        {
            "core_exchange_rate": {
                "quote": {
                    "amount": 1044,
                    "asset_id": "1.3.626"
                },
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }
            }, 
            "settlement_price": {
                "quote": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }, 
                "base": {
                    "amount": 995,
                    "asset_id": "1.3.626"
                }   
            }   
        }   
    ], 
    [
        "EUR",
        {
            "core_exchange_rate": {
                "quote": {
                    "amount": 928,
                    "asset_id": "1.3.424"
                }, 
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }   
            }, 
            "settlement_price": {
                "quote": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }, 
                "base": {
                    "amount": 884,
                    "asset_id": "1.3.424"
                }   
            }   
        }   
    ]   
]
.. I have successfully modified my current price feed script to run for graphene ..
@bytemaster: if 1 BTS is worth $0.007 .. would the following price feed be correct?

Code: [Select]

                "quote": {
                    "amount": 700,
                    "asset_id": "1.3.424"
                }, 
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }   
For now, I used the precision as base amount and the price multiplied by base amount as quote amount ..
Is this how it is supposed to be or am I understanding something wrong?


Code:
https://github.com/xeroc/python-graphenelib/blob/develop/scripts/pricefeeds.py
It requires installation of the python-graphenelib

What you have is almost correct, except it fails to consider the precision difference between BITUSD and BTS.   BTS has 5 digits and BitUSD has 4 so you are off by a factor of 10.

[xeroc]

hmm .. thinking about this approach .. it should be possible to just represent the float as a rational (which is what you had in mind I suppose) and put nominator and denominator into base and quote ..
I will pimp this script this week ..

[xeroc]

So ... assuming the following to be correct:

Code: [Select]

[
    [   
        "USD",
        {
            "core_exchange_rate": {
                "quote": {
                    "amount": 1044,
                    "asset_id": "1.3.626"
                },
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }
            }, 
            "settlement_price": {
                "quote": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }, 
                "base": {
                    "amount": 995,
                    "asset_id": "1.3.626"
                }   
            }   
        }   
    ], 
    [
        "EUR",
        {
            "core_exchange_rate": {
                "quote": {
                    "amount": 928,
                    "asset_id": "1.3.424"
                }, 
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }   
            }, 
            "settlement_price": {
                "quote": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }, 
                "base": {
                    "amount": 884,
                    "asset_id": "1.3.424"
                }   
            }   
        }   
    ]   
]
.. I have successfully modified my current price feed script to run for graphene ..
@bytemaster: if 1 BTS is worth $0.007 .. would the following price feed be correct?

Code: [Select]

                "quote": {
                    "amount": 700,
                    "asset_id": "1.3.424"
                }, 
                "base": {
                    "amount": 100000,
                    "asset_id": "1.3.0"
                }   
For now, I used the precision as base amount and the price multiplied by base amount as quote amount ..
Is this how it is supposed to be or am I understanding something wrong?


Code:
https://github.com/xeroc/python-graphenelib/blob/develop/scripts/pricefeeds.py
It requires installation of the python-graphenelib

[abit]

I did successfully published price feed for my bitAsset "ABITUSDA". 

Code: [Select]

publish_asset_feed in.abit ABITUSDA {"settlement_price":{"base":{"amount":50000,"asset_id":"1.3.662"},"quote":{"amount":1000000000,"asset_id":"1.3.0"}},"core_exchange_rate":{"base":{"amount":100000,"asset_id":"1.3.0"},"quote":{"amount":10000,"asset_id":"1.3.662"}}} true


[xeroc]

@abit: thanks for figuring this out .. ill try to finish the script when back home (unless someone else can do more quickly) .. i'd be happy to share the bounty ..

[abit]

I am stuck ..
What's the syntax to hand over a price_feed object?
I am getting:

Code: [Select]

unlocked >>> publish_asset_feed init0 CORE {settlement_price:"0.4"} false
publish_asset_feed init0 CORE {settlement_price:"0.4"} false
4 parse_error_exception: Parse Error
Expected ':' after key "settlement_price"
    {"key":"settlement_price"}
    th_a  json_relaxed.hpp:589 objectFromStream
Error parsing object
    {}
    th_a  json_relaxed.hpp:611 objectFromStream

    {"str":"publish_asset_feed init0 CORE {settlement_price:\"0.4\"} false"}
    th_a  json.cpp:494 variants_from_string


Here is my command which was able to create a transaction, but the transaction got refused by the chain:

Code: [Select]

publish_asset_feed in.abit USD {"settlement_price":{"base":{"amount":50000,"asset_id":"1.3.626"},"quote":{"amount":1000000000,"asset_id":"1.3.0"}},"core_exchange_rate":{"base":{"amount":100000,"asset_id":"1.3.0"},"quote":{"amount":10000,"asset_id":"1.3.626"}}} true

Looks like there is a bug. With current implementation, it's only possible to publish price feed for "committee issued assets" or "private assets", but all bitAssets are "witness issued assets". Submitted an issue here https://github.com/cryptonomex/graphene/issues/328.

p.s.
1. I think the command syntax isn't user-friendly (at all).
2. Can I get some part of the bounty? 

[xeroc]

I am stuck ..
What's the syntax to hand over a price_feed object?
I am getting:

Code: [Select]

unlocked >>> publish_asset_feed init0 CORE {settlement_price:"0.4"} false
publish_asset_feed init0 CORE {settlement_price:"0.4"} false
4 parse_error_exception: Parse Error
Expected ':' after key "settlement_price"
    {"key":"settlement_price"}
    th_a  json_relaxed.hpp:589 objectFromStream
Error parsing object
    {}
    th_a  json_relaxed.hpp:611 objectFromStream

    {"str":"publish_asset_feed init0 CORE {settlement_price:\"0.4\"} false�"}
    th_a  json.cpp:494 variants_from_string


[betax]

Have you developed a split forum personality, xeroc?

except that there are now 3!!!! prices per bitasset.

Unfortunatelly, I have no clue why you say there are three prices in price_feed ..  I only see settlement_price, and core_exchange_rate. The other two are 'constants'

lol one forum account isn't enough let's get another one that talks to me

LOL that is what I thought!

[xeroc]

hehe ... it's stated in the comments of the source code ..
https://github.com/cryptonomex/graphene/blob/master/libraries/wallet/include/graphene/wallet/wallet.hpp#L916-L939

[santaclause102]

Have you developed a split forum personality, xeroc?

except that there are now 3!!!! prices per bitasset.

Unfortunatelly, I have no clue why you say there are three prices in price_feed ..  I only see settlement_price, and core_exchange_rate. The other two are 'constants'

lol one forum account isn't enough let's get another one that talks to me

[xeroc]

More related info:
https://github.com/cryptonomex/graphene/blob/3eedabbac251b8afa585cb644f4b2b4596f2d37a/libraries/chain/include/graphene/chain/protocol/asset.hpp#L147-L194

Unfortunatelly, I have no clue why you say there are three prices in price_feed ..  I only see settlement_price, and core_exchange_rate. The other two are 'constants'

[xeroc]

It means bitusd works like it does in 1.0. 


Sent from my iPhone using Tapatalk

That statement will definitelly be misinterprered. Bitusd works differently in bts2 .. just the price feeds for bitusd work the same way .. except that there are now 3!!!! prices per bitasset.

[sittingduck]

It means bitusd works like it does in 1.0. 


Sent from my iPhone using Tapatalk

[xeroc]

I'm still unclear on who will be publishing feeds for assets?  I was under the impression that this would be something workers would do.  Is this something I will be expected to do as a witness?  Not that I mind or anything, just looking for clarity.

Witnesses publish for the network.

What that means is (iiuc) that witnesses produce feeds for bitassets .. there are also privatized smartcoins which mmay have different producers ..

[bytemaster]

I'm still unclear on who will be publishing feeds for assets?  I was under the impression that this would be something workers would do.  Is this something I will be expected to do as a witness?  Not that I mind or anything, just looking for clarity.

Witnesses publish for the network.

[puppies]

I'm still unclear on who will be publishing feeds for assets?  I was under the impression that this would be something workers would do.  Is this something I will be expected to do as a witness?  Not that I mind or anything, just looking for clarity.

[svk]

is there bta in genesis block in bts 2.0?

Yes, all assets transfer over as is, including bit assets.

[BTSdac]

is there bta in genesis block in bts 2.0?

[xeroc]

This is what you need

Code: [Select]

/** Publishes a price feed for the named asset. * * Price feed providers use this command to publish their price feeds for market-issued assets. A price feed is * used to tune the market for a particular market-issued asset. For each value in the feed, the median across all * committee_member feeds for that asset is calculated and the market for the asset is configured with the median of that * value. * * The feed object in this command contains three prices: a call price limit, a short price limit, and a settlement price. * The call limit price is structured as (collateral asset) / (debt asset) and the short limit price is structured * as (asset for sale) / (collateral asset). Note that the asset IDs are opposite to eachother, so if we're * publishing a feed for USD, the call limit price will be CORE/USD and the short limit price will be USD/CORE. The * settlement price may be flipped either direction, as long as it is a ratio between the market-issued asset and * its collateral. * * @param publishing_account the account publishing the price feed * @param symbol the name or id of the asset whose feed we're publishing * @param feed the price_feed object containing the three prices making up the feed * @param broadcast true to broadcast the transaction on the network * @returns the signed transaction updating the price feed for the given asset */ signed_transaction publish_asset_feed(string publishing_account, string symbol, price_feed feed, bool broadcast = false);

I'll try to work out a basic script tomorrow .. but I dont have much time this week ..


wallet.hpp line 930

[bytemaster]

We are looking for testers who will publish a BitUSD price feed in graphene.  You will need to be an elected witness to publish the feed.

You can either provide the script to an existing witness tester or become a tester yourself. 

Unfortunately there is little documentation on HOW to do this right now, hence the bounty.   Xeroc may be able to help.