BitShares Forum
Main => General Discussion => Topic started by: bytemaster on November 24, 2015, 02:34:02 pm
-
Everyone who wants a better API, please give us detailed description of what you want the API to be.
-
HTTP access to all parts.
Exchanges need these functions:
* GetBlockHeight
* ListSinceBlock - especially important, to avoid jump through hoops as we do now
* GetBalance
* Send
Every transaction sending function should return a TXID immediately, which is then visible whenever that transaction turns up in future - and transaction must be queryable using this TXID at any time.
-
Any time a balance or amount is returned it should be a consistent format (instead of sometimes strings sometimes integers)
Any time a balance or amount is returned it should be adjusted to the proper precision or the precision should be included(so multiple calls are not required)
When returning an order book, i'd like to see it separated in to an array of "bids": and an array of "asks": ideally ordered most to least relevant (standard for exchanges)
like monsterer mentioned access to all commands via http. I believe its currently still not possible to cancel an order this way.
having a separate call for placing a bid and placing an ask that accepts the standard fields "price" and "amount" instead of the current "amount" and "minimum amount expected" (it works just fine and makes sense as it is with just 1 call but its not the standard way of doing it.)
in the return from get balances, it would be nice to have for each asset an "amount_in_orders" value
with the exception of access to cancel_order and other http requests, all the above have work arounds, just suggesting for simplicity and similarity to other exchanges.
-
The ability to automatically return funds if a minimum amount isn't reached. Think Kickstarter.
-
Basically what we need is to be able to access the wallet from a 3rd trading platform.
1. Need to receive "tick" data (this is the last reported price and volume for an asset). We need to take these "ticks" and create price charts with them.
2. Need to be able to access the exchange from the 3rd party platform. This includes account data such as open positions balances.
3. The platform needs to be able to create and send orders to the DEX on behalf of the user. The dex needs to be able to take this command and execute the order. The dex will need to be able to accommodate limit and stop orders on the block chain level.
-
Another list of feature requests:
https://github.com/cryptonomex/graphene/issues/382
Thanks for considering to improve the API!!! +5%
-
- get_market_history via RPC & in the CLI wallet appears to return the first 200 buckets, not the most recent 200 buckets.
- Consistency in returning object ids from the API - tx's, ops, accounts, etc. Ideally whenever an object is returned, its id is included.
- More/bigger batch methods. Until the API is more fleshed out, simply making it easier to pull data in bulk from witness_node can really help fill in the gaps. e.g.:
- get_account_history - ideally could return more than 100 items
- get_full_blocks(start, limit) - would be nice to have this. blocks can only be pulled 1 at a time
- Returning virtual ops where expected (get_block?). Many virtual ops are only seen in get_account_history, and never show up in get_block
- Governance features: important for this data to be as rich as possible.. for example, the ability to query how much a worker has been paid *in total*. Currently we can only pull the "un-withdrawn" balance object.
- Would be nice if proposals were not removed from the db, until we get the plugin architecture for this stuff. And to be able to see if a proposal was in actually approved or not, after the fact.
I could come up with a lot of "nice-to-haves" but in the short term just exposing as much data as possible helps a lot.
-
There's this also:
https://github.com/cryptonomex/graphene/issues/339
I am actually going to submit a pull request for it (hopefully tonight) for the first 3 ones.
The last one (blockchain_get_witness_slot_records) required some sort of optional indexing in bts1, and I'm not sure it is implemented in graphene. A workaround (for me) would be to allow to query blocks in batch, as suggested by roadscape, and/or have a way to know which block was produced/missed by a witness.
-
Oh yes - another point:
*) Make a single point of entry into the system for API calls, rather than having the API commands duplicated (but with different functionality) between the cli_wallet and the witness_node.
-
pull request here: https://github.com/cryptonomex/graphene/pull/462
-
I'm not sure if this is an API improvement or something that is already there but not accessible from the wallet. It would be nice to make cold storage simple.
-
I'd like to allow plugins to define API calls, and configuration file options to specify which plugins to enable, and which plugins require login to access their API calls.
- It allows us to break the API up into smaller units. Currently database_api is huge because we have to put everything accessible without login in there.
- It allows a path for API upgrade. If the old API and new API are plugins which can be enabled / disabled by a specific site, then a site can upgrade the API on their schedule, not ours.
- It allows different security profiles. For example, we could create a "HTTP plugin" which makes login and secure API calls of other plugins accessible over HTTP. Sites that want to interface with the wallet over HTTP can use that plugin (and then it is on them to ensure the wallet is properly firewalled, containerised or otherwise isolated), while the default config has the plugin disabled for security reasons.
Related issues: #245, #246, #422.
-
I also think the new API should have a single object type for input and output, and do input parsing and output annotation by reflection.
For example:
- On the input side, we want API to accept an asset name wherever we accept an asset ID.
- On the output side, whenever an asset is returned by API, we return a 3-field object of the form {"sym" : symbol, "prec" : precision, "amount" : x}
We currently accomplish this by requiring each asset method to take asset as a string and then manually call parsing functions. Which is done inconsistently.
A better approach is to have every API call take an input struct, then use reflection to find any (potentially deeply nested) JSON fragment of the incoming object which has an asset_id_type, essentially moving the responsibility to parse asset ID's from the API method to the API framework (where "API framework" means some new, Graphene-specific glue code, as it will have to be aware of chain_db).
Likewise, output annotation requirements like the second requirement above can be handled similarly: Any place the output object has an asset_id_type, the framework automatically annotates it with the name and precision of the asset. Or possibly instead fills in a secondary "asset LUT" in the response containing such information for all assets (which makes the response smaller).
Accounts could obviously similarly benefit.
-
Also on my wishlist for the API: Generate a JSON blob representing full type information for each API call, which can be used to auto-generate language bindings for the API.
You can think of this as like splitting James's JSON operation serializer into a frontend which reflects all Graphene types, and a backend which outputs various useful JS code, such as a JS class for all C++ structs. The frontend would then output a JSON blob which is taken by the backend.
I'm pretty sure JS language object model is sufficiently powerful that the backend could be completely written in JS as a generic library, which would take the JSON blob and construct the classes at runtime. Python could be done similarly.
-
Make a single point of entry into the system for API calls, rather than having the API commands duplicated (but with different functionality) between the cli_wallet and the witness_node.
I think what needs to happen is that API calls need to be automatically forwarded from the cli_wallet to the witness_node.
This ties into my previous points, it's better if we have ways to refer to API methods, or groups of API methods, in a config file, so we'd be able to configure enabling/disabling this forwarding on a per-call or per-API basis. And if the cli_wallet can ask the witness_node for type information on all API methods it supports, then it can improve the forwarding, even if the cli_wallet is not implemented in the same language as the backend.
Which leads me to the next item on my wishlist, writing a cli_wallet in Python so you can use proper scripting instead of the very clunky transaction_builder for creating proposed transactions, but that's sort-of off-topic. There's a lot I'd want to do to improve Graphene -- doing it all would probably take a year or two and several junior devs.
-
To the developers, was this feedback in line with your thinking?
-
To the developers, was this feedback in line with your thinking?
Yes, @theoretical nailed it.. all great ideas, but as he said it will take time & resources to implement "properly".
So I don't mind if the devs take a few shortcuts in the short-term before re-architecting again :)
-
Unfortunately, the feedback is not helpful. I want specific requests INPUT ARGUMENTS and RETURN VALUES.
The API already allows you to do everything or can easily be wrapped in one that can. Examples are help, vague descriptions are not.
-
Unfortunately, the feedback is not helpful. I want specific requests INPUT ARGUMENTS and RETURN VALUES.
The API already allows you to do everything or can easily be wrapped in one that can. Examples are help, vague descriptions are not.
@monsterer @Xeldal @lil_jay890 @xeroc @roadscape @wackou @theoretical
Just a headsup, felt this was an important thread.
-
Unfortunately, the feedback is not helpful. I want specific requests INPUT ARGUMENTS and RETURN VALUES.
The API already allows you to do everything or can easily be wrapped in one that can. Examples are help, vague descriptions are not.
I'm not sure if this quite falls under "Better API" since what I think is needed the most of a wiki. Let me give you an example why. Where is is easily stated that for example, consider when I call get_account_history where's a documentation telling me what op.op[0] (the type of history data) represents? What's telling me that op.amount.amount needs to be normalized to its base-10 representation by querying get_asset with op.amount.asset_id to determine the precision value, then I need to compute op.amount.amount / 10^(precision)?
To a new developer working with bitshares all of this seems very vague and confusing, and answers to these kinds of questions aren't very responsive, it takes days, maybe weeks before getting a proper answer posting on the forums.
-
Unfortunately, the feedback is not helpful. I want specific requests INPUT ARGUMENTS and RETURN VALUES.
The API already allows you to do everything or can easily be wrapped in one that can. Examples are help, vague descriptions are not.
I'm not sure if this quite falls under "Better API" since what I think is needed the most of a wiki. Let me give you an example why. Where is is easily stated that for example, consider when I call get_account_history where's a documentation telling me what op.op[0] (the type of history data) represents? What's telling me that op.amount.amount needs to be normalized to its base-10 representation by querying get_asset with op.amount.asset_id to determine the precision value, then I need to compute op.amount.amount / 10^(precision)?
To a new developer working with bitshares all of this seems very vague and confusing, and answers to these kinds of questions aren't very responsive, it takes days, maybe weeks before getting a proper answer posting on the forums.
There is something to be said for providing helpers to do all of this. Right now the "API" simply returns internal implementation details which means understanding those details. I will try to improve this aspect.
-
There is something to be said for providing helpers to do all of this. Right now the "API" simply returns internal implementation details which means understanding those details. I will try to improve this aspect.
I'm all for convenience as long as overhead is negligible. Please keep the API calls fast :)
Unfortunately, the feedback is not helpful. I want specific requests INPUT ARGUMENTS and RETURN VALUES.
something like:
uint32_t get_block_height()
vector<signed_block> list_blocks(uint32_t lower_bound, limit = 100)
vector<operation_detail> get_account_history(string account, operation_history_id_type lower_bound, limit = 100)
vector<operation_detail> list_operations(operation_history_id_type lower_bound, limit = 100)
less practical as an api call:
graphene::chain::asset get_total_worker_pay(worker_id_type id) - how much the worker has received in total
bool get_final_proposal_state(proposal_id_type id) - to retroactively look at the final results of a proposed op
string suggest_fortune() - generates a random fortune cookie message using a mad-lib algo
-
vector<string> list_asset_owners(asset_id_type asset_id, string lower_bound, limit=100)
return asset owners list so that we can calculate asset distribution for things like dividend.
-
vector<string> list_asset_owners(asset_id_type asset_id, string lower_bound, limit=100)
return asset owners list so that we can calculate asset distribution for things like dividend.
http://cryptofresh.com/api/holders?asset=BROWNIE.PTS shout out to @roadscape (we requested this for our sharedrop system).
-
http://cryptofresh.com/api/holders?asset=BROWNIE.PTS shout out to @roadscape (we requested this for our sharedrop system).
Actually I've asked @roadscape and got his kind reply as below:
you will need to iterate through the accounts and call list_account_balances for each one. You can use list_accounts to get all the accounts, but you'll need to paginate through the results.
That's why I ask for list_asset_owners to filter the accounts. I thought of returning balances also but for simplicity I think list_asset_owners is enough to get started.
-
Please take a look at BITMEX. Their view API is on front page and its very comprehensive.
https://www.bitmex.com/app/restAPI
-
http://cryptofresh.com/api/holders?asset=BROWNIE.PTS shout out to @roadscape (we requested this for our sharedrop system).
Actually I've asked @roadscape and got his kind reply as below:
you will need to iterate through the accounts and call list_account_balances for each one. You can use list_accounts to get all the accounts, but you'll need to paginate through the results.
That's why I ask for list_asset_owners to filter the accounts. I thought of returning balances also but for simplicity I think list_asset_owners is enough to get started.
Yeah, I agree that if we could (easily) determine share holders and their balances accurately then this would be ideal, but for the time being the cached data roadscapes api produces is sufficient (at least for my own purposes). I think in terms of time-complexity (performance), balances should be returned as well. It'd be a waste to determine the owners list, and not return their balances at the same time.
-
Unfortunately, the feedback is not helpful. I want specific requests INPUT ARGUMENTS and RETURN VALUES.
The API already allows you to do everything or can easily be wrapped in one that can. Examples are help, vague descriptions are not.
You must be kidding! theoretical's feedback is solid and very useful. It helps for making the API consistent and far easier to learn. It is simply a matter of providing a more standardized format for parameters and return values. With a little thought the API can be structured FAR better than it is. I've seen plenty of API's for lots of applications in my time (like performance critical CAD APIs for circuit representation for example) but quality DOES take time. From what I see everybody is always in such a big rush, skipping standard engineering practices (like UI mockups and use case testing with real users), and we can see the results of that approach.
I also think the new API should have a single object type for input and output, and do input parsing and output annotation by reflection.
For example:
- On the input side, we want API to accept an asset name wherever we accept an asset ID.
- On the output side, whenever an asset is returned by API, we return a 3-field object of the form {"sym" : symbol, "prec" : precision, "amount" : x}
We currently accomplish this by requiring each asset method to take asset as a string and then manually call parsing functions. Which is done inconsistently.
A better approach is to have every API call take an input struct, then use reflection to find any (potentially deeply nested) JSON fragment of the incoming object which has an asset_id_type, essentially moving the responsibility to parse asset ID's from the API method to the API framework (where "API framework" means some new, Graphene-specific glue code, as it will have to be aware of chain_db).
Likewise, output annotation requirements like the second requirement above can be handled similarly: Any place the output object has an asset_id_type, the framework automatically annotates it with the name and precision of the asset. Or possibly instead fills in a secondary "asset LUT" in the response containing such information for all assets (which makes the response smaller).
Accounts could obviously similarly benefit.
Not sure about the level of standardization possible, or the use of reflection (that has a negative performance impact), but looking at the CLI wallet API tells me no attempt was made to group similar calls based on input or output parameters. I doubt it's feasible to have a single input or output structure, but I do believe it could be improved from the plethora of variants I see.
Make a single point of entry into the system for API calls, rather than having the API commands duplicated (but with different functionality) between the cli_wallet and the witness_node.
I think what needs to happen is that API calls need to be automatically forwarded from the cli_wallet to the witness_node.
This ties into my previous points, it's better if we have ways to refer to API methods, or groups of API methods, in a config file, so we'd be able to configure enabling/disabling this forwarding on a per-call or per-API basis. And if the cli_wallet can ask the witness_node for type information on all API methods it supports, then it can improve the forwarding, even if the cli_wallet is not implemented in the same language as the backend.
Which leads me to the next item on my wishlist, writing a cli_wallet in Python so you can use proper scripting instead of the very clunky transaction_builder for creating proposed transactions, but that's sort-of off-topic. There's a lot I'd want to do to improve Graphene -- doing it all would probably take a year or two and several junior devs.
I don't see any reason that can't be done, though it would need to be evaluated for it's impact on performance.
Also on my wishlist for the API: Generate a JSON blob representing full type information for each API call, which can be used to auto-generate language bindings for the API.
You can think of this as like splitting James's JSON operation serializer into a frontend which reflects all Graphene types, and a backend which outputs various useful JS code, such as a JS class for all C++ structs. The frontend would then output a JSON blob which is taken by the backend.
I'm pretty sure JS language object model is sufficiently powerful that the backend could be completely written in JS as a generic library, which would take the JSON blob and construct the classes at runtime. Python could be done similarly.
This is the best piece of feedback theoretical provided. Not only will this improve the API, it will also improve it's adoption by eliminating language barriers. I've been programming in JScript since it was invented back in the 90s, using it to build database driven web aps on the server when everybody else was using cold fusion or Visual Basic (yuk!). JScript (ECMA or Javascript, basically all the same) is a robust, touring complete and object oriented scripting language efficient and adequate for just about any task a script language is suitable for. Witness just how versatile JSON is. It is a concise, easy to read, highly malleable interface notation that has replaced other contenders like XML and YAML for many uses.
I find the CLI API cumbersome, difficult to learn and not very well engineered.
About a month ago I did a bit of research on the pros and cons of angular vs. react-js, and learned that node.js is a package manager for javascript code. I like the improvements react-js has made over angular, it really helps the code readability a lot as well as making it easier to write UI code. It looked like a natural fit and helps to overcome limitations of CSS, for one thing extending procedural functionality to the declarative only CSS space. I was a bit skeptical of making all CSS inline references, but I see the benefits of that with react.