Author Topic: Better API - Please Help Define It  (Read 9503 times)

0 Members and 1 Guest are viewing this topic.

Offline Thom

Unfortunately, the feedback is not helpful.  I want specific requests   INPUT ARGUMENTS and RETURN VALUES.   

The API already allows you to do everything or can easily be wrapped in one that can.  Examples are help, vague descriptions are not.

You must be kidding! theoretical's feedback is solid and very useful. It helps for making the API consistent and far easier to learn. It is simply a matter of providing a more standardized format for parameters and return values. With a little thought the API can be structured FAR better than it is. I've seen plenty of API's for lots of applications in my time (like performance critical CAD APIs for circuit representation for example) but quality DOES take time. From what I see everybody is always in such a big rush, skipping standard engineering practices (like UI mockups and use case testing with real users), and we can see the results of that approach.

I also think the new API should have a single object type for input and output, and do input parsing and output annotation by reflection.

For example:

  • On the input side, we want API to accept an asset name wherever we accept an asset ID.
  • On the output side, whenever an asset is returned by API, we return a 3-field object of the form {"sym" : symbol, "prec" : precision, "amount" : x}

We currently accomplish this by requiring each asset method to take asset as a string and then manually call parsing functions.  Which is done inconsistently.

A better approach is to have every API call take an input struct, then use reflection to find any (potentially deeply nested) JSON fragment of the incoming object which has an asset_id_type, essentially moving the responsibility to parse asset ID's from the API method to the API framework (where "API framework" means some new, Graphene-specific glue code, as it will have to be aware of chain_db).

Likewise, output annotation requirements like the second requirement above can be handled similarly:  Any place the output object has an asset_id_type, the framework automatically annotates it with the name and precision of the asset.  Or possibly instead fills in a secondary "asset LUT" in the response containing such information for all assets (which makes the response smaller).

Accounts could obviously similarly benefit.

Not sure about the level of standardization possible, or the use of reflection (that has a negative performance impact), but looking at the CLI wallet API tells me no attempt was made to group similar calls based on input or output parameters. I doubt it's feasible to have a single input or output structure, but I do believe it could be improved from the plethora of variants I see.

Make a single point of entry into the system for API calls, rather than having the API commands duplicated (but with different functionality) between the cli_wallet and the witness_node.

I think what needs to happen is that API calls need to be automatically forwarded from the cli_wallet to the witness_node.

This ties into my previous points, it's better if we have ways to refer to API methods, or groups of API methods, in a config file, so we'd be able to configure enabling/disabling this forwarding on a per-call or per-API basis.  And if the cli_wallet can ask the witness_node for type information on all API methods it supports, then it can improve the forwarding, even if the cli_wallet is not implemented in the same language as the backend.

Which leads me to the next item on my wishlist, writing a cli_wallet in Python so you can use proper scripting instead of the very clunky transaction_builder for creating proposed transactions, but that's sort-of off-topic.  There's a lot I'd want to do to improve Graphene -- doing it all would probably take a year or two and several junior devs.

I don't see any reason that can't be done, though it would need to be evaluated for it's impact on performance.


Also on my wishlist for the API:  Generate a JSON blob representing full type information for each API call, which can be used to auto-generate language bindings for the API.

You can think of this as like splitting James's JSON operation serializer into a frontend which reflects all Graphene types, and a backend which outputs various useful JS code, such as a JS class for all C++ structs.  The frontend would then output a JSON blob which is taken by the backend.

I'm pretty sure JS language object model is sufficiently powerful that the backend could be completely written in JS as a generic library, which would take the JSON blob and construct the classes at runtime.  Python could be done similarly.

This is the best piece of feedback theoretical provided. Not only will this improve the API, it will also improve it's adoption by eliminating language barriers. I've been programming in JScript since it was invented back in the 90s, using it to build database driven web aps on the server when everybody else was using cold fusion or Visual Basic (yuk!). JScript (ECMA or Javascript, basically all the same) is a robust, touring complete and object oriented scripting language efficient and adequate for just about any task a script language is suitable for. Witness just how versatile JSON is. It is a concise, easy to read, highly malleable interface notation that has replaced other contenders like XML and YAML for many uses.

I find the CLI API cumbersome, difficult to learn and not very well engineered.

About a month ago I did a bit of research on the pros and cons of angular vs. react-js, and learned that node.js is a package manager for javascript code. I like the improvements react-js has made over angular, it really helps the code readability a lot as well as making it easier to write UI code. It looked like a natural fit and helps to overcome limitations of CSS, for one thing extending procedural functionality to the declarative only CSS space. I was a bit skeptical of making all CSS inline references, but I see the benefits of that with react.
Injustice anywhere is a threat to justice everywhere - MLK |  Verbaltech2 Witness Reports: https://bitsharestalk.org/index.php/topic,23902.0.html

Offline hybridd

  • Full Member
  • ***
  • Posts: 164
    • View Profile
  • BitShares: hybr1d
http://cryptofresh.com/api/holders?asset=BROWNIE.PTS shout out to @roadscape (we requested this for our sharedrop system).

Actually I've asked @roadscape and got his kind reply as below:
you will need to iterate through the accounts and call list_account_balances for each one. You can use list_accounts to get all the accounts, but you'll need to paginate through the results.
That's why I ask for list_asset_owners to filter the accounts. I thought of returning balances also but for simplicity I think list_asset_owners is enough to get started.
Yeah, I agree that if we could (easily) determine share holders and their balances accurately then this would be ideal, but for the time being the cached data roadscapes api produces is sufficient (at least for my own purposes). I think in terms of time-complexity (performance), balances should be returned as well. It'd be a waste to determine the owners list, and not return their balances at the same time.
http://sharebits.io - #sharebits "person" amount asset - Start tipping on bitsharestalk today!
Developer @ Freebie

Offline mike623317

  • Hero Member
  • *****
  • Posts: 637
    • View Profile
Please take a look at BITMEX. Their view API is on front page and its very comprehensive.

https://www.bitmex.com/app/restAPI

Offline botfund

  • Full Member
  • ***
  • Posts: 174
    • View Profile
  • BitShares: botfund
http://cryptofresh.com/api/holders?asset=BROWNIE.PTS shout out to @roadscape (we requested this for our sharedrop system).

Actually I've asked @roadscape and got his kind reply as below:
you will need to iterate through the accounts and call list_account_balances for each one. You can use list_accounts to get all the accounts, but you'll need to paginate through the results.
That's why I ask for list_asset_owners to filter the accounts. I thought of returning balances also but for simplicity I think list_asset_owners is enough to get started.

Offline hybridd

  • Full Member
  • ***
  • Posts: 164
    • View Profile
  • BitShares: hybr1d
vector<string> list_asset_owners(asset_id_type asset_id, string lower_bound, limit=100)
  return asset owners list so that we can calculate asset distribution for things like dividend.

http://cryptofresh.com/api/holders?asset=BROWNIE.PTS shout out to @roadscape (we requested this for our sharedrop system).
http://sharebits.io - #sharebits "person" amount asset - Start tipping on bitsharestalk today!
Developer @ Freebie

Offline botfund

  • Full Member
  • ***
  • Posts: 174
    • View Profile
  • BitShares: botfund
vector<string> list_asset_owners(asset_id_type asset_id, string lower_bound, limit=100)
  return asset owners list so that we can calculate asset distribution for things like dividend.

Offline roadscape

There is something to be said for providing helpers to do all of this.  Right now the "API" simply returns internal implementation details which means understanding those details.   I will try to improve this aspect.

I'm all for convenience as long as overhead is negligible. Please keep the API calls fast :)

Unfortunately, the feedback is not helpful.  I want specific requests   INPUT ARGUMENTS and RETURN VALUES.   

something like:
uint32_t  get_block_height()
vector<signed_block> list_blocks(uint32_t lower_bound, limit = 100)
vector<operation_detail> get_account_history(string account, operation_history_id_type lower_bound, limit = 100)
vector<operation_detail> list_operations(operation_history_id_type lower_bound, limit = 100)


less practical as an api call:
graphene::chain::asset get_total_worker_pay(worker_id_type id) - how much the worker has received in total
bool get_final_proposal_state(proposal_id_type id) - to retroactively look at the final results of a proposed op
string suggest_fortune() - generates a random fortune cookie message using a mad-lib algo
http://cryptofresh.com  |  witness: roadscape

Offline bytemaster

Unfortunately, the feedback is not helpful.  I want specific requests   INPUT ARGUMENTS and RETURN VALUES.   

The API already allows you to do everything or can easily be wrapped in one that can.  Examples are help, vague descriptions are not.

I'm not sure if this quite falls under "Better API" since what I think is needed the most of a wiki. Let me give you an example why. Where is is easily stated that for example, consider when I call get_account_history where's a documentation telling me what op.op[0] (the type of history data) represents? What's telling me that op.amount.amount needs to be normalized to its base-10 representation by querying get_asset with op.amount.asset_id to determine the precision value, then I need to compute op.amount.amount / 10^(precision)?

To a new developer working with bitshares all of this seems very vague and confusing, and answers to these kinds of questions aren't very responsive, it takes days, maybe weeks before getting a proper answer posting on the forums.

There is something to be said for providing helpers to do all of this.  Right now the "API" simply returns internal implementation details which means understanding those details.   I will try to improve this aspect.
For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.

Offline hybridd

  • Full Member
  • ***
  • Posts: 164
    • View Profile
  • BitShares: hybr1d
Unfortunately, the feedback is not helpful.  I want specific requests   INPUT ARGUMENTS and RETURN VALUES.   

The API already allows you to do everything or can easily be wrapped in one that can.  Examples are help, vague descriptions are not.

I'm not sure if this quite falls under "Better API" since what I think is needed the most of a wiki. Let me give you an example why. Where is is easily stated that for example, consider when I call get_account_history where's a documentation telling me what op.op[0] (the type of history data) represents? What's telling me that op.amount.amount needs to be normalized to its base-10 representation by querying get_asset with op.amount.asset_id to determine the precision value, then I need to compute op.amount.amount / 10^(precision)?

To a new developer working with bitshares all of this seems very vague and confusing, and answers to these kinds of questions aren't very responsive, it takes days, maybe weeks before getting a proper answer posting on the forums.
http://sharebits.io - #sharebits "person" amount asset - Start tipping on bitsharestalk today!
Developer @ Freebie

TravelsAsia

  • Guest
Unfortunately, the feedback is not helpful.  I want specific requests   INPUT ARGUMENTS and RETURN VALUES.   

The API already allows you to do everything or can easily be wrapped in one that can.  Examples are help, vague descriptions are not.

@monsterer @Xeldal @lil_jay890 @xeroc @roadscape @wackou @theoretical

Just a headsup, felt this was an important thread.

Offline bytemaster

Unfortunately, the feedback is not helpful.  I want specific requests   INPUT ARGUMENTS and RETURN VALUES.   

The API already allows you to do everything or can easily be wrapped in one that can.  Examples are help, vague descriptions are not.
For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.

Offline roadscape

To the developers, was this feedback in line with your thinking?

Yes, @theoretical nailed it.. all great ideas, but as he said it will take time & resources to implement "properly".

So I don't mind if the devs take a few shortcuts in the short-term before re-architecting again :)
http://cryptofresh.com  |  witness: roadscape

TravelsAsia

  • Guest
To the developers, was this feedback in line with your thinking?

Offline theoretical

Make a single point of entry into the system for API calls, rather than having the API commands duplicated (but with different functionality) between the cli_wallet and the witness_node.

I think what needs to happen is that API calls need to be automatically forwarded from the cli_wallet to the witness_node.

This ties into my previous points, it's better if we have ways to refer to API methods, or groups of API methods, in a config file, so we'd be able to configure enabling/disabling this forwarding on a per-call or per-API basis.  And if the cli_wallet can ask the witness_node for type information on all API methods it supports, then it can improve the forwarding, even if the cli_wallet is not implemented in the same language as the backend.

Which leads me to the next item on my wishlist, writing a cli_wallet in Python so you can use proper scripting instead of the very clunky transaction_builder for creating proposed transactions, but that's sort-of off-topic.  There's a lot I'd want to do to improve Graphene -- doing it all would probably take a year or two and several junior devs.
« Last Edit: November 25, 2015, 04:35:30 pm by theoretical »
BTS- theoretical / PTS- PZxpdC8RqWsdU3pVJeobZY7JFKVPfNpy5z / BTC- 1NfGejohzoVGffAD1CnCRgo9vApjCU2viY / the delegate formerly known as drltc / Nothing said on these forums is intended to be legally binding / All opinions are my own unless otherwise noted / Take action due to my posts at your own risk

Offline theoretical


Also on my wishlist for the API:  Generate a JSON blob representing full type information for each API call, which can be used to auto-generate language bindings for the API.

You can think of this as like splitting James's JSON operation serializer into a frontend which reflects all Graphene types, and a backend which outputs various useful JS code, such as a JS class for all C++ structs.  The frontend would then output a JSON blob which is taken by the backend.

I'm pretty sure JS language object model is sufficiently powerful that the backend could be completely written in JS as a generic library, which would take the JSON blob and construct the classes at runtime.  Python could be done similarly.
BTS- theoretical / PTS- PZxpdC8RqWsdU3pVJeobZY7JFKVPfNpy5z / BTC- 1NfGejohzoVGffAD1CnCRgo9vApjCU2viY / the delegate formerly known as drltc / Nothing said on these forums is intended to be legally binding / All opinions are my own unless otherwise noted / Take action due to my posts at your own risk