Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - ElMato

Pages: 1 [2] 3 4 5 6 7 8 9 ... 20
Technical Support / Re: New API for witness_node
« on: December 26, 2016, 07:51:53 pm »
There is a lot more to be discussed but i want to get some feedback from the community.
Also, I would like to join efforts in this because all of us (projects on top of BitShares) need this kind of tools.

Technical Support / New API for witness_node
« on: December 26, 2016, 07:50:44 pm »
Continuing  the discussion on changing / updating the API of witness_node.

The BitShares blockchain has a more diverse STATE than the Bitcoin blockchain (UTXO/Address Balance).
We have a lot of entities: accounts, balances, assets, orders, feeds, etc.

We need maximum flexibility when querying the state.

The state doesn't have all the appropriated indexes or doesn't have the specific function to query that specific part of the state.

We needed to list all the holders of a specific asset.

The index was already there.

... ordered_unique< tag<by_asset_balance>,
               member<account_balance_object, asset_id_type, &account_balance_object::asset_type>,
               member<account_balance_object, share_type, &account_balance_object::balance>,
               member<account_balance_object, account_id_type, &account_balance_object::owner>

So, we just write the function using that index.

Code: [Select]
vector<account_asset_balance> database_api_impl::get_asset_holders( asset_id_type asset_id )const
   const auto& bal_idx = _db.get_index_type< account_balance_index >().indices().get< by_asset_balance >();
   const auto range = bal_idx.equal_range( boost::make_tuple( asset_id ) );
   vector<account_asset_balance> result;
   for( const account_balance_object& bal : boost::make_iterator_range( range.first, range.second ) )
     assert( bal.asset_type == asset_id );
     auto account = _db.find(bal.owner);

     account_asset_balance aab;       = account->name;
     aab.account_id = account->id;
     aab.amount     = bal.balance.value;


   return result;

So if we want to query the state in any imaginable way we have many options each one of them with some trade-offs.

1) Add more boost_multi_index_container for the state with the corresponding functions when needed.
     This will increase memory usage in all nodes since indexes occupy memory space.
     But can be set at compile time or with a custom plugin.

2) Replicate the state in a MySQL/neo4J database.
    Add a function to dump the FULL STATE of the object db.
    And then apply state changes (add/edit/delete objects).
    This process need to lock the block processing until we apply the state change in our underlying database.
    Otherwise if we miss a delta-diff our outside-state will be different from the real state.

    Eg: if we just connect to the WS interface, and wait for "object_changed" callback and affect the outside-state accordingly, then our process go down, the blockchain keeps updating objects, we reconnect later and .... we are in problems.

    We have been experimenting with this approach developing a plugin that will call a python function with the changed_objects content.

Code: [Select]
void db_update_plugin::plugin_initialize(const boost::program_options::variables_map& options)
  auto script_str ="python-script").as<std::string>();
  if ( !script_str.size() )


  my->_main_module = bp::import("__main__");
  my->_global      = my->_main_module.attr("__dict__");

  auto script_path =  bp::str(script_str);

  bp::exec_file(script_path, my->_global, my->_global);

  my->_handler = my->_global["Handler"]();

  database().changed_objects.connect([&]( const std::vector<graphene::db::object_id_type>& ids) {

    bp::list list_ids;
    bp::list list_values;

    for( auto id : ids ) {

      fc::variant vo;
      to_variant(id, vo);
      list_ids.append( vo.get_string() );

      const graphene::db::object *obj = database().find_object(id);
      if ( obj != nullptr ) {
        list_values.append( fc::json::to_string(obj->to_variant()) );
      } else {
        list_values.append( "" );

    bp::call_method<void>(my->_handler.ptr(), "changed_objects", list_ids, list_values);


    (We have to take care of possible forks in the chain? Or the changed_objects callback also tell us [in case of undo] which objects need to be deleted?) 

3) Create a pluggable db backend.
     Take graphene::db::object_database as the base, create an interface and implement object_database_mysql, object_database_neo4j ... etc.

General Discussion / Re: [Evaluation] Need for a better python library?
« on: December 21, 2016, 04:54:28 pm »
In fact the architecture of witness_node + graphene_gui is  good for private node & wallet, but not suitable for public wallet, it's too inefficient.
the client subscribe all chain block data, and process all low level data struct. the api node can't handle too many clients.

btsbots use a different architecture.
server side pre process the origin low level data, generate a new database suitable for btsbots's use case,  and provide these data through meteor's ddp protocol.
no more api request for witness_node are needed, when client get all the public data include: market orders, operation history, encrypt memo
in fact only broadcast the transaction need a new api request for witness_node.
so I guess this way more better for public wallet, and can handle much more clients.

[member=4465]alt[/member], I agree with your view but we need that API inside the witness_node.
Otherwise btsbots or any other application will have to rely on a custom backend.

What we are doing is similar to what you are doing but using GraphQL to make it generic.

Since the underlying database is an object database it makes sense to use a query language like that.

As an example this is how we query for an account balance + history.

Code: [Select]
query getAll($v1 : String!) {
  account(name:$v1) {
    balance {
      asset {
    history(start:0, limit:20) {
      block {
      ... on NoDetailOp {
      ... on Transfer {
        from {
        to {
        memo {
        amount {
          asset {
        fee {
          asset {

In my opionion it would be amazing to have the witnes_node expose a GraphQL API.
Because In that case you just remove all the "special" functions defined in database_api.hpp/database_api.cpp and just rely in the query language to perform all your data extraction needs.

There is a C++ library that process a graphql query.

The remaining work would be to process the AST and resolve using the graphene::chain library. 

General Discussion / Re: more professional price feed?
« on: December 21, 2016, 04:41:10 am »
I don't know it's because of  a wrong configure file or a bug of the script.
but this is not the first time it happened. I can't remember how many times I have mentioned the witness.
include this forum and telegram.
nobody want to take responsibility  for this error, so I have to remove the witness every time this happen.
I can find some old post about this:,20704.msg267158.html#msg267158,21315.msg276995.html#msg276995

I get this type of reply every time:
many witness publish a wrong price

Since you seem to be an authority on the subject, could you please enlighten us witnesses what the right price is?

I can publish a price of 0.0245 CNY/BTS all week if you think that's better.

I have the same settings as DataSecurityNode,

  "CNY" : {
                    "metric" : "weighted",
                    "sources" : ["btc38",

Should we remove yunbi?
According to coinmarketcap  +80% of the volume is on btc38 and polo (BTS/BTC).

General Discussion / Re: BitShares administrative and maintenance tasks
« on: November 01, 2016, 07:18:53 am »
Hi Vikram! Welcome back to the BTS community!! I'm glad to see you around here again!

Sent from my XT1063 using Tapatalk

General Discussion / Re: btsbots wallet release v0.0.1
« on: October 28, 2016, 02:41:26 pm »
Welcome back alt!
Nice to have you around here again!

Sent from my XT1063 using Tapatalk

+5% will try it out

Sent from my XT1063 using Tapatalk

General Discussion / Re: Bitshares on the economist
« on: May 23, 2016, 06:56:58 am »
Again, lets use every possible opportunity to talk about BitShares whenever someone talks about TheDao or DAO (the concept).

Sent from my XT1063 using Tapatalk

General Discussion / Re: Claims of BM saying BitShares has failed
« on: May 23, 2016, 06:53:55 am »
The Dao is our friend, I'm using every possible situation to talk about BitShares thanks to TheDao

Sent from my XT1063 using Tapatalk

General Discussion / Re: BitARS - SmartCoin Petition
« on: May 13, 2016, 11:13:37 pm »
bitARS SmartCoin has been created and transferred to the committee.

We are now talking with the witnesses to start publishing feeds (and eventually equalize the feed after)

On Agu 14 we will be returning the BTS obtained via the cashback! 

Coming late to the discussion but i have been following this thread with much attention.

Here is what i think in a very disorganized way.

Its a delicate thing to change the offset parameter once its been in a certain value for a time (not so long time .. 5 months?)

Saying robbery its a bold argument since you can still trade bitCNY for BTS (or any other asset) at a fair price, the difference is that you now will get less (CNY value in) BTS if force settle.

[member=23]bitcrab[/member] is one of the major players pushing, developing and creating bitCNY (and the DEX, and BitShares as a whole) even after the problems they suffered in the 1.0 -> 2.0 upgrade they still commit to the project and is something that we can't disregard easily.

Also they are doing 1:1 conversion (or similar) from bitCNY <-> CNY (alipay). So the value of bitCNY won't be affected in a significant way even if we move the offset 1%.

Are "normal" users of bitCNY doing force-settlement periodically? is a common usage pattern or they are using the gateway basically?

I also recall BM saying that the cheapest way to buy BTS without moving the price is doing a FS on a smartcoin ... so adding a little percent to the offset i think is fair.

Bottom line: I do support the change in the settlement offset from 0% to 1%, but we have votes .. so lets see what the community thinks.

Stakeholder Proposals / Re: Proxy: xeroc
« on: April 29, 2016, 07:53:22 pm »

 - Approved the bitARS worker since they seem to work hard to get an ARS smartcoin and only ask for the asset creation fee. It would have been slightly better if they used a multisignature account, but I trust elmato

I urge against doing so.  It is not a matter of trust but a matter of doing it right.  IMHO, this has to be done right and the account should _at least_ be a multisig one with a number of independent and respected community members holding the keys.

[member=7312]ElMato[/member]: To fix this, we could ask you to add the committee-account as a secondary account to your active AND owner key of the `elmato` account. Give your own keys and committee-account a weight of 1 and set a threshold of two .. After payout (e.g. the creation of the asset), the committee can then approve its own removal from your account permissions.
It has another advantage, namely, the whole committee can review the creation of the smartcoin and identify issuers prior to its creation.

Would that work for you? It seems you could get more approval if you do it that way.

We knew there was a trust issue involved if we created the worker, this is why we wanted the Committe to create it for us.

It does seem logical to have created the worker from a multisig account, however this was not brought forward (at least to me) and therefore we delegated [member=7312]ElMato[/member] to do the task since we thought he was the oldest and best known member of our team (he also runs an active witness node).

Having said this, I donĀ“t think there will be a problem in giving committe-account permissions until the asset is created and transferred, I will speak with the team and with [member=7312]ElMato[/member] to get this done as soon as possible.

[member=120]xeroc[/member], done.
elmato account now has:

committee-account added to owner authority, threshold 2.
committee-account added to active authority, threshold 1 (for some reason the change didn't go through, so there is a proposed transaction to change the threshold for the committee to approve)

General Discussion / Re: Working on BitShares..
« on: April 28, 2016, 04:12:50 am »
I am not worried either. We continue building ECHO and the SmartCoins POS based on BitShares. A stable feature set will help us to grew our business and attract new clients.

The code is open source and the blockchain is working fine. We do not need to worry. We would love to see more gateways, though to get in and out of BitShares, but that will come too.

Things are improving and more startups are working with BitShares. It is actually a good time!

I totally second that.
The code is open source, the blockchain is stable and we have the tools to run this DAC.

The products that we have (smartcoins and the dex primarily) are the only ones in the industry and still have a lot of potential.

Even if BM is a world class engineer and  it will be hard to replace him in case he is not involved 100% in Bitshares, I see this time as a call to us, the community, to take the lead, regroup and keep doing BitShares great.

I lost a lot of hard earned money because of not selling because I was for the long run, even with that I still see a lot of potential in the project and that's why I keep spending my time working on BitShares and promoting it.

Sent from my XT1063 using Tapatalk

For anyone that find it useful, the bash script below takes a 0.X json wallet backup and dumps "account name, owner pubkey, owner privkey".

Tested with a 07/2014 backup  (0.4.X?)
Tested with a 03/2015 backup  (0.8.X?)

Only one "rare" dependecy.
npm install -g json

For the rest it uses standard command line tools (openssl, sha512sum, xxd

pip install bitcoin (optional to have the privatekey in Wif format)

Code: [Select]
SHA512=$(echo -n $PASSWORD | sha512sum -)

cat $WALLET_FILE | json -c "this.type == 'account_record_type'" | json -c " == true" | json -a data.owner_key | while read -a line; do
  ENC_PRIVKEY=$(cat $WALLET_FILE | json -c "this.type == 'key_record_type'" | json -c " == '$PUBKEY'" | json -a data.encrypted_private_key)

  PRIVKEY=$(echo -n $ENC_PRIVKEY | xxd -r -p | openssl enc -d -aes-256-cbc -K $KEY -iv $IV | xxd -p | tr -d '\n')

  which pybtctool > /dev/null
  if [ $? -eq 0 ]; then
    PRIVKEY=$(echo -n $PRIVKEY | pybtctool -s encode_privkey wif)


General Discussion / Re: Important questions from the business perspective
« on: December 16, 2015, 03:12:48 pm »
[member=7312]ElMato[/member] - could u please give a short update on the limewallet project? thanks in advance :)

People, sorry for not being very active lately.

The delay in the new release of limewallet is just a matter of resources on our side.

We are doing consulting projects (some crypto related, some not) because we need funds to keep us alive and that allows us to keep working on the wallet.

Anyway, Im still working on the wallet (not at the speed i would like to) but we will be releasing a new version in the coming months.

Pages: 1 [2] 3 4 5 6 7 8 9 ... 20