Author [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] Topic: Maintain blockchain balances in DB  (Read 553 times)

Offline ElMato

  • Sr. Member
  • ****
  • Posts: 280
    • View Profile
Maintain blockchain balances in DB
« on: November 04, 2014, 11:24:11 PM »

Can someone review this pseudo code and tell me if the fork handling is right?

Quote
def function_called_periodically()

    # My last block imported in DB
    my_last_block = get_last_block_from_db()

    # Network last block (head of master chain)
    network_last_block = blockchain_get_block_count()

    # Is network ahead of us?
    while network_last_block > my_last_block

      # Get next block based on my last imported block
      next_block = blockchain_get_block(my_last_block+1)

      # I'm on the main chain?
      if next_block['previous'] != my_last_block
        # NO => my block is on a forked chain, undo changes.
        undo_balances_changed_in_block(my_last_block)
        delete_from_db(my_last_block)
        my_last_block = get_last_block_from_db()
      else
        # YES => Import
        my_last_block = import_new_block_in_db_and_update_balances(next_block)

Offline ElMato

  • Sr. Member
  • ****
  • Posts: 280
    • View Profile
Re: Maintain blockchain balances in DB
« Reply #1 on: December 16, 2014, 08:35:04 AM »
Bump.

Our fork handling code (the one that is running on bsw.latincoin.com backend) wasn't able to cope with the forks the network suffered this days.

We had to reimport the complete blockchain.
I would like if someone can look at the pseudo-code and tell me if they see something wrong.

Offline vikram

Re: Maintain blockchain balances in DB
« Reply #2 on: December 16, 2014, 11:21:52 PM »
Code: [Select]
# Is network ahead of us?
    while network_last_block > my_last_block

What happens if network_last_block <= my_last_block? This happened during the fork on 12/12 when delegates switched back to support the minority (shorter) fork.

Offline ElMato

  • Sr. Member
  • ****
  • Posts: 280
    • View Profile
Re: Maintain blockchain balances in DB
« Reply #3 on: December 17, 2014, 07:38:50 AM »
Thanks for the reply Vikram.

The first imported block in our db is the block number 900000.
I don't think the fork got that far.
 

Offline vikram

Re: Maintain blockchain balances in DB
« Reply #4 on: December 17, 2014, 11:37:06 PM »
I'm not sure I understand; on 12/12 a fork happened around block 1246469. After some time, there were two chains: most delegates and other users who upgraded were on the longer chain with head block something like 1249000, and the rest of users were on a shorter chain with head block something like 1247000. Delegates then chose to switch chains and go back to supporting the shorter chain, so it is possible that you could have seen your head block become lower if you had upgraded on that day. Can you give more details on how your fork handling failed that day?

Offline ElMato

  • Sr. Member
  • ****
  • Posts: 280
    • View Profile
Re: Maintain blockchain balances in DB
« Reply #5 on: December 19, 2014, 08:44:14 AM »
Thanks again for your reply Vikram.

I will try to be more specific.

We are maintaining in a mysql DB every balance in the blockchain.
Since blockchain_list_address_balances was not available by the time we started, we maintain the balances looking for every deposit/withdraw operation inside every transaction.

For every deposit we add the amount to the corresponding balance, and for every withdraw we subtract.

So, when we find a fork we go back to the common ancestor block applying withdraw and deposits (from the forked chain) the other way round. (subtract for deposits, add for withdraw)

The question is.

Is this the correct way to handle a chain fork?
What is the bitshares_client doing when he has to revert a fork?

ps: we will be changing to blockchain_list_address_balances/blockchain_list_address_transactions as soon as they are available




Offline bytemaster

Re: Maintain blockchain balances in DB
« Reply #6 on: December 19, 2014, 01:08:53 PM »
That process will not capture the result of market operations. 
For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.

Offline ElMato

  • Sr. Member
  • ****
  • Posts: 280
    • View Profile
Re: Maintain blockchain balances in DB
« Reply #7 on: December 19, 2014, 03:53:30 PM »
Thanks bytemaster.

I know that limitation.
In general terms it will only capture results from operations declared in a transaction. (no market, no yield, etc)

We are not thinking in let the user trade inside this wallet, in case he wants to exchange bitUSD for bitCNY the wallet provider can give that service.

------
I just want to know what is the correct way to handle a chain fork for someone that is importing states using the rpc interface.

Offline vikram

Re: Maintain blockchain balances in DB
« Reply #8 on: December 20, 2014, 07:28:10 PM »
Thanks again for your reply Vikram.

I will try to be more specific.

We are maintaining in a mysql DB every balance in the blockchain.
Since blockchain_list_address_balances was not available by the time we started, we maintain the balances looking for every deposit/withdraw operation inside every transaction.

For every deposit we add the amount to the corresponding balance, and for every withdraw we subtract.

So, when we find a fork we go back to the common ancestor block applying withdraw and deposits (from the forked chain) the other way round. (subtract for deposits, add for withdraw)

The question is.

Is this the correct way to handle a chain fork?
What is the bitshares_client doing when he has to revert a fork?

ps: we will be changing to blockchain_list_address_balances/blockchain_list_address_transactions as soon as they are available

At a high-level this is correct strategy and what the client does as well. When the client decides to switch to another fork, it will rewind the database one block at a time until it returns back to the point where the fork happened. Then it will begin applying the blocks one-at-a-time on the new fork until it is synced again. It is difficult to say what bugs could be in your system without understanding it in more detail.

 

Google+