Author Topic: [RESOLVED] Missing 30K - Help troubleshoot this.  (Read 6620 times)

0 Members and 1 Guest are viewing this topic.

Offline bitmeat

  • Hero Member
  • *****
  • Posts: 1116
    • View Profile
These fixes will be in the next release. You guys will have to do a full rescan (backup your wallet first, just in case) and we can see if it fixes your transaction histories.

Do you guys see any improvement using the latest version? https://github.com/dacsunlimited/bitsharesx/releases/tag/0.2.3

Great! Using wallet_account_history and I now see a correct balance with the missing transactions from before shown as UNKNOWN.

However in the UI the balance column in the transaction history is now completely blank. as in not even numbers are seen there.

I'm closing this issue. I hope exchanges don't ever get to miss a transaction, that would be unfortunate.

Offline vikram

These fixes will be in the next release. You guys will have to do a full rescan (backup your wallet first, just in case) and we can see if it fixes your transaction histories.

Do you guys see any improvement using the latest version? https://github.com/dacsunlimited/bitsharesx/releases/tag/0.2.3

Offline liondani

  • Hero Member
  • *****
  • Posts: 3737
  • Inch by inch, play by play
    • View Profile
    • My detailed info
  • BitShares: liondani
  • GitHub: liondani
Liondani this didn't work for me. I still have discrepancies. This only fixes transactions that stay pending forever. But if I've lost information about pending transaction it's gone for good.

that must be a priority issue for the team I guess...

Offline bitmeat

  • Hero Member
  • *****
  • Posts: 1116
    • View Profile
Liondani this didn't work for me. I still have discrepancies. This only fixes transactions that stay pending forever. But if I've lost information about pending transaction it's gone for good.

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12922
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Just a reminder for the reason of this:

TITAN transactions are costly and time consuming.. if you have a delegate running on the network you need to sign within 5-10 secs ... otherwise you miss your spot .. however if you have several 'other' accounts active TITAN scanning will probably not finish in time .. yet ... multithreading is on the way as BM told us somewhere :)

Offline liondani

  • Hero Member
  • *****
  • Posts: 3737
  • Inch by inch, play by play
    • View Profile
    • My detailed info
  • BitShares: liondani
  • GitHub: liondani
I have answerred that several times and it worked!

do this:

wallet_delegate_set_block_production ALL false

wallet_rescan_blockchain

wait 1-2 minutes (I think it does not matter )

wallet_delegate_set_block_production ALL true
« Last Edit: July 23, 2014, 06:24:26 pm by liondani »

Offline vikram

Great! Thanks for posting links to code!

Question:

Code: [Select]
new_entry = false;
 +                     // TODO: We should probably really have a map of asset ids to amounts per ledger entry
 +                     if( entry.amount.asset_id == amount.asset_id )
 +                         entry.amount += amount;
 +                     else if( entry.amount.amount == 0 )
 +                         entry.amount = amount;

Will you check this logic.

I see that this will fail when transferring assets other than the base asset, as I must have realized at some level before relegating the thought to the TODO there. Good catch! Will think how to address this.

Transactions can be almost arbitrarily complex but to keep meaningful records in our simple wallet we are making strong simplifying assumptions for now. We also have the problem that everyone already has a wallet using a certain format for storing their existing transaction records.

And also - are there tests associated with this code? I know it's a lot to ask for but we should have good integration tests that try all kinds of things.

E.g. have you thoroughly tested with multiple assets. Or able to simulate a missing transaction on the sending/receiving end, and see how the system behaves?

No tests yet, but I intend to write them when there is some time. Still many other fires to put out at the moment.

I have not thoroughly tested with multiple assets / market functionality yet, as is clear from the problem above. Missing transfers of the base asset on sending and receiving side are correctly reconstructed, however.


I also forgot to mention that in the OP, you seem to have been affected by problems with transaction scanning combined with an issue that causes transactions to be stuck pending. This is a known issue but we are still tracking down the cause. Sometimes they get unstuck after some random period of time, so be cautious about resending transactions.

Offline bitmeat

  • Hero Member
  • *****
  • Posts: 1116
    • View Profile
Great! Thanks for posting links to code!

Question:

Code: [Select]
new_entry = false;
 +                     // TODO: We should probably really have a map of asset ids to amounts per ledger entry
 +                     if( entry.amount.asset_id == amount.asset_id )
 +                         entry.amount += amount;
 +                     else if( entry.amount.amount == 0 )
 +                         entry.amount = amount;

Will you check this logic. And also - are there tests associated with this code? I know it's a lot to ask for but we should have good integration tests that try all kinds of things.

E.g. have you thoroughly tested with multiple assets. Or able to simulate a missing transaction on the sending/receiving end, and see how the system behaves?

Thanks! Cheers! Good job!

Offline vikram

I have been working on the transaction history and wallet rescanning code:

There were quite a number of problems with the wallet transaction scanning causing missing entries, incorrect balances, silently failing rescans, etc. In my testing so far, transaction history reconstruction works much better now. I am able to delete every transaction record (withdrawals, deposits, account operations, asset operations) from my wallet and it correctly brings them all back (sans any irrecoverable information due to TITAN). More testing is required but I think we might be able to fix your issues.

These fixes will be in the next release. You guys will have to do a full rescan (backup your wallet first, just in case) and we can see if it fixes your transaction histories.

Offline bytemaster

For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.

Offline Simeon II

  • Sr. Member
  • ****
  • Posts: 355
    • View Profile
Here is what is going on:

He has received all transactions, but when you "re-scan" you do not detect outgoing transactions. 

So when you cleared pending transactions you deleted your knowledge of those transactions from your wallet.   His transaction record is accurate, your balance is accurate, but your transaction record is missing those 2 transactions.

Are you saying that unless I keep my local hard drive safe, I have no way of ever reconstructing the missing transactions in the ledger?

I mean if I export my wallet, then some transactions occur, then my hard drive dies , will I be able to re-create my account from scratch from that older wallet?
(This happened to me by the way and I lost 400 PTS during donation period, because I thought Qt PTS used the same change address, so my backup didn't work). I also hope your change addresses are deterministic.

All private keys we generate are deterministic and can be regenerated.... as long as you have registered all of your accounts you will have an easy time recovering your balance, but your detailed account history could be gone for ever.  If you have unregistered accounts and do not know the public keys for them, then you will have to brute force scan with all of your keys and that could take an eternity (doable, but painful).   

For this reason I recommend registering all accounts on the blockchain *or* backing up all public keys for the unregistered accounts you do create.

This should be posted some place more prominent!

Offline bytemaster

Here is what is going on:

He has received all transactions, but when you "re-scan" you do not detect outgoing transactions. 

So when you cleared pending transactions you deleted your knowledge of those transactions from your wallet.   His transaction record is accurate, your balance is accurate, but your transaction record is missing those 2 transactions.

Are you saying that unless I keep my local hard drive safe, I have no way of ever reconstructing the missing transactions in the ledger?

I mean if I export my wallet, then some transactions occur, then my hard drive dies , will I be able to re-create my account from scratch from that older wallet?
(This happened to me by the way and I lost 400 PTS during donation period, because I thought Qt PTS used the same change address, so my backup didn't work). I also hope your change addresses are deterministic.

All private keys we generate are deterministic and can be regenerated.... as long as you have registered all of your accounts you will have an easy time recovering your balance, but your detailed account history could be gone for ever.   If you have unregistered accounts and do not know the public keys for them, then you will have to brute force scan with all of your keys and that could take an eternity (doable, but painful).   

For this reason I recommend registering all accounts on the blockchain *or* backing up all public keys for the unregistered accounts you do create.
For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.

Offline toast

  • Hero Member
  • *****
  • Posts: 4001
    • View Profile
  • BitShares: nikolai
Correct, told me that he *does* have the correct balance and is only missing the transaction for the 10k btsx


Quote
krabbypatty must have contacted you regarding this.

 I just want to add that while my incoming transaction are about ~130K my combined accounts  balance is 140K -pretty much where it should be, except I have no way to tell who paid the 10K difference.
Do not use this post as information for making any important decisions. The only agreements I ever make are informal and non-binding. Take the same precautions as when dealing with a compromised account, scammer, sockpuppet, etc.

Offline bitmeat

  • Hero Member
  • *****
  • Posts: 1116
    • View Profile
The transaction with no memo for 20K is missing on receive side... very interesting. 

Does his ledger balance out with his account balance after a rescan?

I've instructed him to do a rescan as well. I think he was able to send the balance, but never saw it in the ledger.

I think he is also missing the transaction with 10K one of the "2nd batch" transactions - there are 2 with the same memo, but he got only one - look at TX ID on the right

Offline bytemaster

The transaction with no memo for 20K is missing on receive side... very interesting. 

Does his ledger balance out with his account balance after a rescan?
For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.