BitShares Forum

Main => Technical Support => Topic started by: 38PTSWarrior on September 22, 2014, 11:36:29 am

Title: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 22, 2014, 11:36:29 am
Hi, I prepare for the worst, because even I know that the blockchain loses nothing it could be that I fuc.. made a mistake. I only have a wallet from 10 days ago but in the meantime I bought some bitBTC and bitUSD, which now do not show up anymore. I thought I saved the recent wallet to my usb, but I just pasted it in a folder and forgot to paste the folder to the usb. Then I installed ubuntu again over the old ubuntu.

:D

How are the chances doc?

E: Some other transactions are also missing. I pray to the sun and the universe, and the multiverses, tat there is a command that can help a brother now.
Title: Re: Balance missing because of old wallet file
Post by: xeroc on September 22, 2014, 01:19:05 pm
Check this out:
http://wiki.bitshares.org/index.php/RecoveringFunds
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 22, 2014, 04:07:59 pm
Thank you, I was too nervous for checking the wiki. Now I have something to do for the evening.
Title: Re: Balance missing because of old wallet file
Post by: xeroc on September 22, 2014, 05:19:22 pm
no need to be nervous .. your funds are still on the blockchain and as long as you have a backup of your wallet your funds will be accessible again ..
Title: Re: Balance missing because of old wallet file
Post by: liondani on September 22, 2014, 08:18:20 pm
any news bro?
Title: Re: Balance missing because of old wallet file
Post by: serejandmyself on September 22, 2014, 08:34:10 pm
any news bro?

yes, im quite interested also how this gets solved.... please do update the topic
Title: Re: Balance missing because of old wallet file
Post by: liondani on September 22, 2014, 09:21:19 pm
whatever your conclusions are right now... keep calm... a solution exists 100%! Just keep safe your old json file.
please check your "wallets" directory if you use windows:  users/yourusername/appdata/roaming/bitsharesx/wallets/   you will find there json backup files because the wallet automatically creates backups since one month or so....
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 23, 2014, 09:54:26 am
Oh hi, I tried the commands from the wiki (with the 1000 regenerated keys) and after rescanning some transactions appeared but still not shown in the balance. I then reset after crash, then it stopped at a block around 550000 and now I just reset again in the hope it will scan through. I also tried the Tx recovery from Unknown, but it said that it's not finding a transaction (I guess mistake on my side). Will update, thanks for asking.
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 23, 2014, 09:46:31 pm
Now I tried a few things.

When using the oldest wallet, it will not show any currencies I bought on the market. But I can see the STARs I got only a few days ago. There is a whole lot of the balance missing. I did the Regenerate-1000-thing, rescan. Also the BTSX and STAR I sent are not showing.

When I use the newest wallet I have, only a smaller part of my balance is missing. I can see the currencies I bought, but they show 0 balance. To be exact, BTC and USD are gone but I still have CNY and STARs.

I also regenerated and rescanned here.

E: After regenerating the keys I get a 0 as reply, does this mean everything went well?
EE: What a question,.. I need to chill. Maybe watch a movie

I know the mistake, I think. Because I installed Ubuntu fresh, I didn't see the BitShares X folder. I thought the new version doesn't have it anymore but the file was hidden. That's why I didn't delete it before reinstalling.

Title: Re: Balance missing because of old wallet file
Post by: cass on September 23, 2014, 11:04:44 pm
oh man sry to hear, but as xeroc mentioned before! It's on the blockchain...
Wish you good luck to solve this issue asap

Sry honestly idk know how to help you on this ... i would
Title: Re: Balance missing because of old wallet file
Post by: xeroc on September 24, 2014, 06:38:23 am
https://github.com/BitShares/bitshares_toolkit/issues/769#issuecomment-56597941

Quote
Yes, this is expected behavior. Do not delete your wallet! There is more private data that needs to be kept in order to claim all of your balances than just the account private keys. Primarily, you need the wallet master private key, from which you can regenerate pretty much everything if necessary (though this should be done in emergencies only). If you just create a new wallet and import your account keys, you will not get all of your balances. If you've kept your original wallet, you should have everything you need.
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 24, 2014, 06:17:55 pm
That was tough news. I lost some money but it's ok. That was an expensive lesson.. I'm still in the game though! Did the money now disappear? Who has my lost BTC and USD now? I hope that at the end it benefited the system somehow.

Title: Re: Balance missing because of old wallet file
Post by: vikram on September 24, 2014, 06:52:23 pm
Are you regenerating keys on an older backup of the same wallet? Or is it a different wallet you created from scratch? If it is an older backup of the same wallet, you should be able to regenerate the keys you need to recover everything.
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 24, 2014, 09:16:34 pm
I always used only one wallet, thirtyeightptswarrior. I created the latest json I have on the 13th this month. Then I changed the hard drive and installed  OS and the client and imported the json file. Then I used the market to buy BTC and USD. Then I deleted the computer without making a backup (forgot to copy the files), installed Ubuntu again, the client, and then I saw that something was missing after using the latest json file I have. Is there anything I could do other than regenerating? The output after regenerating was 998.

Thanks

Edit: I forgot to say that I cannot send funds. It says:"RPC error: JSON parse error:  Property name must be a string literal
Edit: Oh no, this only happens with trying to send BTSX. When trying to send CNY the send field is blue. Noooooooo, why, why nooo
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 26, 2014, 01:26:52 pm
-v.0.4.17
-import newest json I have
-wallet_regenerate_keys thirtyeightptswarrior 1000
-rescan
-USD and BTC missing
-send button for BTSX blue now
- when choosing amount of CNY to send to user which is higher than my balance -> not enough funds
- when choosing amount of BTSX to send to user which is higher than my balance -> minus amount shown, no red "not enough funds"
- wallet_recover_transaction ID (without ASK, only number) -> 20022 transaction_not_found: transaction not found (still have unknown)
- BTSX I sent to a user is not showing
Property name must be a string literal
:(

Is the amount of balance I see in the synced latest client the real balance I have? I need support community, this is so unlucky, thx

Edit:

-I still RPC error Json parse error: Property name must be a string literal ( but then it disappeared, I am sure it will appear again)

-wallet_get_transcation fee assert: exception 10 (I just saw this and then it disappeared again)


 
Title: Re: Balance missing because of old wallet file
Post by: vikram on September 26, 2014, 09:30:26 pm
The balance shown by wallet_account_balance reflects all the balances on the blockchain that your wallet has keys to. We need to get your missing keys back to get any missing funds. This is what wallet_regenerate_keys is supposed to do, but it is not working for some people: https://github.com/BitShares/bitshares_toolkit/issues/712 We are tracking down the issue but unable to reproduce it yet. We are also in the process of rewriting wallet transaction scanning to be more robust, accurate and provide more information. When that is done, it might also help give clues for remembering where funds have gone even if you haven't recovered the keys yet.
Title: Re: Balance missing because of old wallet file
Post by: xeroc on September 26, 2014, 09:43:13 pm
The balance shown by wallet_account_balance reflects all the balances on the blockchain that your wallet has keys to. We need to get your missing keys back to get any missing funds. This is what wallet_regenerate_keys is supposed to do, but it is not working for some people: https://github.com/BitShares/bitshares_toolkit/issues/712 We are tracking down the issue but unable to reproduce it yet. We are also in the process of rewriting wallet transaction scanning to be more robust, accurate and provide more information. When that is done, it might also help give clues for remembering where funds have gone even if you haven't recovered the keys yet.

All this ONLY works if you have a backup of EXACTLY that wallet which you used for your market transactions (ie. buy bitUSD)
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 26, 2014, 11:37:00 pm
Ok, I made a backup of my thirtyeightptswarrior wallet. Afterwards I made market orders. Then I made a backup again. This backup got deleted. So now I only have a backup from before the market orders which means that it is not possible to recover the bitBTC and bitUSD I bought. Correct?

All market orders I made of which I do not have a backup, are somehow under 1 or 2 TxID's . It looks messed up. I am cool with the loss as long as you believe that now the funds I have are belonging to my base.

:)

E: And thank you vikram, xeroc and all others, I smoke one for you ;)

EE: I was very lucky because i still had a canceled order in the backup. I made now 2 major mistakes and lost 30% of my BTSX. Time to stop the madness and focus on work which will not ruin my bora bora fantasies.
Title: Re: Balance missing because of old wallet file
Post by: xeroc on September 26, 2014, 11:45:56 pm
You funds are NOT lost! the necessary private key (the wallet private key) is part of you backup .. the keys for the market transactions are derived from that private key and can ideally be recovered by wallet_regenerate_keys .. however it seems that there are some bugs that make the wallet miss the corresponding private key .. I am not a developer but as long as vikram or BM do not state otherwise I would consider the funds NOT lost .. just inaccessible due to some issue
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 27, 2014, 12:06:43 am
Now I understand! Whatever will happen, I am happy that I am here, still have some funds and that I can relax now.
Title: Re: Balance missing because of old wallet file
Post by: vikram on September 29, 2014, 11:09:13 pm
I've reproduced the issue and found a stupid bug that prevented wallet_regenerate_keys from properly regenerating your market keys: https://github.com/BitShares/bitshares_toolkit/commit/a8b30e2ddc21534f65fe994a511a7ae40180df7b

Hopefully you should be able to reclaim your funds after the next version is released.
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on September 29, 2014, 11:14:54 pm
Thanks vikram, when I meet you I want to buy you a beer :)
Title: Balance missing because of old wallet file
Post by: bitmeat on September 29, 2014, 11:15:32 pm
That's great!!!

I'll report whether the 60k discrepancy I'm seeing goes away. Vikram if you like I can let you log in to my machine via teamviewer to investigate.
Title: Re: Balance missing because of old wallet file
Post by: xeroc on September 30, 2014, 06:49:04 am
lol .. this bugs suck ... can take one weeks to find them .. well done ..

@vikram: the market transactions are still using the wallet private key? So the account key alone cannot recover market transactions! If so, is there any chance this can be changed to account keys, or just impossible due to the protocol?
Title: Re: Balance missing because of old wallet file
Post by: kisa on September 30, 2014, 07:00:21 am
Sounds promising! Appreciate Vikram fixing that, and Xeroc staying on the case. hoping for the resolution of market transactions to uknown and missing BTSX and bitUSD balances with the next release. ☺
Title: Re: Balance missing because of old wallet file
Post by: vikram on September 30, 2014, 03:56:11 pm
lol .. this bugs suck ... can take one weeks to find them .. well done ..

@vikram: the market transactions are still using the wallet private key? So the account key alone cannot recover market transactions! If so, is there any chance this can be changed to account keys, or just impossible due to the protocol?

I have plans to make this change: https://github.com/BitShares/bitshares_toolkit/issues/814
Thought with how complex the current wallet code is, it may ultimately end up waiting until we create a new thin client wallet that doesn't required the entire blockchain.

Sounds promising! Appreciate Vikram fixing that, and Xeroc staying on the case. hoping for the resolution of market transactions to uknown and missing BTSX and bitUSD balances with the next release. ☺

There are still problems with the scanning so they may still show up as UNKNOWN. I am in the process of rewriting the scanning though, so once that is done the proper market transaction names should show up again. After the next update you should hopefully be able to recover any market balances after using wallet_regenerate_keys though.
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 04, 2014, 11:11:38 am
Hi Vikram, just an update - no change yet with wallet_regenerate_keys within 0.4.20
Title: Re: Balance missing because of old wallet file
Post by: vikram on October 04, 2014, 05:06:54 pm
Hi Vikram, just an update - no change yet with wallet_regenerate_keys within 0.4.20

Can you describe your situation like PTSWarrior did here: https://bitsharestalk.org/index.php?topic=9194.msg121778#msg121778

PTSWarrior, were you able to recover your funds?
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on October 04, 2014, 05:15:56 pm
I didn't check. For now I want to forget about wallet, building and so on. In case my balance will not return, it will be my mistake because I didn't safe the backup file.
But now that I think about it, maybe I will do it tonight and tell you tomorrow evening.
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 05, 2014, 12:13:16 am
Hi Vikram, just an update - no change yet with wallet_regenerate_keys within 0.4.20

Can you describe your situation like PTSWarrior did here: https://bitsharestalk.org/index.php?topic=9194.msg121778#msg121778

PTSWarrior, were you able to recover your funds?

please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)
Title: Re: Balance missing because of old wallet file
Post by: bitmeat on October 05, 2014, 02:39:47 am
please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

Pretty much in the same spot. Got about 1000BitUSD and 60KBTSX not showing up in my total. What I'd like to know if they went to collateral or if they will at some point be recovered.
Title: Re: Balance missing because of old wallet file
Post by: 38PTSWarrior on October 06, 2014, 01:34:09 pm
Vikram,
I have now version 0.4.20 running. I waited for "synced". I made "wallet_regenerate_keys thirtyeightptswarrior 1000". The result was 2000. Now all the balances are back.

I can see Recent Transactions in "Overview" and in "My Accounts". They still show "UNKNOWN" sometimes but it looks like most of the market orders got fixed.

When I click on "thirtyeightptswarrior" in "My Accounts", there is nothing in the "Recent Transactions" tab.

In the top right corner it shows the alert bell, stating "JSON Parse error: Property name must be a string literal" with the number in red rising to 400 around 5 minutes of using the program. Do you know what this means?

Is your name "vikram" on the blockchain?
Thank you

 
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 06, 2014, 06:27:21 pm
>> wallet_regenerate_keys kisa0145 1000

1000

then nothing happens (don't see transaction scanning as was the case when i tried this command in previous versions)...
and i dont see any improvements to 10x outflows to UNKNOWN and still missing balance.
Title: Re: Balance missing because of old wallet file
Post by: xeroc on October 06, 2014, 06:32:16 pm
Try 2000 .. and run "rescan" manually
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 06, 2014, 06:43:40 pm
Try 2000 .. and run "rescan" manually

>> wallet_regenerate_keys kisa0145 2000

2999

no transaction scanning in progress...

>> rescan

OK

No changes... Sorry if I misunderstand what you meant by "manually"... (BTW, I hope you received my 250 BTSX to talktip.xeroc ;)
Title: Re: Balance missing because of old wallet file
Post by: xeroc on October 06, 2014, 07:16:55 pm
Let the rescan finish.. takes some time .
 Check progress with getinfo
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 06, 2014, 07:56:14 pm
Okay, I did now the following sequence:
-> wallet_rescan_blockchain -> OK -> 100%
-> wallet_regenerate_keys kisa0145 2000 -> 2000 -> 100%
-> rescan -> OK -> 100%
-> wallet_regenerate_keys kisa0145 3000 -> 4000 -> 100%

no improvement yet :/
Title: Re: Balance missing because of old wallet file
Post by: vikram on October 09, 2014, 09:11:22 pm
Vikram,
I have now version 0.4.20 running. I waited for "synced". I made "wallet_regenerate_keys thirtyeightptswarrior 1000". The result was 2000. Now all the balances are back.

I can see Recent Transactions in "Overview" and in "My Accounts". They still show "UNKNOWN" sometimes but it looks like most of the market orders got fixed.

When I click on "thirtyeightptswarrior" in "My Accounts", there is nothing in the "Recent Transactions" tab.

In the top right corner it shows the alert bell, stating "JSON Parse error: Property name must be a string literal" with the number in red rising to 400 around 5 minutes of using the program. Do you know what this means?

Is your name "vikram" on the blockchain?
Thank you

The important part is that the balances are back. The transaction history will be missing information, but that is expected. I am not sure about the JSON parse error, this may be a bug in the web wallet. If you use the command "wallet_account_transaction_history thirtyeightptswarrior" in the console and it shows up, then it's just a bug in the GUI. Yes, I am vikram on the blockchain.

please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

Pretty much in the same spot. Got about 1000BitUSD and 60KBTSX not showing up in my total. What I'd like to know if they went to collateral or if they will at some point be recovered.

I have not forgotten about these issues; I will be looking into them.
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 21, 2014, 05:51:55 am
Thanks Vikram. Would such issues still be worked on if the merger happens?
Title: Re: Balance missing because of old wallet file
Post by: vikram on October 21, 2014, 11:42:36 pm
please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

Pretty much in the same spot. Got about 1000BitUSD and 60KBTSX not showing up in my total. What I'd like to know if they went to collateral or if they will at some point be recovered.

I have not forgotten about these issues; I will be looking into them.

Thanks Vikram. Would such issues still be worked on if the merger happens?

Yes, I will address as many issues as I can regardless of any DAC merges.

I have started rewriting some of the key (re)generating code. I suspect your wallet(s) somehow got into an inconsistent state where the current version of wallet_regenerate_keys won't properly regenerate everything. My updates won't make it into the next release (0.4.21), but when the changes are ready we will see if hopefully you can regenerate the proper keys to unlock your funds.
Title: Re: Balance missing because of old wallet file
Post by: Mysto on October 22, 2014, 10:37:36 am
Hi Vikram, just an update - no change yet with wallet_regenerate_keys within 0.4.20

Can you describe your situation like PTSWarrior did here: https://bitsharestalk.org/index.php?topic=9194.msg121778#msg121778

PTSWarrior, were you able to recover your funds?

please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

The exact same thing just happened to me.
Some advise if this happens to you DON'T PANIC!
I was typing in commands pressing enter and waiting 2 seconds and if it didn't work I would close the wallet and restart. Just be patient! The wallet is a little bulky right now so just chill and follow this http://wiki.bitshares.org/index.php/RecoveringFunds and it'll probably work out if you backed it up and all.
Title: Re: Balance missing because of old wallet file
Post by: kisa on October 22, 2014, 12:24:33 pm
i am not panicking - and thanks for replies.
Title: Re: Balance missing because of old wallet file
Post by: Mysto on October 22, 2014, 02:05:41 pm
i am not panicking - and thanks for replies.

Oh no I didn't mean you. I meant anyone who is having this problem right now. I assume you resolved it by now.
Sorry for the confusion  :)
Title: Re: Balance missing because of old wallet file
Post by: tonyk on October 26, 2014, 07:26:43 am
please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

Pretty much in the same spot. Got about 1000BitUSD and 60KBTSX not showing up in my total. What I'd like to know if they went to collateral or if they will at some point be recovered.

Now if we can increase those amounts 10 folds, for bitmeat only, then we will have a  feature preventing dumping....

https://bitsharestalk.org/index.php?topic=10473.msg139126#msg139126
Title: Re: Balance missing because of old wallet file
Post by: bitmeat on October 28, 2014, 07:53:40 am
Now if we can increase those amounts 10 folds, for bitmeat only, then we will have a  feature preventing dumping....

https://bitsharestalk.org/index.php?topic=10473.msg139126#msg139126

Yup. Bitshares is really good at keeping you locked in. And it's for your own good. It's like a CD account with unknown terms. One day you realize you have a fortune!

But to date, the new versions won't even open my wallet, they just crash. :)

With that said, I don't know where you draw conclusions that I am dumping or anything like that. It was a joke.
Title: Re: Balance missing because of old wallet file
Post by: vikram on November 11, 2014, 10:46:26 pm
please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

Pretty much in the same spot. Got about 1000BitUSD and 60KBTSX not showing up in my total. What I'd like to know if they went to collateral or if they will at some point be recovered.

I have not forgotten about these issues; I will be looking into them.

Thanks Vikram. Would such issues still be worked on if the merger happens?

Yes, I will address as many issues as I can regardless of any DAC merges.

I have started rewriting some of the key (re)generating code. I suspect your wallet(s) somehow got into an inconsistent state where the current version of wallet_regenerate_keys won't properly regenerate everything. My updates won't make it into the next release (0.4.21), but when the changes are ready we will see if hopefully you can regenerate the proper keys to unlock your funds.

v0.4.24 is out (https://github.com/BitShares/bitshares/releases/tag/v0.4.24) which finally has some updates to the wallet key handling and regeneration.

kisa and bitmeat if you are still missing funds, try the following steps exactly as I've written them:

Let me know if this helps anyone.
Title: Re: Balance missing because of old wallet file
Post by: Spratan on November 14, 2014, 01:36:38 pm
It worked ! My balance is ok at the 6th step.
My coins have been missing for 3 months.

Why do I need to transfer them ?

Title: Re: Balance missing because of old wallet file
Post by: xeroc on November 14, 2014, 01:47:24 pm
Why do I need to transfer them ?
if you move your assets once you take them out of the 'special' market transaction addresses .. they are then traditional transaction addresses and the keys can be derived from the account-key only!

older versions of the wallet required a different key to access market transaction balances ..
Title: Re: Balance missing because of old wallet file
Post by: Spratan on November 14, 2014, 02:05:35 pm
I can't transfer my whole balance because I have orders pending and I can't cancel them !
Title: Re: Balance missing because of old wallet file
Post by: Xeldal on November 14, 2014, 08:40:13 pm
I made a market order today and now I'm missing a large portion of my balance.

I followed the above directions and now in wallet_account_balance there is an account without a registered name just BTSXBvBzmZE..... and it contains my missing balance.

How do I retrieve this?  It not an account listed under "My Accounts" and the address doesn't match any that I created.

wallet_market_order_list USD BTS only shows an order I currently have open(not missing), nothing else.

Also when I do a rescan it gets to about 30% and I get "Failure during transaction scanning"

Title: Re: Balance missing because of old wallet file
Post by: vikram on November 14, 2014, 09:50:14 pm
I made a market order today and now I'm missing a large portion of my balance.

I followed the above directions and now in wallet_account_balance there is an account without a registered name just BTSXBvBzmZE..... and it contains my missing balance.

How do I retrieve this?  It not an account listed under "My Accounts" and the address doesn't match any that I created.

wallet_market_order_list USD BTS only shows an order I currently have open(not missing), nothing else.

Also when I do a rescan it gets to about 30% and I get "Failure during transaction scanning"

If it shows up like that under wallet_account_balance, it means you have the private key for it--just your wallet has become inconsistent and it is not associated with a particular account. I will look at why this happened.

There are probably some convoluted ways of recovering this right now--for example you could export a backup, then manually edit the backup's entry for that key to point the account_address at one of your accounts. But I will see if I can add a command in the next version that will allow easily collecting "orphaned" balances like this: https://github.com/BitShares/bitshares/issues/985

Do you remember what exact steps you took with the market order that made the balance go missing?
Title: Re: Balance missing because of old wallet file
Post by: Xeldal on November 14, 2014, 10:20:54 pm
I made a market order today and now I'm missing a large portion of my balance.

I followed the above directions and now in wallet_account_balance there is an account without a registered name just BTSXBvBzmZE..... and it contains my missing balance.

How do I retrieve this?  It not an account listed under "My Accounts" and the address doesn't match any that I created.

wallet_market_order_list USD BTS only shows an order I currently have open(not missing), nothing else.

Also when I do a rescan it gets to about 30% and I get "Failure during transaction scanning"

If it shows up like that under wallet_account_balance, it means you have the private key for it--just your wallet has become inconsistent and it is not associated with a particular account. I will look at why this happened.

There are probably some convoluted ways of recovering this right now--for example you could export a backup, then manually edit the backup's entry for that key to point the account_address at one of your accounts. But I will see if I can add a command in the next version that will allow easily collecting "orphaned" balances like this.

Do you remember what exact steps you took with the market order that made the balance go missing?

Its hard to say.  I didn't notice it until later.

My wallet has other issues like the transaction history has not been visible in gui since 0.4.16
console 'history xeldal'  shows transactions from 07-19 as the first in the list with a negative balance
then after 8-09 it skips to 11-14(today)  I've definitely made transactions between then and now.

the balance shows negative for all transactions incrementing negatively by .5 BTS until I canceled a market order today and the balance increases correctly by the amount canceled.
None of the balances are correct for this accounts history.

Heres what I remember of the process:
I installed 0.4.24 yesterday?
this morning? client requested an update. I clicked ok. (it spammed this message about 100 times over night)
Closed the client.
Clicked the desktop BitSharesX Icon
It doesn't see my wallet because it askes me to create a new one(evidently icon is linked to the BitsharesX.exe instead of Bitshares.exe)
I re-installed from original download.
clicked check for updates (oddly there aren't any)
I then immediately go to MARKET and start placing orders, client may have been in the process of doing a rescan or some other process (I don't remember)
I place several orders not waiting for 1 to complete before placing the next, and the next (didn't think this would cause issue)
I also canceled quite a few orders that mostly filled but had little bits left over (.95 BTS , .54 BTS , .53 BTS etc)
Then on returning to MY ACCOUNTS I notice balance is much lower than it should be.

I can send you any outputs you might find useful.
Title: Re: Balance missing because of old wallet file
Post by: vikram on November 15, 2014, 02:28:55 am
I made a market order today and now I'm missing a large portion of my balance.

I followed the above directions and now in wallet_account_balance there is an account without a registered name just BTSXBvBzmZE..... and it contains my missing balance.

How do I retrieve this?  It not an account listed under "My Accounts" and the address doesn't match any that I created.

wallet_market_order_list USD BTS only shows an order I currently have open(not missing), nothing else.

Also when I do a rescan it gets to about 30% and I get "Failure during transaction scanning"

If it shows up like that under wallet_account_balance, it means you have the private key for it--just your wallet has become inconsistent and it is not associated with a particular account. I will look at why this happened.

There are probably some convoluted ways of recovering this right now--for example you could export a backup, then manually edit the backup's entry for that key to point the account_address at one of your accounts. But I will see if I can add a command in the next version that will allow easily collecting "orphaned" balances like this.

Do you remember what exact steps you took with the market order that made the balance go missing?

Its hard to say.  I didn't notice it until later.

My wallet has other issues like the transaction history has not been visible in gui since 0.4.16
console 'history xeldal'  shows transactions from 07-19 as the first in the list with a negative balance
then after 8-09 it skips to 11-14(today)  I've definitely made transactions between then and now.

the balance shows negative for all transactions incrementing negatively by .5 BTS until I canceled a market order today and the balance increases correctly by the amount canceled.
None of the balances are correct for this accounts history.

Heres what I remember of the process:
I installed 0.4.24 yesterday?
this morning? client requested an update. I clicked ok. (it spammed this message about 100 times over night)
Closed the client.
Clicked the desktop BitSharesX Icon
It doesn't see my wallet because it askes me to create a new one(evidently icon is linked to the BitsharesX.exe instead of Bitshares.exe)
I re-installed from original download.
clicked check for updates (oddly there aren't any)
I then immediately go to MARKET and start placing orders, client may have been in the process of doing a rescan or some other process (I don't remember)
I place several orders not waiting for 1 to complete before placing the next, and the next (didn't think this would cause issue)
I also canceled quite a few orders that mostly filled but had little bits left over (.95 BTS , .54 BTS , .53 BTS etc)
Then on returning to MY ACCOUNTS I notice balance is much lower than it should be.

I can send you any outputs you might find useful.

There is little we can do about that messed up transaction history right now; it will look better when the scanning code is finished being rewritten.

There's a lot going on here to try to track down what happened. I'll give it some thought.

I've made some more fixes that will be in v0.4.25 to help prevent more of this kind of thing, and also to recover orphaned balances like yours:
https://github.com/BitShares/bitshares/commit/12a7075d1ac2e08d494f5100e9a58cfc17d84b0d
https://github.com/BitShares/bitshares/commit/67fa61b76a6588ed8da4b07c4b6071dfc126bfe1
https://github.com/BitShares/bitshares/commit/c580c301f40578f16c55921ee21d9e02c76ff58e
Title: Re: Balance missing because of old wallet file
Post by: Xeldal on November 24, 2014, 11:05:59 pm
Any idea when we might expect the next release?

Today I decided to vote with some of what was left of my non-inaccessible funds and now they are gone/missing/inaccessible as well.

I'm sure they are there but my client can only make it to about 30% rescan before 'failure scanning transactions' .. so they remain out of my reach.

Title: Re: Balance missing because of old wallet file
Post by: vikram on November 25, 2014, 06:03:04 pm
Any idea when we might expect the next release?

Today I decided to vote with some of what was left of my non-inaccessible funds and now they are gone/missing/inaccessible as well.

I'm sure they are there but my client can only make it to about 30% rescan before 'failure scanning transactions' .. so they remain out of my reach.

Since your scanning is failing for some reason, it is not able to pick up the TITAN transaction to see your funds. You might be able to get around this by manually starting a scan partway through the chain--after the block it fails but before the block you made the transfer. See the output of "help wallet_rescan_blockchain":

Code: [Select]
(wallet closed) >>> help scan
Usage:
wallet_rescan_blockchain [first_block_number] [num_blocks] [fast_scan]                                Scans the blockchain history for operations relevant to this wallet.
Scans the blockchain history for operations relevant to this wallet.

Parameters:
  first_block_number (uint32_t, optional, defaults to 0): the first block to scan
  num_blocks (uint32_t, optional, defaults to -1): the number of blocks to scan
  fast_scan (bool, optional, defaults to false): true to scan as fast as possible but freeze the rest of your computer, and false otherwise

Returns:
  void

aliases: scan, rescan

I will fix this in the next version so that it does not abort scanning on a single failure: https://github.com/BitShares/bitshares/issues/1060

The next release has many fixes, but is unfortunately taking some time due to the amount of features we are working on. If you urgently need your funds we can talk on Skype and try to recover them with the current version. I've pm'ed you my username.
Title: Re: Balance missing because of old wallet file
Post by: Xeldal on November 25, 2014, 06:30:05 pm
Hot dog!

This worked^  wallet_rescan_blockchain 1000000

Balance is restored.  Odd though, first I used simply rescan 1000000 and it appeared to do nothing for some time. so I tried wallet_rescan_blockchain 900000 200000 1 .  It then started scanning, slowly, finished with 100% and then after a couple seconds I started getting windows notifications about market orders/receiving/sending etc and noticed it had started scanning again, this time much faster.

I'm guessing the first scan was just highly delayed and scanned twice because I had told it twice, thinking the first one was in error.

Thanks for your help.
Title: Re: Balance missing because of old wallet file
Post by: kisa on December 08, 2014, 03:30:12 pm
please see here the original thread, Vikram...
https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079 (https://bitsharestalk.org/index.php?topic=8371.msg114079#msg114079)

to briefly summarize - on september 8th i bought 1000 bitUSD and left some market orders. later that day my windows crashed and i had to run system recovery. unfortunately i only had old wallet backup. all my market transactions and orders of September 8th show outflows to unknown, so i am missing some 120K BTSX and 1000bitUSD in my balance. xeroc kindly posted this issue on https://github.com/BitShares/bitshares_toolkit/issues/769 (https://github.com/BitShares/bitshares_toolkit/issues/769)

Pretty much in the same spot. Got about 1000BitUSD and 60KBTSX not showing up in my total. What I'd like to know if they went to collateral or if they will at some point be recovered.

I have not forgotten about these issues; I will be looking into them.

Thanks Vikram. Would such issues still be worked on if the merger happens?

Yes, I will address as many issues as I can regardless of any DAC merges.

I have started rewriting some of the key (re)generating code. I suspect your wallet(s) somehow got into an inconsistent state where the current version of wallet_regenerate_keys won't properly regenerate everything. My updates won't make it into the next release (0.4.21), but when the changes are ready we will see if hopefully you can regenerate the proper keys to unlock your funds.

v0.4.24 is out (https://github.com/BitShares/bitshares/releases/tag/v0.4.24) which finally has some updates to the wallet key handling and regeneration.

kisa and bitmeat if you are still missing funds, try the following steps exactly as I've written them:
  • Make sure you have the wallet or wallet backup of the wallet you originally used for the market orders that you are missing funds from.
  • Open the wallet (or restore the backup) with v0.4.24.
  • Unlock the wallet if you haven't already.
  • Make a new backup of the wallet that we will restore to later.
  • We will now regenerate a large number of keys. This will make the wallet slow afterwards, so after we are done we will restore from the most recent backup.
  • Use the command "wallet_regenerate_keys <account_name> 10500", replacing <account_name> with your relevant account name. This will take some time to generate all those keys and then rescan the blockchain.
  • Check "wallet_account_balance". Does that balance include your missing funds now?
  • Check "wallet_market_order_list". Do you have any orders which you weren't able to access before?
  • If you are able to access any missing funds, it's easiest to just transfer your entire balance back to yourself.
  • Restore the backup we created in step (4) and if you transferred any new funds back to yourself, it should show up as an incoming transaction and you should be able to access those funds now.

Let me know if this helps anyone.

Finally I had time to implement the above instructions.

First, I deinstalled all previous client versions and deleted bitshares directory under appshare/ roaming.

Then I downloaded v.0.4.24 from bitshares.org and created an empty wallet.

Syncing blocks didn't work first, got stuck now and then, after I agreed to patch updates. I had to reinstall clean and rejected patch updates. Sync crashed half time, but resumed next day and completed. Then I installed patch updates.

Importing old .json backup did work, and I created new backup before regenerating keys.

Then - great news - regenerating keys 10500 did work and restored missing balances from old orders!

Then I transfered part of BTS to myself (as I was afraid to transfer all funds yet). Outgoing balance was substracted but incoming balance didn't show up, also not after wallet_account_balance.

When I restored to the newly created backup, still the incoming balance from the latest self transfer was missing.

 When I tried to quit client crashed. At re-opening the client showed empty wallet, was not connecting. I agreed to reindexing database.

Then client at default picked up a strange version of wallet, with somewhat incorrect balance and without transaction history. I attempted to regenerate 10500 keys again, and transaction history appeared again after a very slow rescan. Still the recent self-transfer part of balance was missing and self - transfer didn't show under transaction history.

Perhaps I better wait until a new version and then try again, as the client keeps crashing at this stage and I don't want to invest more time into this struggle, as I am not tech person.

Anyway, when I saw restored balance at some point during that process, this gave me confidence that at some future point I get all my funds and will be able to trade on market seamless :)

good luck guys with the new release!
Title: Re: Balance missing because of old wallet file
Post by: Xeldal on December 09, 2014, 06:45:57 pm
running 0.4.25 RC1

With 0.4.24 I had 1 orphaned address with a balance I couldn't access.  With this update I now have 15 orphaned addresses with that balance split among them.  I'm not sure whether this update was meant to fix this or not.  No worries, just reporting.

Thankfully with latest version I am now finally able to see transaction history. 
Title: Re: Balance missing because of old wallet file
Post by: vikram on December 09, 2014, 10:13:37 pm
running 0.4.25 RC1

With 0.4.24 I had 1 orphaned address with a balance I couldn't access.  With this update I now have 15 orphaned addresses with that balance split among them.  I'm not sure whether this update was meant to fix this or not.  No worries, just reporting.

Thankfully with latest version I am now finally able to see transaction history.

Does this help? https://bitsharestalk.org/index.php?topic=12105.msg160172#msg160172
Title: Re: Balance missing because of old wallet file
Post by: vikram on December 09, 2014, 10:17:46 pm
Then I transfered part of BTS to myself (as I was afraid to transfer all funds yet). Outgoing balance was substracted but incoming balance didn't show up, also not after wallet_account_balance.

I suspect you might have been hit with the transaction scanning bug that forced us to release 0.4.24.1--give it another shot once 0.4.25 is released and let us know what happens.
Title: Re: Balance missing because of old wallet file
Post by: educatedwarrior on June 10, 2017, 02:15:51 am
no need to be nervous .. your funds are still on the blockchain and as long as you have a backup of your wallet your funds will be accessible again ..

Please help me.  @xeroc    Somebody?

I"ve tried all of this...

Migration to BitShares 2.0 - Incorrect Balances - "Know How"
https://bitsharestalk.org/index.php/topic,20822.0.html

Balance Missing Because of Old Wallet File
https://bitsharestalk.org/index.php/topic,9194.30.html
Title: Re: Balance missing because of old wallet file
Post by: mrbildo on June 10, 2017, 11:38:27 am
Same problem, where are the devs on this? Clearly this is a huge issue for many of their "Angel" and early adopters. The lack of support here in recovering lost funds is a joke tbh. I feel like I am having to hack my own wallet just to get my own money back, not cool.
Title: Re: Balance missing because of old wallet file
Post by: educatedwarrior on June 10, 2017, 11:57:22 am
Same problem, where are the devs on this? Clearly this is a huge issue for many of their "Angel" and early adopters. The lack of support here in recovering lost funds is a joke tbh. I feel like I am having to hack my own wallet just to get my own money back, not cool.

I made a post here in Steemit.  https://steemit.com/bitshares/@educatedwarrior/my-experience-upgrading-for-bitshares-x-version-0-4-23-1-to-bitshares-2-0-instructions-included
@mrbildo , maybe we need to start a list of all early adopters having issues upgrading their wallets.   Let me know if you find out anything and I'll do the same.
Title: Re: Balance missing because of old wallet file
Post by: mrbildo on June 10, 2017, 11:59:52 am
I think my issue is with rescanning the chain before exporting, I have tried EVERYTHING i can find online and nothing works, this is really not cool.
Title: Re: Balance missing because of old wallet file
Post by: vikram on June 10, 2017, 10:32:07 pm
@educatedwarrior Do you still have the original backup made using v0.4.23.1? And to be clear this is a backup JSON you are recovering from and not an existing full wallet directory from that old version? And you are sure you did not make any further transactions after this backup was created so it should contain all your keys (including potential AGS/PTS keys that were never claimed)?

I'm looking at some of the old wallet releases and it seems like there might be a separate issue with the format of certain wallet records changing in later versions. These records should have automatically updated when upgrading--but to be safe I'm thinking that for very old backups it might help to:


More information on where the breakdown seems to first occur would also be helpful. For example, you are able to see your expected balance at point X but cannot migrate it to version Y?

Edit: @mrbildo Sounds like you are in a similar situation with a very old wallet backup JSON? Do you have an idea of where the breakdown might be occurring when following the above steps?
Title: Re: Balance missing because of old wallet file
Post by: educatedwarrior on June 10, 2017, 10:44:28 pm
@educatedwarrior Do you still have the original backup made using v0.4.23.1? And to be clear this is a backup JSON you are recovering from and not an existing full wallet directory from that old version? And you are sure you did not make any further transactions after this backup was created so it should contain all your keys (including potential AGS/PTS keys that were never claimed)?

I'm looking at some of the old wallet releases and it seems like there might be a separate issue with the format of certain wallet records changing in later versions. These records should have automatically updated when upgrading--but to be safe I'm thinking that for very old backups it might help to:

  • Install v0.4.24.1, import and unlock the backup wallet (you might see a wallet format upgrade message)
  • Upgrade to v0.6.2, open and unlock the same wallet (I don't think this step is necessary but just in case)
  • Upgrade to v0.9.3c, import the final blockchain and do a full rescan following the 2.0 migration instructions

More information on where the breakdown seems to first occur would also be helpful. For example, you are able to see your expected balance at point X but cannot migrate it to version Y?

Edit: @mrbildo Sounds like you are in a similar situation with a very old wallet backup JSON? Do you have an idea of where the breakdown might be occurring when following the above steps?

I can see my balance after upgrading to v0.9.3c and doing a full rescan.  It's when I import the v0.9.3c wallet export file into bitshares 2.0 my balance doesn't transfer over.
Title: Re: Balance missing because of old wallet file
Post by: mrbildo on June 10, 2017, 10:45:51 pm
fluxer555 helped me recover mine, took a while but we got it done, I recommend sending him a pm. Few steps in the process and quite a bit of trouble shooting but he knows what he's doing. Hope you get sorted
Title: Re: Balance missing because of old wallet file
Post by: abit on June 10, 2017, 11:14:46 pm
@educatedwarrior Do you still have the original backup made using v0.4.23.1? And to be clear this is a backup JSON you are recovering from and not an existing full wallet directory from that old version? And you are sure you did not make any further transactions after this backup was created so it should contain all your keys (including potential AGS/PTS keys that were never claimed)?

I'm looking at some of the old wallet releases and it seems like there might be a separate issue with the format of certain wallet records changing in later versions. These records should have automatically updated when upgrading--but to be safe I'm thinking that for very old backups it might help to:

  • Install v0.4.24.1, import and unlock the backup wallet (you might see a wallet format upgrade message)
  • Upgrade to v0.6.2, open and unlock the same wallet (I don't think this step is necessary but just in case)
  • Upgrade to v0.9.3c, import the final blockchain and do a full rescan following the 2.0 migration instructions

More information on where the breakdown seems to first occur would also be helpful. For example, you are able to see your expected balance at point X but cannot migrate it to version Y?

Edit: @mrbildo Sounds like you are in a similar situation with a very old wallet backup JSON? Do you have an idea of where the breakdown might be occurring when following the above steps?

I can see my balance after upgrading to v0.9.3c and doing a full rescan.  It's when I import the v0.9.3c wallet export file into bitshares 2.0 my balance doesn't transfer over.
1. please make sure you have latest 0.x blockchain data before the rescan. Although the network is down, there is a link to download the full data. Otherwise the balance in your screenshot can't be trusted.
2. if the balance is still there after a good rescan, please make sure the "v0.9.3c wallet export file " is created from the command in the console, but not from the gui/menu.
3. if it's still missing, I guess you did something wrong in 2.0, or perhaps have claimed the funds in 2.0 earlier.

Your issue hasn't been described clearly. Paste more screenshots, step by step, how much time cost on one step, etc.

An easier way is to give your public keys to one of the people who can help. Or ask someone to guide you via Skype or so.