Importing Keys into Graphene

[bytemaster]

Im trying to compile tag bts/0.9.3d so that I can export my wallet backup to Graphene, but FC is failing to compile:

Code: [Select]

/home//source/bitshares/libraries/fc/include/fc/fwd_impl.hpp:58:25: error: static assertion failed: Failed to reserve enough space in fc::fwd<T,S>
     void check_size() { static_assert( (ProvidedSize >= RequiredSize), "Failed to reserve enough space in fc::fwd<T,S>" ); }
Any ideas?

Yeah, it looks like you are using a new version of GCC.   Try installing a slightly older version or using CLANG.     If you can tell me what the RequiredSize was (should be in the extended error message) then I can update FC to make it work for you.

[speedy]

Im trying to compile tag bts/0.9.3d so that I can export my wallet backup to Graphene, but FC is failing to compile:

Code: [Select]

/home//source/bitshares/libraries/fc/include/fc/fwd_impl.hpp:58:25: error: static assertion failed: Failed to reserve enough space in fc::fwd<T,S>
     void check_size() { static_assert( (ProvidedSize >= RequiredSize), "Failed to reserve enough space in fc::fwd<T,S>" ); }
Any ideas?

[JWF]

Want everyone to know I finally managed to get access to my accounts today.. I will be posting a full step by step tonight... I could not use openledger in the end though.

curious how many keys you had to import. I've 55000 and I have yet to get my balances to show in the light wallet. I started this import attempt at 8:35am this morning at block 57050 and still nothing now 6 hours later at block 64250. This shouldn't just be a problem I'm having as there must be others with large numbers of keys.

I'm dumb... just actually looked at above posts...

so I'm curious to see what it took for you.

[BunkerChainLabs-DataSecurityNode]

Want everyone to know I finally managed to get access to my accounts today.. I will be posting a full step by step tonight... I could not use openledger in the end though.

[svk]

We are aware of it for sure, James is the main man on this subject though and he's been travelling today.

Now I have an idea, not sure it'll work or not, but assuming the encrypted private keys and public keys are all in order in the json file, wouldn't it be possible to split the file into several smaller json files? That way you could import 1/10 of your keys at a time for example?

And if you have several different accounts in one file then I'm fairly certain you could separate them and make one file per account, just keep the password checksum at the top for each file.

@xeroc does that make sense?

[EstefanTT]

I'm still having a black sceen with "Wallet Management Console - Importing Keys (waiting gif icon spinning)" in the light client. It seems that it can't finished the process of importing the keys or balances,

It seems that I won't be able to veify that that keys.json work properly and I'm able to import my balances correctely before 2.0.

Are we the only ones with Xeldal to have this issue ? Is that gonna be solve when we launch the 2.0 ? Do I have to worry (more) ?

I guess the issue here is that your old wallets have a ton of keys, maybe because you've been trading. We used to display the number of keys in a wallet, is that still there? I remember trying my main wallet and it had 95k keys, it took a while to import but got there in the end.

Maybe the easiest solution for you is to simply transfer your balances to another account with less keys before the snapshot, then import that instead. I assume you could import the private key only from the account with too many keys in order to regain control of it in 2.0, but not 100% sure on that.

I am not clear what you mean here.. but I am having a similar problem where all my accounts are over 30k in private keys. the import function finally gets through them all to display them.. but when I attempt to import it times out. To make matters worse, when I do see the list of account to import I see this at the end:

Unclaimed balances belonging to these keys:
ASSET TOTALS
No Balances

Which is wrong.

Every single active account I have is like this.. I cannot import into a regular wallet or into a witness wallet with CLI.

What can be done to make this work?

This is just a small sample below of all the accounts in the wallet.. which is why the private keys dont add up to what is shown in the image.

I have the exact same problem in two different computers with two different set of accounts and different keys.json

I found in the forum threads several persons in the same situation but no solution. Is this beeing solved by the devs or are they aware of it ?

[svk]

James is looking into it

https://github.com/cryptonomex/graphene-ui/issues/319

[BunkerChainLabs-DataSecurityNode]

Yea so you have 130k keys and I guess your computer isn't fast enough to finish the import in a reasonable time. The solution I outlined should work but hopefully someone like @jcalfee or @bytemaster could confirm that.

I did suggest to James at one point that he could use web workers for the number crunching, might be time to look into that as well in order to use more cores.

Well.. I attempted to import my wallet into a testnet witness node .. the problem I had though is that my drive space kept on filling up with the backup wallet data before it could finish.

I ended up setting up a cron job to delete the files every minute while the import proceeded.

It has now been 3 1/2 hours and it has deleted well over 1TB in backup files as the import has proceeded. This all came out of a 22mb wallet_export_keys json file for about 14 accounts.

It'st still processing.. so we'll see if it works in the end.

If I am having this much processing going on for my wallet others might have similar trouble.

Ouch! I didn't know it wrote backups like that, but you're right, most likely lots of people will have issues like this. I'm scared to import my delegate wallet now, after all those feed transactions it must have tons of keys too.

I'll make sure James sees this thread since he's the one writing all the wallet and key handling code.

It ran for another hour.. at the end of the process these were some of the final messages I had on my screen:

Code: [Select]

1554884ms th_a       wallet.cpp:2650               import_accounts      ] successfully imported 37518 keys for account payment.datasecuritynode-com
1554884ms th_a       wallet.cpp:2646               import_accounts      ] Couldn't import key due to exception 18 aes_exception: AES error
error during aes 256 cbc decrypt final
    {"s":"error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length"}
    th_a  aes.cpp:263 aes_decrypt
1554885ms th_a       wallet.cpp:438                copy_wallet_file     ] backing up wallet test6 to before-import-key-3c4b00c1.wallet
1555347ms th_a       wallet.cpp:727                save_wallet_file     ] saving wallet to file test6
1559695ms th_a       wallet.cpp:438                copy_wallet_file     ] backing up wallet test6 to after-import-key-3c4b00c1.wallet
1559719ms th_a       wallet.cpp:2650               import_accounts      ] successfully imported 1 keys for account start-now
1559719ms th_a       wallet.cpp:2652               import_accounts      ] failed to import 1 keys for account start-now
1559721ms th_a       wallet.cpp:438                copy_wallet_file     ] backing up wallet test6 to before-import-key-ac40e213.wallet
1559862ms th_a       wallet.cpp:727                save_wallet_file     ] saving wallet to file test6
1563951ms th_a       wallet.cpp:438                copy_wallet_file     ] backing up wallet test6 to after-import-key-ac40e213.wallet
1563978ms th_a       wallet.cpp:2646               import_accounts      ] Couldn't import key due to exception 18 aes_exception: AES error
error during aes 256 cbc decrypt final
    {"s":"error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length"}
    th_a  aes.cpp:263 aes_decrypt
1563979ms th_a       wallet.cpp:2650               import_accounts      ] successfully imported 1 keys for account starts-now
1563979ms th_a       wallet.cpp:2652               import_accounts      ] failed to import 1 keys for account starts-now
[[
    "",
    false
  ],[
    "bitshares-ca",
    true
  ],[
    "bunkermining",
    true
  ],[
It looks as though some did not work while others did.

This is a bit concerning now.

The reason I went this route is because of the tens of thousands of keys I couldn't follow the regular procedure of pulling the owner keys and then getting the balance keys etc because I have no idea out of the 30k+ keys to an account which one actually has the balance. I know that xeroc has a python script to help find this..but that is only good if you are running linux.. this is with the windows client.

The few that appear to have worked in importing when I did a dump_private_keys command actually spit out all 30K or so keys back at me. So it seems like all that bloat is just getting dumped back in. Couldn't the process perhaps check for balances and then import with only those keys?

[svk]

Yea so you have 130k keys and I guess your computer isn't fast enough to finish the import in a reasonable time. The solution I outlined should work but hopefully someone like @jcalfee or @bytemaster could confirm that.

I did suggest to James at one point that he could use web workers for the number crunching, might be time to look into that as well in order to use more cores.

Well.. I attempted to import my wallet into a testnet witness node .. the problem I had though is that my drive space kept on filling up with the backup wallet data before it could finish.

I ended up setting up a cron job to delete the files every minute while the import proceeded.

It has now been 3 1/2 hours and it has deleted well over 1TB in backup files as the import has proceeded. This all came out of a 22mb wallet_export_keys json file for about 14 accounts.

It'st still processing.. so we'll see if it works in the end.

If I am having this much processing going on for my wallet others might have similar trouble.

Ouch! I didn't know it wrote backups like that, but you're right, most likely lots of people will have issues like this. I'm scared to import my delegate wallet now, after all those feed transactions it must have tons of keys too.

I'll make sure James sees this thread since he's the one writing all the wallet and key handling code.

[BunkerChainLabs-DataSecurityNode]

Yea so you have 130k keys and I guess your computer isn't fast enough to finish the import in a reasonable time. The solution I outlined should work but hopefully someone like @jcalfee or @bytemaster could confirm that.

I did suggest to James at one point that he could use web workers for the number crunching, might be time to look into that as well in order to use more cores.

Well.. I attempted to import my wallet into a testnet witness node .. the problem I had though is that my drive space kept on filling up with the backup wallet data before it could finish.

I ended up setting up a cron job to delete the files every minute while the import proceeded.

It has now been 3 1/2 hours and it has deleted well over 1TB in backup files as the import has proceeded. This all came out of a 22mb wallet_export_keys json file for about 14 accounts.

It'st still processing.. so we'll see if it works in the end.

If I am having this much processing going on for my wallet others might have similar trouble.

[Pheonike]

You gonna need to max out the specs on the VPS machines for the web wallets. They are going be doing lot of crunching when everyone is importing their wallets for the launch.  Or least have some kind of progress indicator so ppl want assume it has stalled.

[svk]

Yea so you have 130k keys and I guess your computer isn't fast enough to finish the import in a reasonable time. The solution I outlined should work but hopefully someone like @jcalfee or @bytemaster could confirm that.

I did suggest to James at one point that he could use web workers for the number crunching, might be time to look into that as well in order to use more cores.

[BunkerChainLabs-DataSecurityNode]

I'm still having a black sceen with "Wallet Management Console - Importing Keys (waiting gif icon spinning)" in the light client. It seems that it can't finished the process of importing the keys or balances,

It seems that I won't be able to veify that that keys.json work properly and I'm able to import my balances correctely before 2.0.

Are we the only ones with Xeldal to have this issue ? Is that gonna be solve when we launch the 2.0 ? Do I have to worry (more) ?

I guess the issue here is that your old wallets have a ton of keys, maybe because you've been trading. We used to display the number of keys in a wallet, is that still there? I remember trying my main wallet and it had 95k keys, it took a while to import but got there in the end.

Maybe the easiest solution for you is to simply transfer your balances to another account with less keys before the snapshot, then import that instead. I assume you could import the private key only from the account with too many keys in order to regain control of it in 2.0, but not 100% sure on that.

I am not clear what you mean here.. but I am having a similar problem where all my accounts are over 30k in private keys. the import function finally gets through them all to display them.. but when I attempt to import it times out. To make matters worse, when I do see the list of account to import I see this at the end:

Unclaimed balances belonging to these keys:
ASSET TOTALS
No Balances

Which is wrong.

Every single active account I have is like this.. I cannot import into a regular wallet or into a witness wallet with CLI.

What can be done to make this work?

This is just a small sample below of all the accounts in the wallet.. which is why the private keys dont add up to what is shown in the image.

[svk]

I'm still having a black sceen with "Wallet Management Console - Importing Keys (waiting gif icon spinning)" in the light client. It seems that it can't finished the process of importing the keys or balances,

It seems that I won't be able to veify that that keys.json work properly and I'm able to import my balances correctely before 2.0.

Are we the only ones with Xeldal to have this issue ? Is that gonna be solve when we launch the 2.0 ? Do I have to worry (more) ?

I guess the issue here is that your old wallets have a ton of keys, maybe because you've been trading. We used to display the number of keys in a wallet, is that still there? I remember trying my main wallet and it had 95k keys, it took a while to import but got there in the end.

Maybe the easiest solution for you is to simply transfer your balances to another account with less keys before the snapshot, then import that instead. I assume you could import the private key only from the account with too many keys in order to regain control of it in 2.0, but not 100% sure on that.

[EstefanTT]

I'm still having a black sceen with "Wallet Management Console - Importing Keys (waiting gif icon spinning)" in the light client. It seems that it can't finished the process of importing the keys or balances,

It seems that I won't be able to veify that that keys.json work properly and I'm able to import my balances correctely before 2.0.

Are we the only ones with Xeldal to have this issue ? Is that gonna be solve when we launch the 2.0 ? Do I have to worry (more) ?