Author Topic: Web release for importing large wallets (testers needed)  (Read 2235 times)

0 Members and 1 Guest are viewing this topic.

Offline jamesc

Web release for importing large wallets (testers needed)
« on: October 28, 2015, 05:24:43 pm »
TEST COMPLETE.. the filter is in the main wallets now.

   https://bitshares.org/worker

** this is temporary, later you'll file the same wallet under the typical place:  https://bitshares.org/wallet
EDIT: please post browser name and version

This uses the BitShares 2.0 Genesis bloom filter to reduce the size of wallets when importing BTS 0.9.x key files.  The filter is applied to all BTS 0.9.x exported keys (by file format) upon import.  Also "worker" refers to the HTML5 worker threads...

The /worker release has a 1MB file in it (a bloom filter) that was built using all public keys and public addresses used in the BTS 2.0 genesis.json file.  When you import a BTS 0.9.x keys file (or backup), the public keys and addresses are checked against the filter.  With 100% certainty any private key has a public key or address in the genesis block will be included.  Almost all keys that do not appear in the genesis block will be filtered out and do not import..

If you have a small wallet and all the keys are used you may not even notice it happened..  We are mostly interested in: did it work?

This will remove any spent TITAN addresses and remove any unused keys that were created by BTS < 2.0 generate key commands, etc..  You should save and backup your original BTS 0.9.x keys export file as that contains all your keys.

Typically you will use this url or a partner's URL instead:
  https://bitshares.org/wallet

Both /wallet and /worker share the same local storage
(after we remove the /worker directory you will will remain at /wallet)

This release is not available in an installer yet.  You may wait or try to filter yourself:
  GUIDE: How to remove unneeded keys from Bitshares 0.9.3 export_keys json
  https://bitsharestalk.org/index.php/topic,19390.0.html

I'm doing a separate temporary release because I need to know if the HTML5 worker threads are going to work for most browsers.  There are varying degrees of support so I tried to stick to the very basic stuff.  Worker threads lets us get into multi-core processing so this is a good thing to know about...

EDIT:  Filtering 130K keys started almost instantly (about 1 sec to download the filter as shown in the network tab) and took 3 minutes to filter.  In my case about 500 keys remained.  After clicking import the wallet was ready in about 5 seconds.
« Last Edit: November 04, 2015, 02:02:02 pm by jcalfee1 »

Xeldal

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #1 on: October 28, 2015, 06:41:50 pm »
Not sure how long it should take.

I'm using a 15MB keys.json from 0.9.3c in Firefox, with roughly 90k keys. I selected the file and now 15mins in.  Still have the spinning circle, not sure if its doing anything.

I've already imported these key with another method, just testing.

Offline jamesc

Re: Web release for importing large wallets (testers needed)
« Reply #2 on: October 28, 2015, 06:50:10 pm »
Not sure how long it should take.

I'm using a 15MB keys.json from 0.9.3c in Firefox, with roughly 90k keys. I selected the file and now 15mins in.  Still have the spinning circle, not sure if its doing anything.

I've already imported these key with another method, just testing.

It should spin when downloading the 1MB filter.. There is probably a console error that caused it to be stuck in the spinning state.  Can you please send me any errors if you see them.

Xeldal

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #3 on: October 28, 2015, 06:52:49 pm »
Code: [Select]
"connecting to wss://bitshares.openledger.info/ws" app.js:2:183711
Configured for BitShares Object { core_asset: "BTS", address_prefix: "BTS", chain_id: "4018d7844c78f6a6c41c6a552b898022310…" } app.js:2:189461
db init done app.js:2:69245
ChainStore Init app.js:12:9104
synced and subscribed, chainstore ready app.js:12:9589
Warning: setState(...): Can only update a mounted or mounting component. This usually means you called setState() on an unmounted component. This is a no-op. vendors.js:1:20286
GenesisFilterWorker start 1ae2f77e0472c8b52ebd.worker.js:1:560
GenesisFilterWorker InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable 1ae2f77e0472c8b52ebd.worker.js:1:786

Offline jamesc

Re: Web release for importing large wallets (testers needed)
« Reply #4 on: October 28, 2015, 07:08:47 pm »
Code: [Select]
"connecting to wss://bitshares.openledger.info/ws" app.js:2:183711
Configured for BitShares Object { core_asset: "BTS", address_prefix: "BTS", chain_id: "4018d7844c78f6a6c41c6a552b898022310…" } app.js:2:189461
db init done app.js:2:69245
ChainStore Init app.js:12:9104
synced and subscribed, chainstore ready app.js:12:9589
Warning: setState(...): Can only update a mounted or mounting component. This usually means you called setState() on an unmounted component. This is a no-op. vendors.js:1:20286
GenesisFilterWorker start 1ae2f77e0472c8b52ebd.worker.js:1:560
GenesisFilterWorker InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable 1ae2f77e0472c8b52ebd.worker.js:1:786

A problem appeared under Firefox, I see the error and I'm working on it.. thank you

Offline luckybit

  • Hero Member
  • *****
  • Posts: 2921
    • View Profile
  • BitShares: Luckybit
Re: Web release for importing large wallets (testers needed)
« Reply #5 on: October 28, 2015, 09:27:35 pm »
Testing it. So far it looks fantastic. Lets see if it lets me import my balances.
https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

Offline luckybit

  • Hero Member
  • *****
  • Posts: 2921
    • View Profile
  • BitShares: Luckybit
Re: Web release for importing large wallets (testers needed)
« Reply #6 on: October 28, 2015, 10:04:51 pm »
Success!

It found all balances and went perfect.
https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

Offline nomoreheroes7

  • Hero Member
  • *****
  • Posts: 756
  • King of all the land
    • View Profile
  • BitShares: nomoreheroes7
Re: Web release for importing large wallets (testers needed)
« Reply #7 on: October 29, 2015, 11:29:21 am »
Holy crap that worked amazingly well. Had everything filtered and claimed in a good 10 minutes. Started with 190,000 keys lol. Used in Chrome, latest version on Windows 8.1.

Thanks for the great work!

Offline EstefanTT

Re: Web release for importing large wallets (testers needed)
« Reply #8 on: October 29, 2015, 12:08:25 pm »
Is it about to be integrated in the light client ? It's just to know if I wait a couple of days or I try the web browser (I would prefer to use the light client ;)  )
Bit20, the cryptocurrency index fund http://www.bittwenty.com
(BitShares French ConneXion - www.bitsharesfcx.com)

Offline merockstar

Re: Web release for importing large wallets (testers needed)
« Reply #9 on: October 29, 2015, 01:58:12 pm »
I'm playing with it right now.

Unfortunately, I've been tethering from my phone and all my attempts to download the 0.9.3c blockchain have screwed me out of data. So I'm at 2g and this might take a long time.

I'm pretty sure my 9mb wallet is uploading right now. I seem to have gotten the same console error as somebody else did earlier, but I'm not convinced that it isn't working for that reason.

Code: [Select]
"connecting to wss://bitshares.openledger.info/ws" app.js:2:183711
Configured for BitShares Object { core_asset: "BTS", address_prefix: "BTS", chain_id: "4018d7844c78f6a6c41c6a552b898022310…" } app.js:2:189461
db init done app.js:2:69245
ChainStore Init app.js:12:9104
synced and subscribed, chainstore ready app.js:12:9589
Use of getPreventDefault() is deprecated.  Use defaultPrevented instead. worker
Warning: setState(...): Can only update a mounted or mounting component. This usually means you called setState() on an unmounted component. This is a no-op. vendors.js:1:20286
GenesisFilterWorker start 1ae2f77e0472c8b52ebd.worker.js:1:560
GenesisFilterWorker InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable

will keep you apprised. using firefox for ubuntu 41.0.2
« Last Edit: October 29, 2015, 02:01:54 pm by merockstar »

Offline chryspano

  • Hero Member
  • *****
  • Posts: 610
    • View Profile
Re: Web release for importing large wallets (testers needed)
« Reply #10 on: October 29, 2015, 04:00:03 pm »
This also fixed an issue I had with vesting balances.

Offline jamesc

Re: Web release for importing large wallets (testers needed)
« Reply #11 on: October 29, 2015, 04:17:12 pm »
Code: [Select]
"connecting to wss://bitshares.openledger.info/ws" app.js:2:183711
Configured for BitShares Object { core_asset: "BTS", address_prefix: "BTS", chain_id: "4018d7844c78f6a6c41c6a552b898022310…" } app.js:2:189461
db init done app.js:2:69245
ChainStore Init app.js:12:9104
synced and subscribed, chainstore ready app.js:12:9589
Warning: setState(...): Can only update a mounted or mounting component. This usually means you called setState() on an unmounted component. This is a no-op. vendors.js:1:20286
GenesisFilterWorker start 1ae2f77e0472c8b52ebd.worker.js:1:560
GenesisFilterWorker InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable 1ae2f77e0472c8b52ebd.worker.js:1:786

This appears in Firefox versions under 41.  I had to abandon the worker-thread for the Genesis filter.  Make sure to really tell FF to reload the page and this issue should not show up.  Unfortunately, the nice UI message "Filtering X %" does not appear anymore but at least this should be fixed...

You can follow the progress in the browser's console window, I added a message there. 

Let me know if this is good to merge into the main branch!

Xeldal

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #12 on: October 29, 2015, 04:42:07 pm »
After selecting the file, browser when "not responding" for about 3 mins, then it successfully cut 90k keys down to 476 keys, found balances and imported.

maybe give a warning about not responding but otherwise it works great!
Win7
Firefox 41.0.2

Offline jamesc

Re: Web release for importing large wallets (testers needed)
« Reply #13 on: October 29, 2015, 06:15:54 pm »
After selecting the file, browser when "not responding" for about 3 mins, then it successfully cut 90k keys down to 476 keys, found balances and imported.

maybe give a warning about not responding but otherwise it works great!
Win7
Firefox 41.0.2

You should see an animated spinner now (well it does not really animate in Firefox)...

Offline youlonghun

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: Web release for importing large wallets (testers needed)
« Reply #14 on: October 30, 2015, 07:17:57 am »
success to import,but i get some account that cny-collateral-holder-157 in my wallet,how  to delete them。win10, firefox 41.02。

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12892
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: Web release for importing large wallets (testers needed)
« Reply #15 on: October 30, 2015, 07:41:41 am »
success to import,but i get some account that cny-collateral-holder-157 in my wallet,how  to delete them。win10, firefox 41.02。
Those are your open short positions from bitshares 0.9.3 ... you can't actually delete them .. I though asked for a checkbox to let "empty" accounts hide in your overview page
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

Offline youlonghun

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: Web release for importing large wallets (testers needed)
« Reply #16 on: October 30, 2015, 08:48:03 am »
success to import,but i get some account that cny-collateral-holder-157 in my wallet,how  to delete them。win10, firefox 41.02。
Those are your open short positions from bitshares 0.9.3 ... you can't actually delete them .. I though asked for a checkbox to let "empty" accounts hide in your overview page
i have 10 shorts in 0.9.3,but only get 6 in 2.0,what's wrong with it.

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12892
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: Web release for importing large wallets (testers needed)
« Reply #17 on: October 30, 2015, 09:16:46 am »
Maybe they have been called in the meantime and the funds have been returned to your account keys?
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH

38PTSWarrior

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #18 on: October 30, 2015, 06:17:08 pm »
I have a .json file and it states 'invalid format'.
I think I must rename it right?

Offline chryspano

  • Hero Member
  • *****
  • Posts: 610
    • View Profile
Re: Web release for importing large wallets (testers needed)
« Reply #19 on: October 30, 2015, 06:40:40 pm »
I have a .json file and it states 'invalid format'.
I think I must rename it right?

Make sure you exported with 0.9.3c

If you used the windows version you need to export from console, GUI in windows was buggy

38PTSWarrior

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #20 on: October 30, 2015, 06:49:01 pm »
I have a .json file and it states 'invalid format'.
I think I must rename it right?

Make sure you exported with 0.9.3c

If you used the windows version you need to export from console, GUI in windows was buggy
Yes Windows and gui. jcalfee wrote 0.9.x in the opener..

I need your help. Someone posted the download link for the full chain and I cannot find it.

I will install 0.9.3c now.

I have some keys from the electrum wallet I need to import too!

I need:

1. Link to full chain
2. Nerves for building
3. Help to import single electrum keys (i have them in a csv file)


38PTSWarrior

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #21 on: October 30, 2015, 06:52:06 pm »
But I used windows 0.9.2 to recover keys. Then exported. My netbook is not stong enough to recover keys. Maybe with 0.9.3c it will work.

Offline chryspano

  • Hero Member
  • *****
  • Posts: 610
    • View Profile
Re: Web release for importing large wallets (testers needed)
« Reply #22 on: October 30, 2015, 07:09:44 pm »
I'm a little confused...

I believe that if you export your keys using 0.9.3c (use the console to do so if you have windows) and then use the exported file on  https://bitshares.org/worker you will have succes.

I can't help with  electrum keys and I don't know if they have anything to do with this thread, if yes, someone will help you.

If you have any other issue maybe it could better to open a diferent thread and give as much info as posible so someone can help.

Offline merockstar

Re: Web release for importing large wallets (testers needed)
« Reply #23 on: October 30, 2015, 11:41:00 pm »
No doubt this will work great when I get better internet.

Have you thought about having it automatically regenerate a certain number of keys in case people sent themselves bts without having the chain synced (if it doesnt)? then it would an all in one self-contained key import solution.

38PTSWarrior

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #24 on: October 31, 2015, 12:32:43 am »
I am running 0.9.3c. The last export from Windows GUI 0.9.2 is missing some assets. You told me about that. When I try to import an older backup, it states 'wrong password'. OK back to 0.5.2 just to get that fixed. That should have been fixed in 0.9.3c but OK.

Now I have to delete 0.9.3c again and pray that my netbook is strong enough to recover keys in 0.5.2, then export and import into 0.9.3c and then export and import into 2.0.
  :'(

I build therefore I am sparrtaaa

38PTSWarrior

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #25 on: October 31, 2015, 05:25:51 pm »
Everything alright here. 'loading balance claims' now. But already since 1 hour and I am using the /worker URL. Ubuntu Vivid 32bit. Chromium-Browser and Firefox. Chromium is more stable. I hope it will be finished with loading zoon.

Offline jz831

  • Jr. Member
  • **
  • Posts: 46
    • View Profile
    • Customized ASIC Controllers
Re: Web release for importing large wallets (testers needed)
« Reply #26 on: November 01, 2015, 08:00:47 pm »
I have recovered some of my accounts with this method, but a couple of my accounts are missing some BTS balances - any idea of what to try?  I just recovered one, which is missing ~ 75k BTS from it, and another account has more than that missing.  I took a screenshot on 10/13 of my 0.9.3c balances pre-fork, to use for comparison, but i'm kinda stuck as to what to try. I found an xfer from polo in the 1st week of Aug which would account for 90%+ of the ~75k missing funds in the one account's 2.0 import, is there a way to scan a v1 block in v2?  My other assets' balances are correct, however.  A bit concerning about the missing BTS tho, i'm holding off on importing the 2nd account with the greater missing BTS until I learn more.
https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

38PTSWarrior

  • Guest
Re: Web release for importing large wallets (testers needed)
« Reply #27 on: November 02, 2015, 10:59:55 pm »
Ubuntu Vivid, Firefox 41.0.2
Transaction could not be published (when claiming the balance).

Filtering the keys worked nicely.


Chromium-Browser Version 45.0.2454.101 on Ubuntu 15.04 (Vivid)
There is no filter showing, it goes straight to claim balance and loads forever (I think it's not working or not as fast and good as with Firefox).

Someone knows a combination that works for 32bit Ubuntu 15.04? On Android Chrome did not work but Firefox. On Ubuntu Firefox works until 'transaction not broadcasted'.





« Last Edit: November 03, 2015, 12:36:48 am by 38PTSWarrior »

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 3820
    • View Profile
    • Steemit Blog
  • BitShares: abit
  • GitHub: abitmore
Re: Web release for importing large wallets (testers needed)
« Reply #28 on: December 31, 2015, 08:59:40 pm »
I have recovered some of my accounts with this method, but a couple of my accounts are missing some BTS balances - any idea of what to try?  I just recovered one, which is missing ~ 75k BTS from it, and another account has more than that missing.  I took a screenshot on 10/13 of my 0.9.3c balances pre-fork, to use for comparison, but i'm kinda stuck as to what to try. I found an xfer from polo in the 1st week of Aug which would account for 90%+ of the ~75k missing funds in the one account's 2.0 import, is there a way to scan a v1 block in v2?  My other assets' balances are correct, however.  A bit concerning about the missing BTS tho, i'm holding off on importing the 2nd account with the greater missing BTS until I learn more.
This post may help you: https://bitsharestalk.org/index.php/topic,20822.0.html
BTS account: abit
BTS committee member: abit
BTS witness: in.abit