GUIDE: How to remove unneeded keys from Bitshares 0.9.3 export_keys json

[svk]

Instructions for the novice Windows user:

You will need to download nodejs, git, and Python 3 first and foremost.

• Open a command prompt by typing "cmd" in your windows search in the start menu
• Clone the graphene-ui repoby typing "git clone https://github.com/cryptonomex/graphene-ui.git" in the cmd window. (remember that what ever directory/file location is showing on the prompt is where the graphen-ui download will be located
• Now change the directory to the graphene-ui file location, type

Code: [Select]

cd C:\your\file\location\graphene-ui\bloom_filter

• then type

Code: [Select]

npm install

• Now find the wallet file you downloaded from Bitshares 0.93 and change the wallet name to

Code: [Select]

wallet.json

• Copy the newly named wallet.json file
• open the graphene-ui file location (the actual file, not in the cmd). Then open the "bloom_filter" file and paste the new wallet.json file inside
• Go back to the command prompt and type

Code: [Select]

npm run start-windows

In the cmd window you will see it start to run and depending on how many keys you have it could take a little while. Also you should see the sha 1 sum at the beginning too (3cee441d8d28.......)
When it is finished you will have the a file with only the necessary keys linked to accounts and balances.

The new key file is in the graphene-ui/bloom_filter file and it is called

Code: [Select]

filtered_wallet.json
This just worked for me again to make sure my directions were correct. Unless someone changes things in the graphene-ui file you download from git.

Thanks for the write-up!

[Bitshiz]

Instructions for the novice Windows user:

You will need to download nodejs, git, and Python 3 first and foremost.

• Open a command prompt by typing "cmd" in your windows search in the start menu
• Clone the graphene-ui repoby typing "git clone https://github.com/cryptonomex/graphene-ui.git" in the cmd window. (remember that what ever directory/file location is showing on the prompt is where the graphen-ui download will be located
• Now change the directory to the graphene-ui file location, type

Code: [Select]

cd C:\your\file\location\graphene-ui\bloom_filter

• then type

Code: [Select]

npm install

• Now find the wallet file you downloaded from Bitshares 0.93 and change the wallet name to

Code: [Select]

wallet.json

• Copy the newly named wallet.json file
• open the graphene-ui file location (the actual file, not in the cmd). Then open the "bloom_filter" file and paste the new wallet.json file inside
• Go back to the command prompt and type

Code: [Select]

npm run start-windows

In the cmd window you will see it start to run and depending on how many keys you have it could take a little while. Also you should see the sha 1 sum at the beginning too (3cee441d8d28.......)
When it is finished you will have the a file with only the necessary keys linked to accounts and balances.

The new key file is in the graphene-ui/bloom_filter file and it is called

Code: [Select]

filtered_wallet.json
This just worked for me again to make sure my directions were correct. Unless someone changes things in the graphene-ui file you download from git.

[Bitshiz]

Thanks!

[svk]

Well I just got this error now:

C:\Users\skysc\graphene-ui\bloom_filter>npm run start-windows

> bitshares-keys-bloom-filter@1.0.0 start-windows C:\Users\skysc\graphene-ui\bloom_filter
> type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json

bloom_bitshares.dat (1048576 bytes) sha1 3cee441d8d28ab3b26aea149630fa2a96a91845c

C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:59
        var inputJSON = inputChunks.join("")
                                    ^

RangeError: Invalid string length
    at Array.join (native)
    at Socket.<anonymous> (C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:59:37)
    at emitNone (events.js:91:20)
    at Socket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "run" "start-windows"
npm ERR! node v6.9.4
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! bitshares-keys-bloom-filter@1.0.0 start-windows: `type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the bitshares-keys-bloom-filter@1.0.0 start-windows script 'type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the bitshares-keys-bloom-filter package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs bitshares-keys-bloom-filter
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls bitshares-keys-bloom-filter
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\skysc\graphene-ui\bloom_filter\npm-debug.log

I deleted graphene-ui and cloned it again before I ran this. I then ran "npm install" and copied my wallet.json(I changed the name to wallet.json) file to the bloom_filter folder.

I figured it out! Kind of. My json file is over a 1,013,686KB. So it is too big of a json file. I tried on another wallet file that is only 90,252KB and it is working. Is there any way to get this to work on a larger file? I don't think I am missing any balances but just curious what the limit or why there would be a limit on the json file size?

kept 2099 of 534165 keys

kept 2102 of 534168 keys

kept 2103 of 534169 keys

Thanks @svk

Wow that is a lot of keys! I have a 16MB json file myself but it only has around 85k keys.

If you're able to edit the file it's certainly possible to cut and paste the keys inside and assemble multiple smaller files. You'll have to do that by hand though, I recommend sublime editor for big JSON files like that.

[Bitshiz]

Well I just got this error now:

C:\Users\skysc\graphene-ui\bloom_filter>npm run start-windows

> bitshares-keys-bloom-filter@1.0.0 start-windows C:\Users\skysc\graphene-ui\bloom_filter
> type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json

bloom_bitshares.dat (1048576 bytes) sha1 3cee441d8d28ab3b26aea149630fa2a96a91845c

C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:59
        var inputJSON = inputChunks.join("")
                                    ^

RangeError: Invalid string length
    at Array.join (native)
    at Socket.<anonymous> (C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:59:37)
    at emitNone (events.js:91:20)
    at Socket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "run" "start-windows"
npm ERR! node v6.9.4
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! bitshares-keys-bloom-filter@1.0.0 start-windows: `type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the bitshares-keys-bloom-filter@1.0.0 start-windows script 'type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the bitshares-keys-bloom-filter package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     type wallet.json | node bloom_filter_wallet.js > filtered_wallet.json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs bitshares-keys-bloom-filter
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls bitshares-keys-bloom-filter
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\skysc\graphene-ui\bloom_filter\npm-debug.log

I deleted graphene-ui and cloned it again before I ran this. I then ran "npm install" and copied my wallet.json(I changed the name to wallet.json) file to the bloom_filter folder.

I figured it out! Kind of. My json file is over a 1,013,686KB. So it is too big of a json file. I tried on another wallet file that is only 90,252KB and it is working. Is there any way to get this to work on a larger file? I don't think I am missing any balances but just curious what the limit or why there would be a limit on the json file size?

kept 2099 of 534165 keys

kept 2102 of 534168 keys

kept 2103 of 534169 keys

Thanks @svk 

[svk]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Ok I had a look and it was indeed trying to load modules that no longer exist. I pushed an update and also included a bloom_bitshares.dat file that you can use if don't want to compile your own.

To run the filter now, just go to the "bloom_filter" folder, run "npm i" to get the dependencies, name your old wallet file "wallet.json" then  do "npm start". The output will be in a file called "filtered_wallet.json"

I didn't have an old wallet.json file to test it with so the npm start method might not work properly, if so just run it manually like the original instructions said.

so I tried it and got this error because of the 'cat' command:

Code: [Select]

'cat' is not recognized as an internal or external command,
operable program or batch file.

npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "start"
npm ERR! node v6.9.4
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! bitshares-keys-bloom-filter@1.0.0 start: `cat wallet.json | node bloom_filter_wallet.js > filtered_wallet.json`
npm ERR! Exit status 255
npm ERR!
npm ERR! Failed at the bitshares-keys-bloom-filter@1.0.0 start script 'cat wallet.json | node bloom_filter_wallet.js > filtered_wallet.json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the bitshares-keys-bloom-filter package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     cat wallet.json | node bloom_filter_wallet.js > filtered_wallet.json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs bitshares-keys-bloom-filter
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls bitshares-keys-bloom-filter
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\skysc\graphene-ui\bloom_filter\npm-debug.log

C:\Users\skysc\graphene-ui\bloom_filter>

I then tried to follow instructions from the original post and got error:

Code: [Select]

C:\Users\skysc\graphene-ui\bloom_filter>wallet.json | node ./bloom_filter_wallet.js > filtered_wallet.json'
C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:29
    if (err) throw err
             ^

Error: ENOENT: no such file or directory, open 'C:\Users\skysc\graphene-ui\bloom_filter\bloom.dat'
    at Error (native)


So then i decided to switch out my original bloom.dat file I created yesterday instaed of using the bloom_bitshares.dat and then got this error:

Code: [Select]

C:\Users\skysc\graphene-ui\bloom_filter>wallet.json | node ./bloom_filter_wallet.js > filtered_wallet.json
bloom_bitshares.dat (1048576 bytes) sha

undefined:1



SyntaxError: Unexpected end of JSON input
    at Object.parse (native)
    at Socket.<anonymous> (C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:59:27)
    at emitNone (events.js:91:20)
    at Socket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)


Those instructions assume a Linux environment, the cat command is not available on Windows. You also need to fetch the latest version, there is no long a reference to web/lib/common/hash so apparently you're not using the latest release.

I've identified a bug however so I've pushed a new version just now, and this time I've tested it with an old wallet.json I had lying around so I know it works. I've also added a npm script for windows that you can use, try this in your node console:

Code: [Select]

npm run start-windows
Your wallet  json file must be called wallet.json for this to work.

[Bitshiz]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Ok I had a look and it was indeed trying to load modules that no longer exist. I pushed an update and also included a bloom_bitshares.dat file that you can use if don't want to compile your own.

To run the filter now, just go to the "bloom_filter" folder, run "npm i" to get the dependencies, name your old wallet file "wallet.json" then  do "npm start". The output will be in a file called "filtered_wallet.json"

I didn't have an old wallet.json file to test it with so the npm start method might not work properly, if so just run it manually like the original instructions said.

so I tried it and got this error because of the 'cat' command:

Code: [Select]

'cat' is not recognized as an internal or external command,
operable program or batch file.

npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "start"
npm ERR! node v6.9.4
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! bitshares-keys-bloom-filter@1.0.0 start: `cat wallet.json | node bloom_filter_wallet.js > filtered_wallet.json`
npm ERR! Exit status 255
npm ERR!
npm ERR! Failed at the bitshares-keys-bloom-filter@1.0.0 start script 'cat wallet.json | node bloom_filter_wallet.js > filtered_wallet.json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the bitshares-keys-bloom-filter package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     cat wallet.json | node bloom_filter_wallet.js > filtered_wallet.json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs bitshares-keys-bloom-filter
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls bitshares-keys-bloom-filter
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\skysc\graphene-ui\bloom_filter\npm-debug.log

C:\Users\skysc\graphene-ui\bloom_filter>

I then tried to follow instructions from the original post and got error:

Code: [Select]

C:\Users\skysc\graphene-ui\bloom_filter>wallet.json | node ./bloom_filter_wallet.js > filtered_wallet.json'
C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:29
    if (err) throw err
             ^

Error: ENOENT: no such file or directory, open 'C:\Users\skysc\graphene-ui\bloom_filter\bloom.dat'
    at Error (native)


So then i decided to switch out my original bloom.dat file I created yesterday instaed of using the bloom_bitshares.dat and then got this error:

Code: [Select]

C:\Users\skysc\graphene-ui\bloom_filter>wallet.json | node ./bloom_filter_wallet.js > filtered_wallet.json
bloom_bitshares.dat (1048576 bytes) sha

undefined:1



SyntaxError: Unexpected end of JSON input
    at Object.parse (native)
    at Socket.<anonymous> (C:\Users\skysc\graphene-ui\bloom_filter\bloom_filter_wallet.js:59:27)
    at emitNone (events.js:91:20)
    at Socket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)


[Bitshiz]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Ok I had a look and it was indeed trying to load modules that no longer exist. I pushed an update and also included a bloom_bitshares.dat file that you can use if don't want to compile your own.

To run the filter now, just go to the "bloom_filter" folder, run "npm i" to get the dependencies, name your old wallet file "wallet.json" then  do "npm start". The output will be in a file called "filtered_wallet.json"

I didn't have an old wallet.json file to test it with so the npm start method might not work properly, if so just run it manually like the original instructions said.

Thank you! I will try it today.

[Bitshiz]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Like Chris mentioned you could try the light wallet as it includes the bloom filter directly, as does https://bitshares.org/wallet I believe.

The bloom filter program in the graphene-ui repo is probably out of sync with the location of files in the repo right now after I moved a lot of things around, I'll have a look and update it if necessary.

The light wallet crashes every time I try to load it. I suspect it is because my account has so many keys as in BTSX I was trading every day, many trades a day. I also have about 35 accounts but only used probably 4 of them although I did send small amounts to about 10  others out of the 35. The wallet file I am trying to import only had 3 acounts attached and is 1,013,686 KB, so it is a large json file. But every time i try to load it into the light client, after about 10 minutes the entire screen goes white. I have smaller wallet files with more accounts because it was before I regenerated many keys and when I load those it will eventually list all my accounts in the light client but when I hit claim balances, after an hour or more the client freezes up too.

[svk]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Ok I had a look and it was indeed trying to load modules that no longer exist. I pushed an update and also included a bloom_bitshares.dat file that you can use if don't want to compile your own.

To run the filter now, just go to the "bloom_filter" folder, run "npm i" to get the dependencies, name your old wallet file "wallet.json" then  do "npm start". The output will be in a file called "filtered_wallet.json"

I didn't have an old wallet.json file to test it with so the npm start method might not work properly, if so just run it manually like the original instructions said.

[svk]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Like Chris mentioned you could try the light wallet as it includes the bloom filter directly, as does https://bitshares.org/wallet I believe.

The bloom filter program in the graphene-ui repo is probably out of sync with the location of files in the repo right now after I moved a lot of things around, I'll have a look and update it if necessary.

[Bitshiz]

Any help with this error. I cant upload my .json to the web client because I have too many keys and it crashes every fucking time. been trying to access my BTS for over a year and finally tried this manual but am getting this error. ANyone know what this is? And how to fix? This happens after I run

Code: [Select]

OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json but with my wallet names. Doing this through windows 10 64 and I have downloaded everything this tutorial says.

Code: [Select]

module.js:471
    throw err;
    ^

Error: Cannot find module '../web/lib/common/hash'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Users\skysc\graphene-ui\programs\bloom_filter_wallet.js:20:9)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

[Chris4210]

Yes, the filter is integrated in the light wallet too.

Just download the standard Windows light client and upload your backup to integrate to 2.0 .

[ripplexiaoshan]

Is there is window version?

[cass]

works like a charme .. at least for me .. thx for this script!

[unreadPostsSinceLastVisit]

I have a question.

Is the process different for muse? or would the same resulting wallet be okay to use for muse?

if it should be okay, then I have a bigger problem, because when I attempted to import my pruned wallet into muse, it only say my ags/pts notes, and not the ones I bought from the exchange, and it didn't import any of my names.

If you read the three posts before yours you'll have the answer.

Aww man. Sorry.

Going to go try that now. wish me luck.

edit: looks like it worked. about to test.

[svk]

I have a question.

Is the process different for muse? or would the same resulting wallet be okay to use for muse?

if it should be okay, then I have a bigger problem, because when I attempted to import my pruned wallet into muse, it only say my ags/pts notes, and not the ones I bought from the exchange, and it didn't import any of my names.

If you read the three posts before yours you'll have the answer.

[unreadPostsSinceLastVisit]

I have a question.

Is the process different for muse? or would the same resulting wallet be okay to use for muse?

if it should be okay, then I have a bigger problem, because when I attempted to import my pruned wallet into muse, it only say my ags/pts notes, and not the ones I bought from the exchange, and it didn't import any of my names.

[mdj]

This worked for me however I am not seeing any of my NOTES in the import.

This guide is for BTS, you can use the procedure with MUSE as well but some things need to be changed.

First you need to use the MUSE genesis.json file. Secondly, there are some BTS specific variables in one of the scripts. Look for GPH and replace it with BTS, and also BTS should be replaced with MUSE.

Thanks! Will give it a try later

[svk]

This worked for me however I am not seeing any of my NOTES in the import.

This guide is for BTS, you can use the procedure with MUSE as well but some things need to be changed.

First you need to use the MUSE genesis.json file. Secondly, there are some BTS specific variables in one of the scripts. Look for GPH and replace it with BTS, and also BTS should be replaced with MUSE.

[mdj]

This worked for me however I am not seeing any of my NOTES in the import.

[jamesc]

We are testing the web version .. If it works well I can merge it and it will make its way into the installers:

Web release for importing large wallets (testers needed)
https://bitsharestalk.org/index.php/topic,19557.msg251364.html#top

[youlonghun]

waiting for an easy way，it hard for me in windows。

[EstefanTT]

That's great news !!!

+500% !

[jamesc]

I'm not sure if I wait a simpler solution or I dig into this one.

Anyone knows how much time it could take to have a wallet base simpler solution ?

Anyone could give me an estimation of the time it could takes to me to acheive it with the explanations above ? I've wrote a few lines of html, css and autohotkey but my code knowledge is not very deep. I'm on windows 10 / light client.

Thanks

Today for notes and tomorrow to test it.  So tomorrow or the day after...

[Riverhead]

This process is very useful thank you.

kept 168 of 25103 keys
kept 61 of 10157 keys

Needless to say the import process worked much better with the reduced number of keys.

[bitcrab]

on windows the point is the last step: how to execute the cat command on windows?

type

thanks, I failed under Ubuntu but succeeded under windows.

[Chris4210]

push the poll:

https://bitsharestalk.org/index.php/topic,19432.0.html

Please vote here

[EstefanTT]

I'm not sure if I wait a simpler solution or I dig into this one.

Anyone knows how much time it could take to have a wallet base simpler solution ?

Anyone could give me an estimation of the time it could takes to me to acheive it with the explanations above ? I've wrote a few lines of html, css and autohotkey but my code knowledge is not very deep. I'm on windows 10 / light client.

Thanks

[cube]

on windows the point is the last step: how to execute the cat command on windows?

type

[bitcrab]

Instructions are for *nix/OSX

...can this be done on Windows? I have some basic CLI-knowledge, but not much beyond simply copy/pasting commands.

I think so, its just a bit more work to get git, python and node installed, but apart from that there's nothing stopping this from working on windows.

on windows the point is the last step: how to execute the cat command on windows?

[Musewhale]

cool

[roadscape]

The export format is just a "minimal" version of the (encrypted) wallet, which is why you don't need to have your wallet unlocked when you export the keys. The keys file is as safe as the locked wallet itself.. not that I'd recommend sharing it

[NewMine]

I don't seem to need a password to do the wallet prune.

newmine and chris, do you guys want me to follow the instructions on your wallets for you?

edit: I should point out that I haven't successfully claimed the balances on from the keys on the new wallet i've created for myself. but I got the script to run, and the hash checked out. runnign the balance import now and its taking forever still with 278 keys

Sure. Here is my private key: 5Hfkfi85akfmHPL12890aiyeW1xX6aLs9gdS

i assumed the private keys would be encrypted.

Why?

[unreadPostsSinceLastVisit]

I don't seem to need a password to do the wallet prune.

newmine and chris, do you guys want me to follow the instructions on your wallets for you?

edit: I should point out that I haven't successfully claimed the balances on from the keys on the new wallet i've created for myself. but I got the script to run, and the hash checked out. runnign the balance import now and its taking forever still with 278 keys

Sure. Here is my private key: 5Hfkfi85akfmHPL12890aiyeW1xX6aLs9gdS

i assumed the private keys would be encrypted.

[NewMine]

I don't seem to need a password to do the wallet prune.

newmine and chris, do you guys want me to follow the instructions on your wallets for you?

edit: I should point out that I haven't successfully claimed the balances on from the keys on the new wallet i've created for myself. but I got the script to run, and the hash checked out. runnign the balance import now and its taking forever still with 278 keys

Sure. Here is my private key: 5Hfkfi85akfmHPL12890aiyeW1xX6aLs9gdS

[unreadPostsSinceLastVisit]

I don't seem to need a password to do the wallet prune.

newmine and chris, do you guys want me to follow the instructions on your wallets for you?

edit: I should point out that I haven't successfully claimed the balances on from the keys on the new wallet i've created for myself. but I got the script to run, and the hash checked out. runnign the balance import now and its taking forever still with 278 keys

[NewMine]

Instructions are for *nix/OSX

...can this be done on Windows? I have some basic CLI-knowledge, but not much beyond simply copy/pasting commands.

I think so, its just a bit more work to get git, python and node installed, but apart from that there's nothing stopping this from working on windows.

I think he or she was looking for a guide. Something not in dev speak. I'm waiting too.  I am glad you guys are working this problem but what you are proposing is something like if I ask you for directions to the supermarket and you respond with actual gps coordinates. We all know what they mean, just not all of us have the means to interpret or implement them.

[unreadPostsSinceLastVisit]

Can someone please create a walk through guide here? I have absolutely now idea what you guys are talking about. Some help would be great! Thank you

this guide is for linux. i think it's different if you're using windows.

[unreadPostsSinceLastVisit]

Hooray I got it to work and now I have a wallet with some 258 keys!

[Chris4210]

Can someone please create a walk through guide here? I have absolutely now idea what you guys are talking about. Some help would be great! Thank you

[svk]

Instructions are for *nix/OSX

...can this be done on Windows? I have some basic CLI-knowledge, but not much beyond simply copy/pasting commands.

I think so, its just a bit more work to get git, python and node installed, but apart from that there's nothing stopping this from working on windows.

[nomoreheroes7]

Instructions are for *nix/OSX

...can this be done on Windows? I have some basic CLI-knowledge, but not much beyond simply copy/pasting commands.

[svk]

made it to step 2

Code: [Select]

merockstar@merockstar-HP-Pavilion-dv6-Notebook-PC:~/Desktop/key pruning$ cat mig6.json | node ./bloom_filter_wallet.js > pruned1.json

module.js:340
    throw err;
          ^
Error: Cannot find module '../src/common/hash'
  at Function.Module._resolveFilename (module.js:338:15)
  at Function.Module._load (module.js:280:25)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/merockstar/Desktop/key pruning/bloom_filter_wallet.js:20:9)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Function.Module.runMain (module.js:497:10)
  at startup (node.js:119:16)
  at node.js:902:3

coffee script is installe

The instructions do not give you all the files you need. One way is to clone the repository. I have a sample maybe by for Monday so you can do this in the web wallet instead.

clone what repository?

He means the graphene-ui repo. It looks like I missed the fact that the second script depends on some files from the graphene-ui repo, my bad. I'll update the instructions in the OP

[unreadPostsSinceLastVisit]

made it to step 2

Code: [Select]

merockstar@merockstar-HP-Pavilion-dv6-Notebook-PC:~/Desktop/key pruning$ cat mig6.json | node ./bloom_filter_wallet.js > pruned1.json

module.js:340
    throw err;
          ^
Error: Cannot find module '../src/common/hash'
  at Function.Module._resolveFilename (module.js:338:15)
  at Function.Module._load (module.js:280:25)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/merockstar/Desktop/key pruning/bloom_filter_wallet.js:20:9)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Function.Module.runMain (module.js:497:10)
  at startup (node.js:119:16)
  at node.js:902:3

coffee script is installe

The instructions do not give you all the files you need. One way is to clone the repository. I have a sample maybe by for Monday so you can do this in the web wallet instead.

clone what repository?

[rajarush]

Oh man this is so simple! Thank you for the good work!

[NewMine]

This is all you have to do? There is no way anyone will have a problem with this method. /s

[jamesc]

made it to step 2

Code: [Select]

merockstar@merockstar-HP-Pavilion-dv6-Notebook-PC:~/Desktop/key pruning$ cat mig6.json | node ./bloom_filter_wallet.js > pruned1.json

module.js:340
    throw err;
          ^
Error: Cannot find module '../src/common/hash'
  at Function.Module._resolveFilename (module.js:338:15)
  at Function.Module._load (module.js:280:25)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/merockstar/Desktop/key pruning/bloom_filter_wallet.js:20:9)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Function.Module.runMain (module.js:497:10)
  at startup (node.js:119:16)
  at node.js:902:3

coffee script is installe

The instructions do not give you all the files you need. One way is to clone the repository. I have a sample maybe by for Monday so you can do this in the web wallet instead.

[unreadPostsSinceLastVisit]

made it to step 2

Code: [Select]

merockstar@merockstar-HP-Pavilion-dv6-Notebook-PC:~/Desktop/key pruning$ cat mig6.json | node ./bloom_filter_wallet.js > pruned1.json

module.js:340
    throw err;
          ^
Error: Cannot find module '../src/common/hash'
  at Function.Module._resolveFilename (module.js:338:15)
  at Function.Module._load (module.js:280:25)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/merockstar/Desktop/key pruning/bloom_filter_wallet.js:20:9)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Function.Module.runMain (module.js:497:10)
  at startup (node.js:119:16)
  at node.js:902:3

coffee script is installed

[roadscape]

Much needed, thanks!!

[jamesc]

...backup your original key file as it contains all your keys.

[jamesc]

This is from Step 1: https://www.dropbox.com/s/jw0r603u1klm7s5/bloom.dat
The sha1 sum will print when you run step two... It should print:

Code: [Select]

bloom.dat (1048576 bytes) sha1 3cee441d8d28ab3b26aea149630fa2a96a91845c

[svk]

Thanks to awesome work by theoretical and jcalfee we now have two simple command line tools that will remove keys that are not in the Bitshares 2 genesis file from your keys json file generated by the wallet_export_keys command. I've just tested it on a wallet with 95k keys (16MB) and it was reduced to 375 keys (64kB). This might get embedded in the Bitshares 2 wallet and in a future 0.9.4 release of legacy Bitshares, but I wanted to share it with those of you who are struggling with importing your keys.

DISCLAIMER: try at your own risk, I'm not responsible for any loss of keys or balances, etc. Instructions are for *nix/OSX
DISCAIMER 2: Please backup and keep safe your original keys file first

The tools were written using Python and Nodejs so you'll need those installed.

Step 1: Generate a bloom filter file from Bitshares 2 genesis file

• Create a new folder somewhere to keep all the files you will download
• Download the genesis file from github: https://raw.githubusercontent.com/bitshares/bitshares-2/bitshares/genesis.json
• Download the bloom filter script: https://raw.githubusercontent.com/bitshares/bitshares-2/bitshares/programs/genesis_util/create_bloom_filter.py
• Run the script using the following command

Code: [Select]

python create_bloom_filter.py -i ./genesis.json --o bloom.dat
Step 2: Filter your old wallet.json using the bloom filter

• Clone the graphene-ui repo: "git clone https://github.com/cryptonomex/graphene-ui.git"
• In a terminal, go inside the cloned repo: "cd graphene-ui/dl/" and install dependencies "npm install"
• Copy the bloom.dat file you generated as well as your wallet.json file to "graphene-ui/dl/programs"
• From inside the folder "graphene-ui/dl/programs", apply the filter to your wallet.json using:

Code: [Select]

cat OLD_WALLET_NAME.json | node ./bloom_filter_wallet.js > NEW_WALLET_NAME.json
The script will run, telling you as it goes how many keys it's removed and has processed.

Step 3: Profit
Once that's done, you should have a MUCH smaller json file containing the keys you need to import in the Bitshares 2.0 wallet of your choice, light wallet, OpenLedger, localhost, etc.