BitShares Forum

Other => Graveyard => Marketplace => Topic started by: bytemaster on February 11, 2014, 04:14:07 pm

Title: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet [CLOSED]
Post by: bytemaster on February 11, 2014, 04:14:07 pm
In the spirit of supporting all wallets that can contribute to AGS I would like to start a bounty to extract the private keys from any wallet. 

Assuming Armory doesn't use the bitcoinqt wallet.dat, then it qualifies.  Otherwise it does not.

This 300 PTS bounty is for EITHER Electrum or Armory and both can be won for a total of 600 PTS

To win this bounty you must produce a pull request that successfully implements the following import method in a single .cpp and .hpp file with minimal dependencies.   The pull request must work on all platforms.

Code: [Select]
namespace bts  {
      std::vector<fc::ecc::private_key> import_electrum_wallet( const fc::path& wallet_dat, const std::string& passphrase );
}


toast edit: s/electrium/electrum/g
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: thisisausername on February 13, 2014, 12:07:35 am
I'm almost finished with this for electrum.

I have cython generating

Code: [Select]
std::vector<std::string> pyimport_electrum_wallet(std::string &__pyx_v_path, std::string &__pyx_v_pp)
and am calling it like

Code: [Select]
namespace bts
{
   std::vector<fc::ecc:private_key> import_electrum_wallet( const fc::path& wallet_dat, const string& passphrase )
   {
      size_t i;
      std::vector<fc::ecc::private_key> keysRet;

      if (!fc::exists(wallet_dat)
         return std::vector<fc::ecc::private_key>();

      std::vector<string> keys = pyimport_electrum_wallet(wallet_dat.to_native_ansi_path(), passphrase);
      for( i = 0; i < keys.size(); i++ )
      {
         fc::datastream<const char*> stream(keys[i].c_str(), keys[i].size());
         fc::sha256 bits;
         stream >> bits;
         keysRet.push_back(fc::ecc::private_key::regenerate(bits));
      }
      return keysRet;
   }

I'm not so sure about the C++ part.  I still can't compile anything under BTS, waiting for my boost to finish upgrading to 1.53 (Gentoo...)

Also have to figure out how to do pull requests and whatnot.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: bytemaster on February 13, 2014, 12:38:12 am
I'm almost finished with this for electrum.

I have cython generating

Code: [Select]
std::vector<std::string> pyimport_electrum_wallet(std::string &__pyx_v_path, std::string &__pyx_v_pp)
and am calling it like

Code: [Select]
namespace bts
{
   std::vector<fc::ecc:private_key> import_electrum_wallet( const fc::path& wallet_dat, const string& passphrase )
   {
      size_t i;
      std::vector<fc::ecc::private_key> keysRet;

      if (!fc::exists(wallet_dat)
         return std::vector<fc::ecc::private_key>();

      std::vector<string> keys = pyimport_electrum_wallet(wallet_dat.to_native_ansi_path(), passphrase);
      for( i = 0; i < keys.size(); i++ )
      {
         fc::datastream<const char*> stream(keys[i].c_str(), keys[i].size());
         fc::sha256 bits;
         stream >> bits;
         keysRet.push_back(fc::ecc::private_key::regenerate(bits));
      }
      return keysRet;
   }

I'm not so sure about the C++ part.  I still can't compile anything under BTS, waiting for my boost to finish upgrading to 1.53 (Gentoo...)

Also have to figure out how to do pull requests and whatnot.

FC_ASSERT( fc::exists( wallet_dat ), "unable to find ${wallet}", ("wallet",wallet_dat) ) is better than returning nothing...
or better yet use FC_THROW_EXCEPTION( file_not_found... ) 

 
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: thisisausername on February 13, 2014, 01:12:52 am
FC_ASSERT( fc::exists( wallet_dat ), "unable to find ${wallet}", ("wallet",wallet_dat) ) is better than returning nothing...
or better yet use FC_THROW_EXCEPTION( file_not_found... )

Done.  Here are my changes as they currently stand, https://github.com/tiau/BitShares/compare .  I haven't initiated a pull request since I'm still waiting on boost to be able to test the changes.  There are almost certainly bugs in the not-cython-generated C++ portion and I doubt I did the CMakeList.txt changes right either...

One question: Is electrum_python.cpp okay as it stands (50,000 lines of mostly dead code) since the C++ compiler will remove all the cruft anyways or should it be cleaned up?
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: bytemaster on February 13, 2014, 01:32:25 am
FC_ASSERT( fc::exists( wallet_dat ), "unable to find ${wallet}", ("wallet",wallet_dat) ) is better than returning nothing...
or better yet use FC_THROW_EXCEPTION( file_not_found... )

Done.  Here are my changes as they currently stand, https://github.com/tiau/BitShares/compare .  I haven't initiated a pull request since I'm still waiting on boost to be able to test the changes.  There are almost certainly bugs in the not-cython-generated C++ portion and I doubt I did the CMakeList.txt changes right either...

One question: Is electrum_python.cpp okay as it stands (50,000 lines of mostly dead code) since the C++ compiler will remove all the cruft anyways or should it be cleaned up?

For 300 PTS it should be cleaned up.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: thisisausername on February 13, 2014, 07:09:07 am
Well, I spoke a bit soon.  Everything works great under python, but attempting to use the cython-generated code I get,
Code: [Select]
Exception NameError: "name 'WalletStorage' is not defined" in 'lib.wallet.pyimport_electrum_wallet' ignored at runtime.

Anyone have any idea what's up with that?  I've previously only ever cython'd much more simple programs and things have 'just worked'.  It works using nuitka, but I'm not sure how to specify nicely callable C++ functions using it (calling an external executable just seems too ugly.)

I'll hack on this a bit more tomorrow; see if I can figure it out.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: BrownBear on February 13, 2014, 09:10:35 pm
I've added support for Armory:
https://github.com/InvictusInnovations/BitShares/pull/39
https://github.com/InvictusInnovations/fc/pull/9

Note:
this is compatible with DreKrobs PTSArmory wallets, which have a different structure due to compressed public keys
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: bytemaster on February 14, 2014, 12:52:30 am
I've added support for Armory:
https://github.com/InvictusInnovations/BitShares/pull/39
https://github.com/InvictusInnovations/fc/pull/9

Note:
this is compatible with DreKrobs PTSArmory wallets, which have a different structure due to compressed public keys

Is it compatible with BTCArmory  wallets which may have given to the ags address?
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: BrownBear on February 14, 2014, 03:19:17 am
It can read BTC wallets as well as PTS wallets.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: thisisausername on February 14, 2014, 11:38:22 am
Sorry for not getting to this yesterday, had to deal with some snow.

In any case, I think I'm done.  See pull request here (https://github.com/InvictusInnovations/BitShares/pull/42).

A few caveats:


0) Haven't tested it yet (beyond that it compiles and that the generated C++ code properly dumps keys -- haven't tested the glue C++.)  I'll get on that as time permits in the next day or two.


1) I'm especially unsure about
Code: [Select]
fc::datastream<const char*> stream(keys[i].c_str(), keys[i].size());
fc::sha256 bits;
stream >> bits;
keysRet.push_back(fc::ecc::private_key::regenerate(bits));

I took it from DerKorb's multibit wallet import implementation.  In my case keys{i} is going to be a 51-character, starts-with-a-5 style uncompressed key -- but I haven't looked into whether that's what he was dealing with as well.


2) The cython-generated C++ file isn't very clean.  I realized that a better way to do things than to make a minimal private key extractor from the electrum code would be to minimally modify the electrum code, thereby making it easier to keep the import function working should electrum change the way they handle wallets.  In any case I added to vendor (wrong place, I know, there wasn't anywheres that fit too well) a makefile script and patch that lets one copy over the electrum lib directory and easily regenerate the C++ code.


3) I made the cmake file rely upon python 2.7.5 exactly.  This isn't right, we just need python 2.x, but I cannot for the life of me figure out how to set a maximum (minimum required versions are easy) that cmake will find and it always goes for 3.3 on my system.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: BrownBear on February 14, 2014, 11:44:59 am
Regenerate keys needs 32 byte raw key data.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: thisisausername on February 14, 2014, 12:20:15 pm
Regenerate keys needs 32 byte raw key data.

Awesomeness, thanks for the info.  Pull request updated with code that may work, much better than the previous code that certainly didn't!
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: batmaninpink on February 15, 2014, 08:02:48 am
I am working on the Electrum support as well, should be done by tonight and does not rely on Python.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: bytemaster on February 15, 2014, 08:08:53 am
Regenerate keys needs 32 byte raw key data.

Awesomeness, thanks for the info.  Pull request updated with code that may work, much better than the previous code that certainly didn't!

Can we get unit tests for this code.  I need more assurances than 'may work' :)
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: batmaninpink on February 16, 2014, 08:40:23 am
Here is my take on the support for importing keys from Electrum wallets  - no external dependencies - uses Boost::Spirit to parse the python wallet dictionary:

https://github.com/InvictusInnovations/BitShares/pull/45
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: thisisausername on February 16, 2014, 11:30:00 am
Can we get unit tests for this code.  I need more assurances than 'may work' :)

Done.  Though badmaninpink's solution is probably better.  Mine definitely depends heavily upon python.

The benefit of nigh-instantly adapting to electrum changing the way they do wallets...  eh, not very likely.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: batmaninpink on February 28, 2014, 07:08:06 pm
Both my Electrum wallet support and Brownbear's Armory support has now been successfully merge into main tree - so time to close and pay now that it happens to be the 28th? ;)
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet
Post by: bytemaster on February 28, 2014, 07:35:49 pm
CLOSED & PAID
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet [CLOSED]
Post by: BrownBear on March 01, 2014, 04:15:44 pm
I can see the payment to batmaninpink, but not the one to me for the armory wallet.
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet [CLOSED]
Post by: bytemaster on March 01, 2014, 07:15:22 pm
I can see the payment to batmaninpink, but not the one to me for the armory wallet.

Dang, I read the prior bounty regarding the Armory and completely missed yours.   So here is what I can do.  I can either send you 600 PTS today or I can send you 300 PTS today and 400 BTS when the wallet is ready.   
Title: Re: 2 x 300 PTS Bounty to Import Electrum or Armory Wallet [CLOSED]
Post by: BrownBear on March 03, 2014, 11:00:25 am
300 PTS and 400 BTS is fine thanks.