250 PTS - BOUNTY - Import wallet.dat Private Keys via API call (2 PTS Referral)

[leeyt]

hey，bytemaster!，Why not publish an address using in BITMESSAGE or KEYHOTEE ,so someone who is still in darkness could connect with you and yous in secret?After all,they will help iii,and on the other side,they are helping themself!

[batmaninpink]

  ...
Please provide me your PTS addresses.

Thanks.

[batmaninpink]

well awesome, welcome to the forum and thanks for letting everyone know they are wasting their time on this because you are nearly finished. that's a two person nightshift and 20+ hours of coding beeing utterly useless now.

Sorry, I started this yesterday when being stuck at an airport - fully aware that I was probably too late anyway. So any status update would've been "I am looking into this" - which I am sure would not have deterred you or anyone else from continuing (I see a couple of those status updates earlier in this thread...).

Batmaninpink, your code was superior upon inspection due to proper use of the fc library for managing encryption.  The other submission had potential memory leaks and used OpenSSL calls directly without clear tracking of memory or exception safety.   Your submission also had better abstraction of data parsing. 

I have merged your code and will be evaluating it prior to paying the bounty.

Drekrob,  your submission was also very good and absent the competition may have been accepted after some robustness enhancements.  I will give you a 30 PTS tip for your submission (~$360 dollars) which should partially recognize your time.    These bounties are a kind of interview and so redundant work actually has value to us.

It is very clear to me that you are both talented developers and I would like to discuss how we can work together on a non-bounty basis to eliminate redundant effort.  I am looking for some serious c++ full time developers.

Please provide me your PTS addresses.

PTS address below.

[bytemaster]

well awesome, welcome to the forum and thanks for letting everyone know they are wasting their time on this because you are nearly finished. that's a two person nightshift and 20+ hours of coding beeing utterly useless now.

Sorry, I started this yesterday when being stuck at an airport - fully aware that I was probably too late anyway. So any status update would've been "I am looking into this" - which I am sure would not have deterred you or anyone else from continuing (I see a couple of those status updates earlier in this thread...).

Batmaninpink, your code was superior upon inspection due to proper use of the fc library for managing encryption.  The other submission had potential memory leaks and used OpenSSL calls directly without clear tracking of memory or exception safety.   Your submission also had better abstraction of data parsing. 

I have merged your code and will be evaluating it prior to paying the bounty.

Drekrob,  your submission was also very good and absent the competition may have been accepted after some robustness enhancements.  I will give you a 30 PTS tip for your submission (~$360 dollars) which should partially recognize your time.    These bounties are a kind of interview and so redundant work actually has value to us.

It is very clear to me that you are both talented developers and I would like to discuss how we can work together on a non-bounty basis to eliminate redundant effort.  I am looking for some serious c++ full time developers.

Please provide me your PTS addresses.

[batmaninpink]

I have made pull requests for this, see:

https://github.com/InvictusInnovations/fc/pull/4
https://github.com/InvictusInnovations/BitShares/pull/21

I just saw that they were merged into the main repos, let me know if there are any additional issues that need addressing.

When you are happy, bounty can be paid to: Pe8tGCH7uGjjQjfBGAbJ6FjQRwgK5u9rUK

[drekrob]

Here it is:
https://github.com/InvictusInnovations/fc/pull/5
https://github.com/InvictusInnovations/BitShares/pull/22

[bytemaster]

Well sorry for sounding kind of rough, it's hardly your fault. I specificaly told arlen i do not like to work on bounties as i saw exactly this problem coming up. Even if you would not have done it and our solution had been the first, there were still three or four others puting time into this. In the end you are always spending way more time on a task than it is needed.

We agree bounties have many problems including heavy management overhead on our end.  We will probably be using bounties more specifically in the future.   The problem we have is that if we don't leave it open to anyone someone will come along, claim they are working on it which will stop everyone else from working on it.   

The good news is that everyone who competes a bounty is a good candidate for us to hire and we can eventually get a team of talented developers without the need for redundant waisted effort.

[drekrob]

Well sorry for sounding kind of rough, it's hardly your fault. I specificaly told arlen i do not like to work on bounties as i saw exactly this problem coming up. Even if you would not have done it and our solution had been the first, there were still three or four others puting time into this. In the end you are always spending way more time on a task than it is needed.

[batmaninpink]

well awesome, welcome to the forum and thanks for letting everyone know they are wasting their time on this because you are nearly finished. that's a two person nightshift and 20+ hours of coding beeing utterly useless now.

Sorry, I started this yesterday when being stuck at an airport - fully aware that I was probably too late anyway. So any status update would've been "I am looking into this" - which I am sure would not have deterred you or anyone else from continuing (I see a couple of those status updates earlier in this thread...).

[drekrob]

So more redundant work for less money? I don't think this bounty modell is the most efficient way to manage the human resources at your disposal, but i guess i already wrote that in my last mail.

[testz]

well awesome, welcome to the forum and thanks for letting everyone know they are wasting their time on this because you are nearly finished. that's a two person nightshift and 20+ hours of coding beeing utterly useless now.

Please submit you code also.
I think, bytemaster will look into both solutions and select the winner.
Anyway because we will have 2 solutions, i think, bytemaster will split the bounty.

[drekrob]

well awesome, welcome to the forum and thanks for letting everyone know they are wasting their time on this because you are nearly finished. that's a two person nightshift and 20+ hours of coding beeing utterly useless now.

[batmaninpink]

I have made pull requests for this, see:

https://github.com/InvictusInnovations/fc/pull/4
https://github.com/InvictusInnovations/BitShares/pull/21

[drekrob]

Yes this works with encryption. I am a as good as done with everything. Unit tests are finished aswell. worked on it until 6am yesterday. I will need to fix that pts wallet bug then I can send the pull request. Last thing I noticed was that the pts private keys are within the DER encoded strings from the dB but after converting it they are lost. So it is not a problem with Dr coding the address but a difference in the way the wallet.DAT must be  read.

[pc]

working on this, too

[bytemaster]

How do you want the function to respond to soft errors (corrupt data in db, decryption error etc). Should it throw an exception or ignore and continue with the remaining keys?
I'm able to import the correct private keys from a bitcoin wallet (verified it with a address utility, the public key matches the address in the client). But the public keys i import from pts wallets dont get me the right addresses with

Code: [Select]

auto test_pub = imported_keys[n].get_public_key();
bts::pts_address test_addr(test_pub);So if the pts_address class is supposed to work i somehow don't get the right public keys from pts wallets while i get the right public keys from bitcoin wallets.

Good work.  Does this even work for wallets that are encrypted?

The pts_address class *should work* but has not been throughly tested.   Could it be a version byte is different in the wallet?

All errors should use FC_THROW_EXCEPTION( ... ) or FC_ASSERT(...) to capture proper debug information to be passed up the stack.

[drekrob]

How do you want the function to respond to soft errors (corrupt data in db, decryption error etc). Should it throw an exception or ignore and continue with the remaining keys?
I'm able to import the correct private keys from a bitcoin wallet (verified it with a address utility, the public key matches the address in the client). But the public keys i import from pts wallets dont get me the right addresses with

Code: [Select]

auto test_pub = imported_keys[n].get_public_key();
bts::pts_address test_addr(test_pub);So if the pts_address class is supposed to work i somehow don't get the right public keys from pts wallets while i get the right public keys from bitcoin wallets.

[bytemaster]

Pts and btc wallet are the same format.

As long as everything is forward declared and OpenSSL headers are not exposed in the header. 


Sent from my iPhone using Tapatalk

[drekrob]

The function is called import_bitcoin_wallet but you say the unit test should check for proper PTS addresses. Can you clarify? Also i would like to change the private_key constructor so it can take an optional EC_KEY if that is ok for you.

[bytemaster]

The unit tests should include two wallet files one with and one without a password and import multiple keys from both.  Include hidden addresses used for change. 


Sent from my iPhone using Tapatalk

[drekrob]

I am currently working on this bounty. Can you tell me more details about your requirements for unit tests? I've already made some good progress. The code will add libdb_cxx to the dependencies but will not depend on any of the bitcoin-qt sources.

[busygin]

Ok, I can parse wallet.dat file in C++ now with db_cxx library. Need to learn about the algorithm used for private key encryption and then I think I can handle this task.

[5chdn]

do I understand correctly that wallet.dat is a Berkeley database file?

Yes it is.

[busygin]

I am going to increase this bounty to 250 PTS as this is a high priority.

I want to try this task but I've never done any bitcoin or crypto-related development before. So, perhaps a silly noob question: do I understand correctly that wallet.dat is a Berkeley database file? What library (if any) should I use in C++ to read it without bringing something that would contradict bitshares software license?

[bytemaster]

I am going to increase this bounty to 250 PTS as this is a high priority.

[bytemaster]

I am going to increase this bounty to 150 PTS... I suspect that toast has decided to tackle the other bounties in the short term such as the Mac Keyhotee bounty.   Keep up the good work guys.

[toast]

This project on github claims to be able to import private keys among other things. It is written i Python

Some code I found on the repository

Code: [Select]

def i2d_ECPrivateKey(pkey, compressed=False):#, crypted=True):
    part3='a081a53081a2020101302c06072a8648ce3d0101022100'  # for uncompressed keys
    if compressed:
        if True:#not crypted:  ## Bitcoin accepts both part3's for crypted wallets...
            part3='a08185308182020101302c06072a8648ce3d0101022100'  # for compressed keys
        key = '3081d30201010420' + \
            '%064x' % pkey.secret + \
            part3 + \
            '%064x' % _p + \
            '3006040100040107042102' + \
            '%064x' % _Gx + \
            '022100' + \
            '%064x' % _r + \
            '020101a124032200'
    else:
        key = '308201130201010420' + \
            '%064x' % pkey.secret + \
            part3 + \
            '%064x' % _p + \
            '3006040100040107044104' + \
            '%064x' % _Gx + \
            '%064x' % _Gy + \
            '022100' + \
            '%064x' % _r + \
            '020101a144034200'

    return key.decode('hex') + i2o_ECPublicKey(pkey, compressed)
I am going to test pywallet to see if this works.

Yo, that's awesome. I'm gonna get on IRC in 45 minutes, this looks we could finish really quick if that works.

Edit: One note of caution. I cant find a license accompanying the code on github.

[arcke]

This project on github claims to be able to import private keys among other things. It is written i Python

Some code I found on the repository

Code: [Select]

def i2d_ECPrivateKey(pkey, compressed=False):#, crypted=True):
    part3='a081a53081a2020101302c06072a8648ce3d0101022100'  # for uncompressed keys
    if compressed:
        if True:#not crypted:  ## Bitcoin accepts both part3's for crypted wallets...
            part3='a08185308182020101302c06072a8648ce3d0101022100'  # for compressed keys
        key = '3081d30201010420' + \
            '%064x' % pkey.secret + \
            part3 + \
            '%064x' % _p + \
            '3006040100040107042102' + \
            '%064x' % _Gx + \
            '022100' + \
            '%064x' % _r + \
            '020101a124032200'
    else:
        key = '308201130201010420' + \
            '%064x' % pkey.secret + \
            part3 + \
            '%064x' % _p + \
            '3006040100040107044104' + \
            '%064x' % _Gx + \
            '%064x' % _Gy + \
            '022100' + \
            '%064x' % _r + \
            '020101a144034200'

    return key.decode('hex') + i2o_ECPublicKey(pkey, compressed)
I am going to test pywallet to see if this works.

Edit: One note of caution. I cant find a license accompanying the code on github.

[bytemaster]

Just  install it with apt-get
apt-get install libboost1.54-all-dev

Ubuntu (I assume you're on 13.10) installs boost 1.53 by default, bitshares require 1.54. You can install it with apt-get or you can stay on 1.53 and just downgrade required boost version in CMakeList.txt and also remove boost_coroutine requirement from it. It compiles and runs fine with 1.53, I compiled and ran Keyhotee a few days ago.

Yes, I am running Ubuntu 13.10.

I tried to install Boost 1.55, with this sequence:

Code: [Select]

wget http://sourceforge.net/projects/boost/files/boost/1.55.0/boost_1_55_0.tar.bz2
tar --bzip2 -xf /path/to/boost_1_55_0.tar.bz2
cd boost_1_55_0/
./bootstrap.sh --prefix=/usr/local/
./b2 install
./bjam
sudo ./bjam install
The same error about 1.53 still appears.

Code: [Select]

CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1126 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.53.0

  Boost include path: /usr/include

  The following Boost libraries could not be found:

          boost_coroutine

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:57 (FIND_PACKAGE)


-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!


[/quote]

You need to upgrade to boost 1.54

[alt]

Just  install it with apt-get
apt-get install libboost1.54-all-dev

Ubuntu (I assume you're on 13.10) installs boost 1.53 by default, bitshares require 1.54. You can install it with apt-get or you can stay on 1.53 and just downgrade required boost version in CMakeList.txt and also remove boost_coroutine requirement from it. It compiles and runs fine with 1.53, I compiled and ran Keyhotee a few days ago.

Yes, I am running Ubuntu 13.10.

I tried to install Boost 1.55, with this sequence:

Code: [Select]

wget http://sourceforge.net/projects/boost/files/boost/1.55.0/boost_1_55_0.tar.bz2
tar --bzip2 -xf /path/to/boost_1_55_0.tar.bz2
cd boost_1_55_0/
./bootstrap.sh --prefix=/usr/local/
./b2 install
./bjam
sudo ./bjam install
The same error about 1.53 still appears.

Code: [Select]

CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1126 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.53.0

  Boost include path: /usr/include

  The following Boost libraries could not be found:

          boost_coroutine

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:57 (FIND_PACKAGE)


-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!
[/quote]

[alt]

Bitshares/CMakeLists.txt

diff --git a/CMakeLists.txt b/CMakeLists.txt
index fc8870a..d3724e7 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -49,8 +49,8 @@ ELSE(WIN32)
   add_subdirectory( vendor/leveldb-1.12.0 )
 
   MESSAGE(status "Compiling on UNIX")
-  SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x -Wall -Wno-unused-local-typedefs -fmax-errors=3 -msse4.2" )
-  SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c99 -Wall -Wno-unused-local-typedefs -fmax-errors=3 -msse4.2" )
+  SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x -Wall -Wno-unused-local-typedefs -fmax-errors=3" )
+  SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c99 -Wall -Wno-unused-local-typedefs -fmax-errors=3" )
   #SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x -Wall -Wno-unused-local-typedefs" )
   #SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c99 -Wall -Wno-unused-local-typedefs" )
   SET(Boost_USE_STATIC_LIBS ON)

Almost... compiled.
Got 3 errors. Any idea on how to fix this?

Code: [Select]

compilation terminated due to -fmax-errors=3.
make[2]: ** [fc/CMakeFiles/fc.dir/src/crypto/city.cpp.o] Erro 1
make[1]: ** [fc/CMakeFiles/fc.dir/all] Erro 2
make: ** [all] Erro 2


[arcke]

Almost... compiled.
Got 3 errors. Any idea on how to fix this?

Code: [Select]

compilation terminated due to -fmax-errors=3.
make[2]: ** [fc/CMakeFiles/fc.dir/src/crypto/city.cpp.o] Erro 1
make[1]: ** [fc/CMakeFiles/fc.dir/all] Erro 2
make: ** [all] Erro 2


Can you post the error. I believe GCC should have printed an error before what you post.

Edit: Can we try to keep this thread about new development, maybe you can ask questions about compiling in another thread or create a new one somewhere other than the bounty discussion.

[pvaladares]

Almost... compiled.
Got 3 errors. Any idea on how to fix this?

Code: [Select]

compilation terminated due to -fmax-errors=3.
make[2]: ** [fc/CMakeFiles/fc.dir/src/crypto/city.cpp.o] Erro 1
make[1]: ** [fc/CMakeFiles/fc.dir/all] Erro 2
make: ** [all] Erro 2


[arcke]

Ubuntu (I assume you're on 13.10) installs boost 1.53 by default, bitshares require 1.54. You can install it with apt-get or you can stay on 1.53 and just downgrade required boost version in CMakeList.txt and also remove boost_coroutine requirement from it. It compiles and runs fine with 1.53, I compiled and ran Keyhotee a few days ago.

Yes, I am running Ubuntu 13.10.

I tried to install Boost 1.55, with this sequence:

Code: [Select]

wget http://sourceforge.net/projects/boost/files/boost/1.55.0/boost_1_55_0.tar.bz2
tar --bzip2 -xf /path/to/boost_1_55_0.tar.bz2
cd boost_1_55_0/
./bootstrap.sh --prefix=/usr/local/
./b2 install
./bjam
sudo ./bjam install
The same error about 1.53 still appears.

Code: [Select]

CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1126 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.53.0

  Boost include path: /usr/include

  The following Boost libraries could not be found:

          boost_coroutine

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:57 (FIND_PACKAGE)


-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!
[/quote]
Your prefix is /usr/local. Your boost include path is /usr/include. Try running cmake like this

Code: [Select]

cmake -DBoost_INCLUDE_DIR:PATH=/usr/local/include -DBoost_LIBRARY_DIR:PATH=/usr/local/lib .
Now it should find the boost libraries during compile.

[toast]

I think the --prefix flag doesn't work as you expect.

Just use apt-get, worked for others in this thread

[pvaladares]

Ubuntu (I assume you're on 13.10) installs boost 1.53 by default, bitshares require 1.54. You can install it with apt-get or you can stay on 1.53 and just downgrade required boost version in CMakeList.txt and also remove boost_coroutine requirement from it. It compiles and runs fine with 1.53, I compiled and ran Keyhotee a few days ago.
[/quote]

Yes, I am running Ubuntu 13.10.

I tried to install Boost 1.55, with this sequence:

Code: [Select]

wget http://sourceforge.net/projects/boost/files/boost/1.55.0/boost_1_55_0.tar.bz2
tar --bzip2 -xf /path/to/boost_1_55_0.tar.bz2
cd boost_1_55_0/
./bootstrap.sh --prefix=/usr/local/
./b2 install
./bjam
sudo ./bjam install
The same error about 1.53 still appears.

Code: [Select]

CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1126 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.53.0

  Boost include path: /usr/include

  The following Boost libraries could not be found:

          boost_coroutine

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:57 (FIND_PACKAGE)


-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!


[arcke]

Yep of course, the idea was to just get it to work *at all* and the prune

If someone comes across specifications for bitcoin/protoshares wallet.dat-files this can help big time. Right now we have been using bitcoin and protoshares as reference implementations for importing wallets.

[toast]

Yep of course, the idea was to just get it to work *at all* and the prune

[bytemaster]

Status update: First we tried starting with CBitcoinSecret and pulling out as little as possible to make this work (I think we only need the stuff in fc::ecc and possibly in openssl), then we gave up and tried mass-importing everything from bitcoin it built with CBitcoinSecret but ran into build problems involving boost.

Some points of interest:

https://github.com/bitcoin/bitcoin/blob/086d7ec2b8c6ee352af820fe57908d43ec86f586/src/rpcdump.cpp#L159
https://github.com/bitcoin/bitcoin/blob/f76c122e2eac8ef66f69d142231bd33c88a24c50/src/key.cpp#L150

The repo (note, two branches trying top-down and bottom-up approaches):
https://github.com/nmushegian/BitShares

I would like to import as little code from Bitcoin as possible to minimize future maintenance and dependencies.  If there is more than one solution to this bounty, this will be a major factor in breaking the ties.

[toast]

^^ that's no longer an issue, look at the most recent state in the repo. Would be awesome if anyone could build the base58.h-import branch.

[enodata]

When I try to compile I get the boost error

Code: [Select]

CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1126 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.53.0

  Boost include path: /usr/include

  The following Boost libraries could not be found:

          boost_coroutine

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:57 (FIND_PACKAGE)


-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!

Is there any guide on how to properly install latest version?

Ubuntu (I assume you're on 13.10) installs boost 1.53 by default, bitshares require 1.54. You can install it with apt-get or you can stay on 1.53 and just downgrade required boost version in CMakeList.txt and also remove boost_coroutine requirement from it. It compiles and runs fine with 1.53, I compiled and ran Keyhotee a few days ago.

[toast]

Status update: First we tried starting with CBitcoinSecret and pulling out as little as possible to make this work (I think we only need the stuff in fc::ecc and possibly in openssl), then we gave up and tried mass-importing everything from bitcoin it built with CBitcoinSecret but ran into build problems involving boost.

Some points of interest:

https://github.com/bitcoin/bitcoin/blob/086d7ec2b8c6ee352af820fe57908d43ec86f586/src/rpcdump.cpp#L159
https://github.com/bitcoin/bitcoin/blob/f76c122e2eac8ef66f69d142231bd33c88a24c50/src/key.cpp#L150

The repo (note, two branches trying top-down and bottom-up approaches):
https://github.com/nmushegian/BitShares

[toast]

Want to sync up and work in my repo? I'm a bit ahead of you I think

gonna hop on IRC (#bitshares on freenode)

[arcke]

I have some code for reading the wallet. How do I execute the unit tests or a specific unit test?

Edit: Nevermind, I just run them directly.

Edit2: Working with code from https://github.com/bitcoin/bitcoin/blob/master/src/rpcdump.cpp#L131

[toast]

sudo apt-get libboost-all-dev?

not sure, that's just what the first google search gives

[pvaladares]

What platform? For me it was just "brew install boost" on osx

I am under Ubuntu.

[toast]

It is cool if I add some more files? Specifically, base58.h and some of its dependencies

https://github.com/bitcoin/bitcoin/blob/f76c122e2eac8ef66f69d142231bd33c88a24c50/src/base58.h

Edit: I see you have base58 stuff in fc/crypto already, what I actually need is CBitcoinSecret, I can stick that in a more clearly named file if you prefer

edit 2: fuck it I'll just put everything in the new bitcoin_wallet

[toast]

What platform? For me it was just "brew install boost" on osx

[pvaladares]

When I try to compile I get the boost error

Code: [Select]

CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1126 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.53.0

  Boost include path: /usr/include

  The following Boost libraries could not be found:

          boost_coroutine

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:57 (FIND_PACKAGE)


-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!

Is there any guide on how to properly install latest version?

[toast]

Finally got everything to build (when in doubt, nuke the directory and redo "cmake ."). Added nearly-empty files / tests / updated cmake. Now we can get rolling.

https://github.com/nmushegian/BitShares

[arcke]

Yeah, I've looked through the code and it shouldn't be too hard, I think once I actually start working I'll be able to finish in less than a day. arcke, want to split the work somehow?

Right now I'm almost done getting this to build, successfully got it to use g++ for CXX objects but still working on getting it to use gcc instead of clang for C objects. Very good learning experience. Turns out OSX mavericks quitely links g++/gcc to clang, how nice

Yes, when I find an opportunity to split work I will let you know. For now I am still just investigating.

First method of interest I found is CWallet::Unlock. Reverse engineering this will tell us how we can decrypt an encrypted wallet with a passphrase. To be done: investigate if we can reuse code from bitcoin/protoshares to do this.

I am working on Linux, cant help you with the OSX build specifics.

[bytemaster]

Yeah, I've looked through the code and it shouldn't be too hard, I think once I actually start working I'll be able to finish in less than a day. arcke, want to split the work somehow?

Right now I'm almost done getting this to build, successfully got it to use g++ for CXX objects but still working on getting it to use gcc instead of clang for C objects. Very good learning experience. Turns out OSX mavericks quitely links g++/gcc to clang, how nice

I haven't upgraded to Mavericks yet on my primary development machine.   FYI, once this task is done I will have more follow-on tasks as fast as I can specify them, which is becoming my full time job

[toast]

Yeah, I've looked through the code and it shouldn't be too hard, I think once I actually start working I'll be able to finish in less than a day. arcke, want to split the work somehow?

Right now I'm almost done getting this to build, successfully got it to use g++ for CXX objects but still working on getting it to use gcc instead of clang for C objects. Very good learning experience. Turns out OSX mavericks quitely links g++/gcc to clang, how nice

[arcke]

This means we can reuse code from bitcoin. I have been looking at bitcoin/src/wallet.{h,cpp}. I might be able to do this basing my solution on bitcoin code. For now we should make unit tests firsts and put them online.

[bytemaster]

To be accepted, a unit test case must be provided using boost unit tests that verifies that the keys can be imported and converted to the proper PTS address.

I am in doubt regarding the task demanded by the bounty. What types of wallets should the method import? Should it import protoshares wallets into bitshares or is this method universally importing wallets of all bitcoin-forks or just bitcoin?

Bitcoin-QT and ProtoShares use the same wallet format, these are the only wallets required by this bounty. 

[arcke]

To be accepted, a unit test case must be provided using boost unit tests that verifies that the keys can be imported and converted to the proper PTS address.

I am in doubt regarding the task demanded by the bounty. What types of wallets should the method import? Should it import protoshares wallets into bitshares or is this method universally importing wallets of all bitcoin-forks or just bitcoin?

[bytemaster]

You are using XCodes clang++ compiler rather than g++. 

[toast]

Code: [Select]

boost-1.54.0 already installed
I'm using brew though.


I did get this during cmake, is it relevant?:

Code: [Select]

(~/projects/BitShares (cmake .
statusCompiling on UNIX
-- Boost version: 1.54.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
--   coroutine
-- Boost version: 1.54.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context
CMake Warning (dev) at CMakeLists.txt:87 (set):
  Cannot set "BOOST_LIBRARIES": current scope has no parent.
the full errors I'm getting during make

Code: [Select]

[  1%] Building CXX object CMakeFiles/bshare.dir/src/network/server.cpp.o
clang: warning: argument unused during compilation: '-fmax-errors=3'
warning: unknown warning option '-Wno-unused-local-typedefs' [-Wunknown-warning-option]
In file included from /Users/nikolai/projects/BitShares/src/network/server.cpp:1:
In file included from /Users/nikolai/projects/BitShares/include/bts/network/server.hpp:2:
In file included from /Users/nikolai/projects/BitShares/include/bts/network/message.hpp:2:
In file included from /Users/nikolai/projects/BitShares/include/bts/network/channel_id.hpp:76:
In file included from /Users/nikolai/projects/BitShares/fc/include/fc/reflect/reflect.hpp:9:
In file included from /Users/nikolai/projects/BitShares/fc/include/fc/utility.hpp:3:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/new:56:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/exception:81:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/type_traits:770:38: error: implicit instantiation of undefined template
      'std::__1::hash<fc::ip::endpoint>'
    : public integral_constant<bool, __is_empty(_Tp)> {};
                                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/unordered_map:327:54: note: in instantiation of template class
      'std::__1::is_empty<std::__1::hash<fc::ip::endpoint> >' requested here
template <class _Key, class _Tp, class _Hash, bool = is_empty<_Hash>::value
                                                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/unordered_map:678:13: note: in instantiation of default argument for
      '__unordered_map_hasher<fc::ip::endpoint, std::__1::shared_ptr<bts::network::connection>, std::__1::hash<fc::ip::endpoint> >' required here
    typedef __unordered_map_hasher<key_type, mapped_type, hasher>   __hasher;
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/nikolai/projects/BitShares/src/network/server.cpp:62:71: note: in instantiation of template class 'std::__1::unordered_map<fc::ip::endpoint, std::__1::shared_ptr<bts::network::connection>,
      std::__1::hash<fc::ip::endpoint>, std::__1::equal_to<fc::ip::endpoint>, std::__1::allocator<std::__1::pair<const fc::ip::endpoint, std::__1::shared_ptr<bts::network::connection> > > >' requested
      here
          std::unordered_map<fc::ip::endpoint,connection_ptr>         connections;
                                                                      ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:3081:29: note: template is declared here
template <class _Tp> struct hash;
                            ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2147:15: error: no matching constructor for initialization of 'bts::network::connection'
              __second_(_VSTD::forward<_Args2>(get<_I2>(__second_args))...)
              ^         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2421:15: note: in instantiation of function template specialization
      'std::__1::__libcpp_compressed_pair_imp<std::__1::allocator<bts::network::connection>, bts::network::connection, 1>::__libcpp_compressed_pair_imp<std::__1::allocator<bts::network::connection> &,
      const std::__1::shared_ptr<bts::network::stcp_socket> &, bts::network::detail::server_impl *&&, 0, 0, 1>' requested here
            : base(__pc, _VSTD::move(__first_args), _VSTD::move(__second_args),
              ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:3703:16: note: in instantiation of function template specialization
      'std::__1::__compressed_pair<std::__1::allocator<bts::network::connection>, bts::network::connection>::__compressed_pair<std::__1::allocator<bts::network::connection> &, const
      std::__1::shared_ptr<bts::network::stcp_socket> &, bts::network::detail::server_impl *&&>' requested here
            :  __data_(piecewise_construct, _VSTD::forward_as_tuple(__a),
               ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4277:26: note: in instantiation of function template specialization
      'std::__1::__shared_ptr_emplace<bts::network::connection, std::__1::allocator<bts::network::connection> >::__shared_ptr_emplace<const std::__1::shared_ptr<bts::network::stcp_socket> &,
      bts::network::detail::server_impl *>' requested here
    ::new(__hold2.get()) _CntrlBlk(__a2, _VSTD::forward<_Args>(__args)...);
                         ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4636:12: note: in instantiation of function template specialization
      'std::__1::shared_ptr<bts::network::connection>::make_shared<const std::__1::shared_ptr<bts::network::stcp_socket> &, bts::network::detail::server_impl *>' requested here
    return shared_ptr<_Tp>::make_shared(_VSTD::forward<_Args>(__args)...);
           ^
/Users/nikolai/projects/BitShares/src/network/server.cpp:116:28: note: in instantiation of function template specialization 'std::__1::make_shared<bts::network::connection, const
      std::__1::shared_ptr<bts::network::stcp_socket> &, bts::network::detail::server_impl *>' requested here
                auto con = std::make_shared<connection>(s,this);
                           ^
/Users/nikolai/projects/BitShares/include/bts/network/connection.hpp:61:9: note: candidate constructor not viable: no known conversion from 'bts::network::detail::server_impl *' to
      'bts::network::connection_delegate *' for 2nd argument
        connection( const stcp_socket_ptr& c, connection_delegate* d);
        ^
/Users/nikolai/projects/BitShares/include/bts/network/connection.hpp:62:9: note: candidate constructor not viable: requires single argument 'd', but 2 arguments were provided
        connection( connection_delegate* d );
        ^
/Users/nikolai/projects/BitShares/include/bts/network/connection.hpp:58:10: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
   class connection : public std::enable_shared_from_this<connection>
         ^
In file included from /Users/nikolai/projects/BitShares/src/network/server.cpp:1:
In file included from /Users/nikolai/projects/BitShares/include/bts/network/server.hpp:2:
In file included from /Users/nikolai/projects/BitShares/include/bts/network/message.hpp:2:
In file included from /Users/nikolai/projects/BitShares/include/bts/network/channel_id.hpp:76:
In file included from /Users/nikolai/projects/BitShares/fc/include/fc/reflect/reflect.hpp:18:
In file included from /Users/nikolai/projects/BitShares/fc/include/fc/reflect/typename.hpp:2:
In file included from /Users/nikolai/projects/BitShares/fc/include/fc/string.hpp:7:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/string:434:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/algorithm:594:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2147:15: error: no matching constructor for initialization of 'bts::network::connection'
              __second_(_VSTD::forward<_Args2>(get<_I2>(__second_args))...)
              ^         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:2421:15: note: in instantiation of function template specialization
      'std::__1::__libcpp_compressed_pair_imp<std::__1::allocator<bts::network::connection>, bts::network::connection, 1>::__libcpp_compressed_pair_imp<std::__1::allocator<bts::network::connection> &,
      bts::network::detail::server_impl *&&, 0, 0>' requested here
            : base(__pc, _VSTD::move(__first_args), _VSTD::move(__second_args),
              ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:3703:16: note: in instantiation of function template specialization
      'std::__1::__compressed_pair<std::__1::allocator<bts::network::connection>, bts::network::connection>::__compressed_pair<std::__1::allocator<bts::network::connection> &,
      bts::network::detail::server_impl *&&>' requested here
            :  __data_(piecewise_construct, _VSTD::forward_as_tuple(__a),
               ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4277:26: note: in instantiation of function template specialization
      'std::__1::__shared_ptr_emplace<bts::network::connection, std::__1::allocator<bts::network::connection> >::__shared_ptr_emplace<bts::network::detail::server_impl *>' requested here
    ::new(__hold2.get()) _CntrlBlk(__a2, _VSTD::forward<_Args>(__args)...);
                         ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:4636:12: note: in instantiation of function template specialization
      'std::__1::shared_ptr<bts::network::connection>::make_shared<bts::network::detail::server_impl *>' requested here
    return shared_ptr<_Tp>::make_shared(_VSTD::forward<_Args>(__args)...);
           ^
/Users/nikolai/projects/BitShares/src/network/server.cpp:251:29: note: in instantiation of function template specialization 'std::__1::make_shared<bts::network::connection,
      bts::network::detail::server_impl *>' requested here
       connection_ptr con = std::make_shared<connection>( my.get() );
                            ^
/Users/nikolai/projects/BitShares/include/bts/network/connection.hpp:58:10: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from
      'bts::network::detail::server_impl *' to 'const bts::network::connection' for 1st argument
   class connection : public std::enable_shared_from_this<connection>
         ^
/Users/nikolai/projects/BitShares/include/bts/network/connection.hpp:62:9: note: candidate constructor not viable: no known conversion from 'bts::network::detail::server_impl *' to
      'bts::network::connection_delegate *' for 1st argument
        connection( connection_delegate* d );
        ^
/Users/nikolai/projects/BitShares/include/bts/network/connection.hpp:61:9: note: candidate constructor not viable: requires 2 arguments, but 1 was provided
        connection( const stcp_socket_ptr& c, connection_delegate* d);
        ^
1 warning and 3 errors generated.
make[2]: *** [CMakeFiles/bshare.dir/src/network/server.cpp.o] Error 1
make[1]: *** [CMakeFiles/bshare.dir/all] Error 2
make: *** [all] Error 2

[bytemaster]

got it, getting other build errors but it's probably due to my environment

anyone developing this on OSX that I can chat with?

Me    

g++4.8 from mac ports
build boost 1.54 from source and install

[toast]

got it, getting other build errors but it's probably due to my environment

anyone developing this on OSX that I can chat with?

[bytemaster]

BitShares
https://github.com/InvictusInnovations/BitShares/blob/master/.gitignore#L46

Igorning fc from the bitshares repo is because fc is a foreign repository checked out separately...

git clone bitshares
cd bitshares
git clone fc
cmake .
make

[toast]

BitShares
https://github.com/InvictusInnovations/BitShares/blob/master/.gitignore#L46

[bytemaster]

Can you take fc/ out of the .gitignore, or at least explain what's supposed to be in there?

bitshares/fc  -> https://github.com/InvictusInnovations/fc

Which repository are you referring to for the .gitignore

[toast]

Can you take fc/ out of the .gitignore, or at least explain what's supposed to be in there?

[toast]

I'm starting this, contact me if anyone wants to work together

[bytemaster]

Our wallets will need to import from many different wallet.dat files which may be protected by a password.  For easy of development we would like an API call implemented in the bitshares code base that can import a wallet.dat file.

Files to be created:
include/bts/bitcoin_wallet.hpp
src/bitcoin_wallet.cpp

Code: [Select]

namespace bts  {
      std::vector<fc::ecc::private_key> import_bitcoin_wallet( const fc::path& wallet_dat, const std::string& passphrase );
}
The bounty will be awarded to the first person who can complete the task with pull request that is accepted by our team.  To be accepted, a unit test case must be provided using boost unit tests that verifies that the keys can be imported and converted to the proper PTS address.   All code must follow the same conventions found in the bitshares repository.   Code must compile with g++ and VC 2010 or newer.   A 1 PTS penalty will be assessed (deducted from ultimate award) for every submission attempt that is sent back for coding style violations, compilation errors, or failed unit tests.   

There is an additional 2 PTS bounty paid to the person who refers the winner of this bounty to this thread.  You must be the first to post the name of the person who ultimately wins the bounty to this thread.   You cannot refer yourself.

Any questions regarding the implementation of this method or about the requirements must be posted in this thread.  More bounties will be forth coming.