BitShares Forum

Main => General Discussion => Topic started by: bytemaster on March 06, 2015, 12:09:10 am

Title: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: bytemaster on March 06, 2015, 12:09:10 am
I would like to have  a drop in replacement that implements this api:
https://github.com/BitShares/fc/blob/master/include/fc/crypto/elliptic.hpp

Using:
https://github.com/bitcoin/secp256k1

To get the bounty the pull request must be merged and proven to work with BitShares main client.   
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: eagleeye on March 06, 2015, 01:20:06 am
Bytemaster read Rgcryptos blog https://rgcrypto.wordpress.com

He may have found the secret sauce for bitshares success or failure.  I understand this is not related.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: testz on March 06, 2015, 02:09:55 am
Good starting point/example can be Auroracoin sources where it's implemented (see define USE_SECP256K1):
https://compilr.com/einsteinz/auroracoin-core/src/key.cpp

Whole sources from compilr.com can be found here: https://mega.co.nz/#!VJYRXQ7A!7LjU3M8iXWZlf4WIoRdrZw7gYXL_abtNgjxeQTyYxPQ
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: bitmarket on March 06, 2015, 03:53:23 am
For the non-techies.... what is this about?
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: jsidhu on March 06, 2015, 07:32:17 am
Good starting point/example can be Auroracoin sources where it's implemented (see define USE_SECP256K1):
https://compilr.com/einsteinz/auroracoin-core/src/key.cpp

Whole sources from compilr.com can be found here: https://mega.co.nz/#!VJYRXQ7A!7LjU3M8iXWZlf4WIoRdrZw7gYXL_abtNgjxeQTyYxPQ
They just based off of latest btc.... Best to look here instead https://github.com/bitcoin/bitcoin/commit/16a58a86442ad587449f321c0dbab08cc028c2bd

Btw secp256k1  is experimental and bitcoin hasnt given it the nod yet.. Why do this?
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: xeroc on March 06, 2015, 07:54:31 am
Btw secp256k1  is experimental and bitcoin hasnt given it the nod yet.. Why do this?
secp256k1 is a set of parameters for a elliptic curve and not a piece of software
https://en.bitcoin.it/wiki/Secp256k1
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: cube on March 06, 2015, 08:16:27 am
For the non-techies.... what is this about?

Bitcoin uses secp256k1 to digitally sign and verify transactions. This link may give a clue - http://bitcoin.stackexchange.com/questions/36136/is-libsecp256k1-faster-than-openssl
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: pc on March 06, 2015, 08:40:08 am
I'll take the challenge.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: svk on March 06, 2015, 08:50:51 am
I'll take the challenge.

Awesome! :) I asked about this on github a while back, it looks like it could be a real gamechanger in terms of scanning speed for the client.

This is a quote from Gavin Andresen in his blog post talking about increasing the block size of Bitcoin:

Quote
Of course, we won't be running current code on tomorrow's hardware; we'll be running better code. CPU usage should go down by a factor of about eight in the next release when we switch to Pieter's libsecp256k1 library for validating transactions.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: cass on March 06, 2015, 09:16:19 am
I'll take the challenge.

 +5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: cube on March 06, 2015, 10:25:10 am
I'll take the challenge.

 +5%

Way to go!
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: testz on March 06, 2015, 10:46:16 am
I'll take the challenge.

 +5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: 38PTSWarrior on March 06, 2015, 10:49:32 am
Yeah
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: TurkeyLeg on March 06, 2015, 11:26:52 am
I'll take the challenge.

Awesome! +5% 
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: jsidhu on March 06, 2015, 04:37:58 pm
Btw secp256k1  is experimental and bitcoin hasnt given it the nod yet.. Why do this?
secp256k1 is a set of parameters for a elliptic curve and not a piece of software
https://en.bitcoin.it/wiki/Secp256k1
There is some verification so it needs to be fully tested... Its not yet.. But as long as it passes some integration and unit tests it should be ok i think.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: xeroc on March 06, 2015, 05:37:05 pm
Ah .. I understand .. we are talking about the implementation libsecp256k1 ..
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: santaclause102 on March 06, 2015, 07:17:49 pm
https://groups.google.com/forum/#!topic/raiblocks/z6z2x_uvrH0
https://blake2.net/

Is the above relevant and a subset the category Secp256k1 also belongs to?
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: liondani on March 06, 2015, 07:35:32 pm
It's getting interesting again.  :)

Sent from my ALCATEL ONE TOUCH 997D
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: jsidhu on March 06, 2015, 08:15:10 pm
Good starting point/example can be Auroracoin sources where it's implemented (see define USE_SECP256K1):
https://compilr.com/einsteinz/auroracoin-core/src/key.cpp

Whole sources from compilr.com can be found here: https://mega.co.nz/#!VJYRXQ7A!7LjU3M8iXWZlf4WIoRdrZw7gYXL_abtNgjxeQTyYxPQ
They just based off of latest btc.... Best to look here instead https://github.com/bitcoin/bitcoin/commit/16a58a86442ad587449f321c0dbab08cc028c2bd

Btw secp256k1  is experimental and bitcoin hasnt given it the nod yet.. Why do this?

Latest Bitcoin Core 0.10 has switched to libsecp256k1 ONLY for signing: https://github.com/bitcoin/bitcoin/blob/aeb92792281b4cb9958f3defc9e36f63e65b778a/doc/release-notes.md#improved-signing-security

If we choose to integrate this into mainline BitShares I will never use it for any more functionality than latest Bitcoin uses it for.

Is the intention to use it for more? If so, then does bitshares do it now or wait?
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: julian1 on March 06, 2015, 09:29:58 pm
This one is supposed to be fast as well, with inline gcc assembler primitives for ARM.

https://github.com/kmackay/micro-ecc
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: abit on March 07, 2015, 12:01:32 am
It's said that LibSecp256k1 is much faster than OpenSSL while verifying signatures. By using it we can improve the client sync speed much, which is a bottleneck by now.

http://www.reddit.com/r/Bitcoin/comments/2weymr/experiment_bitcoin_core_0100_initial_sync_time/coqghm2
Quote
Just did a benchmark on libsecp256k1's current master, without GMP, without hand-written assembly, and it's around 3.6x faster than OpenSSL on my machine (64-bit code, i7 cpu). When the assembly is compiled in (which does not require any extra dependencies anymore), it's 4.9 times faster.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: pc on March 09, 2015, 10:33:01 am
I would like to have  a drop in replacement that implements this api:
https://github.com/BitShares/fc/blob/master/include/fc/crypto/elliptic.hpp

Using:
https://github.com/bitcoin/secp256k1

I'm making good progress. Results are promising, the ecc_test program runs about 20 times faster with libsecp256k1 than with openssl.

I noticed the current implementation of public_key::mult is broken - it computes the same result as public_key::add, only in a different way. Fortunately, it doesn't seem to be used anywhere in the bitshares code.

Latest Bitcoin Core 0.10 has switched to libsecp256k1 ONLY for signing: https://github.com/bitcoin/bitcoin/blob/aeb92792281b4cb9958f3defc9e36f63e65b778a/doc/release-notes.md#improved-signing-security

If we choose to integrate this into mainline BitShares I will never use it for any more functionality than latest Bitcoin uses it for.

I can provide a "mixed" variant, where signing is done using openssl and verification using libsecp256k1. (My plan is to use a CMake cache var for selecting the ecc implementation to use.)
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: svk on March 09, 2015, 12:46:00 pm
I would like to have  a drop in replacement that implements this api:
https://github.com/BitShares/fc/blob/master/include/fc/crypto/elliptic.hpp

Using:
https://github.com/bitcoin/secp256k1

I'm making good progress. Results are promising, the ecc_test program runs about 20 times faster with libsecp256k1 than with openssl.

I noticed the current implementation of public_key::mult is broken - it computes the same result as public_key::add, only in a different way. Fortunately, it doesn't seem to be used anywhere in the bitshares code.

Latest Bitcoin Core 0.10 has switched to libsecp256k1 ONLY for signing: https://github.com/bitcoin/bitcoin/blob/aeb92792281b4cb9958f3defc9e36f63e65b778a/doc/release-notes.md#improved-signing-security

If we choose to integrate this into mainline BitShares I will never use it for any more functionality than latest Bitcoin uses it for.

I can provide a "mixed" variant, where signing is done using openssl and verification using libsecp256k1. (My plan is to use a CMake cache var for selecting the ecc implementation to use.)

Good stuff, keep up the good work! :)
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: cube on March 09, 2015, 12:53:12 pm

Good stuff, keep up the good work! :)

For all who find his work good, please vote for Delegate dev-pc.bitcube! 

Delegate dev-pc.bitcube has been registered for some time now. Please show some love to it. :)
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: bytemaster on March 09, 2015, 01:08:07 pm
I would like to have  a drop in replacement that implements this api:
https://github.com/BitShares/fc/blob/master/include/fc/crypto/elliptic.hpp

Using:
https://github.com/bitcoin/secp256k1

I'm making good progress. Results are promising, the ecc_test program runs about 20 times faster with libsecp256k1 than with openssl.

I noticed the current implementation of public_key::mult is broken - it computes the same result as public_key::add, only in a different way. Fortunately, it doesn't seem to be used anywhere in the bitshares code.

Latest Bitcoin Core 0.10 has switched to libsecp256k1 ONLY for signing: https://github.com/bitcoin/bitcoin/blob/aeb92792281b4cb9958f3defc9e36f63e65b778a/doc/release-notes.md#improved-signing-security

If we choose to integrate this into mainline BitShares I will never use it for any more functionality than latest Bitcoin uses it for.

I can provide a "mixed" variant, where signing is done using openssl and verification using libsecp256k1. (My plan is to use a CMake cache var for selecting the ecc implementation to use.)

Good work!   That should help greatly reduce our sync CPU load.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: vlight on March 09, 2015, 09:44:47 pm
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: jsidhu on March 09, 2015, 10:57:21 pm
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.

Sorry it already uses too much RAM lol
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: vlight on March 09, 2015, 11:02:33 pm
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.

Sorry it already uses too much RAM lol

Well, for me when syncing less than 100 hours left, it takes only 1GB of RAM(2GB unused), but uses HDD heavily.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: jsidhu on March 09, 2015, 11:08:08 pm
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.

Sorry it already uses too much RAM lol

Well, for me when syncing less than 100 hours left, it takes only 1GB of RAM(2GB unused), but uses HDD heavily.

On my ubuntu installations I always run out of 2 GB ram.. even after sync and restarting.. maybe jus tmemory leak in latest release... and my delegate always runs out i had to create a 4 gb swap file and that gets half used but then i start to miss blocks because its slow.. so i just restart it once eveyr few weeks
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: bytemaster on March 10, 2015, 12:05:15 am
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.

Sorry it already uses too much RAM lol

We are working to reduce RAM and CPU. 
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: muse-umum on March 10, 2015, 12:28:20 am
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.

Sorry it already uses too much RAM lol

We are working to reduce RAM and CPU.

Very glad to see you come back to the most critical topic.
+5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: alt on March 10, 2015, 02:56:55 am
But the main part that slows sync is HDD not CPU. The software could also use more RAM that is left unused.

Sorry it already uses too much RAM lol

We are working to reduce RAM and CPU.

Very glad to see you come back to the most critical topic.
+5%
+5% +5% +5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: pc on March 11, 2015, 10:07:17 pm
PR submitted: https://github.com/BitShares/fc/pull/4

Speedup of --rebuild-index is insignificant on my system, around 5%. Possibly more on an SSD. The indexing is heavy on IO, so not much speedup to be expected here.

Speedup of wallet_rescan_blockchain is 4% for the mixed implementation and 25% for pure libsecp256k1.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: cass on March 11, 2015, 10:33:36 pm
wow that was fast @pc :)  +5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: robrigo on March 11, 2015, 10:35:19 pm
wow that was fast @pc :)  +5%

Yeah what a beast. Good work PC!
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: BTSdac on March 12, 2015, 01:26:42 am
very good work  +5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: cube on March 12, 2015, 01:50:52 am
very good work  +5%

Please remember to vote for dev-pc.bitcube. Give him your encouragement and love. :)
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: ElMato on March 12, 2015, 04:12:47 am
great work @pc!!
 +5% +5%
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: vikram on March 12, 2015, 07:06:04 pm
PR submitted: https://github.com/BitShares/fc/pull/4

Speedup of --rebuild-index is insignificant on my system, around 5%. Possibly more on an SSD. The indexing is heavy on IO, so not much speedup to be expected here.

Speedup of wallet_rescan_blockchain is 4% for the mixed implementation and 25% for pure libsecp256k1.

Note that block/transaction signature validation is skipped until after the most recent checkpoint, otherwise IO may not necessarily dominate.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: Akado on November 16, 2015, 09:34:29 pm
https://www.reddit.com/r/Bitcoin/comments/3t0kff/core_switched_to_libsecp256k1_for_verification/
LOL. Delayed a few months.
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: topcandle on November 16, 2015, 09:53:28 pm
https://www.reddit.com/r/Bitcoin/comments/3t0kff/core_switched_to_libsecp256k1_for_verification/
LOL. Delayed a few months.

What's there to laugh about.  Bitshares has delayed many features it promised....
Title: Re: [BOUNTY] $5000 BitUSD - LibSecp256k1 integration with FC
Post by: Akado on November 16, 2015, 11:21:52 pm
https://www.reddit.com/r/Bitcoin/comments/3t0kff/core_switched_to_libsecp256k1_for_verification/
LOL. Delayed a few months.

What's there to laugh about.  Bitshares has delayed many features it promised....

Many features that haven't been implemented by anyone. What's there to laugh about is people adore when Bitcoin implements some features and despise other projects that have those exact same features. That's what there's to laugh about. The irony of bitcoin having a great libertarian spirit behind it yet, their community is closed like the same banks they hate.