BitShares Forum

Other => Graveyard => Marketplace => Topic started by: bytemaster on December 29, 2013, 01:13:11 am

Title: [$5000 worth in PTS] BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on December 29, 2013, 01:13:11 am
So far BitShares has been developed with a cowboy coding mentality.  I would design, document, and implement as I went.  We are now looking to grow this into something that can be maintained by the community and understood by all.  To this end I would like to produce a professional quality design document that is maintained as a website that can be updated by accepting pull requests via Github.    This document should be content focused and built for clarity.   Someone should be able to take this specification and implement a working BitShares blockchain with a high degree of compatibility with other independent implementations.

I have already defined the blockchain and many unit tests and there exists documentation embedded in the code right now.  I would like to capture the ideas expressed in the code and make them available and accessible to everyone.   I will be available to answer questions here in the forum about anything that is unclear.

Because BitShares has not been finalized at this point, the process of documenting the current design should help us explore undiscovered corner cases.   

This is a cooperative bounty and all submissions must be developed publicly.  I want to encourage cooperation so if you copy contributions from other members be sure to agree on how to split the bounty. If the accepted design document has any disputes about how the bounty is to be divided then Invictus will decide and take a 25% cut.  In other words, it is in everyones interest to fairly access their own contribution and resolve disputes.

This bounty is PENDING until all questions regarding the scope and requirements of this bounty can be settled.  At that point we will lock in the terms and begin the bounty.  Of course, getting a head start will benefit all.

There is also a referral bounty paid to the individual who refers anyone who wins the bounty. 

https://github.com/InvictusInnovations/BitShares/blob/master/include/bts/blockchain/block.hpp
https://github.com/InvictusInnovations/BitShares/blob/master/include/bts/blockchain/outputs.hpp

Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on December 29, 2013, 01:13:59 am
I referred forum member naturalog to this bounty.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on December 29, 2013, 01:19:52 am
Interesting job. I'm starting to work on it.
How do you imagine the document?
Explaining in fluent English the classes and their relations, or more a reference-type document giving details on every class?
Initial version can be found here https://github.com/naturalog/BitShares/blob/master/doc/BitSharesBlockchainDesign.html
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on December 29, 2013, 02:14:38 am
I would like the spec to be language independent.    Talk about the data format and meaning rather than class names. 




Sent from my iPhone using Tapatalk (http://tapatalk.com/m?id=1)
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on December 29, 2013, 02:20:47 am
Interesting job. I'm starting to work on it.
How do you imagine the document?
Explaining in fluent English the classes and their relations, or more a reference-type document giving details on every class?
Initial version can be found here https://github.com/naturalog/BitShares/blob/master/doc/BitSharesBlockchainDesign.html

Ack. May I suggest using markdown instead of HTML? There are plenty render-md-as-html tools, and in the meantime we can have a readable format which github will render nicely for us.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on December 29, 2013, 02:23:28 am
The more professional the look the better.  The easier to maintain the better.  Must work well with version control and images.


Sent from my iPhone using Tapatalk (http://tapatalk.com/m?id=1)
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on December 29, 2013, 02:26:05 am
Ack. May I suggest using markdown instead of HTML? There are plenty render-md-as-html tools, and in the meantime we can have a readable format which github will render nicely for us.

Sounds like a great idea, admits all requirements as can be seen here https://help.github.com/articles/github-flavored-markdown
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on December 31, 2013, 05:57:02 pm
Ack. May I suggest using markdown instead of HTML? There are plenty render-md-as-html tools, and in the meantime we can have a readable format which github will render nicely for us.

Sounds like a great idea, admits all requirements as can be seen here https://help.github.com/articles/github-flavored-markdown

I would expect there to be regular questions and a place where you are publishing your work?
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on January 01, 2014, 12:04:45 am
Hijack!

I have ported what naturalog has written so far to markdown and put it on github ("spec" branch). I'm gonna be working on this / BTS explorer tonight and didn't have access to the repo. Also I added naturalog as a contributor (both "naturalog" and "ohadasor").

https://github.com/nmushegian/BitShares/blob/spec/doc/spec.md

bitshare explorer bounty thread
https://bitsharestalk.org/index.php?topic=1829.0;topicseen


edit: bytemaster, since this is a separate branch, want to just pull it into the II/BitShares repo right now?
edit 2: nvm then you need to grant repo permissions
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 01, 2014, 01:18:39 am
Hijack!

I have ported what naturalog has written so far to markdown and put it on github ("spec" branch). I'm gonna be working on this / BTS explorer tonight and didn't have access to the repo. Also I added naturalog as a contributor (both "naturalog" and "ohadasor").

https://github.com/nmushegian/BitShares/blob/spec/doc/spec.md

bitshare explorer bounty thread
https://bitsharestalk.org/index.php?topic=1829.0;topicseen


edit: bytemaster, since this is a separate branch, want to just pull it into the II/BitShares repo right now?
edit 2: nvm then you need to grant repo permissions

I am going to make this my priority task once I get Keyhotee ID registration done.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on January 02, 2014, 12:05:19 pm
bytemaster, can you please supply some explanation about the block state structure?
toast, do you plan working on this doc? Otherwise I'll work on my fork.
see my work so far here
https://github.com/naturalog/BitShares/blob/master/doc/BitSharesBlockchainDesign.md
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on January 02, 2014, 07:32:15 pm
I saw the doc but it barely had anything at the time. Let's pick one repo and work there. I'm fine with using yours if you do the work of merging (but you should check out a new branch IMO, not use master). Conversely, I'll gladly do the merge if we want to work in my branch.

https://github.com/nmushegian/BitShares/blob/spec/doc/spec.md
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 02, 2014, 07:47:31 pm
Hey, today I want to help bootstrap this process some. 
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on January 02, 2014, 07:48:13 pm
getting on irc bitshares@freenode
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on January 03, 2014, 12:37:05 pm
Conversely, I'll gladly do the merge if we want to work in my branch.

https://github.com/nmushegian/BitShares/blob/spec/doc/spec.md

please do :) i'm new to github..
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 03, 2014, 04:36:11 pm
bytemaster, can you please supply some explanation about the block state structure?
toast, do you plan working on this doc? Otherwise I'll work on my fork.
see my work so far here
https://github.com/naturalog/BitShares/blob/master/doc/BitSharesBlockchainDesign.md

The block_state structure does not exist in the current github repo. 
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on January 03, 2014, 07:16:05 pm
bytemaster you said you wanted to help bootstrap this, can you say some more words?
I'm going to work on merging in whatever parts of naturalog's work that I can and then the most up-to-date work will be in the spec branch on my repo.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 03, 2014, 07:17:11 pm
Yes I can.  I am busy on keyhotee release right now


Sent from my iPhone using Tapatalk (http://tapatalk.com/m?id=1)
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on January 05, 2014, 08:18:27 pm
bytemaster you said you wanted to help bootstrap this, can you say some more words?
I'm going to work on merging in whatever parts of naturalog's work that I can and then the most up-to-date work will be in the spec branch on my repo.

is it ready already? can i continue on your branch?
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 05, 2014, 08:54:56 pm
bytemaster you said you wanted to help bootstrap this, can you say some more words?
I'm going to work on merging in whatever parts of naturalog's work that I can and then the most up-to-date work will be in the spec branch on my repo.

is it ready already? can i continue on your branch?

I am working on the blockchain today. 
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on January 05, 2014, 10:34:17 pm
bytemaster you said you wanted to help bootstrap this, can you say some more words?
I'm going to work on merging in whatever parts of naturalog's work that I can and then the most up-to-date work will be in the spec branch on my repo.

is it ready already? can i continue on your branch?

Yes, go ahead
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on January 20, 2014, 12:19:08 am
I checked in a version that finishes (hopefully) all the explanation of the classes in block.hpp and transaction.hpp.
Please guide me for more work.
https://github.com/nmushegian/BitShares/blob/master/doc/spec.md
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 20, 2014, 12:22:50 am
I checked in a version that finishes (hopefully) all the explanation of the classes in block.hpp and transaction.hpp.
Please guide me for more work.
https://github.com/nmushegian/BitShares/blob/master/doc/spec.md

For an example of you Bitcoin documented this:
https://en.bitcoin.it/wiki/Protocol_specification

It would be very difficult for me to build BitShares X starting only with the spec you have provided.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on January 20, 2014, 12:30:50 am
I checked in a version that finishes (hopefully) all the explanation of the classes in block.hpp and transaction.hpp.
Please guide me for more work.
https://github.com/nmushegian/BitShares/blob/master/doc/spec.md

For an example of you Bitcoin documented this:
https://en.bitcoin.it/wiki/Protocol_specification

It would be very difficult for me to build BitShares X starting only with the spec you have provided.

Ok so we're talking about the whole protocol specification, not only blockchain structure.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on January 20, 2014, 12:34:33 am
I checked in a version that finishes (hopefully) all the explanation of the classes in block.hpp and transaction.hpp.
Please guide me for more work.
https://github.com/nmushegian/BitShares/blob/master/doc/spec.md

For an example of you Bitcoin documented this:
https://en.bitcoin.it/wiki/Protocol_specification

It would be very difficult for me to build BitShares X starting only with the spec you have provided.

Ok so we're talking about the whole protocol specification, not only blockchain structure.

The blockchain structure is the protocol, the messages for communicating blocks, transactions, over the wire etc are not considered part of this.  Someone should be able to follow these direction to build and serialize a valid transaction, sign it, and validate a block chain.   A good start would be to fully document the serialization and fields in an easy to understand way. 
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: naturalog on January 20, 2014, 12:44:19 am
I see so you want it down to the byte level. OK. Will work on this more. Another workers around? Let's collaborate.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: vertoe on February 05, 2014, 11:00:57 pm
Bump. Whats the status on this? I need this.  8)
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on February 05, 2014, 11:39:57 pm
I'm curious as well
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: vertoe on February 12, 2014, 09:29:35 am
If nobody completes it this week i will start working on this bounty because i need the specification really bad :)

Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: toast on February 12, 2014, 04:14:47 pm
There is a work in progress, I think this bounty is better done cooperatively than competitively. Equal expected return per hour but higher chance of getting the bounty before feb 28th

https://github.com/nmushegian/BitShares/blob/spec/doc/spec.md
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: vertoe on February 12, 2014, 09:41:31 pm
There is a work in progress, I think this bounty is better done cooperatively than competitively. Equal expected return per hour but higher chance of getting the bounty before feb 28th

https://github.com/nmushegian/BitShares/blob/spec/doc/spec.md

Yes I've seen this. I didnt mean to start a competition here. I just dont see anything happening for several days now...

I dont want to do this. But If I start working on bitshares I can do it because I need it.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: theoretical on August 11, 2014, 02:14:03 am

Over the next two weeks, I'd like to work on documenting the blockchain format.  In particular, going from a block as a sequence of bytes; to fields with particular data types (uint32, hash); to semantic information about entities like accounts, balances, delegates, orders, and votes.  To keep the scope manageable, I'd like to limit my contribution to focusing on parsing the serialized blockchain.

I'll include information on the following:

- Byte-by-byte documentation of the serialized block format
- Constraints between various fields (e.g. the value in each block's parent block hash field must be equal to the value obtained by computing the hash of the serialization of the parent block).
- Reasons for design decisions
- Genesis-block parameters that can easily be tweaked
- Highlight major BitShares Toolkit features that either mimic features of other cryptocoins, or are different from other cryptocoins
- Describe in detail the implementation of high-level semantic nouns and verbs (e.g.:  What is a short position and how do you cover it?  How do we make TITAN transactions invisible?)

I'll post here with the Github URL when I start.

Also, since the bounty was posted before snapshot was taken, I'd like to ask if the bounty will include the BTSX spawned by the 500 PTS?

Hopefully this post satisfies the "scope and requirements" laid out in the opening post, and we can change from pending to active.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: xeroc on August 11, 2014, 09:23:48 am

Over the next two weeks, I'd like to work on documenting the blockchain format.  In particular, going from a block as a sequence of bytes; to fields with particular data types (uint32, hash); to semantic information about entities like accounts, balances, delegates, orders, and votes.  To keep the scope manageable, I'd like to limit my contribution to focusing on parsing the serialized blockchain.

I'll include information on the following:

- Byte-by-byte documentation of the serialized block format
- Constraints between various fields (e.g. the value in each block's parent block hash field must be equal to the value obtained by computing the hash of the serialization of the parent block).
- Reasons for design decisions
- Genesis-block parameters that can easily be tweaked
- Highlight major BitShares Toolkit features that either mimic features of other cryptocoins, or are different from other cryptocoins
- Describe in detail the implementation of high-level semantic nouns and verbs (e.g.:  What is a short position and how do you cover it?  How do we make TITAN transactions invisible?)

I'll post here with the Github URL when I start.

Also, since the bounty was posted before snapshot was taken, I'd like to ask if the bounty will include the BTSX spawned by the 500 PTS?

Hopefully this post satisfies the "scope and requirements" laid out in the opening post, and we can change from pending to active.
+5% for the initiative ..

just my 2 PTS:

 - could you use the bitshares wiki?
 - I like how the bitcoin guys are writing it up: https://bitcoin.org/en/developer-guide

if you need assistance (maybe drawing some pictures/blockdiagrams) I can give you a hand ..
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: bytemaster on August 11, 2014, 01:54:11 pm

Over the next two weeks, I'd like to work on documenting the blockchain format.  In particular, going from a block as a sequence of bytes; to fields with particular data types (uint32, hash); to semantic information about entities like accounts, balances, delegates, orders, and votes.  To keep the scope manageable, I'd like to limit my contribution to focusing on parsing the serialized blockchain.

I'll include information on the following:

- Byte-by-byte documentation of the serialized block format
- Constraints between various fields (e.g. the value in each block's parent block hash field must be equal to the value obtained by computing the hash of the serialization of the parent block).
- Reasons for design decisions
- Genesis-block parameters that can easily be tweaked
- Highlight major BitShares Toolkit features that either mimic features of other cryptocoins, or are different from other cryptocoins
- Describe in detail the implementation of high-level semantic nouns and verbs (e.g.:  What is a short position and how do you cover it?  How do we make TITAN transactions invisible?)

I'll post here with the Github URL when I start.

Also, since the bounty was posted before snapshot was taken, I'd like to ask if the bounty will include the BTSX spawned by the 500 PTS?

Hopefully this post satisfies the "scope and requirements" laid out in the opening post, and we can change from pending to active.

I'll pay $5000 worth of PTS at the time of completion.
Title: Re: 500 PTS BitShares Blockchain Technical Specification [PENDING]
Post by: vikram on August 12, 2014, 05:01:05 am

Over the next two weeks, I'd like to work on documenting the blockchain format.  In particular, going from a block as a sequence of bytes; to fields with particular data types (uint32, hash); to semantic information about entities like accounts, balances, delegates, orders, and votes.  To keep the scope manageable, I'd like to limit my contribution to focusing on parsing the serialized blockchain.

I'll include information on the following:

- Byte-by-byte documentation of the serialized block format
- Constraints between various fields (e.g. the value in each block's parent block hash field must be equal to the value obtained by computing the hash of the serialization of the parent block).
- Reasons for design decisions
- Genesis-block parameters that can easily be tweaked
- Highlight major BitShares Toolkit features that either mimic features of other cryptocoins, or are different from other cryptocoins
- Describe in detail the implementation of high-level semantic nouns and verbs (e.g.:  What is a short position and how do you cover it?  How do we make TITAN transactions invisible?)

I'll post here with the Github URL when I start.

Also, since the bounty was posted before snapshot was taken, I'd like to ask if the bounty will include the BTSX spawned by the 500 PTS?

Hopefully this post satisfies the "scope and requirements" laid out in the opening post, and we can change from pending to active.

 +5% +5% +5%
Title: Re: [$5000 worth in PTS] BitShares Blockchain Technical Specification [PENDING]
Post by: theoretical on August 29, 2014, 08:13:43 pm

Sorry, it's taking a little longer than I expected, I've been spending most of my time testing and reporting bugs for the client, and of course trading :)

I should be able to get some done this weekend (but if there are any more fantastic margin crises or arbitrage opportunities in store, I may just trade instead).
Title: Re: [$5000 worth in PTS] BitShares Blockchain Technical Specification [PENDING]
Post by: theoretical on September 09, 2014, 05:10:40 am

I've started on this bounty.  Initial progress is here:  https://github.com/drltc/docblock

I have some questions here:  https://github.com/drltc/docblock/blob/master/questions.md

Since bytemaster is most familiar with the core code, I suspect he'll be answering most of them.  But any other developers or random forum lurkers are encouraged to answer to conserve bytemaster's time :)
Title: Re: [$5000 worth in PTS] BitShares Blockchain Technical Specification [PENDING]
Post by: xeroc on September 09, 2014, 06:57:28 am
can't wait to see this document finished ..
good job so far +5%
Title: Re: [$5000 worth in PTS] BitShares Blockchain Technical Specification [PENDING]
Post by: vikram on September 09, 2014, 07:52:09 pm

I've started on this bounty.  Initial progress is here:  https://github.com/drltc/docblock

I have some questions here:  https://github.com/drltc/docblock/blob/master/questions.md

Since bytemaster is most familiar with the core code, I suspect he'll be answering most of them.  But any other developers or random forum lurkers are encouraged to answer to conserve bytemaster's time :)

+1 +1 +6
Title: Re: [$5000 worth in PTS] BitShares Blockchain Technical Specification [PENDING]
Post by: theoretical on September 20, 2014, 03:37:25 am

I added some updates.  https://github.com/drltc/docblock/blob/master/block.md#on-the-wire-format is most useful to those writing parsers.

My next goal is to document the individual parameter formats and precise semantics of each operation.