BitShares Forum

Main => Technical Support => Topic started by: cryptosile on August 13, 2015, 01:23:58 am

Title: Automated build and test server for graphene
Post by: cryptosile on August 13, 2015, 01:23:58 am

I started off by making this docker build box: https://hub.docker.com/r/sile16/graphene-build/ (https://hub.docker.com/r/sile16/graphene-build/)

After that I thought, it can't be too hard to automate the build process so when a new commit hits github I'll run the build capture the output and print to html.  Took about 10 times longer than it probably should but here it is:

https://www.everydaycrypto.com/?page_id=107 (https://www.everydaycrypto.com/?page_id=107)

Every 5 minutes my server checks for a new commit, will download build and update the webpage with the results.  It also builds docker images once a day for the witness node and cli, but those aren't quite ready for use as I still need to tweak the docker configs to allow the correct ports through etc.  But theoretically i'll be able to deploy witness nodes from the auto generated docker images:

https://hub.docker.com/r/sile16/graphene-cli/ (https://hub.docker.com/r/sile16/graphene-cli/)
and
https://hub.docker.com/r/sile16/graphene-witness/ (https://hub.docker.com/r/sile16/graphene-witness/)

Once we actually have a taged release on github for graphene those tags should propagate into the docker image as tags.   I'd like to do this for the testnet / devshares as well but not sure what github repo will be used for that whether bitshares/devshares will be used or cryptonomex/devshares   ... 

I'm new to all this docker stuff so if you have a good idea how to make it better let me know.

Oh,, and all the code i'm using the automate the build process is at: https://github.com/sile16/bts2 (https://github.com/sile16/bts2)

I'm sure someone is going to tell me you can do this with jenkins in like 2 lines of code or something.....  hahaha

Title: Re: Automated build and test server for graphene
Post by: xeroc on August 13, 2015, 07:01:32 am

wow .. this is cool ..
I always though that is what "travis" is doing .. but never took a closer look as to why graphene seems to not use it ..

Title: Re: Automated build and test server for graphene
Post by: cass on August 13, 2015, 09:24:24 am

wow really great!  +5%

Title: Re: Automated build and test server for graphene
Post by: Fox on August 13, 2015, 11:56:09 am

This is definitely Brownie.PTS worthy! Very useful codebase. Many thanks for your contributions.

Title: Re: Automated build and test server for graphene
Post by: bytemaster on August 13, 2015, 12:06:37 pm

Quote from: Fox on August 13, 2015, 11:56:09 am

This is definitely Brownie.PTS worthy! Very useful codebase. Many thanks for your contributions.

I agree!  Some major Brownie.PTS heading your way.   

May I request that you run ./tests/chain_tests and report the number of pass/failure and duration? 

Title: Re: Automated build and test server for graphene
Post by: cryptosile on August 13, 2015, 12:41:58 pm

yeah, I can do that tonight probably.  It already looks at the return code and colors the test box green if the return code is 0 for all test passed.  If it's red 1 or more tests failed.  You can click the log link to look at the full build log or individual test logs. 

I can probably make the test cell link directly to it corresponding log and also on failure report the # of failed tests.

Title: Re: Automated build and test server for graphene
Post by: sudo on August 14, 2015, 01:52:49 am

awesome job

Title: Re: Automated build and test server for graphene
Post by: ElMato on August 14, 2015, 01:56:33 am

+5 excellent!

Title: Re: Automated build and test server for graphene
Post by: Tuck Fheman on August 14, 2015, 02:19:03 am

 +5%

Title: Re: Automated build and test server for graphene
Post by: clayop on August 15, 2015, 03:29:16 am

Got an error... I ran

Code: [Select]

sudo docker run -v /home/clayop:/build sile16/graphene-build --make_cli_wallet --make_witness_node


Code: [Select]

[ 65%] Built target graphene_net
make[1]: *** [programs/cli_wallet/CMakeFiles/cli_wallet.dir/rule] Error 2
make: *** [cli_wallet] Error 2
running ['/usr/bin/git', 'clone', 'https://github.com/cryptonomex/graphene.git']
running: /usr/bin/git submodule update --init --recursive
running /usr/bin/git pull --recurse-submodules origin master
['/usr/bin/cmake', '-DBOOST_ROOT="/usr/local/"', '-DCMAKE_BUILD_TYPE=Debug', '.']
['/usr/bin/make', '-j4', 'cli_wallet', 'witness_node']


Title: Re: Automated build and test server for graphene
Post by: cryptosile on August 15, 2015, 09:29:25 pm

I'm taking a look now.

#Update1
It had some weird error about not having dependencies.  I'd just try running it again or try it without either of the --make options, by default it will build both cli and witness.

Just running again with no options worked for me.  I'll see if I can do anything to avoid the make error in the future.  That will take some more investigating.

try:

Code: [Select]

sudo docker run -v /home/clayop:/build sile16/graphene-build
#Update 2
It looks like the combo of -j4 and specifying both cli and witness is what breaks some dependency in the make process.  I did two things:
I made the default script to not specify -j4 .  Going for less speed but more dependable.  I added an option to specify the jobs for the people who want to speed up the build process.  I also change the target selection so by default it builds all unless you just want to specify a single target.

with these two changes I don't think this problem can happen for anyone else.