Author Topic: October 5 Test Network  (Read 127128 times)

0 Members and 1 Guest are viewing this topic.

Offline Thom

Okay. Here's how I build. Can you compare it with yours?

Code: [Select]
git clone https://github.com/cryptonomex/graphene.git
cd graphene
git submodule update --init --recursive
cmake -DCMAKE_BUILD_TYPE=Release
make witness_node cli_wallet

Yep, the std way like everybody else that doesn't use some type of automation.

But like I said, irrelevant for the reasons stated.
Injustice anywhere is a threat to justice everywhere - MLK |  Verbaltech2 Witness Reports: https://bitsharestalk.org/index.php/topic,23902.0.html

Offline Bhuz

  • Committee member
  • Sr. Member
  • *
  • Posts: 467
    • View Profile
  • BitShares: bhuz
I got this on my cli_wallet, in red:

Code: [Select]
7364ms th_a       wallet.cpp:1717               sign_transaction     ] Caught exception while broadcasting transaction with id 0be4839dfa602f3be290b7d2a5524a318f889e50

Still, it is not crashed...

Offline theoretical

Can't seem to get another instance up and running. I've verified all of the wallets, config and binaries with md5sum to working versions. I've deleted the object_database and blockchain. I've tried --resync-blockchain, --replay-blockchain options. Everytime I start the witness I get this output:

Code: [Select]
2114494ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1"]
2114531ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1"]
2118682ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1","000001a1895e7bb499ff3f12bd5eda0a050488de","000001a4236eb3795a2cfa9c5d9f00177fd0b14c","000001a64a8f85ec0c46910dc9eb8ad1ad6a5d10"]
2118852ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb922c6c7c8e72d1b09c04425e0c1","000001a1895e7bb499ff3f12bd5eda0a050488de","000001a4236eb3795a2cfa9c5d9f00
177fd0b14c","000001a64a8f85ec0c46910dc9eb8ad1ad6a5d10"]

cli_wallet does not show correct "info" at all. Using the same seed as working VPS (104.236.51.238:2005)

How does the info command output look incorrect?  If you are doing --resync-blockchain then it nukes all the downloaded blocks and starts at the beginning, so you will see block times / block numbers in the past until you've finished downloading the chain.  The get_blockchain_synop messages are normal, they show your node exchanging block ID's with peers.

BTW - I still haven't found a way to stop the witness cleanly. The only way I know to stop it at all is with ^C. Sometimes it reports "Aborted ..." in the output but most of the time when it is restarted it complains it was shutdown uncleanly.

Yeah, our shutdown code isn't that robust.  I filed a ticket.

Is there a signal that will do it properly? St least in the old code there was a "quit" command, although it often resulted in a corrupted blockchain database. Can this fundamental issue not be fixed?

SIGINT is your best bet.
BTS- theoretical / PTS- PZxpdC8RqWsdU3pVJeobZY7JFKVPfNpy5z / BTC- 1NfGejohzoVGffAD1CnCRgo9vApjCU2viY / the delegate formerly known as drltc / Nothing said on these forums is intended to be legally binding / All opinions are my own unless otherwise noted / Take action due to my posts at your own risk

Offline clayop

  • Hero Member
  • *****
  • Posts: 2033
    • View Profile
    • Bitshares Korea
  • BitShares: clayop
Can you post your build commands?

Hey clayop, thanks for the response. To answere your question, no, not easily. I'm using a python script (bts_tools) and some of it is buried in the config files.

Besides, I don't think that's relevant, given that the binaries are in operation now and I haven't missed a block in several hours on VPS seed07 in Singnapore.

Also, as I mentioned, I verified the binaries, config etc were all transfered from seed07 to seed06 in LA without changes using md5sum.

I just tried to start it as a seed node on seed06 (no block production) and it fails in the same way, but I expected that.

I will try to start a seed on another VPS in Germany and see if I can get it to sync. If I can it will be some issue on the VPS, in which case I'll rebuild it from scratch. All 4 of the VPSs should be the same, but I have noticed subtle differences in the images each host uses, tho they are all Ubuntu 14.04, and I used the same setup script on each one.

It's been fast pace work, and I may have overlooked something tho. It only goes to show if it's this difficult to determine why I can't bring up a new witness instance with all of these checks and verifications it's not only a suble difference but also that the voluminous withness_node output is not easy to distill information from.

Okay. Here's how I build. Can you compare it with yours?

Code: [Select]
git clone https://github.com/cryptonomex/graphene.git
cd graphene
git submodule update --init --recursive
cmake -DCMAKE_BUILD_TYPE=Release
make witness_node cli_wallet
Bitshares Korea - http://www.bitshares.kr
Vote for me and see Korean Bitshares community grows
delegate-clayop

Offline Thom

Can you post your build commands?

Hey clayop, thanks for the response. To answere your question, no, not easily. I'm using a python script (bts_tools) and some of it is buried in the config files.

Besides, I don't think that's relevant, given that the binaries are in operation now and I haven't missed a block in several hours on VPS seed07 in Singnapore.

Also, as I mentioned, I verified the binaries, config etc were all transfered from seed07 to seed06 in LA without changes using md5sum.

I just tried to start it as a seed node on seed06 (no block production) and it fails in the same way, but I expected that.

I will try to start a seed on another VPS in Germany and see if I can get it to sync. If I can it will be some issue on the VPS, in which case I'll rebuild it from scratch. All 4 of the VPSs should be the same, but I have noticed subtle differences in the images each host uses, tho they are all Ubuntu 14.04, and I used the same setup script on each one.

It's been fast pace work, and I may have overlooked something tho. It only goes to show if it's this difficult to determine why I can't bring up a new witness instance with all of these checks and verifications it's not only a suble difference but also that the voluminous withness_node output is not easy to distill information from.

Injustice anywhere is a threat to justice everywhere - MLK |  Verbaltech2 Witness Reports: https://bitsharestalk.org/index.php/topic,23902.0.html

Offline Bhuz

  • Committee member
  • Sr. Member
  • *
  • Posts: 467
    • View Profile
  • BitShares: bhuz
I am on commit: e68e99ed3ae11ddac607e983e7549e8278fdecc4 by BM, I miss the last two from emfrias... are they important for the hardfork?

Offline clayop

  • Hero Member
  • *****
  • Posts: 2033
    • View Profile
    • Bitshares Korea
  • BitShares: clayop
Stress test will begin at 19:00 UTC (in 2 hours). I expect to see the performance of the new code!



Hopefully, the network won't be broken  :P
Bitshares Korea - http://www.bitshares.kr
Vote for me and see Korean Bitshares community grows
delegate-clayop

Offline Ander

  • Hero Member
  • *****
  • Posts: 3506
    • View Profile
  • BitShares: Ander
Stress test will begin at 19:00 UTC (in 2 hours). I expect to see the performance of the new code!

https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

Offline clayop

  • Hero Member
  • *****
  • Posts: 2033
    • View Profile
    • Bitshares Korea
  • BitShares: clayop
Can't seem to get another instance up and running. I've verified all of the wallets, config and binaries with md5sum to working versions. I've deleted the object_database and blockchain. I've tried --resync-blockchain, --replay-blockchain options. Everytime I start the witness I get this output:

Code: [Select]
2114494ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1"]
2114531ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1"]
2118682ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1","000001a1895e7bb499ff3f12bd5eda0a050488de","000001a4236eb3795a2cfa9c5d9f00177fd0b14c","000001a64a8f85ec0c46910dc9eb8ad1ad6a5d10"]
2118852ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb922c6c7c8e72d1b09c04425e0c1","000001a1895e7bb499ff3f12bd5eda0a050488de","000001a4236eb3795a2cfa9c5d9f00
177fd0b14c","000001a64a8f85ec0c46910dc9eb8ad1ad6a5d10"]

cli_wallet does not show correct "info" at all. Using the same seed as working VPS (104.236.51.238:2005)

I will repeat the attempt on another VPS with the same exact files and see what happens. I will make one last attempt to restart this VPS as a seed.

Can you post your build commands?
Bitshares Korea - http://www.bitshares.kr
Vote for me and see Korean Bitshares community grows
delegate-clayop

Offline Thom

Can't seem to get another instance up and running. I've verified all of the wallets, config and binaries with md5sum to working versions. I've deleted the object_database and blockchain. I've tried --resync-blockchain, --replay-blockchain options. Everytime I start the witness I get this output:

Code: [Select]
2114494ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1"]
2114531ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1"]
2118682ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb92
2c6c7c8e72d1b09c04425e0c1","000001a1895e7bb499ff3f12bd5eda0a050488de","000001a4236eb3795a2cfa9c5d9f00177fd0b14c","000001a64a8f85ec0c46910dc9eb8ad1ad6a5d10"]
2118852ms th_a       application.cpp:712           get_blockchain_synop ] synopsis: ["0000019a905fb922c6c7c8e72d1b09c04425e0c1","000001a1895e7bb499ff3f12bd5eda0a050488de","000001a4236eb3795a2cfa9c5d9f00
177fd0b14c","000001a64a8f85ec0c46910dc9eb8ad1ad6a5d10"]

cli_wallet does not show correct "info" at all. Using the same seed as working VPS (104.236.51.238:2005)

I will repeat the attempt on another VPS with the same exact files and see what happens. I will make one last attempt to restart this VPS as a seed.


BTW - I still haven't found a way to stop the witness cleanly. The only way I know to stop it at all is with ^C. Sometimes it reports "Aborted ..." in the output but most of the time when it is restarted it complains it was shutdown uncleanly.

Is there a signal that will do it properly? St least in the old code there was a "quit" command, although it often resulted in a corrupted blockchain database. Can this fundamental issue not be fixed?
« Last Edit: October 09, 2015, 05:49:04 pm by Thom »
Injustice anywhere is a threat to justice everywhere - MLK |  Verbaltech2 Witness Reports: https://bitsharestalk.org/index.php/topic,23902.0.html

Offline clayop

  • Hero Member
  • *****
  • Posts: 2033
    • View Profile
    • Bitshares Korea
  • BitShares: clayop
Stress test will begin at 19:00 UTC (in 2 hours).

Can you postpone it to 19:10 UTC or so?  Some new code goes into effect at 18:45 UTC, and 19:00 UTC will be the first maintenance interval after the new code goes into effect.  It might be helpful for debugging to have a way to tell the difference between witnesses going down due to the new code (18:45-19:10) and witnesses going down due to the stress (19:10 or later).

Sure. What about 19:30 UTC?
Bitshares Korea - http://www.bitshares.kr
Vote for me and see Korean Bitshares community grows
delegate-clayop

Offline theoretical

Stress test will begin at 19:00 UTC (in 2 hours).

Can you postpone it to 19:10 UTC or so?  Some new code goes into effect at 18:45 UTC, and 19:00 UTC will be the first maintenance interval after the new code goes into effect.  It might be helpful for debugging to have a way to tell the difference between witnesses going down due to the new code (18:45-19:10) and witnesses going down due to the stress (19:10 or later).
BTS- theoretical / PTS- PZxpdC8RqWsdU3pVJeobZY7JFKVPfNpy5z / BTC- 1NfGejohzoVGffAD1CnCRgo9vApjCU2viY / the delegate formerly known as drltc / Nothing said on these forums is intended to be legally binding / All opinions are my own unless otherwise noted / Take action due to my posts at your own risk

Offline clayop

  • Hero Member
  • *****
  • Posts: 2033
    • View Profile
    • Bitshares Korea
  • BitShares: clayop
Stress test will begin at 19:00 19:30 UTC (in 2 hours). I expect to see the performance of the new code!
« Last Edit: October 09, 2015, 05:52:33 pm by clayop »
Bitshares Korea - http://www.bitshares.kr
Vote for me and see Korean Bitshares community grows
delegate-clayop

Offline puppies

  • Hero Member
  • *****
  • Posts: 1659
    • View Profile
  • BitShares: puppies
I would not assume that all missed blocks are the fault of your witness or an indication that you need to switch nodes.    I would probably use the "last_confirmed_block_num" property on the witness and only switch if it is more than 3 rounds behind the head block number.
Wouldn't waiting for 3 missed blocks before switching have the same effect?  Either way the number of missed blocks before you switch could pretty easily be configured in the config.

***edit*** I do get that 3 missed blocks will not always be equal to 3x active witnesses since last generated block.  I was just curious if there is a reason why measuring missed blocks by number of rounds completed since last block is a better way of measuring.
« Last Edit: October 09, 2015, 05:08:07 pm by puppies »
https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

Offline bytemaster

Witness spectral stopped producing blocks and started producing error messages at the witness output. Here is a log from the screen buffer of the witness output:

https://drive.google.com/file/d/0BzEEZBS7b6xPUnJvQXFZUHZQZW8/view?usp=sharing

Did this also happen to the others?

I deleted oct5 and object_database, restarted witness, and it's back to producing blocks and price feed.

Looks like we missed the real error.
For the latest updates checkout my blog: http://bytemaster.bitshares.org
Anything said on these forums does not constitute an intent to create a legal obligation or contract between myself and anyone else.   These are merely my opinions and I reserve the right to change them at any time.