Test Net for Advanced Users

[abit]

Maybe we still have chance? According to the log, the 'recently_missed' number is increasing.

Code: [Select]

...
{"recently_missed":47937,"max_undo":1000}
...
{"recently_missed":47981,"max_undo":1000}
...
{"recently_missed":48685,"max_undo":1000}

Retrying..

[mudshark79]

It's as easy as adding this to the witness_node command:

Code: [Select]

--checkpoint '[40902,"00009fc6d4e24db66e13af461b94063165999bfa"]' --enable-stale-production

But it seems too late to rescue the network, I still get this error:

Code: [Select]

1122000ms th_a       witness.cpp:240               block_production_loo ] slot: 11203 scheduled_witness: 1.6.5247 scheduled_[967/1963]
-08-20T10:18:42 now: 2015-08-20T10:18:42
1122000ms th_a       witness.cpp:243               block_production_loo ] Witness 1.6.5247 production slot has arrived; generating a b
lock now...
1122001ms th_a       db_block.cpp:170              _push_block          ] Failed to push new block:
3070000 undo_database_exception: undo database exception
The database does not have enough undo history to support a blockchain with so many missed blocks. Please add a checkpoint if you woul
d like to continue applying blocks beyond this point.
    {"recently_missed":44813,"max_undo":1000}
    th_a  db_update.cpp:72 update_global_dynamic_data

    {"next_block.block_num()":40903}
    th_a  db_block.cpp:439 _apply_block
1122001ms th_a       witness.cpp:266               block_production_loo ] Got exception while generating block:
3070000 undo_database_exception: undo database exception
The database does not have enough undo history to support a blockchain with so many missed blocks. Please add a checkpoint if you woul
d like to continue applying blocks beyond this point.
    {"recently_missed":44813,"max_undo":1000}
    th_a  db_update.cpp:72 update_global_dynamic_data

    {"next_block.block_num()":40903}
    th_a  db_block.cpp:439 _apply_block

    {"new_block":{"previous":"00009fc6d4e24db66e13af461b94063165999bfa","timestamp":"2015-08-20T10:18:42","witness":"1.6.5247","next_s
ecret_hash":"a151bcbe08eac49feb4eb86a5a68f332b4e7cf9d","previous_secret":"2fa1e05db52f0c421e335648c21bb3707383c1d6","transaction_merkl
e_root":"0000000000000000000000000000000000000000","extensions":[],"witness_signature":"206ea3856d3cf6317f19ab6b5d3af6648409ac092eae9c
c8d943d7357e3ac9c753141733131d2312c4644d44125835afce24190d295c21a6a620920ab660ec5f5c","transactions":[]}}
    th_a  db_block.cpp:176 _push_block

    {"witness_id":"1.6.5247"}
    th_a  db_block.cpp:315 _generate_block



Did it get you going again? Seems to be still stuck or maybe you are on your own fork now? Which is your adress to use as seed-node?

[xeroc]

Checkpoints are supposed to be part of config.ini .. it may be the same syntax as checkpoints.json from bitshares0.9

[abit]

It's as easy as adding this to the witness_node command:

Code: [Select]

--checkpoint '[40902,"00009fc6d4e24db66e13af461b94063165999bfa"]' --enable-stale-production

But it seems too late to rescue the network, I still get this error:

Code: [Select]

1122000ms th_a       witness.cpp:240               block_production_loo ] slot: 11203 scheduled_witness: 1.6.5247 scheduled_[967/1963]
-08-20T10:18:42 now: 2015-08-20T10:18:42
1122000ms th_a       witness.cpp:243               block_production_loo ] Witness 1.6.5247 production slot has arrived; generating a b
lock now...
1122001ms th_a       db_block.cpp:170              _push_block          ] Failed to push new block:
3070000 undo_database_exception: undo database exception
The database does not have enough undo history to support a blockchain with so many missed blocks. Please add a checkpoint if you woul
d like to continue applying blocks beyond this point.
    {"recently_missed":44813,"max_undo":1000}
    th_a  db_update.cpp:72 update_global_dynamic_data

    {"next_block.block_num()":40903}
    th_a  db_block.cpp:439 _apply_block
1122001ms th_a       witness.cpp:266               block_production_loo ] Got exception while generating block:
3070000 undo_database_exception: undo database exception
The database does not have enough undo history to support a blockchain with so many missed blocks. Please add a checkpoint if you woul
d like to continue applying blocks beyond this point.
    {"recently_missed":44813,"max_undo":1000}
    th_a  db_update.cpp:72 update_global_dynamic_data

    {"next_block.block_num()":40903}
    th_a  db_block.cpp:439 _apply_block

    {"new_block":{"previous":"00009fc6d4e24db66e13af461b94063165999bfa","timestamp":"2015-08-20T10:18:42","witness":"1.6.5247","next_s
ecret_hash":"a151bcbe08eac49feb4eb86a5a68f332b4e7cf9d","previous_secret":"2fa1e05db52f0c421e335648c21bb3707383c1d6","transaction_merkl
e_root":"0000000000000000000000000000000000000000","extensions":[],"witness_signature":"206ea3856d3cf6317f19ab6b5d3af6648409ac092eae9c
c8d943d7357e3ac9c753141733131d2312c4644d44125835afce24190d295c21a6a620920ab660ec5f5c","transactions":[]}}
    th_a  db_block.cpp:176 _push_block

    {"witness_id":"1.6.5247"}
    th_a  db_block.cpp:315 _generate_block



[mudshark79]

I'm stuck at 40902 now. Looks like the testnet is dead again?

Yes, same for me, I was late for the party. If you already are a block producer maybe you can retsart your witness and try checkpointing option just for fun? Maybe you can get it back to life?
Option's obviously it's already there:
https://github.com/cryptonomex/graphene/blob/9c0c588ed62165886a510775d76c8524c50c09c5/libraries/app/application.cpp

Regards

Good idea. I am going to check how to set checkpoints. If you or anybody know please post here, thanks!

Line 594 of application.ccp says:
("checkpoint,c", bpo::value<vector<string>>()->composing(), "Pairs of [BLOCK_NUM,BLOCK_ID] that should be enforced as checkpoints.")

so adding "--checkpoint  [BLOCK_NUM,BLOCK_ID]" or '-c  [BLOCK_NUM,BLOCK_ID]' for the witness startup should do it....

[abit]

I'm stuck at 40902 now. Looks like the testnet is dead again?

Yes, same for me, I was late for the party. If you already are a block producer maybe you can retsart your witness and try checkpointing option just for fun? Maybe you can get it back to life?
Option's obviously it's already there:
https://github.com/cryptonomex/graphene/blob/9c0c588ed62165886a510775d76c8524c50c09c5/libraries/app/application.cpp

Regards

Good idea. I am going to check how to set checkpoints. If you or anybody know please post here, thanks!

[betax]

For those eagerly waiting .. here is another howto
https://github.com/cryptonomex/graphene/wiki/Howto-import-an-existing-delegate-as-witness-in-BitShares-2.0

Nice! I have just done a small correction on

Code: [Select]

--witness-id '"<witnessid>"'

[betax]

Happy forking 
After forked, if restart without --resync-blockchain, then my node won't sync.
If restart with --resync-blockchain, it won't produce block after in sync.

So it's best remove the data directory before start.

I'm online again now.

If restarted with --resync, needs to restarted again after sync. I have to do it also if I deleted the data directory.

[mudshark79]

I'm stuck at 40902 now. Looks like the testnet is dead again?

Code: [Select]

{
  "head_block_num": 40902,
  "head_block_id": "00009fc6d4e24db66e13af461b94063165999bfa",
  "head_block_age": "4 minutes old",
  "next_maintenance_time": "31 seconds ago",
  "chain_id": "5508f5f743717fe2c78445364f62a72badd7532974d26f089af2062228b532eb",
  "active_witnesses": [
    "1.6.0",
    "1.6.1",
    "1.6.2",
    "1.6.1525",
    "1.6.1536",
    "1.6.4435",
    "1.6.5245",
    "1.6.5246",
    "1.6.5247",
    "1.6.5248",
    "1.6.5249"
  ],
  "active_committee_members": [
    "1.5.0",
    "1.5.1",
    "1.5.2",
    "1.5.3",
    "1.5.4",
    "1.5.5",
    "1.5.6",
    "1.5.7",
    "1.5.8",
    "1.5.9"
  ],
  "entropy": "8927f3f36c4599a6a7af5a3cdfb70a8f57d463ae"
}


Code: [Select]

2015-08-20T07:11:59 th_a:invoke handle_block         handle_block ] Got block #40902 from network                       application.cp
p:343
2015-08-20T07:12:00 th_a:Witness Block Production block_production_loo ] slot: 1 scheduled_witness: 1.6.4435 scheduled_time: 2015-08-2
0T07:12:00 now: 2015-08-20T07:12:00                       witness.cpp:240
2015-08-20T07:12:01 th_a:Witness Block Production block_production_loo ] slot: 2 scheduled_witness: 1.6.5245 scheduled_time: 2015-08-2
0T07:12:01 now: 2015-08-20T07:12:01                       witness.cpp:240
2015-08-20T07:12:02 th_a:Witness Block Production block_production_loo ] slot: 3 scheduled_witness: 1.6.1 scheduled_time: 2015-08-20T0
7:12:02 now: 2015-08-20T07:12:02                  witness.cpp:240
2015-08-20T07:12:03 th_a:Witness Block Production block_production_loo ] slot: 4 scheduled_witness: 1.6.5247 scheduled_time: 2015-08-2
0T07:12:03 now: 2015-08-20T07:12:03                       witness.cpp:240
2015-08-20T07:12:03 th_a:Witness Block Production block_production_loo ] Witness 1.6.5247 production slot has arrived; generating a bl
ock now...                        witness.cpp:243
2015-08-20T07:12:03 th_a:Witness Block Production          _push_block ] Failed to push new block:
3070000 undo_database_exception: undo database exception
The database does not have enough undo history to support a blockchain with so many missed blocks. Please add a checkpoint if you would like to continue applying blocks beyond this point.
    {"recently_missed":1006,"max_undo":1000}
    th_a  db_update.cpp:72 update_global_dynamic_data

    {"next_block.block_num()":40903}
    th_a  db_block.cpp:439 _apply_block                 db_block.cpp:170


Yes, same for me, I was late for the party. If you already are a block producer maybe you can retsart your witness and try checkpointing option just for fun? Maybe you can get it back to life?
Option's obviously it's already there:
https://github.com/cryptonomex/graphene/blob/9c0c588ed62165886a510775d76c8524c50c09c5/libraries/app/application.cpp

Regards

[xeroc]

For those eagerly waiting .. here is another howto
https://github.com/cryptonomex/graphene/wiki/Howto-import-an-existing-delegate-as-witness-in-BitShares-2.0

[abit]

I'm stuck at 40902 now. Looks like the testnet is dead again?

Code: [Select]

{
  "head_block_num": 40902,
  "head_block_id": "00009fc6d4e24db66e13af461b94063165999bfa",
  "head_block_age": "4 minutes old",
  "next_maintenance_time": "31 seconds ago",
  "chain_id": "5508f5f743717fe2c78445364f62a72badd7532974d26f089af2062228b532eb",
  "active_witnesses": [
    "1.6.0",
    "1.6.1",
    "1.6.2",
    "1.6.1525",
    "1.6.1536",
    "1.6.4435",
    "1.6.5245",
    "1.6.5246",
    "1.6.5247",
    "1.6.5248",
    "1.6.5249"
  ],
  "active_committee_members": [
    "1.5.0",
    "1.5.1",
    "1.5.2",
    "1.5.3",
    "1.5.4",
    "1.5.5",
    "1.5.6",
    "1.5.7",
    "1.5.8",
    "1.5.9"
  ],
  "entropy": "8927f3f36c4599a6a7af5a3cdfb70a8f57d463ae"
}


Code: [Select]

2015-08-20T07:11:59 th_a:invoke handle_block         handle_block ] Got block #40902 from network                       application.cp
p:343
2015-08-20T07:12:00 th_a:Witness Block Production block_production_loo ] slot: 1 scheduled_witness: 1.6.4435 scheduled_time: 2015-08-2
0T07:12:00 now: 2015-08-20T07:12:00                       witness.cpp:240
2015-08-20T07:12:01 th_a:Witness Block Production block_production_loo ] slot: 2 scheduled_witness: 1.6.5245 scheduled_time: 2015-08-2
0T07:12:01 now: 2015-08-20T07:12:01                       witness.cpp:240
2015-08-20T07:12:02 th_a:Witness Block Production block_production_loo ] slot: 3 scheduled_witness: 1.6.1 scheduled_time: 2015-08-20T0
7:12:02 now: 2015-08-20T07:12:02                  witness.cpp:240
2015-08-20T07:12:03 th_a:Witness Block Production block_production_loo ] slot: 4 scheduled_witness: 1.6.5247 scheduled_time: 2015-08-2
0T07:12:03 now: 2015-08-20T07:12:03                       witness.cpp:240
2015-08-20T07:12:03 th_a:Witness Block Production block_production_loo ] Witness 1.6.5247 production slot has arrived; generating a bl
ock now...                        witness.cpp:243
2015-08-20T07:12:03 th_a:Witness Block Production          _push_block ] Failed to push new block:
3070000 undo_database_exception: undo database exception
The database does not have enough undo history to support a blockchain with so many missed blocks. Please add a checkpoint if you would like to continue applying blocks beyond this point.
    {"recently_missed":1006,"max_undo":1000}
    th_a  db_update.cpp:72 update_global_dynamic_data

    {"next_block.block_num()":40903}
    th_a  db_block.cpp:439 _apply_block                 db_block.cpp:170


[abit]

I added a default.log by editing config.ini so that screen output could be logged into file. I'm curious why it doesn't be enabled by default.

Code: [Select]

# declare an appender named "default" that writes messages to default.log
[log.file_appender.default]
filename=logs/default/default.log
# filename can be absolute or relative to this config file

# route any messages logged to the default logger to the "stderr" logger
# and "default" logger we
# declared above, if they are info level are higher
[logger.default]
level=info
appenders=stderr,default


[abit]

Happy forking 
After forked, if restart without --resync-blockchain, then my node won't sync.
If restart with --resync-blockchain, it won't produce block after in sync.

So it's best remove the data directory before start.

I'm online again now.

[betax]

I was able to delete my files and resync the witness, then I turned block production back on and it seems to be working now.

Here are some examples of how to use docker to run the witness:

Start a witness, all logs and data will be dumped right after execution stops:
docker run -it --rm sile16/graphene-witness:aug19test --genesis-json /aug-19-puppies-test-genesis.json -s 45.55.6.216:1776
(this docker image i included the genesis json in)

Start a witness with persistent data dir:
docker run -it --rm -v <local data dir>:/witness_node_data_dir sile16/graphene-witness:aug19test --genesis-json /aug-19-puppies-test-genesis.json -s 45.55.6.216:1776

to make it easier to expose ports you can also include --net=host
docker run -it --rm --net=host -v /witness_node/test_net2:/witness_node_data_dir sile16/graphene-witness:aug19test --genesis-json /aug-19-puppies-test-genesis.json -s 45.55.6.216:1776 --rpc-endpoint 127.0.0.1:8090


the docker image is pushed so these commands should work for anyone.

 

[betax]

Probably my spam transactions caused the problem.

Yes, and I can't resync to the network, witness_node crashes, so if devs will look to our test net they will found something interesting. 

I launched with --resync and everything was ok afterwards.