3508896ms th_a       application.cpp:265           startup              ] Detected unclean shutdown. Replaying blockchain...
It takes too much time to replay blockchain.. How to "cleanly" shutdown the witness node?

I'm at work and only have access to my vps through my phone.  Corporate vpn blocks port 22.  This is what I've got

Code: [Select]

2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ] Currently have 8 of [20/200] connections               $2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]    my id is 9ba089ffa726097f30b96f85510d79f6e4c2a53af47$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 178.62.88.151:59148 with 00bdfa3f396cf7$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 178.62.88.151:41574 with 5d33f74a83e257$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 176.9.234.167:34858 with 86a353604829f9$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 104.156.226.183:60715 with 24f8ce11c3bd$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 45.115.36.171:57281 with 256a70048ea155$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 188.226.252.109:58843 with 7ea834eafbcd$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 216.252.204.69:65026 with e8f88c156e162$2015-08-23T21:00:07 p2p:p2p_network_connect_loop display_current_conn ]         active: 128.68.89.226:55445 with 8b9a48d3943899$2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(178.62.91.161:2009)     $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(46.226.109.66:1778)     $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(5.189.131.201:2009)     $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(46.226.15.26:40706)     $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(46.226.12.230:2009)     $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(185.25.22.21:2776)      $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(69.172.229.183:2883)    $2015-08-23T21:00:07 p2p:p2p_network_connect_loop  connect_to_endpoint ] node_impl::connect_to_endpoint(66.172.11.223:1700)     $
And apparently juicessh saves the output all on one line.  Scroll to the right or quote of its easier.

OK, I've checked them. You have 8 connections, the first 6 are in my seed node list, the last 2 are incoming.

The latter 'connect_to_endpoint' ones seem to be 'bitshares_client' but not 'graphene'.
p2p.log.20150822T220000:2015-08-22T22:04:04 p2p:message read_loop disconnect_from_peer ] Disconnecting from 178.62.91.161:2009 for You are on a different chain from me                 node.cpp:4493


45.55.6.216:1776 is still up

Code: [Select]

}
unlocked >>> info
info
{
  "head_block_num": 245425,
  "head_block_id": "0003beb1a638ed50d318117d014424d44c2b2360",
  "head_block_age": "0 second old",
  "next_maintenance_time": "5 minutes in the future",
  "chain_id": "d011922587473757011118587f93afcc314fbaea094fc1055574721b27975083",
  "active_witnesses":

Yes, the port is open, but it doesn't response with a handshake message when connected.
2015-08-23T21:27:10 p2p:terminate_inactive_connections_loop terminate_inactive_c ] Forcibly disconnecting from handshaking peer 45.55.6.216:1776 due to inactivity of at least 5 seconds                        node.cpp:1270
2015-08-23T21:27:10   p2p:connect_to_task           connect_to ] fatal: error connecting to peer 45.55.6.216:1776: 0 exception: unspecified

Can you check p2p.log and see if there is any successful incoming connection?
grep inbound p2p.log|grep active

By check p2p.log we can also found what seed nodes we're connected with:
grep onbound p2p.log|grep active


I have found this error in my witness:

Code: [Select]

1249999ms th_a       witness.cpp:243               block_production_loo ] Witness 1.6.4231 production slot has arrived; generating a block now...
1250000ms th_a       witness.cpp:266               block_production_loo ] Got exception while generating block:
10 assert_exception: Assert Exception
_consecutive_production_enabled || db.get_dynamic_global_properties().current_witness != scheduled_witness: Last block was generated by the same witness, this node is probably disconnect
ed from the network so block production has been disabled.  Disable this check with --allow-consecutive option.
    {}
    th_a  witness.cpp:248 block_production_loop

Is it related to p2p problem?

Looks like you're on a fork.. Try restart, if it happen again try restart with --resync-blockchain.

I'm going to leave my witness node running until tomorrow, but I'n tired of waiting for puppies to vote me in or for the maintenance interval to pass that should do the same.

I'll have to try again Monday.

Good luck to your efforts puppies!

You've been voted in. 

Quote from: abit on August 22, 2015, 09:52:58 am

Run out of disk space 
p2p.log too large.

Same here, p2p.log grows like 1Gb/h
have you tried setting log level from debug to info? I'll try it now.

On my side it's 4~5GB/h for one node. Hadn't had time to tweak the settings. Just mounted a larger partition, will try.

Here is my theory:
Current testnet is too centralized.
Because the init node have about 90% of active witnesses, if it didn't receive a block produced by other witness (networking issue or so), most time next slot is the init node itself, so it just produce one, and next, next, next, certainly it will be the longest chain. That's why the network is 'robust', but it's not true, it's not safe, we can say we're encountering 51% attack indeed.