Author [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] Topic: What the Fork??  (Read 770 times)

0 Members and 1 Guest are viewing this topic.

Offline Riverhead

What the Fork??
« on: September 22, 2014, 11:38:19 AM »

One of the good things about this alpha stage is that it's allowing the delegates to shake out system issues, build tools around their delegates, and better understand the technology.

So in that spirit I'm looking for some insight into what happened a few hours ago that caused me to miss 16 consecutive blocks. Is it related to the forks in the list after the picture? I noticed that after my fork the next delegate has a huge latency spike. Am I on the wrong fork because of them or visa versa? Why did it eventually figure itself out?



Code: [Select]
   FORKED BLOCK              FORKING BLOCK ID              SIGNING DELEGATE      TXN COUNT      SIZE           TIMESTAMP   LATENCY   VALID    IN CURRENT CHAIN
--------------------------------------------------------------------------------------------------------------------------------------------------------------
         544146
     6dfadfc4e391f46184fcae08fce1eceb627a762f              a.delegate.xeroc              0       166 2014-09-21T13:38:40       598     YES                 YES
     954e5975848e98aba6ee7f80bc95fb9ad764d440             delegate.liondani             13      3019 2014-09-21T13:55:00         0     N/A                  NO
         544359
     30b0a1bf7566055f8cbabd8435d937d513934ea9                    dele-puppy              0       166 2014-09-21T14:15:10       609     YES                 YES
     4ea6c8ee71f7b6f98a4e70678f8724d304bc9818        www2.minebitshares-com             13      3374 2014-09-21T14:31:50         0     N/A                  NO
         544512
     ab2897abecd0d35b8c00af8a70a576395b0b1201               delegate.taolje              0       166 2014-09-21T14:40:50         0     YES                 YES
     90b8609d8ec0ac76adc8bff9c7b897c22fd06cd5                           fox              7      1930 2014-09-21T14:49:00         0     N/A                  NO
         545307
     21431f4eeb608faec4598af0a31d14a504491ae4               delegate.taolje              1       410 2014-09-21T16:54:00         0     YES                 YES
     07e77010de0e66e766737774571c38083917c5b2                           fox             13      4778 2014-09-21T17:10:30         1     N/A                  NO
         545552
     26e8fb9cae7f31249e1314c34fba192fa8f8c345                     spartako1              1       410 2014-09-21T17:35:30         0     YES                 YES
     ac8ba2a73a3dac799fe54cb1ad7fccb5bc29a8f1        www2.minebitshares-com              3      2267 2014-09-21T17:37:50         0     N/A                  NO
         545580
     7682aacc16c2b4077ff37ab36febe07e658d81cb        riverhead-del-server-1              5      1276 2014-09-21T17:46:20         0     YES                  NO
     34136cdfc995af6b79e59a86336e26da04ad974c                 dpos.crazybit              1       382 2014-09-21T17:40:20      3921     YES                 YES
         545636
     b34c1c611e50cf400c2e023d876ac7810dac87c3         btsx.chinesecommunity              0       166 2014-09-21T17:50:10         0     YES                 YES
     d9b679d434be6ef11576deac06bf1569b21b0bfd               delegate.webber              2       722 2014-09-21T17:54:30         0     N/A                  NO
         546260
     6cc0550eb4e5ea13c642b1098815916823e2b6ed                titan.crazybit              1       410 2014-09-21T19:37:10         0     YES                 YES
     0cde6d50bd417fc07dbad0c8c90daf4a55463230                     calabiyau              2      2019 2014-09-21T19:49:20        -1     N/A                  NO
         546293
     fa4ac65214a818f387bb8eb91ee548985d871151              delegate.cgafeng              0       166 2014-09-21T19:42:50         0     YES                 YES
     993f7692d68e136ff2575c9b1bca63f99ae271b0                    maqifrnswa              1      1775 2014-09-21T19:48:30         0     N/A                  NO
         548904
     6ba84dc7b49762b367c9a63d1b9148f7c8ad8f4e             delegate.liondani              0       166 2014-09-22T03:05:20         0     YES                 YES
     34cce5013e5c865e0c1b018a45e042a247870a93              delegate.cgafeng              9      5576 2014-09-22T03:21:00         1     N/A                  NO
         549041
     de000ca0a862b328c05d92b765b5bae42d1a864a                      crazybit              1       506 2014-09-22T03:28:20         0     YES                 YES
     e688612bc696b009d65449902e9dbaf7f4592e9d               delegate.bitder              4      1430 2014-09-22T03:33:40         0     N/A                  NO
         549231
     76c478280584e04b46b376e516eb8daff3f8941c        riverhead-del-server-1              2      2217 2014-09-22T04:08:10         0     YES                  NO
     e5e9ddc2b67817f0d9edaf5d3bf5744cb8eab0ed                    cny.bts500              0       166 2014-09-22T04:01:10     15660     YES                 YES
         549287
     cbbd03bc312c9267a1a2803c90d2959461cadc0a                  delegate-alt              0       166 2014-09-22T04:11:00         0     YES                 YES
     0747752a078447aa26685890164e000f62e9a46b                delegate1-galt              0       166 2014-09-22T04:17:00         0     N/A                  NO
         549429
     421abe39c5f1769c22c8d42385558f15b6e2a08f                     coolspeed              0       166 2014-09-22T04:36:50         0     YES                 YES
     7edfbf3eb8d94bc6e8df66166c71803af18716ba             delegate.liondani              6      3162 2014-09-22T04:48:30         0     N/A                  NO
         549526
     72c89d29eb6a586db10dbe0c88394bf4f8b7979b                   dc-delegate              0       166 2014-09-22T04:54:20         0     YES                 YES
     f9d5c5270fd95335a27907b1a72d4b23edb0c3ed                       chinese              3      1172 2014-09-22T04:58:50         0     N/A                  NO
         549641
     c8dbea7384f7c834ff92d7e7d178608bfc9cbc25                delegate-baozi              0       166 2014-09-22T05:15:40         0     YES                 YES
     9a295c98a77be97b4fbf6b72268ccc2ba58f32ad                       kokojie              6      3186 2014-09-22T05:31:30         0     N/A                  NO
         549700
     56f4c2bcf413df9cb4ad047ffcdc3f624fd31363          sun.delegate.service              1       410 2014-09-22T05:26:10         1     YES                 YES
     8a40ccba6e6dcc85d4749358c24443215e74e37e               delegate.bitder              2       567 2014-09-22T05:26:40         0     N/A                  NO
         550492
     8db516f5dfe768a524abc00375c24cc705c5afbf                 dpos.crazybit              0       166 2014-09-22T07:51:00         0     YES                 YES
     da70aa6eabbb31b2f8c2b9711feab65705cc68aa               delegate.bitder              0       166 2014-09-22T07:51:20         0     N/A                  NO

Offline svk

Re: What the Fork??
« Reply #1 on: September 22, 2014, 12:04:53 PM »
Interesting, this happened to me a few times as well and would be nice to understand why. BM said at the time it could be due to the delegate behind or in front of me having a high latency and thus ignoring my block, so guess that could be why in your case.

I looked up the blocks where your fork happened on my site (545580+1 and 549231+1), and I've got them down as signed by "dpos.crazybit" and "cny.bts500" with a latency of 0s. It looks like you tried to produce that first block 6 minutes after and the second 7 minutes after it was produced, which I guess is why you created forks.

Maybe the latency spike was actually on your end, or your system clock was unsynchronized?



Worker: dev.bitsharesblocks

Offline Riverhead

Re: What the Fork??
« Reply #2 on: September 22, 2014, 12:13:15 PM »
Thank you for the research, svk. That is an interesting point as a few times I have found that on another wallet (where I'm playing with bots) I sometimes see that my client is five to 10 minutes behind.

So I guess the next question is....why is my client experiencing these stoppages in block syncing? I wonder if there's enough information in the logs to correlate connection count and the missing blocks.

Offline tonyk

  • Hero Member
  • *****
  • Posts: 3309
    • View Profile
Re: What the Fork??
« Reply #3 on: September 22, 2014, 12:27:27 PM »
Thank you for the research, svk. That is an interesting point as a few times I have found that on another wallet (where I'm playing with bots) I sometimes see that my client is five to 10 minutes behind.

So I guess the next question is....why is my client experiencing these stoppages in block syncing? I wonder if there's enough information in the logs to correlate connection count and the missing blocks.

Experiencing similar things since some delegates upgraded to 0.4.16 (cr1/2) with my non delegate client. My client was still on 0.4.15; Now with me on 0.4.16 I have seen this behavior 3 or 4 times. Restarting seems to fix the problem for me but then again it is easy when one is not delegate... Will post more info the next time this happens...will be good if somebody points me to what info will be helpful.
My initial post/info: https://bitsharestalk.org/index.php?topic=9099.msg118456#msg118456
Lack of arbitrage is the problem, isn't it. And this 'should' solves it.

Offline Riverhead

Re: What the Fork??
« Reply #4 on: September 22, 2014, 12:39:25 PM »
Thanks tonyk, glad I'm not alone on this. The reason I noticed it with my bots is that they would a long time without a price update. When I checked the wallet I noticed it was minutes behind. A restart fixed it then as well. However, it seems the client does eventually catch up on its own.

Offline svk

Re: What the Fork??
« Reply #5 on: September 22, 2014, 12:54:59 PM »
Just thought of something, are you running your client with "--accept-incoming-connections false"? That's how I missed blocks last time, after a certain time you start losing connections and your client gets out of sync. That doesn't explain the latency though..
Worker: dev.bitsharesblocks

Offline Riverhead

Re: What the Fork??
« Reply #6 on: September 22, 2014, 01:02:40 PM »
I am running with it that way; but I always have been (it's behind a strict firewall). Perhaps things have changes. I'll try it with that flag set to true and see what happens.

Update: it is set to true...must have missed that in an upgrade. Anyway, I get this at start which looks normal.

Code: [Select]
./run_delegate.sh
Loading blockchain from: .delegate_0.4.16/chain
Loading config from file: .delegate_0.4.16/config.json
Starting JSON RPC server on port 49333 (localhost only)
Starting HTTP JSON RPC server on port 1579 (localhost only)
UPnP: ExternalIPAddress = xxx.xxx.xxx.xxx
UPnP Port Mapping successful
« Last Edit: September 22, 2014, 01:05:28 PM by Riverhead »

Offline Riverhead

Re: What the Fork??
« Reply #7 on: September 22, 2014, 01:21:19 PM »
Just caught this. Restarted and didn't miss a block. I'll have to keep an eye on it until we figure out what's going on.

"blockchain_head_block_age": "14 minutes old",

Code: [Select]
info
{
  "blockchain_head_block_num": 552313,
  "blockchain_head_block_age": "14 minutes old",
  "blockchain_head_block_timestamp": "2014-09-22T13:03:30",
  "blockchain_average_delegate_participation": "55.19 %",
  "blockchain_confirmation_requirement": 1,
  "blockchain_delegate_pay_rate": "2.23912 BTSX",
  "blockchain_share_supply": "1,999,861,866.73016 BTSX",
  "blockchain_blocks_left_in_round": 56,
  "blockchain_next_round_time": "at least 9 minutes in the future",
  "blockchain_next_round_timestamp": "2014-09-22T13:26:30",
  "blockchain_random_seed": "61c71a80b3c2f54aedc92f0c72c8f321352fcd0b",
  "client_data_dir": "/home/riverhead/.delegate_0.4.16",
  "client_version": "v0.4.16",
  "network_num_connections": 0,
  "network_num_connections_max": 200,
  "ntp_time": "2014-09-22T13:17:16",
  "ntp_time_error": -0.022846000000000002,
  "wallet_open": true,
  "wallet_unlocked": true,
  "wallet_unlocked_until": "3 years 2 months in the future",
  "wallet_unlocked_until_timestamp": "2017-11-22T22:50:43",
  "wallet_last_scanned_block_timestamp": "2014-07-21T07:02:30",
  "wallet_scan_progress": "? %",
  "wallet_block_production_enabled": true,
  "wallet_next_block_production_time": "at least 9 minutes in the future",
  "wallet_next_block_production_timestamp": "2014-09-22T13:33:50"
« Last Edit: September 22, 2014, 01:25:03 PM by Riverhead »

Offline svk

Re: What the Fork??
« Reply #8 on: September 22, 2014, 01:25:48 PM »
I meant to say I missed blocks when I set it to true, sorry about that. It's when you're blocking incoming connections that you end up losing connections and getting out of sync.
Worker: dev.bitsharesblocks

Offline bitder

  • Full Member
  • ***
  • Posts: 65
    • View Profile
Re: What the Fork??
« Reply #9 on: September 22, 2014, 02:00:15 PM »
Thank you for the research, svk. That is an interesting point as a few times I have found that on another wallet (where I'm playing with bots) I sometimes see that my client is five to 10 minutes behind.

So I guess the next question is....why is my client experiencing these stoppages in block syncing? I wonder if there's enough information in the logs to correlate connection count and the missing blocks.

Experiencing similar things since some delegates upgraded to 0.4.16 (cr1/2) with my non delegate client. My client was still on 0.4.15; Now with me on 0.4.16 I have seen this behavior 3 or 4 times. Restarting seems to fix the problem for me but then again it is easy when one is not delegate... Will post more info the next time this happens...will be good if somebody points me to what info will be helpful.
My initial post/info: https://bitsharestalk.org/index.php?topic=9099.msg118456#msg118456

I can confirm that since upgrading to 0.4.16 my delegate has missed blocks.
My non-delegate client (also running 0.4.16) stopped syncing.
Code: [Select]
   "blockchain_head_block_age": "5 hours old",
   "client_version": "v0.4.16",
   "network_num_connections": 5,

Restarted the non-delegate client and it made some progress but stopped syncing again

Code: [Select]
  "blockchain_head_block_num": 552498,
  "blockchain_head_block_age": "22 minutes old",
  "blockchain_head_block_timestamp": "2014-09-22T13:34:40",
  "client_version": "v0.4.16",
  "network_num_connections": 14,
  "network_num_connections_max": 200,
  "ntp_time": "2014-09-22T13:57:02",
  "ntp_time_error": -0.024471,
wallet_account_set_approval delegate.bitder 1

Offline bitder

  • Full Member
  • ***
  • Posts: 65
    • View Profile
Re: What the Fork??
« Reply #10 on: September 22, 2014, 02:02:40 PM »
Thank you for the research, svk. That is an interesting point as a few times I have found that on another wallet (where I'm playing with bots) I sometimes see that my client is five to 10 minutes behind.

So I guess the next question is....why is my client experiencing these stoppages in block syncing? I wonder if there's enough information in the logs to correlate connection count and the missing blocks.

Experiencing similar things since some delegates upgraded to 0.4.16 (cr1/2) with my non delegate client. My client was still on 0.4.15; Now with me on 0.4.16 I have seen this behavior 3 or 4 times. Restarting seems to fix the problem for me but then again it is easy when one is not delegate... Will post more info the next time this happens...will be good if somebody points me to what info will be helpful.
My initial post/info: https://bitsharestalk.org/index.php?topic=9099.msg118456#msg118456

I can confirm that since upgrading to 0.4.16 my delegate has missed blocks.
My non-delegate client (also running 0.4.16) stopped syncing.
Code: [Select]
   "blockchain_head_block_age": "5 hours old",
   "client_version": "v0.4.16",
   "network_num_connections": 5,

Restarted the non-delegate client and it made some progress but stopped syncing again

Code: [Select]
  "blockchain_head_block_num": 552498,
  "blockchain_head_block_age": "22 minutes old",
  "blockchain_head_block_timestamp": "2014-09-22T13:34:40",
  "client_version": "v0.4.16",
  "network_num_connections": 14,
  "network_num_connections_max": 200,
  "ntp_time": "2014-09-22T13:57:02",
  "ntp_time_error": -0.024471,

Restarted my non-delegate client again and it caught up.
wallet_account_set_approval delegate.bitder 1

Offline dannotestein

Re: What the Fork??
« Reply #11 on: September 22, 2014, 02:43:28 PM »
Thank you for the research, svk. That is an interesting point as a few times I have found that on another wallet (where I'm playing with bots) I sometimes see that my client is five to 10 minutes behind.

So I guess the next question is....why is my client experiencing these stoppages in block syncing? I wonder if there's enough information in the logs to correlate connection count and the missing blocks.

Experiencing similar things since some delegates upgraded to 0.4.16 (cr1/2) with my non delegate client. My client was still on 0.4.15; Now with me on 0.4.16 I have seen this behavior 3 or 4 times. Restarting seems to fix the problem for me but then again it is easy when one is not delegate... Will post more info the next time this happens...will be good if somebody points me to what info will be helpful.
My initial post/info: https://bitsharestalk.org/index.php?topic=9099.msg118456#msg118456

I can confirm that since upgrading to 0.4.16 my delegate has missed blocks.
My non-delegate client (also running 0.4.16) stopped syncing.
Code: [Select]
   "blockchain_head_block_age": "5 hours old",
   "client_version": "v0.4.16",
   "network_num_connections": 5,

Restarted the non-delegate client and it made some progress but stopped syncing again

Code: [Select]
  "blockchain_head_block_num": 552498,
  "blockchain_head_block_age": "22 minutes old",
  "blockchain_head_block_timestamp": "2014-09-22T13:34:40",
  "client_version": "v0.4.16",
  "network_num_connections": 14,
  "network_num_connections_max": 200,
  "ntp_time": "2014-09-22T13:57:02",
  "ntp_time_error": -0.024471,
Can you try deleting the node_config.json in your data directory for that client and see if the problem goes away? I want to check if it's this issue: https://github.com/BitShares/bitshares_toolkit/issues/794
http://blocktrades.us Fast/Safe/High-Liquidity Crypto Coin Converter

Offline CalabiYau

Re: What the Fork??
« Reply #12 on: September 22, 2014, 04:32:39 PM »
Code: [Select]
       552367
     2c084d3ebd8bbfdbdfdbd7bf8785d3c83778af4e              a.delegate.xeroc              0       166 2014-09-22T13:12:40         0     YES                 YES
     b7c66ee7c18d952c2243e7254265b12e28bb6b3f               anchor.crazybit              0       166 2014-09-22T13:14:20         0     N/A                  NO
         552563
     eae05293ddcb22a2cef8b1e196fe646f69c99441                 delegate.adam              0       166 2014-09-22T13:45:40         0     YES                 YES
     422b027cf7424c9e08c7b9aef35dc79522eddec5                   dc-delegate             13      2344 2014-09-22T13:59:40         1     N/A                  NO
         553071
     f83e68347c0cbc04270023632ad3364acfa831ba                      crazybit              1       410 2014-09-22T15:11:10         1     YES                 YES
     67036b28d3890e11c18b096e1d4b0016d90adf01              delegate.cgafeng              1       410 2014-09-22T15:12:00         0     N/A                  NO
         553310
     0c0b4b133d29340801c1a10dc1950d1127d8b014                     calabiyau              8      9577 2014-09-22T16:03:40         0     N/A                  NO
     da5e0d99b7dbff3524ba6244b0cac9ea8d24b465      delegate.nathanhourt.com              0       166 2014-09-22T15:51:20      1701     YES                 YES

for the second time in 24 hours the delegate client lost the chain and was missing a block.
Deleted  node_config.json and restart.
Watching.

Offline Riverhead

Re: What the Fork??
« Reply #13 on: September 22, 2014, 04:46:00 PM »


Can you try deleting the node_config.json in your data directory for that client and see if the problem goes away? I want to check if it's this issue: https://github.com/BitShares/bitshares_toolkit/issues/794

Deleted.


Offline tonyk

  • Hero Member
  • *****
  • Posts: 3309
    • View Profile
Re: What the Fork??
« Reply #14 on: September 22, 2014, 08:18:03 PM »
Code: [Select]
>> get_info

{
  "blockchain_head_block_num": 554852,
  "blockchain_head_block_age": "6 minutes old",
  "blockchain_head_block_timestamp": "2014-09-22T20:10:50",
  "blockchain_average_delegate_participation": "74.26 %",
  "blockchain_confirmation_requirement": 1,
  "blockchain_delegate_pay_rate": "2.20136 BTSX",
  "blockchain_share_supply": "2,000,187,518.50463 BTSX",
  "blockchain_blocks_left_in_round": 42,
  "blockchain_next_round_time": "at least 7 minutes in the future",
  "blockchain_next_round_timestamp": "2014-09-22T20:23:40",
  "blockchain_random_seed": "cca1278fda16821b701399d4b061a331c259008d",
  "client_data_dir": "C:/Users/Toni/AppData/Roaming/BITSHA~1",
  "client_version": "v0.4.16",
  "network_num_connections": 14,
  "network_num_connections_max": 200,
  "ntp_time": "2014-09-22T20:16:42",
  "ntp_time_error": 1.9602310000000001,
  "wallet_open": true,
  "wallet_unlocked": true,
  "wallet_unlocked_until": "12 days in the future",
  "wallet_unlocked_until_timestamp": "2014-10-04T08:49:10",
  "wallet_last_scanned_block_timestamp": "2014-09-22T20:10:50",
  "wallet_scan_progress": "100.00 %",
  "wallet_block_production_enabled": false,
  "wallet_next_block_production_time": null,
  "wallet_next_block_production_timestamp": null
}

Deleted.
« Last Edit: September 22, 2014, 08:22:50 PM by tonyk »
Lack of arbitrage is the problem, isn't it. And this 'should' solves it.

 

Google+