BitShares Forum
Main => Stakeholder Proposals => Topic started by: alt on July 30, 2014, 05:33:00 am
-
hi, delegates. we have a solution for hot standby.
set up delegate wallet in 2 machines. one is master, and the other is slave.
master run delegate_loop task at seconds 0,10,20 ....
slave run the task with 5 seconds delay, at 5, 15, 25 .....
Normally master will generate block in time, slave will receive the block less than 5 seconds, than slave will not generate block.
If for some reason, master missed generation, slave didn't receive block from master in 5 seconds, slave will generate.
here is the patch for slave program:
--- a/libraries/client/client.cpp
+++ b/libraries/client/client.cpp
@@ -729,6 +729,7 @@ config load_config( const fc::path& datadir )
uint32_t slot_number = blockchain::get_slot_number( now );
time_point_sec next_slot_time = blockchain::get_slot_start_time( slot_number + 1 );
+ next_slot_time += 5;
ilog( "Rescheduling delegate loop for time: ${t}", ("t",next_slot_time) );
time_point scheduled_time = next_slot_time;
-
+5% +5%
I want to clarify again that for anyone who are going to try this code, please pay attention that there are *two* nodes, master and slave, with different schedule times but the same slot time!
Looking forward more review ...
-
Awesome! Never miss a block anymore ..
i guess best practice for the beginning would be to directly connect the mashines to each other
When will this patch be on github?
-
Awesome! Never miss a block anymore ..
i guess best practice for the beginning would be to directly connect the mashines to each other
When will this patch be on github?
yes, should add node to the other side.
If it's ok to run like this, maybe we should have a parameter(from 0 to 5) to setup the delay time.
-
nice.i will try this out
hi, delegates. we have a solution for hot standby.
set up delegate wallet in 2 machines. one is master, and the other is slave.
master run delegate_loop task at seconds 0,10,20 ....
slave run the task with 5 seconds delay, at 5, 15, 25 .....
Normally master will generate block in time, slave will receive the block less than 5 seconds, than slave will not generate block.
If for some reason, master missed generation, slave didn't receive block from master in 5 seconds, slave will generate.
here is the patch for slave program:
--- a/libraries/client/client.cpp
+++ b/libraries/client/client.cpp
@@ -729,6 +729,7 @@ config load_config( const fc::path& datadir )
uint32_t slot_number = blockchain::get_slot_number( now );
time_point_sec next_slot_time = blockchain::get_slot_start_time( slot_number + 1 );
+ next_slot_time += 5;
ilog( "Rescheduling delegate loop for time: ${t}", ("t",next_slot_time) );
time_point scheduled_time = next_slot_time;
-
excellent. can even make a new command "wallet_delegate_set_slave {IP_of_master} {true/false}" which will add the node and add a 5 second delay to production
-
excellent. can even make a new command "wallet_delegate_set_slave {IP_of_master} {true/false}" which will add the node and add a 5 second delay to production
Oh yhea .. would make things incredibly easy
-
This will likely cause you to sign 2 different blocks.
How you ensure that they will not sign 2 different blocks ?
-
This will likely cause you to sign 2 different blocks.
How you ensure that they will not sign 2 different blocks ?
This!
and if not, why would bytemaster not decrease the block production time from 10 to 5 sec. on the next BTSX hard fork?
IS IT POSSIBLE ? ;D
(http://images4.fanpop.com/image/photos/23800000/Doc-Brown-back-to-the-future-23840539-720-488.jpg)
-
Usage:
wallet_delegate_set_slave <delegate> <master_node> <delay> <enable>
Enable or disable block production for delegate following delayed block production on master node
Parameters:
delegate (string, required): The delegate name to slave for (may use ALL for multiple delegates)
master_node (string, required): The IP and port number of the master delegate node. (Format IP:PORT)
delay (int, optional) {0-10} : The delay in seconds the slave will wait for receipt of block production from master (default 5)
enabled (bool, optional): true to enable block production, else false (default: true)
Returns:
void
-
This will likely cause you to sign 2 different blocks.
How you ensure that they will not sign 2 different blocks ?
Networking the master/slave pair nearest each other will mitigate risks due to propagation delays. However, I do not see an elegant fail safe way with a single master/slave pair to ensure delegate block production. Introducing additional slaves for consensus building will likely add too much time before the block production window is breached and the next delegate produces on time.
-
This will likely cause you to sign 2 different blocks.
How you ensure that they will not sign 2 different blocks ?
yes, this can happen.
If the rule forbid do this, we'll change to another solution, ensure only 1 node active at the same time.
-
This will likely cause you to sign 2 different blocks.
How you ensure that they will not sign 2 different blocks ?
This!
and if not, why would bytemaster not decrease the block production time from 10 to 5 sec. on the next BTSX hard fork?
IS IT POSSIBLE ? ;D
(http://images4.fanpop.com/image/photos/23800000/Doc-Brown-back-to-the-future-23840539-720-488.jpg)
In fact, I run master and slave at the same machine now.
And I add the node to each other to make sure they community with less latency.
for now, most missed block is because of software issue, no need to run in two different machine.
And another thing, it's better to restart slave every 12 hours.
I forgot to set auto restart slave yesterday. my slave run long time, and met the old problem, connection lost much, and sign some wrong blocks...
-
Ok .. I am finally setting up a back delegate with this solution ... python polling is shit :)
-
Not sure if I did sth. wrong but by backup delegate produced a fork .. so it seems it does not work as intended... does the backup notice that a valid block is allready there and stops the production?
-
Not sure if I did sth. wrong but by backup delegate produced a fork .. so it seems it does not work as intended... does the backup notice that a valid block is allready there and stops the production?
yes it should stop the production.
I run with version 0.3.1, and it work find.
I'll check it when I upgrade to 0.4.1
-
ok .. so probably I did sth wrong .. maybe mixed up my machines (currently running 3 VPS nodes :) )
Gonna recheck this afternoon
-
I think maybe it's better that people should stop taking this approach, because it increase the chance of producing two blocks which is not for reaching consensus, later versions will have operation to fire the delegate if someone can prove this delegate broadcast two different blocks.
-
I think maybe it's better that people should stop taking this approach, because it increase the chance of producing two blocks which is not for reaching consensus, later versions will have operation to fire the delegate if someone can prove this delegate broadcast two different blocks.
I have think this again.
with this rule, I think the hot standby is less useful.
It can't handle the situation that master is totally unavailable to slave. (maybe program crash, maybe network down, maybe lost power ....)
the slave can't replace the master without master's empower.
All we can do is to check the state of the node, like the sync block number, the network connects, and adjust to a more suitable node.
-
I wonder if the orphan block is harmless or not?
If it's harmless, May I ask to delay the punish rule, until we have confirm there is no issue about missed blocks?
here is the blocks which should have missed without hot standby.
you can see all these blocks are generated by the slave node, with a latency about 5 seconds
20140801T005845.095164 th_a delegate_loop ] Produced block #106529! client.cpp:765
20140807T040155.143352 th_a delegate_loop ] Produced block #158623! client.cpp:765
20140811T150025.185202 th_a delegate_loop ] Produced block #196508! client.cpp:765
20140813T020315.208326 th_a delegate_loop ] Produced block #208869! client.cpp:765
20140816T030035.228557 th_a delegate_loop ] Produced block #234622! client.cpp:765
20140817T100325.246930 th_a delegate_loop ] Produced block #245657! client.cpp:765
20140817T102915.243216 th_a delegate_loop ] Produced block #245808! client.cpp:765
20140817T104105.244196 th_a delegate_loop ] Produced block #245879! client.cpp:765
20140817T110245.247619 th_a delegate_loop ] Produced block #246006! client.cpp:765
20140817T112915.242517 th_a delegate_loop ] Produced block #246161! client.cpp:765
20140817T113155.239689 th_a delegate_loop ] Produced block #246176! client.cpp:765
20140817T125335.259069 th_a delegate_loop ] Produced block #246656! client.cpp:765
20140817T131305.254306 th_a delegate_loop ] Produced block #246771! client.cpp:765
20140817T132925.246843 th_a delegate_loop ] Produced block #246866! client.cpp:765
20140817T133915.336232 th_a delegate_loop ] Produced block #246924! client.cpp:765
And I still don't know why the master node have missed these blocks. here is the log from master node
maybe some is strange at date 20140817
delegate (unlocked) >>> blockchain_list_blocks 246920 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246920 2014-08-17T13:38:30 eur.bts500 0 166 0.00000 BTSX 1 0.002601
246921 2014-08-17T13:38:40 delegate.charity 0 166 0.00000 BTSX 1 0.001493
246922 2014-08-17T13:38:50 bitcoiners 0 166 0.00000 BTSX 32 0.001067
246923 2014-08-17T13:39:00 coolspeed 1 410 0.10000 BTSX 23 0.021211
246924 2014-08-17T13:39:10 delegate-watchman 0 166 0.00000 BTSX 13 0.002413
delegate (unlocked) >>> blockchain_list_blocks 246862 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246862 2014-08-17T13:28:40 clout-delegate5 0 166 0.00000 BTSX 1 0.001768
246863 2014-08-17T13:28:50 skyscraperfarms 0 166 0.00000 BTSX 2 0.001662
246864 2014-08-17T13:29:00 delegate.bitsuperlab 0 166 0.00000 BTSX 0 0.002276
246865 2014-08-17T13:29:10 init73 0 166 0.00000 BTSX 21 0.000418
246866 2014-08-17T13:29:20 delegate-baozi 0 166 0.00000 BTSX 11 0.004477
delegate (unlocked) >>> blockchain_list_blocks 246767 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246767 2014-08-17T13:12:20 init85 0 166 0.00000 BTSX 1 0.013516
246768 2014-08-17T13:12:30 calabiyau 1 365 0.10000 BTSX 1 0.007375
246769 2014-08-17T13:12:40 init28 0 166 0.00000 BTSX 61 0.001217
246770 2014-08-17T13:12:50 delegate.taolje 0 166 0.00000 BTSX 51 0.006725
246771 2014-08-17T13:13:00 delegate.bitsuperlab 0 166 0.00000 BTSX 41 0.009017
delegate (unlocked) >>> blockchain_list_blocks 246652 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246652 2014-08-17T12:52:50 happyshares-2 0 166 0.00000 BTSX 1 0.001722
246653 2014-08-17T12:53:00 spartako 0 166 0.00000 BTSX 1 0.001564
246654 2014-08-17T12:53:10 love.ymca 1 362 0.10000 BTSX 1 0.005774
246655 2014-08-17T12:53:20 delegate.xeldal 0 166 0.00000 BTSX 15 0.002737
246656 2014-08-17T12:53:30 delegate-watchman 0 166 0.00000 BTSX 6 0.004592
delegate (unlocked) >>> blockchain_list_blocks 246172 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246172 2014-08-17T11:31:10 clout-delegate5 0 166 0.00000 BTSX 1 0.001608
246173 2014-08-17T11:31:20 delegate.bitder 0 166 0.00000 BTSX 1 0.001527
246174 2014-08-17T11:31:30 init65 0 166 0.00000 BTSX 31 0.001175
246175 2014-08-17T11:31:40 init28 0 166 0.00000 BTSX 21 0.003292
246176 2014-08-17T11:31:50 delegate.bitsuperlab 0 166 0.00000 BTSX 11 0.001498
delegate (unlocked) >>> blockchain_list_blocks 246157 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246157 2014-08-17T11:28:30 clout-delegate1 0 166 0.00000 BTSX 1 0.001934
246158 2014-08-17T11:28:40 www.minebitshares-com 0 166 0.00000 BTSX 1 0.001773
246159 2014-08-17T11:28:50 riverhead-del-server-1 0 166 0.00000 BTSX 31 0.001019
246160 2014-08-17T11:29:00 cny.bts500 0 166 0.00000 BTSX 21 0.00554
246161 2014-08-17T11:29:10 delegate-watchman 0 166 0.00000 BTSX 11 0.001573
delegate (unlocked) >>> blockchain_list_blocks 246002 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
246002 2014-08-17T11:02:00 liondani-delegate-1 0 166 0.00000 BTSX 1 0.004777
246003 2014-08-17T11:02:10 bitsharesx-delegate 0 166 0.00000 BTSX 3 0.00576
246004 2014-08-17T11:02:20 future.dacwin 0 166 0.00000 BTSX 27 0.00024
246005 2014-08-17T11:02:30 delegate.charity 0 166 0.00000 BTSX 17 0.007007
246006 2014-08-17T11:02:40 delegate-watchman 0 166 0.00000 BTSX 7 0.001626
delegate (unlocked) >>> blockchain_list_blocks 245875 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
245875 2014-08-17T10:40:20 clout-delegate5 1 410 0.10000 BTSX 1 0.005272
245876 2014-08-17T10:40:30 chinese 1 602 0.10000 BTSX 0 0.013464
245877 2014-08-17T10:40:40 mr.agsexplorer 0 166 0.00000 BTSX 0 0.00159
245878 2014-08-17T10:40:50 eur.bts500 1 410 0.10000 BTSX 16 0.001288
245879 2014-08-17T10:41:00 delegate.bitsuperlab 1 410 0.10000 BTSX 7 0.012646
delegate (unlocked) >>> blockchain_list_blocks 245804 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
245804 2014-08-17T10:28:30 google.helloworld 0 166 0.00000 BTSX 1 0.00236
245805 2014-08-17T10:28:40 skyscraperfarms 0 166 0.00000 BTSX 1 0.00291
245806 2014-08-17T10:28:50 init58 1 410 0.10000 BTSX 0 0.015231
245807 2014-08-17T10:29:00 now.dacwin 0 166 0.00000 BTSX 21 0.000282
245808 2014-08-17T10:29:10 delegate-baozi 0 166 0.00000 BTSX 11 0.006493
delegate (unlocked) >>> blockchain_list_blocks 245653 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
245653 2014-08-17T10:02:40 mr.agsexplorer 0 166 0.00000 BTSX 1 0.00362
245654 2014-08-17T10:02:50 delegate.taolje 0 166 0.00000 BTSX 1 0.002551
245655 2014-08-17T10:03:00 delegate-baozi 0 166 0.00000 BTSX 0 0.002826
245656 2014-08-17T10:03:10 maqifrnswa 0 166 0.00000 BTSX 21 0.000229
245657 2014-08-17T10:03:20 delegate-watchman 0 166 0.00000 BTSX 11 0.016401
delegate (unlocked) >>> blockchain_list_blocks 234618 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
234618 2014-08-16T02:59:50 btsnow 0 166 0.00000 BTSX 0 0.001571
234619 2014-08-16T03:00:00 dele-puppy 0 166 0.00000 BTSX 0 0.0022
234620 2014-08-16T03:00:10 angel.bitdelegate 0 166 0.00000 BTSX 0 0.001478
234621 2014-08-16T03:00:20 clout-delegate5 0 166 0.00000 BTSX 0 0.00132
234622 2014-08-16T03:00:30 delegate-watchman 0 166 0.00000 BTSX 5 0.00274
delegate (unlocked) >>> blockchain_list_blocks 208865 5
HEIGHT TIMESTAMP SIGNING DELEGATE # TXS SIZE TOTAL FEES LATENCY PROCESSING TIME
===================================================================================================================
208865 2014-08-13T02:02:30 init58 0 166 0.00000 BTSX 0 0.003361
208866 2014-08-13T02:02:40 bits 0 166 0.00000 BTSX 0 0.003819
208867 2014-08-13T02:02:50 delegate-watchman 0 166 0.00000 BTSX 0 0.001733
208868 2014-08-13T02:03:00 bts.coin 0 166 0.00000 BTSX 0 0.0125
208869 2014-08-13T02:03:10 delegate-watchman 0 166 0.00000 BTSX 6 0.002742