Dry Run 7: The Power of DPOS

[bytemaster]

I go to bed for a few lousy hours and you all let the blockchain die.  What am I going to do with you?

I am trying to figure out what happened and I think we killed it with a series of unfortunate events:

1) Webber flooded us at 22 trx per second which was probably triggered a ton of forks
2) I upgraded the database in an attempt to improve processing speed,  this upgrade was probably ill-advised as it looks like it broke everyone who upgraded.

So what do we do now?  We use this as a dry run on how to handle massive chain disruption.  Our mission is to get delegate participation back up to 90% or more. 

We are now preparing a series of significant enhancements to address the flooding issue:
1) Every client will limit the number of pending transactions it will store.  This will delay propagation of transactions until the blockchain can process them.  In the event the network is saturated you can still broadcast additional transactions at exponential increases in the required fee. 
2) The database bug exposed a bug in network syncing, so we are adding extra validation checks on messages we receive from peers
3) We have implemented rotating logs with compression which should help limit the impact of our verbose test logs.
4) Fixed a bug with importing old BTS wallets which was keeping some of my delegates off line
5) Roll back my database updates.

Once these changes are committed then we will let everyone know and try to recover the test network. 

[wackou]

started having problems, so I built from master, clean home dir, and now I get this:

Code: [Select]

(wallet closed) >>> about
{
  "bitshares_toolkit_revision": "b6aa9950db53ecc9753f7c4cbf2a00a3b6dff800",
  "bitshares_toolkit_revision_age": "50 minutes ago",
  "fc_revision": "6dbcba505b230ee77a0a7fe8af5f5e43c59b0b75",
  "fc_revision_age": "16 hours ago",
  "compile_date": "compiled on Jul  3 2014 at 14:41:17"
}
(wallet closed) >>> get_info
6 key_not_found_exception: Key Not Found

    {"key":31757}
    th_a  cached_level_map.hpp:51 fetch

    {"key":31757}
    th_a  cached_level_map.hpp:52 fetch

    {"block_num":31757}
    th_a  chain_database.cpp:1116 get_block_id

    {"block_num":31757}
    th_a  chain_database.cpp:1101 get_block_header

    {}
    th_a  common_api_client.cpp:1971 get_info

    {"command":"get_info"}
    th_a  cli.cpp:538 execute_command

[xeroc]

Bam .. How could I miss that

BTW. for stronger connectivity use:

network_set_advanced_node_parameters {"desired_number_of_connections":40 "maximum_number_of_connections":64}

[liondani]

Take on of these .. or all of them

network_add_node "54.187.102.138:49216" add
network_add_node "191.235.210.4:1024" add
network_add_node "50.174.230.86:8701" add
network_add_node "107.170.172.14:8701" add
network_add_node "162.243.219.145:8701" add
network_add_node "50.89.106.103:8701" add
network_add_node "95.85.33.16:8764" add

thanks added all nodes

now
7 connections linux
3 connections win7

only 1 added on each...

[xeroc]

Take on of these .. or all of them

network_add_node "54.187.102.138:49216" add
network_add_node "191.235.210.4:1024" add
network_add_node "50.174.230.86:8701" add
network_add_node "107.170.172.14:8701" add
network_add_node "162.243.219.145:8701" add
network_add_node "50.89.106.103:8701" add
network_add_node "95.85.33.16:8764" add

[xeroc]

still have 2 network connections on win7....
6 on linux but (has changed a lot times from 3 to 6  connections)

network_set_advanced_node_parameters {"desired_number_of_connections":40 "maximum_number_of_connections":64}

[liondani]

still have 2 network connections on win7....
6 on linux but (has changed a lot times from 3 to 6  connections)

[sumantso]

Seems like the last tx was in block 7124 .. no tx pending any longer

no idea what happend to all the other delegates though

I don't have any connection anymore even after adding the nodes you provided.

[xeroc]

Seems like the last tx was in block 7124 .. no tx pending any longer

no idea what happend to all the other delegates though

[liondani]

is anyone else having trouble getting connections on the windows gui? i'm only getting 3

I have no connections on windows!
and 5 on linux now...(I had 8-12 last days)

I checked now and my linux delegate "liondani-linux"
missed over night-morning (04.00-10.00 eet Greece) 7 blocks
so I assume even on linux it was due all connections was lost for some hours....

[xeroc]

> blockchain_get_pending_transactions

gives an 'aweful' long list of transactions!

[Webber]

I‘m so sorry everyone. .I made 24261  transactions in 18 minutes aboout one hour ago.And I think bts is not work now.
Flooding attack is so cheap,We need to find some way to avoid.

You are wrong .. network still active ..

default (unlocked) >>> getinfo
{
  "blockchain_head_block_num": 7085,
  "blockchain_head_block_age": "80 seconds old",
  "blockchain_head_block_timestamp": "20140703T095015",
  "blockchain_average_delegate_participation": 41.978387364921034,
  "blockchain_blocks_left_in_round": 86,
  "blockchain_confirmation_requirement": 303,
  "blockchain_share_supply": 199999392383082,
  "blockchain_random_seed": "67de4b090798dd86f594bf236bd637a9a5a615b6",
  "blockchain_database_version": 109,
  "blockchain_version": 104,
  "network_num_connections": 13,
  "network_num_connections_max": 64,
  "network_protocol_version": 104,
  "ntp_time": "20140703T095135",
  "ntp_error": -0.0036280000000000001,
  "wallet_open": true,
  "wallet_unlocked": true,
  "wallet_unlocked_until": "48 years  ago",
  "wallet_unlocked_until_timestamp": "19660102T055113",
  "wallet_block_production_enabled": true,
  "wallet_next_block_production_time": "55 seconds in the future",
  "wallet_next_block_production_timestamp": "20140703T095230",
  "wallet_version": 101
}

Maybe some delegates just screwed ...

maybe,but must be something wrong.

[xeroc]

I‘m so sorry everyone. .I made 24261  transactions in 18 minutes aboout one hour ago.And I think bts is not work now.
Flooding attack is so cheap,We need to find some way to avoid.

You are wrong .. network still active ..

default (unlocked) >>> getinfo
{
  "blockchain_head_block_num": 7085,
  "blockchain_head_block_age": "80 seconds old",
  "blockchain_head_block_timestamp": "20140703T095015",
  "blockchain_average_delegate_participation": 41.978387364921034,
  "blockchain_blocks_left_in_round": 86,
  "blockchain_confirmation_requirement": 303,
  "blockchain_share_supply": 199999392383082,
  "blockchain_random_seed": "67de4b090798dd86f594bf236bd637a9a5a615b6",
  "blockchain_database_version": 109,
  "blockchain_version": 104,
  "network_num_connections": 13,
  "network_num_connections_max": 64,
  "network_protocol_version": 104,
  "ntp_time": "20140703T095135",
  "ntp_error": -0.0036280000000000001,
  "wallet_open": true,
  "wallet_unlocked": true,
  "wallet_unlocked_until": "48 years  ago",
  "wallet_unlocked_until_timestamp": "19660102T055113",
  "wallet_block_production_enabled": true,
  "wallet_next_block_production_time": "55 seconds in the future",
  "wallet_next_block_production_timestamp": "20140703T095230",
  "wallet_version": 101
}

Maybe some delegates just screwed ...

[Webber]

I‘m so sorry everyone. .I made 24261  transactions in 18 minutes aboout one hour ago.And I think bts is not work now.
Flooding attack is so cheap,We need to find some way to avoid.

Code: [Select]

mywallet (unlocked) >>> wallet_transfer 0.1 XTS webber-test1 webber-test5 test24261
wallet_transfer 0.1 XTS webber-test1 webber-test5 test24295
wallet_transfer 0.1 XTS webber-test1 webber-test5 test24296
wallet_transfer 0.1 XTS webber-test1 webber-test5 test24297
wallet_transfer 0.1 XTS    BLK.TRX  TIMESTAMP           FROM                TO                  AMOUNT              FEE                 MEMO                                   ID     
----------------------------------------------------------------------------------------------------------------------------------------------------------------
   PENDING   2014-07-03T08:45:39 webber-test1        webber-test5        0.10000 XTS         0.10000 XTS         test24261                              9372bbcc
 webber-test1 webber-test5 test24298
wallet_transfer 0.1 XTS webber-test1 webber-test5 test24299
--- there are now 7 active connections to the p2p network
--- there are now 6 active connections to the p2p network
--- there are now 5 active connections to the p2p network
--- there are now 4 active connections to the p2p network
--- there are now 3 active connections to the p2p network
--- there are now 2 active connections to the p2p network
--- there are now 3 active connections to the p2p network
--- syncing with p2p network, our last block was created 36 hours ago
--- syncing with p2p network, 1411 blocks left to fetch
--- syncing with p2p network, our last block was created 36 hours ago
--- syncing with p2p network, our last block was created 35 hours ago
--- syncing with p2p network, our last block was created 35 hours ago
Peer 188.138.107.159:8701 disconnected us: You offered us a block that we reject as invalid
--- syncing with p2p network, our last block was created 35 hours ago
--- there are now 2 active connections to the p2p network
--- there are now 3 active connections to the p2p network
--- syncing with p2p network, 3131 blocks left to fetch
--- there are now 2 active connections to the p2p network
--- there are now 1 active connections to the p2p network
--- there are now 2 active connections to the p2p network
--- syncing with p2p network, 2434 blocks left to fetch
Peer 188.138.107.159:8701 disconnected us: You offered us a block that we reject as invalid
--- there are now 1 active connections to the p2p network
mywallet (unlocked) >>>

[xeroc]

network_add_node "54.187.102.138:49216" add
network_add_node "191.235.210.4:1024" add
network_add_node "50.174.230.86:8701" add
network_add_node "107.170.172.14:8701" add
network_add_node "162.243.219.145:8701" add
network_add_node "50.89.106.103:8701" add
network_add_node "95.85.33.16:8764" add