Test Net for Advanced Users

[puppies]

http://stats.bitshares.eu/

that is cool

Just installed an upgrade to this ..
sources are here:
https://github.com/BitSharesEurope/stats.bitshares.eu

definitely more to come once I figured out how trading ops look like

Very nice Xeroc.  It seems much more stable.  It still locks up pretty bad when I am flooding the network, but recovers nicely. 

[xeroc]

http://stats.bitshares.eu/

that is cool

Just installed an upgrade to this ..
sources are here:
https://github.com/BitSharesEurope/stats.bitshares.eu

definitely more to come once I figured out how trading ops look like

[sudo]

http://stats.bitshares.eu/

that is cool

[bytemaster]

Interesting.  I am just running top through ssh from my VPS, and when I either manually spam transactions or do a flood_network command I see a dramatic increase in the CPU usage of witness_node.  Up to 100% (its a dual core VPS, and I think top just references the core its running on)  Would there be a better way to test it locally, or would it be better to find a way to sync better between multiple users?  IRC, or Mumble?

Whenever I tried to flood_network, I met "Segmentation fault (core dumped)" error. How can you do this command?

Ben encountered this yesterday and has a fix for the wallet.

[bytemaster]

Good job everyone.  I will look into these issues.

[abit]

One of my node got out of sync when you're flooding the network. An assertion failure which is never seen before.
Here is some related log:

Code: [Select]

2015-09-03T00:30:20 th_a:invoke handle_block         handle_block ] Got block #34498 from network application.cpp:356
...
...
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_block         handle_block ] Got block #34500 from network application.cpp:356
2015-09-03T00:30:30 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:35 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:35 th_a:invoke handle_block         handle_block ] Got block #34501 from network application.cpp:356
2015-09-03T00:30:35 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Got block #34499 from network application.cpp:3
...
...
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Error when pushing block:
10 assert_exception: Assert Exception
(skip & skip_transaction_dupe_check) || trx_idx.indices().get<by_trx_id>().find(trx_id) == trx_idx.indices().get<by_trx_id>().end():
    {}
    th_a  db_block.cpp:487 _apply_transaction

    {"trx":{"ref_block_num":34497,"ref_block_prefix":2564640669,"expiration":"2015-09-03T00:31:49","operations":[[0,{"fee":{"amount":2000000,"asset_id":"1.3.0"},"from":"1.2.63354","to":"1.2.63355","amount":{"amount":100000,"asset_id":"1.3.0"},"extensions":[]}]],"extensions":[],"signatures":["1f5f41592441c4f40af47b118e80a0c4786a1d768cf6a8d3204d7c30d79d33e971463c4dc78f1c5fc8f2ae7637ec21e8b523b089baef754422fe99108576b343c8"]}}
    th_a  db_block.cpp:543 _apply_transaction

    {"next_block.block_num()":34499}
    th_a  db_block.cpp:448 _apply_block
...
...
2015-09-03T00:30:37 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:40 th_a:invoke handle_block         handle_block ] Got block #34502 from network application.cpp:356
2015-09-03T00:30:40 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57

Full log files are here:
https://drive.google.com/open?id=0B3xrm70jSHn4U3h3V3liOUlOakk
https://drive.google.com/open?id=0B3xrm70jSHn4TmtNYWNaVm1yalU
https://drive.google.com/open?id=0B3xrm70jSHn4SURQZjBZaE1rVGc

The other node works fine.

Strange. Restarting doesn't help. Will try resync.

//Edit: resync worked. Will post here if I found something interesting in the log files.

//Edit2: here is related logs which possibly explains why unable to sync after restarted.
* The head_block of the chain was block 34498, id = 000086c202f51da427ebf0507404bc9b6d192f96
* Witness failed to push block 34499 last time, id = 000086c3bf20d2e5eeb1039f38c221b46880f2f0
* After restarted, the witness ignores block 34499 (id 000086c3bf20d2e5eeb1039f38c221b46880f2f0) because it's considered "seen"
* Following blocks are unable to push to chain because they're not linked (because 34499 is not on chain)

default.log:

Code: [Select]

2015-09-03T09:46:28        th_a:?unnamed?                 main ] Started witness node on a chain with 34498 blocks. main.cpp:165
...
2015-09-03T09:48:39 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57

p2p.log:

Code: [Select]

2015-09-03T09:46:30 p2p:message read_loop fetch_next_batch_of_ ] sync: sending a request for the next items after 000086c202f51da427ebf0507404bc9b6d192f96 to peer 176.9.234.167:40936, (full request is ["000006c280cdf25ca122561981716c74045f16fe","000046c2482fc56dda591b0bea7a47ccbd64d6b2","000066c27fc3abd468f7dd5ad1ce8939614d9d64","000076c2a8ad357d929c0967d98ba955521c11b4","00007ec23da7bcf6188fa3e6e57f0216e78cba1a","000082c22a40c3cbc1152bbd18239e9ffca051dc","000084c22add8beb2d06e3adb13c3573a56ae0b4","000085c29de6e5d439166100ab9d809c730e006c","00008642bd45a1a5711a74bacf4e73cfd5ae69cd","00008682ea2e75d79000e7a4c3411368d7a0a74a","000086a273f0cd7e751e91f6b0795486303009a2","000086b28863612b25b49a1c0a95a3525e42529c","000086ba6b31127d85ba6e1a8b5ea1145a0766e0","000086be74b88f7adb24796112e15c72b04a4e59","000086c0a1c19fb2c9b2d8d92f1caf812fda4bde","000086c19d4fdd989e8e7a62830c6f58e73f33be","000086c202f51da427ebf0507404bc9b6d192f96"]) node.cpp:2294
...
...
2015-09-03T09:46:30 p2p:message read_loop on_blockchain_item_i ] sync: received a list of 669 available items from 192.241.198.6:39809 node.cpp:2310
2015-09-03T09:46:30 p2p:message read_loop on_blockchain_item_i ] is_first_item_for_other_peer: false.  item_hashes_received.size() = 669 node.cpp:2356
2015-09-03T09:46:30 p2p:message read_loop on_blockchain_item_i ] popping item because delegate has already seen it.  peer's last block the delegate has seen is now 000086c202f51da427ebf0507404bc9b6d192f96 (1) node.cpp:2368
2015-09-03T09:46:30 p2p:message read_loop on_blockchain_item_i ] popping item because delegate has already seen it.  peer's last block the delegate has seen is now 000086c3bf20d2e5eeb1039f38c221b46880f2f0 (2) node.cpp:2368
2015-09-03T09:46:30 p2p:message read_loop on_blockchain_item_i ] after removing all items we have already seen, item_hashes_received.size() = 667 node.cpp:2371
2015-09-03T09:46:30 p2p:message read_loop trigger_fetch_sync_i ] Triggering fetch sync items loop now node.cpp:1084
2015-09-03T09:46:30 p2p:fetch_sync_items_loop fetch_sync_items_loo ] beginning another iteration of the sync items loop node.cpp:1023
2015-09-03T09:46:30 p2p:fetch_sync_items_loop request_sync_items_f ] requesting 100 item(s) ["000086c4e3da8babd770344cb8db18ee89a9d09a","000086c50ebdc7f4b5ff303fee02b53f3aec5773",...,"00008727dc9a24b61d0d51454278f2282e89672d"] from peer 192.241.198.6:39809 node.cpp:1007
...
...
2015-09-03T09:48:39 p2p:send_sync_block_to_node_delegate send_sync_block_to_n ] Successfully pushed sync block 34500 (id:000086c4e3da8babd770344cb8db18ee89a9d09a) node.cpp:2793
2015-09-03T09:48:39 p2p:send_sync_block_to_node_delegate send_sync_block_to_n ] sync: client accpted the block, we now have only 5937 items left to fetch before we're in sync node.cpp:2831

Full default.log: https://drive.google.com/open?id=0B3xrm70jSHn4UWJ2N2QzdkVOX2M
Full p2p.log: https://drive.google.com/open?id=0B3xrm70jSHn4QVNoanZfMS13UXM

[abit]

One of my node got out of sync when you're flooding the network. An assertion failure which is never seen before.
Here is some related log:

Code: [Select]

2015-09-03T00:30:20 th_a:invoke handle_block         handle_block ] Got block #34498 from network application.cpp:356
...
...
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_block         handle_block ] Got block #34500 from network application.cpp:356
2015-09-03T00:30:30 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:35 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:35 th_a:invoke handle_block         handle_block ] Got block #34501 from network application.cpp:356
2015-09-03T00:30:35 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Got block #34499 from network application.cpp:3
...
...
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Error when pushing block:
10 assert_exception: Assert Exception
(skip & skip_transaction_dupe_check) || trx_idx.indices().get<by_trx_id>().find(trx_id) == trx_idx.indices().get<by_trx_id>().end():
    {}
    th_a  db_block.cpp:487 _apply_transaction

    {"trx":{"ref_block_num":34497,"ref_block_prefix":2564640669,"expiration":"2015-09-03T00:31:49","operations":[[0,{"fee":{"amount":2000000,"asset_id":"1.3.0"},"from":"1.2.63354","to":"1.2.63355","amount":{"amount":100000,"asset_id":"1.3.0"},"extensions":[]}]],"extensions":[],"signatures":["1f5f41592441c4f40af47b118e80a0c4786a1d768cf6a8d3204d7c30d79d33e971463c4dc78f1c5fc8f2ae7637ec21e8b523b089baef754422fe99108576b343c8"]}}
    th_a  db_block.cpp:543 _apply_transaction

    {"next_block.block_num()":34499}
    th_a  db_block.cpp:448 _apply_block
...
...
2015-09-03T00:30:37 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:40 th_a:invoke handle_block         handle_block ] Got block #34502 from network application.cpp:356
2015-09-03T00:30:40 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57

Full log files are here:
https://drive.google.com/open?id=0B3xrm70jSHn4U3h3V3liOUlOakk
https://drive.google.com/open?id=0B3xrm70jSHn4TmtNYWNaVm1yalU
https://drive.google.com/open?id=0B3xrm70jSHn4SURQZjBZaE1rVGc

The other node works fine.

Do you have enough disk space now?

Yes I have a 500G disk mounted, now used 110G.

[clayop]

One of my node got out of sync when you're flooding the network. An assertion failure which is never seen before.
Here is some related log:

Code: [Select]

2015-09-03T00:30:20 th_a:invoke handle_block         handle_block ] Got block #34498 from network application.cpp:356
...
...
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_block         handle_block ] Got block #34500 from network application.cpp:356
2015-09-03T00:30:30 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:35 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:35 th_a:invoke handle_block         handle_block ] Got block #34501 from network application.cpp:356
2015-09-03T00:30:35 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Got block #34499 from network application.cpp:3
...
...
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Error when pushing block:
10 assert_exception: Assert Exception
(skip & skip_transaction_dupe_check) || trx_idx.indices().get<by_trx_id>().find(trx_id) == trx_idx.indices().get<by_trx_id>().end():
    {}
    th_a  db_block.cpp:487 _apply_transaction

    {"trx":{"ref_block_num":34497,"ref_block_prefix":2564640669,"expiration":"2015-09-03T00:31:49","operations":[[0,{"fee":{"amount":2000000,"asset_id":"1.3.0"},"from":"1.2.63354","to":"1.2.63355","amount":{"amount":100000,"asset_id":"1.3.0"},"extensions":[]}]],"extensions":[],"signatures":["1f5f41592441c4f40af47b118e80a0c4786a1d768cf6a8d3204d7c30d79d33e971463c4dc78f1c5fc8f2ae7637ec21e8b523b089baef754422fe99108576b343c8"]}}
    th_a  db_block.cpp:543 _apply_transaction

    {"next_block.block_num()":34499}
    th_a  db_block.cpp:448 _apply_block
...
...
2015-09-03T00:30:37 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:40 th_a:invoke handle_block         handle_block ] Got block #34502 from network application.cpp:356
2015-09-03T00:30:40 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57

Full log files are here:
https://drive.google.com/open?id=0B3xrm70jSHn4U3h3V3liOUlOakk
https://drive.google.com/open?id=0B3xrm70jSHn4TmtNYWNaVm1yalU

The other node works fine.

Do you have enough disk space now?

[clayop]

I have three things to report

1. CPU usage goes up to 45% from 7:22 AM UTC (without any transactions?)
2. Log files are so gigantic. I have to delete them before they eat up my VPS disk space (around 8 G already)
3. Resync is really fast. It takes less than 5 min to catch up 40000 blocks from scratch.

[abit]

One of my node got out of sync when you're flooding the network. An assertion failure which is never seen before.
Here is some related log:

Code: [Select]

2015-09-03T00:30:20 th_a:invoke handle_block         handle_block ] Got block #34498 from network application.cpp:356
...
...
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:30 th_a:invoke handle_block         handle_block ] Got block #34500 from network application.cpp:356
2015-09-03T00:30:30 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:35 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:35 th_a:invoke handle_block         handle_block ] Got block #34501 from network application.cpp:356
2015-09-03T00:30:35 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Got block #34499 from network application.cpp:3
...
...
2015-09-03T00:30:37 th_a:invoke handle_block         handle_block ] Error when pushing block:
10 assert_exception: Assert Exception
(skip & skip_transaction_dupe_check) || trx_idx.indices().get<by_trx_id>().find(trx_id) == trx_idx.indices().get<by_trx_id>().end():
    {}
    th_a  db_block.cpp:487 _apply_transaction

    {"trx":{"ref_block_num":34497,"ref_block_prefix":2564640669,"expiration":"2015-09-03T00:31:49","operations":[[0,{"fee":{"amount":2000000,"asset_id":"1.3.0"},"from":"1.2.63354","to":"1.2.63355","amount":{"amount":100000,"asset_id":"1.3.0"},"extensions":[]}]],"extensions":[],"signatures":["1f5f41592441c4f40af47b118e80a0c4786a1d768cf6a8d3204d7c30d79d33e971463c4dc78f1c5fc8f2ae7637ec21e8b523b089baef754422fe99108576b343c8"]}}
    th_a  db_block.cpp:543 _apply_transaction

    {"next_block.block_num()":34499}
    th_a  db_block.cpp:448 _apply_block
...
...
2015-09-03T00:30:37 th_a:invoke handle_transaction   handle_transaction ] Got transaction from network application.cpp:391
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:37 th_a:invoke handle_transaction   on_objects_changed ] ids: ["2.5.40","2.5.18","2.6.63354","2.7.4698"] api.cpp:867
2015-09-03T00:30:40 th_a:invoke handle_block         handle_block ] Got block #34502 from network application.cpp:356
2015-09-03T00:30:40 th_a:invoke handle_block           push_block ] Pushing block to fork database that failed to link. fork_database.cpp:57

Full log files are here:
https://drive.google.com/open?id=0B3xrm70jSHn4U3h3V3liOUlOakk
https://drive.google.com/open?id=0B3xrm70jSHn4TmtNYWNaVm1yalU
https://drive.google.com/open?id=0B3xrm70jSHn4SURQZjBZaE1rVGc

The other node works fine.

//Edit: updated link of log files.

[clayop]

Yes you're correct. But my VPS's cpu spikes a little bit (from 3% to 30%) when I was spamming from another box.

Edit: I have to repeat my statement. There's no spike.

My mistake. I was out of active witnesses. With puppies network flooding (from puppies to clayop account... thanks ), my CPU usage went up to 35%

[puppies]

So.  I set  up a keyboard script to send lots of transactions.  I hope clayop is a good account for you.

[clayop]

Interesting.  I am just running top through ssh from my VPS, and when I either manually spam transactions or do a flood_network command I see a dramatic increase in the CPU usage of witness_node.  Up to 100% (its a dual core VPS, and I think top just references the core its running on)  Would there be a better way to test it locally, or would it be better to find a way to sync better between multiple users?  IRC, or Mumble?

Whenever I tried to flood_network, I met "Segmentation fault (core dumped)" error. How can you do this command?

I am using it as

Code: [Select]

unlocked >>> flood_network test34.puppies 200
flood_network test34.puppies 200
3593111ms th_a       wallet.cpp:1936               flood_network        ] Created 66 accounts in 671 milliseconds
3594438ms th_a       wallet.cpp:1946               flood_network        ] Transferred to 132 accounts in 1327 milliseconds
3595016ms th_a       wallet.cpp:1955               flood_network        ] Issued to 66 accounts in 577 milliseconds
null
unlocked >>>
test34.puppies is not a real account, but it seems to error out a lot of you use it with the same name.  I have never had it cause a segmentation fault though.  The errors are very very long, but not serious enough to kill the client.

I used any new name but all gave me the same error. Maybe a bug, but it does not have to be fixed IMO (Will someone use flood_network in real blockchain? )

[puppies]

Sent some CORE to puppies

thanks clayop

I can't look far enough back to see your tranfer.  What is a good account to spam tranfers too?

[puppies]

Interesting.  I am just running top through ssh from my VPS, and when I either manually spam transactions or do a flood_network command I see a dramatic increase in the CPU usage of witness_node.  Up to 100% (its a dual core VPS, and I think top just references the core its running on)  Would there be a better way to test it locally, or would it be better to find a way to sync better between multiple users?  IRC, or Mumble?

Whenever I tried to flood_network, I met "Segmentation fault (core dumped)" error. How can you do this command?

I am using it as

Code: [Select]

unlocked >>> flood_network test34.puppies 200
flood_network test34.puppies 200
3593111ms th_a       wallet.cpp:1936               flood_network        ] Created 66 accounts in 671 milliseconds
3594438ms th_a       wallet.cpp:1946               flood_network        ] Transferred to 132 accounts in 1327 milliseconds
3595016ms th_a       wallet.cpp:1955               flood_network        ] Issued to 66 accounts in 577 milliseconds
null
unlocked >>>
test34.puppies is not a real account, but it seems to error out a lot of you use it with the same name.  I have never had it cause a segmentation fault though.  The errors are very very long, but not serious enough to kill the client.