[Testnet-StressTest] March 15th - 3:00pm UTC

[xeroc]

@xeroc The conclusion of the paper shows some room for improvement within the network and validator code.  Is this something that can be done with a worker proposal?  EOS appears a ways off so why not improve what we have at the current time?

If you can find a developer that wants to get his hands dirty ... funding shouldn't be the problem anymore!

[Brekyrself]

@xeroc The conclusion of the paper shows some room for improvement within the network and validator code.  Is this something that can be done with a worker proposal?  EOS appears a ways off so why not improve what we have at the current time?

[Louis]

Thanks Xeroc

Is there a new stress test coming soon?

[xeroc]

The paper that shows some of the end results of the first stress test is available here:
http://docs.bitshares.eu/bitshares/papers/index.html

[R]

I spoke during the 14th Bitshares hangout regarding the potential Bitshares BOINC project for stress-testing, rewarding full/api nodes, testnet experiments, etc.

Unnofficial recording: https://steemit.com/beyondbitcoin/@glitterfart/bitshares-hangout-2017-03-30-beyond-bitcoin-radioshow-raw-recording-for-impatients

[Thom]

@crypto123 thanks for the clarification. I especially like the last part of your idea that provides a way to measure testnet participation.

Sounds like you have the knowledge to put something like that together. If so I encourage you to do so. I've been looking at docker myself, and think that would indeed be a good way to package "an executable unit" to simplify distribution and insure each unit is consistent and behaves the same.

[R]

Thoughts on utilizing a BOINC project to distribute the load generating software required to push the tx/s to new heights? https://steemit.com/beyondbitcoin/@cm-steem/brainstorming-boinc-projects-006#@cm-steem/re-cm-steem-brainstorming-boinc-projects-006-20170326t013749374z

Mentioned the idea on beyondbitcoin #200 too: https://steemit.com/beyondbitcoin/@glitterfart/beyond-bitcoin-radioshow-2017-03-24-gridcoin-raw-recording-for-impatients

What I took away from that conversation is that BOINC would not be suitable for the BitShares testnet to spam the network for testing b/c BOINC projects are geared towards problems where longer term application of CPU resources is required, as opposed to the burst requirements of the testnet. Testing on the testnet is not a continuous, ongoing operation.

I'm not very knowledgeable about BOINC, just commenting on what I heard in the mumble last Friday.

What I meant was that a Bitshares testnet stress-test BOINC project would be unsuitable for Gridcoin whitelist status as the Gridcoin network focuses on projects with continuous work unit distribution, and the nature of stress-tests is bursty & far apart.

Such a BOINC project is entirely possible, you can distribute any kind of distributed computing application. I'd imagine that for this idea, you would distribute a docker container which contained python dependencies, the testnet stress-test python script, and the bitshares client.

Once running on an user's machine, it would need to create a testnet user account, then the project admin would need to send enough testnet BTS to each participating account, then the main stress-test script would begin bombarding testnet with transactions, so it's not as simple as just distributing the bitshares client and the python script.

With regards to rewarding participating users, you could add an BTS account field within the BOINC profile or within the project registration page, to which you could take into account the credit achieved by participating users during the stress-test phases & distribute an UIA proportionally for their completed work.

[Thom]

Thoughts on utilizing a BOINC project to distribute the load generating software required to push the tx/s to new heights? https://steemit.com/beyondbitcoin/@cm-steem/brainstorming-boinc-projects-006#@cm-steem/re-cm-steem-brainstorming-boinc-projects-006-20170326t013749374z

Mentioned the idea on beyondbitcoin #200 too: https://steemit.com/beyondbitcoin/@glitterfart/beyond-bitcoin-radioshow-2017-03-24-gridcoin-raw-recording-for-impatients

What I took away from that conversation is that BOINC would not be suitable for the BitShares testnet to spam the network for testing b/c BOINC projects are geared towards problems where longer term application of CPU resources is required, as opposed to the burst requirements of the testnet. Testing on the testnet is not a continuous, ongoing operation.

I'm not very knowledgeable about BOINC, just commenting on what I heard in the mumble last Friday.

[R]

Thoughts on utilizing a BOINC project to distribute the load generating software required to push the tx/s to new heights? https://steemit.com/beyondbitcoin/@cm-steem/brainstorming-boinc-projects-006#@cm-steem/re-cm-steem-brainstorming-boinc-projects-006-20170326t013749374z

Mentioned the idea on beyondbitcoin #200 too: https://steemit.com/beyondbitcoin/@glitterfart/beyond-bitcoin-radioshow-2017-03-24-gridcoin-raw-recording-for-impatients

[pc]

Excellent results!

Good job guys!

[Chris4210]

Results of the BitShares Munich Testnet: https://steemit.com/bitshares/@Chris4210/1h30-min-bitshares-stress-test-of-the-15-march-2017-3300txs-14000ops

Other big thanks to all participants and especially for Xeroc and his countless hours to get a great stats page up and moderating the stress test.

I am looking forward to the next stress test soon.

[Permie]

I'd like to thank BTS-Munich for providing some funding to obtain a think machine for the

Thank you BTS Munich and Xeroc, for the benefit of all shareholders

[abit]

@xeroc IIUC only one node will be producing blocks, with all witnesses on it?

I don't quite understand your statement ... the testnet is distributed among 15 witnesses which are placed from South Korea, over Germany and Paris, to AWS in th U.S.

Got it. Thanks.

Update:
test report with video: https://steemit.com/bitshares/@chris4210/1h30-min-bitshares-stress-test-of-the-15-march-2017-3300txs-14000ops

[xeroc]

@xeroc IIUC only one node will be producing blocks, with all witnesses on it?

I don't quite understand your statement ... the testnet is distributed among 15 witnesses which are placed from South Korea, over Germany and Paris, to AWS in th U.S.

[instructor2121]

@Chris Thanks for the tutorial link.

[abit]

@xeroc IIUC only one node will be producing blocks, with all witnesses on it?

[fractalnode]

done!
what's next?

btw.
I changed the logging level from debug to info
debug gives 1GB per 1h

Code: [Select]

Azure DS12 v2
E5-2673 v3 @ 2.40GHz,
28 GB Ram

Hosted by NFOrce Entertainment B.V. (Amsterdam) [2.18 km]: 5.332 ms
Testing download speed........................................
Download: 2126.96 Mbit/s
Testing upload speed..................................................
Upload: 445.69 Mbit/s

Code: [Select]

unlocked >>> info
info
{
  "head_block_num": 6856241,
  "head_block_id": "00689e31e66b5eb24ba8c4ddc1dbc065f4bfc7e0",
  "head_block_age": "1 second old",
  "next_maintenance_time": "2 minutes in the future",
  "chain_id": "39f5e2ede1f8bc1a3a54a7914414e3779e33193f1f5693510e73cb7a87617447",
  "participation": "100.00000000000000000",
  "active_witnesses": [
    "1.6.1",
    "1.6.3",
    "1.6.20",
    "1.6.21",
    "1.6.22",
    "1.6.23",
    "1.6.24",
    "1.6.25",
    "1.6.27",
    "1.6.28",
    "1.6.29",
    "1.6.30",
    "1.6.31",
    "1.6.32",
    "1.6.33",
    "1.6.34",
    "1.6.36"
  ],
  "active_committee_members": [
    "1.5.11",
    "1.5.12",
    "1.5.13",
    "1.5.14",
    "1.5.15",
    "1.5.16",
    "1.5.17",
    "1.5.18",
    "1.5.19",
    "1.5.20",
    "1.5.2"
  ]
}
unlocked >>>


I look forward to further guidance

[Fox]

but I still have a few questions to this list from point 12 ii and above
https://github.com/ryanRfox/BitSharesTESTNET

Use the WIF private key you copied earlier from the ~/brain_key.json file unlocked >>> import_key 5yourPrivateKeyStartsWith5...

>>> import_key accoutname 5KgFzmuA8FXMgFzmuA8FgFzmuA8FgFzmuA8F true
point 14
but I don't have any file here :

tail: cannot open ~/testnet/cli_wallet/wallet.json for reading: No such file or directory

Sorry all, I had poor syntax in my README file. There are three (3) parameters for import_key

Code: [Select]

import_key <account_name> <WIF_private_key> <bool>There for it should look something like:

Code: [Select]

unlocked >>> import_key fox 5ALLprivateKeysStartsWith5andAbunchOfChars true
Also, I neglected to create the /home/$USER_NAME/testnet/cli_wallet folder prior to launching the cli_wallet, so it was unable to save the wallet file. I have now added that to the script so it 'should' function properly. Also, I added the web socket RPC flag to the wallet startup. The relevant code changes are here:

Code: [Select]

##################################################################################################
# Create a script to launch the cli_wallet using a wallet file stored at                         #
# /home/$USER_NAME/$PROJECT/cli_wallet/wallet.json                                               #
##################################################################################################
mkdir /home/$USER_NAME/$PROJECT/cli_wallet/
cat >/home/$USER_NAME/launch-$PROJECT-wallet.sh <<EOL
/usr/bin/$CLI_WALLET -w /home/$USER_NAME/$PROJECT/cli_wallet/wallet.json \
                     -s ws://$LOCAL_IP:$RPC_PORT \
-H 127.0.0.1:$WALLET_HTTP_RPC_PORT \
-r 127.0.0.1:$WALLET_WS_RPC_PORT
EOL
chmod +x /home/$USER_NAME/launch-$PROJECT-wallet.sh


[fractalnode]

it seems like this link is broken:
https://t.me/joinchat/AAAAAA0N-zjsAdcLnUsL6A

I've created and synchronize my new Azure DS12
but I still have a few questions to this list from point 12 ii and above
https://github.com/ryanRfox/BitSharesTESTNET

Use the WIF private key you copied earlier from the ~/brain_key.json file unlocked >>> import_key 5yourPrivateKeyStartsWith5...

>>> import_key accoutname 5KgFzmuA8FXMgFzmuA8FgFzmuA8FgFzmuA8F
true

whether this account must contain TEST tokens?

poin 14
but I don't have any file here :

tail: cannot open ~/testnet/cli_wallet/wallet.json for reading: No such file or directory

[Thom]

A 32GB, 8 CPU core node on vultr.com is priced at $0.238 / hour.

The are two 32GB options on bithost, one designated "(high memory)", not sure what that means:

1) 32GB
    Monthly Price    $448.00
    Hourly Price    $0.6667

2) 32GB (high memory):
    Monthly Price    $336.00
    Hourly Price    $0.5

One of these (I believe it's Vultr) is a Digital Ocean reseller. Both can take Bitcoin as payment.

I will be researching other VPS hosting options this weekend, as well as deployment options.

[Chris4210]

Here is a good tutorial how to set up a test net node thanks to Fox: https://github.com/ryanRfox/BitSharesTESTNET

Good work.

[xeroc]

Some more pictures made during my personal testing:


I'd like to thank BTS-Munich for providing some funding to obtain a think machine for the public API node.
You can find more data about the machine (also during stress testing) on
http://stats.bitshares.eu/netdata/

[xeroc]

I am currently pushing some load on the testnet to see if the frontend and stuff works as desired.

http://stats.bitshares.eu/#test




Looks promising

[xeroc]

Hey there,
I finally picked a date for the testnet stress test:

       15th March beginning around 3pm UTC.

Organization
I would like to invite everyone (in particular the witness node operators) to join this Telegram channel:
https://t.me/joinchat/AAAAAA0N-zjsAdcLnUsL6A

Real-Time Stats
Turns out the new middleware for the stats seem to work pretty well. You can see the real-time stats during the stresstest on
http://stats.bitshares.eu  (make sure to select the testnet!)
After the stresstest, when the transactions are all in the blokchain, we can do some offline analysis

Spamming
Currently, the cli-wallet is the fastest to produce transactions
Python Script For the CLI wallet

Code: [Select]

import time
import json
from grapheneapi.grapheneapi import GrapheneAPI

blockinterval    = 2
numbertxperblock = 500

if __name__ == '__main__':
    client = GrapheneAPI("localhost", 8090, "", "")
    while True :
        for i in range(0,numbertxperblock) :
            print(i)
            res = client.transfer("faucet","xeroc","0.001", "TEST", "", True);
        time.sleep(blockinterval)

I hope I can finish up the basic uptick client and add a temporary flooding call there by next week, then you won't need the cli-wallet for it.

Public Witness API
Now that I have set a date, I can order a fat machine for the public API node and monitor it's RAM and bandwidth usage.
The URL will still be

  wss://node.testnet.bitshares.eu

and unless you run your own witness node (which would be very welcome), this node should hopefully have enough resources to stay up and running during the stresstest.

Is there anything else we should discuss in the meantime?