Okay, here goes. Please help me flesh this plan out. Or alternately if you think its stupid, you could just tell me that. Although I would appreciate it if you were a little bit polite when you made fun of my plan.
Thanks for an excellent plan. I have a few comments/points I like to add:
1) What are we testing?
For the first phase I propose that we attempt a stress test of the test network WAN. This is a real world test of what we will be able to do when 2.0 launches.
Do you mean LAN (Local Area Network) test to begin with? If LAN test failed to achieve 100K tps then we know WAN test will fail too. Besides, WAN depends on the individual witness node's speed and so there are more variables in the testing.
2) What is our goal?
I propose a goal of 1000tps sustained for 5 minutes.
Let's start with 5mins. And let's see how long the network can hold before it breaks.
3) Why test?
First of all it will be good publicity to show what we can actually do, not just the theoretical limit. Secondly, it will provide lots of information about how the binaries behave on different machines, and how the network deals with that amount of traffic.
Yes, an excellent way to 'show off' graphene's 100K tps real power.
4) What do we need?
a) compiled binaries for windows, osx, and linux
I think for a start, and to make thing easier, choose one - Windows and/or Linux since they are available now.
b) instructions on how to download, intall, and start the witness_node, cli_wallet, and a python script (lets focus on getting everyone to the startup screen on cli_wallet, and take care of account creation, and transaction spamming with a script.
This would be especially useful when we start running tests for WAN.
c) a script that will:
1c) create and unlock a wallet
2c) Import a known test private key to get a balance
3c) create an account and upgrade it to lifetime status
4c) spam transactions
5c) keep an eye on account balance
6c) withdraw vesting funds so testing can continue longer
Having scripts to simulate as close to 'real life' transactions as possible - this list is excellent for that. But since writing scripts take time - a whole lot of time, I suggest we forcus on scripting tasks that most frequently used by users and those txs that take up most resources - cpu/mem/network. We want to see how much stress the network can bear. Import private keys are sensitive and private stuff, and so we may not want to script it. Perhaps simplifying to:
1c) create and unlock a wallet
3c) create an account and upgrade it to lifetime status
4c) transfer transactions
7c) log anything that might be interesting. Both to screen and to file.
Yes, we like to log the good results and show it to the world.
Alternately if we can set up a faucet the script can register the new accounts under a lifetime account, and our excess funds vesting will be in a centralized location.
It would be great if this script could be the interaction point for testers. Asking any required information at the beginning, and displaying any information needed. That would prevent testers from having to learn anything about the witness_node or cli_wallet
d) Lots of people to test. If installing 3 programs, and typing in a few commands sounds like something you can do, then you could help us make history.
I am not sure if it is the right time now to 'teach' users who are not technical since preparing documentation/tutorials/simplying installation will take up lots of time and effort. But if we can accomplish that, it would be great.
If you can help us make this test a reality please let me know. We already have binaries available, but will probably need them updated by the time we are ready to test. I can help with instructions. I could write the script, but it would take me 10 times longer than xeroc, and the final product would be 10 times worse than xerocs product would be. I will also of course help run nodes for the test. Both at home, and on some VPS's.
Generating dummy user accounts and transactions will probably involve interacting with a database. How easy to modify xeroc's scripts for that?
I am considering jmeter. How about adding a custom plug-in to jmeter to perform the necessary rpc callings (these create the commands to the witness_node daemon to perform the transactions)?
Xeroc. What do you think in regards to the script Idea I posted? Would you be able to donate that script to the cause, or would you want to set a price up front?
I would like to hear from xeroc too.
No. To save on fees. The hardest part about testing so far has been getting your hands on enough core. 20bts per transaction adds up fast. If the account is upgraded that is lowered to 4bts per transaction after rebate. Since my goal is 1000tps for 5 minutes, if each node can handle 10tps for 5 minutes that would be a savings of 38k bts after the 10k bts fee to upgrade. I hope that made sense.
If tx-fees is a concern, how about starting a new testnet for the load testing? We can modify the source codes to generate tons and tons of CORE for the tests.
1. Current version of Graphene has sync issues, especially when the network is under pressure (when there is transaction flooding). I think it's better to do this wider test after the sync issues fixed.
2. Current test network looks like stable, because it is highly centralized -- 80% of witnesses are running on one or two VPS(es) which controlled by BM/CryptoNomex, every forks made by other witnesses are simply discarded. In a decentralized network it's harder to decide which fork is effective (or say longest?). We'd better prove that it's stable on a decentralized network first.
BM has fixed the sync issues. For witnesses, we could create a number of them and spread them out to different computers/instances.