Scheduling Proof of Scalability

[liondani]

     3c) create an account and upgrade it to lifetime status

to see if it works?

[puppies]

To summarise:

It needs a test plan (what are we testing, how are we testing it, what is what we want to achieved or why are we testing), environment/s setup, scripts to execute tests, and of course who is going to participate.

Once this is done, we can think what money we need to achieve this.

Okay, here goes.  Please help me flesh this plan out.  Or alternately if you think its stupid, you could just tell me that.  Although I would appreciate it if you were a little bit polite when you made fun of my plan.

1) What are we testing?
     For the first phase I propose that we attempt a stress test of the test network WAN.  This is a real world test of what we will be able to do when 2.0 launches. 
2) What is our goal?
     I propose a goal of 1000tps sustained for 5 minutes. 
3) Why test? 
     First of all it will be good publicity to show what we can actually do, not just the theoretical limit.  Secondly, it will provide lots of information about how the binaries behave on different machines, and how the network deals with that amount of traffic.
4) What do we need?
   a) compiled binaries for windows, osx, and linux
   b) instructions on how to download, intall, and start the witness_node, cli_wallet, and a python script (lets focus on getting everyone to the startup screen on cli_wallet, and take care of account creation, and transaction spamming with a script.
   c) a script that will:
     1c) create and unlock a wallet
     2c) Import a known test private key to get a balance
     3c) create an account and upgrade it to lifetime status
     4c) spam transactions
     5c) keep an eye on account balance
     6c) withdraw vesting funds so testing can continue longer
     7c) log anything that might be interesting.  Both to screen and to file.
Alternately if we can set up a faucet the script can register the new accounts under a lifetime account, and our excess funds vesting will be in a centralized location.
It would be great if this script could be the interaction point for testers.  Asking any required information at the beginning, and displaying any information needed.  That would prevent testers from having to learn anything about the witness_node or cli_wallet
   d) Lots of people to test.  If installing 3 programs, and typing in a few commands sounds like something you can do, then you could help us make history. 

What does everyone think?  Is this a good enough starting point? 

If you can help us make this test a reality please let me know.  We already have binaries available, but will probably need them updated by the time we are ready to test.  I can help with instructions.  I could write the script, but it would take me 10 times longer than xeroc, and the final product would be 10 times worse than xerocs product would be.  I will also of course help run nodes for the test.  Both at home, and on some VPS's. 

Xeroc.  What do you think in regards to the script Idea I posted?  Would you be able to donate that script to the cause, or would you want to set a price up front?

Either way if we end up doing this, please consider tipping xeroc, cube, and maqifrnswa.  Please let me know if I have missed any other tip worthy contributors. 

[Fox]

1. I'm contributing with the following amount of BTS: 5000
2. I'm contributing with the following resources (for testers only):  up to 30 x 2 Core VMs 
3. I'm aiming to contribute following amount of transactions per second (for testers only): 500TPS (in short coordinated bursts)
4. I'm willing to contribute scripts for setting up nodes in Azure.

[Thom]

To summarise:

It needs a test plan (what are we testing, how are we testing it, what is what we want to achieved or why are we testing), environment/s setup, scripts to execute tests, and of course who is going to participate.

Once this is done, we can think what money we need to achieve this.

Sorry guys, trying to catch up after the weekend and personal events of last week. Been reading this thread but don't have a clear picture of what is really going on here.

An answer to the above post would help clear my fog. I'll be staying tuned.

1. I'm contributing with the following amount of BTS: 5000
2. I'm contributing with the following resources (for testers only):  Several VPS instances around the world, if useful for testing
3. I'm aiming to contribute following amount of transactions per second (for testers only): Not sure how to answer this

[puppies]

I thought Dan said they already did a LAN test, and achieved something like 186k tps.  Is there a reason we're trying to do this again? other than capturing it on video.  I would think they would have some scripts for this already also.

the WAN test might be more interesting. 

It sounds like there are still a great deal of optimizations that can be implemented.  Is now the best time to spend money on powerful/connected servers, only to do it again later with better optimizations?

Why can we not just use the existing test network in a organized fashion to capture this data. It may not produce the ultimate upper bounds but should still reveal bottlenecks and have something meaningful for video or whatever media angle you're interested in.    Im not fully convinced is best to spend a bunch of money at this time. 

I'm willing to donate some personal funds to aid in this:
1. I'm contributing with the following amount of BTS: 2500
2.  N/A
3.  N/A     

but for any significant amount from BitsharesBreakout, at this time I'd like to hold off for a moment to get a better feel for the value derived and necessity for doing this now, and in the way described.  Get some feed back from devs etc as to whether this is valuable for their work.

I have to agree with this, we are testing for testing. Why focus in an scenario that we can build towards and not focus on the current world. Our testnet is a good way to prove what we can do now, with the current vps / servers we currently need.

Probably best is to organise the testnet in a better way. If we are using Azure / AWS we can ensure we are all in the same region and using better connectivity. If you are using a home server you are still good to go.

We need to ensure that we have enough funds to test . Most of us have used puppies /clayop autohotkey script  or xerocs (I used xerocs as I was using putty) and quickly ran out. If xerocs script is modified so we distribute funds across all named participants, we might not ran out of CORE. Also ensure that we can quickly restart our witness if down.

Maybe scripts should run a counter to verify all the transactions are sent / received.

Once all this is organised, then we can experiment on scaling vertically and horizontally. More transactions, more clients / users to count votes, bigger vms.

We can now withdraw the extra fees collected from lifetime members, so we can reduce the effective fees down to 4 bts per transaction.  https://bitsharestalk.org/index.php/topic,17962.msg234909.html#msg234909  Adding a balance check, vesting balance check, withdraw funds to the script will allow us to extend testing/test with fewer CORE. 

If we had simple instructions on how to set up a node, and begin flooding with a python script how many of the "non technical" users would be willing to participate?

[tbone]

I have no problem contributing to something useful funded directly by the community.  But I have to agree with Xeldal.  What are we trying to accomplish here exactly?  Do we have input from Dan and the devs?  Have we considered that this may backfire on us?  Is it really a good use of funds and a good risk vs. reward?  Personally, I think it makes more sense to start figuring out what we'd like to fund via worker proposals since 2.0 will be a reality soon.  For example, I think 2FA is absolutely critical to implement as soon as possible.

Edit: After reading Akado's latest post I would be more inclined to contribute 5000 BTS to this if done in coordination with the dev team.

[Akado]

I thought Dan said they already did a LAN test, and achieved something like 186k tps.  Is there a reason we're trying to do this again? other than capturing it on video.  I would think they would have some scripts for this already also.

the WAN test might be more interesting. 

It sounds like there are still a great deal of optimizations that can be implemented.  Is now the best time to spend money on powerful/connected servers, only to do it again later with better optimizations?

Why can we not just use the existing test network in a organized fashion to capture this data. It may not produce the ultimate upper bounds but should still reveal bottlenecks and have something meaningful for video or whatever media angle you're interested in.    Im not fully convinced is best to spend a bunch of money at this time. 

I'm willing to donate some personal funds to aid in this:
1. I'm contributing with the following amount of BTS: 2500
2.  N/A
3.  N/A     

but for any significant amount from BitsharesBreakout, at this time I'd like to hold off for a moment to get a better feel for the value derived and necessity for doing this now, and in the way described.  Get some feed back from devs etc as to whether this is valuable for their work.

Yes WAN would be better.
I'm waiting on devs input on this as well, this was merely a suggestion.
It doesn't necessarily need to be done now, just trying to organize stuff, the mentioned dates are up to 3 weeks from now so we should have some time, offical should shouldn't be much later than that (maybe 2 weeks).
The objective would be to replicate a real case scenario as good as possible, from what I've read, via LAN and with no signature verification it might have had better results but in a not so specific case, it might take longer, plus it would be a statement towards all other crypto and give us plenty of attention, not to mention to shut the mouths of many flamers out there saying this is fraud.

We only benefit from this, but obviously needs to be well coordinated and details need to be worked out but since I'm not a technical guy, that is beyond my skill. This thread was created with the exact purpose of arranging that and coming up with something we all agree with.

It will have to be done eventually, so why not discuss? It's not like I'm saying it needs to be done now, but since it will eventually be done, let's at least try to come at a consensus and start to work things out so we're ready when the time comes. We only benefit from this.

To summarise:

It needs a test plan (what are we testing, how are we testing it, what is what we want to achieved or why are we testing), environment/s setup, scripts to execute tests, and of course who is going to participate.

Once this is done, we can think what money we need to achieve this.

 

[unreadPostsSinceLastVisit]

1. I'm contributing with the following amount of BTS: 2500
2. I'm contributing with the following resources (for testers only): N/A
3. I'm aiming to contribute following amount of transactions per second (for testers only): N/A

[betax]

To summarise:

It needs a test plan (what are we testing, how are we testing it, what is what we want to achieved or why are we testing), environment/s setup, scripts to execute tests, and of course who is going to participate.

Once this is done, we can think what money we need to achieve this.

[betax]

I thought Dan said they already did a LAN test, and achieved something like 186k tps.  Is there a reason we're trying to do this again? other than capturing it on video.  I would think they would have some scripts for this already also.

the WAN test might be more interesting. 

It sounds like there are still a great deal of optimizations that can be implemented.  Is now the best time to spend money on powerful/connected servers, only to do it again later with better optimizations?

Why can we not just use the existing test network in a organized fashion to capture this data. It may not produce the ultimate upper bounds but should still reveal bottlenecks and have something meaningful for video or whatever media angle you're interested in.    Im not fully convinced is best to spend a bunch of money at this time. 

I'm willing to donate some personal funds to aid in this:
1. I'm contributing with the following amount of BTS: 2500
2.  N/A
3.  N/A     

but for any significant amount from BitsharesBreakout, at this time I'd like to hold off for a moment to get a better feel for the value derived and necessity for doing this now, and in the way described.  Get some feed back from devs etc as to whether this is valuable for their work.

I have to agree with this, we are testing for testing. Why focus in an scenario that we can build towards and not focus on the current world. Our testnet is a good way to prove what we can do now, with the current vps / servers we currently need.

Probably best is to organise the testnet in a better way. If we are using Azure / AWS we can ensure we are all in the same region and using better connectivity. If you are using a home server you are still good to go.

We need to ensure that we have enough funds to test . Most of us have used puppies /clayop autohotkey script  or xerocs (I used xerocs as I was using putty) and quickly ran out. If xerocs script is modified so we distribute funds across all named participants, we might not ran out of CORE. Also ensure that we can quickly restart our witness if down.

Maybe scripts should run a counter to verify all the transactions are sent / received.

Once all this is organised, then we can experiment on scaling vertically and horizontally. More transactions, more clients / users to count votes, bigger vms.

[xeroc]

I thought Dan said they already did a LAN test, and achieved something like 186k tps.  Is there a reason we're trying to do this again? other than capturing it on video.  I would think they would have some scripts for this already also.

They have not had ECDSA signature verifications enabled .. just the plain blockchain replay ..

It sounds like there are still a great deal of optimizations that can be implemented.  Is now the best time to spend money on powerful/connected servers, only to do it again later with better optimizations?

It's supposed to be a show case for the AS-IS state

[Xeldal]

I thought Dan said they already did a LAN test, and achieved something like 186k tps.  Is there a reason we're trying to do this again? other than capturing it on video.  I would think they would have some scripts for this already also.

the WAN test might be more interesting. 

It sounds like there are still a great deal of optimizations that can be implemented.  Is now the best time to spend money on powerful/connected servers, only to do it again later with better optimizations?

Why can we not just use the existing test network in a organized fashion to capture this data. It may not produce the ultimate upper bounds but should still reveal bottlenecks and have something meaningful for video or whatever media angle you're interested in.    Im not fully convinced is best to spend a bunch of money at this time. 

I'm willing to donate some personal funds to aid in this:
1. I'm contributing with the following amount of BTS: 2500
2.  N/A
3.  N/A     

but for any significant amount from BitsharesBreakout, at this time I'd like to hold off for a moment to get a better feel for the value derived and necessity for doing this now, and in the way described.  Get some feed back from devs etc as to whether this is valuable for their work.

[Slappy]

I'm not very technical so all I can offer is the bts. Mark me down for 2500. Look forward to seeing the results.

[Akado]

I could assist with scripting (when I have time) ... but I need a clear list of things you need implemented

Cube could you elaborate on this so xeroc knows what you and other testers need exactly? A script for account creation and another for transaction spam?

And could someone explain phase 1 and 2 to me as that part for me confused. So we're doing 2 tests?

Already contacted bitsharsbreakout, waiting for a reply https://bitsharestalk.org/index.php/topic,13500.msg234844.html#msg234844

[chryspano]

1. I'm contributing with the following amount of BTS: 2500
2. I'm contributing with the following resources (for testers only): N/A
3. I'm aiming to contribute following amount of transactions per second (for testers only): N/A