Author Topic: Transaction Throughput Testing  (Read 5187 times)

0 Members and 1 Guest are viewing this topic.

Offline JianJolly

  • Newbie
  • *
  • Posts: 14
    • View Profile
Transaction Throughput Testing
« on: December 27, 2016, 02:25:47 pm »
Motivation

Scalability of consensus protocol implementations is a well known and much debated issue, in which the capacity of a blockchain to handle a large number of transactions per second plays a central role. In this regard, to test the performance of the Graphene technology, on which Bitshares is built, a group of people from the community have expressed their willingness to conduct a collaborative performance test on Bitshares. The aim of the test is to get a measure of the platform's transaction throughput by trying to execute a heavy load of transactions on the blockchain.


Design

A script will be run by multiple participants to broadcast transactions. The participants will agree on a time to run the script, and flood the network at that time for a predetermined timespan. There is no upper limit to the load of transactions, which means that more participants and better testing environment will result in a higher load.

There will be multiple phases of the test, described as follows:

Phase I: The participants will broadcast transactions at the same time with the same delay between each transaction for 1 minute.
Phase II: The participants will broadcast transactions at random times with a random delay between transactions for 1 minute.
Phase III: The participants will broadcast as many transactions as possible for 10 minutes.


Implementation

The testing will first be performed on the testnet, for which the conditions will be emulated to the mainnet environment. To this end, a total of 30 witness nodes will be installed and the necessary configurations will be made for them to get an active witness status on the testnet. Afterwards, the testing script will be written and tested by individual participants. The participants will then agree on a time to start the testing on the testnet.

As many statistically significant details as possible about the test results will be collected, compiled, visualized, interpreted, and published. A member or a group of members from the community will be chosen to execute the post-testing work.

The details of the mainnet testing will be determined later.


Roadmap

- Installation & configuration of the necessary number of testnet witness nodes.
- Writing the testing script & individual sanity checks.
- Agreeing on a time & agreeing on the details of the post-testing work.
- Performing the test on the testnet.
- Collection, refinement, and presentation of the test results.
- Deciding for the details of the mainnet testing.


Call for Participants

Please write under this thread to participate in the test for the swift communication and easy resource handling.


Relevant Links

Bitshares Telegram group: https://telegram.me/BitSharesDEX
Graphene testnet repo: https://github.com/BitSharesEurope/graphene-testnet
Documentation for producing witness nodes: http://docs.bitshares.org/bitshares/tutorials/witness-howto.html


Result

The test conducted on the testnet has successfully shown that the Bitshares network could process 3300 transactions per second. For more details, see the following post:

1h30 min BitShares Stress Test of the 15 March 2017 - 3300TXs 14000OPs

Conducting the stress test on the mainnet is canceled because of the conclusiveness of the test on the testnet.
« Last Edit: March 19, 2017, 08:58:19 pm by JianJolly »

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4473
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
Re: Transaction Throughput Testing
« Reply #1 on: December 27, 2016, 04:18:03 pm »
Good initiative  +5%

However I don't think there are many qualified people will join (I for one, won't, sorry). For best simulation, need to setup block producing nodes in different data centers / providers all over the world, and testing nodes (for generating new transactions) as well.

We've done similar testing before launching the chain, see these posts for example:
* https://bitsharestalk.org/index.php/topic,18684.0.html
* https://bitsharestalk.org/index.php/topic,17962.0.html
* https://bitsharestalk.org/index.php/topic,18699.0.html
* https://bitsharestalk.org/index.php/topic,18751.0.html
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline GChicken

  • Sr. Member
  • ****
  • Posts: 231
    • View Profile
Re: Transaction Throughput Testing
« Reply #2 on: December 27, 2016, 04:21:02 pm »
i will help with the testing, will be great to see the results and i think i will learn some valuable knowledge from contributing.

Will start building a node tomorrow, hosted in APAC

Offline Fox

Re: Transaction Throughput Testing
« Reply #3 on: December 27, 2016, 07:45:51 pm »
I will contribute at least one node to the TESTNET for this effort. Please provide a link to the github repository with your transaction generation script(s). As referenced above, previous test have been limited  not by the witness nodes processing the transactions, rather the ability of the clients to sent transactions quickly enough and at scale to generate a high volume. Focus your effort on building the transaction construction and broadcasting.
Witness: fox

Offline rnglab

  • Full Member
  • ***
  • Posts: 171
    • View Profile
  • BitShares: rnglab
Re: Transaction Throughput Testing
« Reply #4 on: December 28, 2016, 01:39:31 pm »
As expressed before, I see no reason to promote bitshares scalability before prooving the yet outstanding throughput we already have, over a sound patform.

There are features being developed that will require a hardfork.
We have [member=4854]vikram[/member] back willing to work for bitshares.
Steem's graphene implementation is free software and actively developed.
If the p2p protocol or something else needs optimizations, this seems to me like the right moment to start working on it.

This would need funding of course. As I get it, dilution not always means devaluation, and even if price drops it can become a good investment if done right.

I can run segregated witness nodes and spamming peers.

And I'd be glad if we can organize this towards an official stable release.

 








Offline kenCode

  • Hero Member
  • *****
  • Posts: 2283
    • View Profile
    • Agorise
Re: Transaction Throughput Testing
« Reply #5 on: December 31, 2016, 08:32:58 am »
BitShares Munich will allocate 2 Dedicated servers (in France and Germany at 2 different hosts), our Lead Sysadmin, and 10,000 BTS to this initiative.
kenCode - Decentraliser @ Agorise
Matrix/Keybase/Hive/Commun/Github: @Agorise
www.PalmPay.chat

Offline ElMato

  • Sr. Member
  • ****
  • Posts: 288
    • View Profile
Re: Transaction Throughput Testing
« Reply #6 on: December 31, 2016, 09:51:56 am »
Im in, i will setup a VPS in Argentina and if i have some time also some scripts to generate spam txs

Sent from my XT1063 using Tapatalk


Offline kimziv

Re: Transaction Throughput Testing
« Reply #7 on: January 01, 2017, 05:43:30 am »
we will contribute at least one Dedicated servers for this Testing with my partner Yao(https://bitsharestalk.org/index.php?action=profile;u=2134).  When will the Testing begin and How long will it be lasting?

I will build the node when my server is ready recently, my server will be hosted in AliYun Cloud , China.

YOYOW---You Own Your Own Words

Offline Yao

  • Hero Member
  • *****
  • Posts: 534
  • QQ/WeChat(微信):664349247
    • View Profile
  • BitShares: yao
  • GitHub: imYao
Re: Transaction Throughput Testing
« Reply #8 on: January 01, 2017, 08:19:21 am »
we will contribute at least one Dedicated servers for this Testing with my partner Yao(https://bitsharestalk.org/index.php?action=profile;u=2134).  When will the Testing begin and How long will it be lasting?

I will build the node when my server is ready recently, my server will be hosted in AliYun Cloud , China.
+5%

Our BitShares witness ([Witness Proposal] 1.6.71 - witness.yao) is active now, we hope to have a better understanding of the witness mechanism and BitShares blockchain by participating the testing. ;)
BTS witness: witness.yao
BTS Proxy: yao

Offline teiva

  • Newbie
  • *
  • Posts: 6
    • View Profile
Re: Transaction Throughput Testing
« Reply #9 on: January 03, 2017, 12:58:59 am »
I'd like to participate too but it seems that these test have already been conducted on test net before ?
I've got a few servers in France and some VPS if needed.
Just send instructions how to proceed

Offline kenCode

  • Hero Member
  • *****
  • Posts: 2283
    • View Profile
    • Agorise
Re: Transaction Throughput Testing
« Reply #10 on: January 03, 2017, 07:33:35 am »
I'd like to participate too but it seems that these test have already been conducted on test net before ?
I've got a few servers in France and some VPS if needed.
Just send instructions how to proceed

Thank you [member=42109]teiva[/member] :)
Are you in our Telegram channel?
http://telegram.me/bitsharesdex
 
There are tons of server people in there who can help you get your servers all setup for the big test.
Thanx again!
  ken
kenCode - Decentraliser @ Agorise
Matrix/Keybase/Hive/Commun/Github: @Agorise
www.PalmPay.chat

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4473
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore
Re: Transaction Throughput Testing
« Reply #11 on: January 07, 2017, 03:33:28 am »
Found this old post https://bitsharestalk.org/index.php/topic,18727.0.html .

Thanks to [member=19219]clayop[/member] and [member=31]fav[/member]
BTS account: abit
BTS committee member: abit
BTS witness: in.abit

Offline aaronic

Re: Transaction Throughput Testing
« Reply #12 on: January 16, 2017, 12:50:53 pm »
Hi JianJolly,

I am interested in setting up a BitShares witness node and this test effort would be a great introduction to the ecosystem. I generally use VPSs in Germany but could look at other providers if you need to be more geographically diverse. This will be my first BitShares server but I'm pretty comfortable with Linux sysadmin and software deployments.

Please let me know if this would be helpful and the timeframe of the test effort.

Cheers, Aaron

Offline kenCode

  • Hero Member
  • *****
  • Posts: 2283
    • View Profile
    • Agorise
Re: Transaction Throughput Testing
« Reply #13 on: January 16, 2017, 01:31:55 pm »
Hi JianJolly,

I am interested in setting up a BitShares witness node and this test effort would be a great introduction to the ecosystem. I generally use VPSs in Germany but could look at other providers if you need to be more geographically diverse. This will be my first BitShares server but I'm pretty comfortable with Linux sysadmin and software deployments.

Please let me know if this would be helpful and the timeframe of the test effort.

Cheers, Aaron

Definitely, and thank you for the assistance! :)
 
[member=43221]JianJolly[/member] [member=120]xeroc[/member] - Do we have a definitive date and UTC time for the event as of yet?
kenCode - Decentraliser @ Agorise
Matrix/Keybase/Hive/Commun/Github: @Agorise
www.PalmPay.chat

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12915
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
Re: Transaction Throughput Testing
« Reply #14 on: January 16, 2017, 03:24:51 pm »
Hi JianJolly,

I am interested in setting up a BitShares witness node and this test effort would be a great introduction to the ecosystem. I generally use VPSs in Germany but could look at other providers if you need to be more geographically diverse. This will be my first BitShares server but I'm pretty comfortable with Linux sysadmin and software deployments.

Please let me know if this would be helpful and the timeframe of the test effort.

Cheers, Aaron
code is over here: https://github.com/BitSharesEurope/graphene-testnet
the setup is similar to BitShares nodes: http://docs.bitshares.eu/installation/Build.html

I don't have a timeline for the stress test yet.. but we can certainly need some more witnesses to jump in. Let me know if you run into any trouble setting up a witness
Give BitShares a try! Use the http://testnet.bitshares.eu provided by http://bitshares.eu powered by ChainSquad GmbH