BitShares Forum

Main => General Discussion => Topic started by: jakub on October 31, 2015, 07:52:34 pm

Title: QA needed in the GUI deployment process
Post by: jakub on October 31, 2015, 07:52:34 pm
Our current GUI deployment cycle looks like this:
(1) Without any prior notification the devs deploy a new release on Open Ledger (our only live platform).
(2) Community members who happen to find out about the upgrade report bugs introduced with the new release.
(3) Devs fix the new bugs and if the bugs are critical they make an emergency redeployment.

IMO such workflow is unacceptable at this stage.
We are live (no longer beta) and aim to be treated seriously.
It's a very bad practice to deploy a new GUI release without several people testing it first in different configurations.

What I propose is introduce QA (Quality Assurance) in the GUI deployment process by making community testing an integral part of it.
We could implement it in these simple steps:
(1) A new GUI release is announced on the forum and then made available on a dedicated (non-public) URL.
(2) We test the upgraded GUI and give feedback about any new bugs found.
(3) Devs fix those new bugs.
(4) After all bugs are fixed and no new ones are found within the next 24 hours, the new release is deployed on Open Ledger.

And just to make it clear - I'm not upset about bugs occurring, as this is normal when code gets upgraded.
What I'm upset about is that we could easily protect normal users from being exposed to those bugs and yet we don't do it.

We definitely need to be more careful about these things.
This is the time when this GUI is being showcased in business meetings and it needs to work without any new unexpected bugs or issues.
Otherwise we risk making fools of ourselves in those crucial meetings.
Title: Re: Improvement to the GUI deployment cycle
Post by: jakub on October 31, 2015, 08:04:30 pm
This is a very recent bug that we currently have after the latest upgrade (I guess):
https://github.com/cryptonomex/graphene-ui/issues/413

And it's a critical one - it makes the GUI crash.
Title: Re: Improvement to the GUI deployment cycle
Post by: Shentist on October 31, 2015, 08:07:32 pm
This is a very recent bug that we currently have after the latest upgrade (I guess):
https://github.com/cryptonomex/graphene-ui/issues/413

And it's a critical one - it makes the GUI crash.

it is not happening on my machine - windows 7 - chrome
Title: Re: Improvement to the GUI deployment cycle
Post by: jakub on October 31, 2015, 08:20:52 pm
This is a very recent bug that we currently have after the latest upgrade (I guess):
https://github.com/cryptonomex/graphene-ui/issues/413

And it's a critical one - it makes the GUI crash.

it is not happening on my machine - windows 7 - chrome

It happens each time I do it on my machine.

(http://i.imgur.com/P4bP1fp.png)
Title: Re: Improvement to the GUI deployment cycle
Post by: Shentist on October 31, 2015, 08:28:31 pm
i noticed that i can not change into "Dashboard, Explore, Trad, Transfer" i have to reload again. Maybe this is the crash you talked about?
Title: Re: Improvement to the GUI deployment cycle
Post by: jakub on October 31, 2015, 08:46:43 pm
i noticed that i can not change into "Dashboard, Explore, Trad, Transfer" i have to reload again. Maybe this is the crash you talked about?

Yes, that's the symptom that the js code has crashed.
You should see the errors once you open the js console in chrome.

Anyway, it might be the case that this particular issue is dependent on some other settings - I cannot recreate it on a different Win 7 machine.

But whatever it is in this case, it even enforces my point - we cannot rely on the devs testing the GUI before releasing it as they are not able to simulate all the different configurations.
Community testing should become part of the deployment cycle.
Title: Re: Improvement to the GUI deployment cycle
Post by: svk on October 31, 2015, 09:00:58 pm
I've pushed a fix for this but not sure when Valentine will update the server. The bug is related to a highcharts plugin that's only used in the price chart but tries to also apply itself to the depth chart. To avoid it until the fix is out, don't open the depth chart..

Tbh what we need is a testnet..
Title: Re: Improvement to the GUI deployment cycle
Post by: jakub on October 31, 2015, 09:18:22 pm
I've pushed a fix for this but not sure when Valentine will update the server. The bug is related to a highcharts plugin that's only used in the price chart but tries to also apply itself to the depth chart. To avoid it until the fix is out, don't open the depth chart..

Tbh what we need is a testnet..

Thanks, svk. I can see you work hard, even during weekends.

But do you agree that we need to coordinate those upgrades in a more orderly manner?
Or at the very least announce those upgrades on the forum so that we know when a new wave of bugs can be expected and prepare ourselves for that.
I'm having a series of business meetings next week where I intend to present the GUI and if something like this happens during those meetings, it can ruin my plans.
Title: Re: Improvement to the GUI deployment cycle
Post by: svk on October 31, 2015, 09:21:44 pm
Yea totally, it's very fast and furious atm since we're trying to add and fix as much stuff as possible as fast as possible, but it would probably be best if we move to a fixed release schedule with time for qa in between releases asap.
Title: Re: Improvement to the GUI deployment cycle
Post by: clayop on October 31, 2015, 09:36:00 pm
The loading problem I have is also related to the depth chart.
Title: Re: Improvement to the GUI deployment cycle
Post by: cass on October 31, 2015, 10:40:49 pm
Yea totally, it's very fast and furious atm since we're trying to add and fix as much stuff as possible as fast as possible, but it would probably be best if we move to a fixed release schedule with time for qa in between releases asap.

 +5%

https://www.zenhub.io/open-source
Title: Re: Improvement to the GUI deployment cycle
Post by: jakub on November 01, 2015, 10:38:38 am
Yea totally, it's very fast and furious atm since we're trying to add and fix as much stuff as possible as fast as possible, but it would probably be best if we move to a fixed release schedule with time for qa in between releases asap.

Could we use https://bitshares.org/wallet/ for QA community testing?
As far as I know, this URL is not currently advertised as an official BTS 2.0 platform so it can safely be used for QA.
In this case the only modification of the current process would be you deploying to bitshares.org/wallet first and then if nothing comes up during the next 24h deploy it to Open Ledger.
Title: Re: QA needed in the GUI deployment process
Post by: GaltReport on November 01, 2015, 03:15:04 pm
Our current GUI deployment cycle looks like this:
(1) Without any prior notification the devs deploy a new release on Open Ledger (our only live platform).
(2) Community members who happen to find out about the upgrade report bugs introduced with the new release.
(3) Devs fix the new bugs and if the bugs are critical they make an emergency redeployment.

IMO such workflow is unacceptable at this stage.
We are live (no longer beta) and aim to be treated seriously.
It's a very bad practice to deploy a new GUI release without several people testing it first in different configurations.

What I propose is introduce QA (Quality Assurance) in the GUI deployment process by making community testing an integral part of it.
We could implement it in these simple steps:
(1) A new GUI release is announced on the forum and then made available on a dedicated (non-public) URL.
(2) We test the upgraded GUI and give feedback about any new bugs found.
(3) Devs fix those new bugs.
(4) After all bugs are fixed and no new ones are found within the next 24 hours, the new release is deployed on Open Ledger.

And just to make it clear - I'm not upset about bugs occurring, as this is normal when code gets upgraded.
What I'm upset about is that we could easily protect normal users from being exposed to those bugs and yet we don't do it.

We definitely need to be more careful about these things.
This is the time when this GUI is being showcased in business meetings and it needs to work without any new unexpected bugs or issues.
Otherwise we risk making fools of ourselves in those crucial meetings.

 +5% - Starting to get frustrating again having to reload...butttons don't work...works different in different browsers.  You need some sort of formal QA process.
Title: Re: QA needed in the GUI deployment process
Post by: DestBest on November 01, 2015, 04:18:20 pm
What I propose is introduce QA (Quality Assurance) in the GUI deployment process by making community testing an integral part of it.
We could implement it in these simple steps:
(1) A new GUI release is announced on the forum and then made available on a dedicated (non-public) URL.
(2) We test the upgraded GUI and give feedback about any new bugs found.
(3) Devs fix those new bugs.
(4) After all bugs are fixed and no new ones are found within the next 24 hours, the new release is deployed on Open Ledger.
+5% and available for testing on win8
Title: Re: QA needed in the GUI deployment process
Post by: noisy on November 01, 2015, 04:46:27 pm
Quote
Community testing should become part of the deployment cycle.
100% agree.

Quote
Tbh what we need is a testnet..

This is also true. We should be able to test all kind of scenarios without having to worry, that this will cost us a fee.

Next step would be to automate testing. And do not merge a Pull Request without all unittest pass, and do not release new version without all selenium tests passed. I could help with that.
Title: Re: QA needed in the GUI deployment process
Post by: CalabiYau on November 02, 2015, 06:33:18 am
What I propose is introduce QA (Quality Assurance) in the GUI deployment process by making community testing an integral part of it.
We could implement it in these simple steps:
(1) A new GUI release is announced on the forum and then made available on a dedicated (non-public) URL.
(2) We test the upgraded GUI and give feedback about any new bugs found.
(3) Devs fix those new bugs.
(4) After all bugs are fixed and no new ones are found within the next 24 hours, the new release is deployed on Open Ledger.

 +5%
Title: Re: Improvement to the GUI deployment cycle
Post by: jakub on November 02, 2015, 07:58:07 pm
I've pushed a fix for this but not sure when Valentine will update the server. The bug is related to a highcharts plugin that's only used in the price chart but tries to also apply itself to the depth chart. To avoid it until the fix is out, don't open the depth chart..
@svk @valzav, please update the server.
This critical bug is still there.
Title: Re: QA needed in the GUI deployment process
Post by: xeroc on November 02, 2015, 08:32:12 pm
All old version can be accesed via http://decentral.exchange .. let me ping the admin so that he updates the latest master .. there seems to be a manual step involved to initiate a recompile
Title: Re: QA needed in the GUI deployment process
Post by: svk on November 02, 2015, 08:44:11 pm
I've pushed a fix for this but not sure when Valentine will update the server. The bug is related to a highcharts plugin that's only used in the price chart but tries to also apply itself to the depth chart. To avoid it until the fix is out, don't open the depth chart..
@svk @valzav, please update the server.
This critical bug is still there.
Valentine will push a new version later today. After this update we'll move to a qa process like you suggested where changes will be tested at bitshares.org/wallet before being pushed to OL.
Title: Re: QA needed in the GUI deployment process
Post by: fav on November 03, 2015, 12:02:16 pm
https://bitsharestalk.org/index.php/topic,19714.msg252996.html#new needs to be fixed asap