[Proposal] - review of new bitshares build testing procedures

[monsterer]

but it wouldn't test the new code that gets unlocked after the hard fork block is reached (at which point ideally all delegates should have already upgraded).

This is true. It would be fine for non-forking changes, though. This is the easiest to implement procedure.

For testing behavior in forking changes it gets a lot more complicated and you'd need to use another 'mirrored' chain, or a least a test network of some kind with a bunch of test delegates.

[arhag]

In light of the recent 'problems' with the accidental hard fork in v0.4.25, I propose that new builds are not pushed to live until a test delegate has run them for 24 hours and confirmed they are still on the main chain with no failures, or memory problems.

Sounds like a decent idea, but it wouldn't test the new code that gets unlocked after the hard fork block is reached (at which point ideally all delegates should have already upgraded). That is why those new features and changes should be properly tested on a test network before pushing the code to the real network. DevShares can also be incredibly useful for this.

Of course that is the ideal, in real life things get more complicated. We should have a discussion about what level of testing and quality assurance should be required for different classes of changes to the code, before the delegates agree to upgrade. For example, what is the minimum threshold of testing that delegates should demand from the devs for a change that fixes a critical security bug discovered in the client? Obviously the standards of testing for that change would be less strict than the standards for a change that adds a new non-essential feature to the blockchain.

Also there should be some accountability. For example, the devs who make the decision to classify a particular upgrade as a critical security upgrade should sign that statement with their private keys. That way the delegates can know it is important to upgrade ASAP and if it turns out later the upgrade was inappropriately classified, the devs' reputations are hurt. Obviously, in these early stages, control of BitShares is very centralized with the devs and we all trust them to do the right thing. But I am just talking about future plans as we try to make BitShares even more decentralized.

[monsterer]

Good idea. Maybe one of our technical delegates can propose to run such a delegate.

It needs to be a core team member, or at least someone in their timezone / office otherwise the lead time will be too long.

[svk]

In light of the recent 'problems' with the accidental hard fork in v0.4.25, I propose that new builds are not pushed to live until a test delegate has run them for 24 hours and confirmed they are still on the main chain with no failures, or memory problems.

These recent problems could have been avoided if such measures were in place.

Good idea. Maybe one of our technical delegates can propose to run such a delegate.

[monsterer]

In light of the recent 'problems' with the accidental hard fork in v0.4.25, I propose that new builds are not pushed to live until a test delegate has run them for 24 hours and confirmed they are still on the main chain with no failures, or memory problems.

These recent problems could have been avoided if such measures were in place.