Recent Network Split Analysis

[monsterer]

Isn't it the exchanges' responsibility to be allowing tx only when they are on the main chain? Or run delayed nodes?  If an exchange did not bother to check whether it was on a folk, it could only expect to risk losing funds.

It is yes. I'm just pointing out the facts.

[cube]

No cause for alarm.  It only affects the witnesses in turn of them having to 'restart' their witness nodes if a split occurred.  No funds are affected in any way.

Doesn't that depend on whether exchanges are following the 2/3 confirmation time rule? If an exchange deposits a transaction on a fork which gets rolled back, that would be a serious problem.

Isn't it the exchanges' responsibility to be allowing tx only when they are on the main chain? Or run delayed nodes?  If an exchange did not bother to check whether it was on a folk, it could only expect to risk losing funds.

[xeroc]

No cause for alarm.  It only affects the witnesses in turn of them having to 'restart' their witness nodes if a split occurred.  No funds are affected in any way.

Doesn't that depend on whether exchanges are following the 2/3 confirmation time rule? If an exchange deposits a transaction on a fork which gets rolled back, that would be a serious problem.

Thata why they are supposed to run the delayed node .. the all they see is FINAL and IRREVERSIBLE

[monsterer]

No cause for alarm.  It only affects the witnesses in turn of them having to 'restart' their witness nodes if a split occurred.  No funds are affected in any way.

Doesn't that depend on whether exchanges are following the 2/3 confirmation time rule? If an exchange deposits a transaction on a fork which gets rolled back, that would be a serious problem.

[cube]

Any cause for alarm here? (BTS vanishing, etc)

No cause for alarm.  It only affects the witnesses in turn of them having to 'restart' their witness nodes if a split occurred.  No funds are affected in any way.

[karnal]

Any cause for alarm here? (BTS vanishing, etc)

[mindphlux]

It just happened again. I withdraw vested balance and 20% of the witnesses forked.

I will restrain from vesting until BM tells us to do so or has fixed the bug.

[xeroc]

Good thing we have delayed nodes ..

[puppies]

lol.  The only reason I withdrew anything was to check if it was working for mindphlux.  In the short term I just won't withdraw any witness pay.  Does this bug affect other vesting balances?

[bytemaster]

I don't expect this to be a security issue because the only people that can trigger this particular bug are "trusted" witnesses.

I define security issue as something that would allow someone to get "something for nothing".   

We have a plan in place to catch this bug if it happens again.   

[wackou]

Around 2015-10-14T19:21:39  dele-puppy attempted to claim a vesting balance (his witness pay) and as a result the network split 60/40 because 40% of the witnesses thought the transaction was invalid.

As a result the last irreversible block hung around block 33845 (the block prior to the one that forked the network).   The blockchain grew to 269 blocks before 2/3's of the witnesses managed to agree and extend the last irreversible block back to the typical 20 to 30 behind the head block number.

In this situation everyone on the network was accurately reporting the state and the user interfaces indicated to the users that their operations were pending confirmation.   Any / all exchanges following the last irreversible block would have been OK.

We have determined the cause is due to some non-deterministic behavior in witness pay that resolves itself with a replay.   After replaying the blockchain all witnesses agreed the transaction was valid. 

We will continue to monitor the blockchain for issues as we look for the underlying cause.   In the meantime this particular failure can be mitigated by witnesses periodically replaying their blockchain. 

Despite the "bug" everything is working just fine and your funds are safe.

Would love to hear details.

me too, got me really curious there... Although I would understand if we had to wait until it is properly resolved, as this might be a security issue I imagine...

[betax]

So every 12 hours we can put another witness up, with a fresh database.

[lakerta06]

Around 2015-10-14T19:21:39  dele-puppy attempted to claim a vesting balance (his witness pay) and as a result the network split 60/40 because 40% of the witnesses thought the transaction was invalid.

As a result the last irreversible block hung around block 33845 (the block prior to the one that forked the network).   The blockchain grew to 269 blocks before 2/3's of the witnesses managed to agree and extend the last irreversible block back to the typical 20 to 30 behind the head block number.

In this situation everyone on the network was accurately reporting the state and the user interfaces indicated to the users that their operations were pending confirmation.   Any / all exchanges following the last irreversible block would have been OK.

We have determined the cause is due to some non-deterministic behavior in witness pay that resolves itself with a replay.   After replaying the blockchain all witnesses agreed the transaction was valid. 

We will continue to monitor the blockchain for issues as we look for the underlying cause.   In the meantime this particular failure can be mitigated by witnesses periodically replaying their blockchain. 

Despite the "bug" everything is working just fine and your funds are safe.

Would love to hear details.

[bytemaster]

Around 2015-10-14T19:21:39  dele-puppy attempted to claim a vesting balance (his witness pay) and as a result the network split 60/40 because 40% of the witnesses thought the transaction was invalid.

As a result the last irreversible block hung around block 33845 (the block prior to the one that forked the network).   The blockchain grew to 269 blocks before 2/3's of the witnesses managed to agree and extend the last irreversible block back to the typical 20 to 30 behind the head block number.

In this situation everyone on the network was accurately reporting the state and the user interfaces indicated to the users that their operations were pending confirmation.   Any / all exchanges following the last irreversible block would have been OK.

We have determined the cause is due to some non-deterministic behavior in witness pay that resolves itself with a replay.   After replaying the blockchain all witnesses agreed the transaction was valid. 

We will continue to monitor the blockchain for issues as we look for the underlying cause.   In the meantime this particular failure can be mitigated by witnesses periodically replaying their blockchain. 

Despite the "bug" everything is working just fine and your funds are safe.