BitShares unplanned maintenance

[xeroc]

I wrote an incident report .. waiting for the lawyer to approve the stuff that comes with that report :D

[Chronos]

Has anyone written a technical overview of what went wrong and what the fix was?

I'd definitely be interested in more details, as well. Bitshares outages have seen so rare since 2.0 was released. This was a major event!

[karnal]

Has anyone written a technical overview of what went wrong and what the fix was?

[R]

Good time to analyze which witnesses were quick to react with this situation.

I know xeldal, lafona and sahkan are updating. no idea about the others

Surely their 'missed blocks' statistic is going to go through the roof soon? Might spur a shift in voting intent!

[fav]

Good time to analyze which witnesses were quick to react with this situation.

I know xeldal, lafona and sahkan are updating. no idea about the others

[yvv]

Good news. The issue was handled pretty well.

[Brekyrself]

Good time to analyze which witnesses were quick to react with this situation.

[fav]

we're back https://steemit.com/bitshares/@ash/beauty-sleep-is-over-bitshares-restarting

[fav]

Update 9 (Aftermath): https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

[fav]

Update 8 (final?): https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

[fav]

Update 7: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

[fav]

Update 6: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

[fav]

Update 5: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

[fav]

Witnesses are incapable of producing blocks b/c there is an issue that prevents verifying the previous chain - a particular transaction is fouling up the works. Dan Larimer and witnesses are working the problem as fast as possible.

[crypto4ever]

How is this even technically possible?

Did all the miners stop mining at the same time?

Is there a bug in the code?


<3
JQ

There's data in a particular block (from what I've read) that caused this problem. It's probably a minor fix. It just takes time to understand it, create a fix, and then test it.  It's a small hiccup, and once fixed, you probably won't see this happen ever again.

The extra time they are taking to be careful they fix it right and the fix is 100% good, I like.  I don't want them to rush writing a cheap fix.   Leave them some time.

...and usually someone will write a blog post explaining what happened and how it was fixed after this is all over.

[jQrgen]

BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Updates: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

How is this even technically possible?

Did all the miners stop mining at the same time?

Is there a bug in the code?


<3
JQ

[fav]

Update 4: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode @bytemaster joined forces

[crypto4ever]

And we use advertise everywhere that a decentralized network can not halt...

Hope this issue will be tracked down and fixed.

That's not true. Show me where it says "it cannot halt".  It says that your funds cannot be stolen since you hold the keys.  The chain is full of encryption. This is still true.

It's probably nothing more than an inode or disk problem. Who knows. We'll find out soon.

Yes of course this issue will be tracked down and fixed. It's obviously their #1 priority.

[yvv]

And we use advertise everywhere that a decentralized network can not halt...

Hope this issue will be tracked down and fixed.

[sudo]

 +5% +5% +5%

[compumatrix]

Thanks for the heads up @fav

[fav]

Updates: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

[fav]

BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Thanks for the info. It explains why the chain stops at 18,154,987.
Do you have more information (links) on the issue?

not atm, pretty sure we'll get an update from witnesses asap

[zhangweis]

BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Thanks for the info. It explains why the chain stops at 18,154,987.
Do you have more information (links) on the issue?

[fav]

Restart: https://steemit.com/bitshares/@ash/beauty-sleep-is-over-bitshares-restarting

BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Updates: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode