BitShares Forum

Main => General Discussion => Topic started by: fav on July 10, 2017, 09:44:21 am

Title: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 09:44:21 am
Restart: https://steemit.com/bitshares/@ash/beauty-sleep-is-over-bitshares-restarting

BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Updates: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: zhangweis on July 10, 2017, 10:10:44 am
BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Thanks for the info. It explains why the chain stops at 18,154,987.
Do you have more information (links) on the issue?
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 10:32:06 am
BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Thanks for the info. It explains why the chain stops at 18,154,987.
Do you have more information (links) on the issue?

not atm, pretty sure we'll get an update from witnesses asap
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 12:26:36 pm
Updates: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: compumatrix on July 10, 2017, 12:32:59 pm
Thanks for the heads up @fav
Title: Re: BitShares unplanned maintenance
Post by: sudo on July 10, 2017, 01:25:20 pm
 +5% +5% +5%
Title: Re: BitShares unplanned maintenance
Post by: yvv on July 10, 2017, 02:38:35 pm
And we use advertise everywhere that a decentralized network can not halt...

Hope this issue will be tracked down and fixed.
Title: Re: BitShares unplanned maintenance
Post by: crypto4ever on July 10, 2017, 02:45:43 pm
And we use advertise everywhere that a decentralized network can not halt...

Hope this issue will be tracked down and fixed.

That's not true. Show me where it says "it cannot halt".  It says that your funds cannot be stolen since you hold the keys.  The chain is full of encryption. This is still true.

It's probably nothing more than an inode or disk problem. Who knows. We'll find out soon.

Yes of course this issue will be tracked down and fixed. It's obviously their #1 priority.
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 02:45:55 pm
Update 4: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode @bytemaster joined forces
Title: Re: BitShares unplanned maintenance
Post by: jQrgen on July 10, 2017, 02:55:33 pm
BitShares Network halted ~40 mins ago

Do not Trade / Withdraw / Deposit

Witnesses / Devs are debugging and working on a fix.

Updates: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode

How is this even technically possible?

Did all the miners stop mining at the same time?

Is there a bug in the code?


<3
JQ
Title: Re: BitShares unplanned maintenance
Post by: crypto4ever on July 10, 2017, 02:57:25 pm

How is this even technically possible?

Did all the miners stop mining at the same time?

Is there a bug in the code?


<3
JQ

There's data in a particular block (from what I've read) that caused this problem. It's probably a minor fix. It just takes time to understand it, create a fix, and then test it.  It's a small hiccup, and once fixed, you probably won't see this happen ever again.

The extra time they are taking to be careful they fix it right and the fix is 100% good, I like.  I don't want them to rush writing a cheap fix.   Leave them some time.

...and usually someone will write a blog post explaining what happened and how it was fixed after this is all over.

Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 03:03:08 pm
Quote
Witnesses are incapable of producing blocks b/c there is an issue that prevents verifying the previous chain - a particular transaction is fouling up the works. Dan Larimer and witnesses are working the problem as fast as possible.
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 03:08:21 pm
Update 5: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 03:52:19 pm
Update 6: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 04:06:00 pm
Update 7: https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 05:19:51 pm
Update 8 (final?): https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 05:32:06 pm
Update 9 (Aftermath): https://steemit.com/bitshares/@ash/bitshares-entered-unplanned-maintenance-mode
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 06:59:44 pm
we're back https://steemit.com/bitshares/@ash/beauty-sleep-is-over-bitshares-restarting
Title: Re: BitShares unplanned maintenance
Post by: Brekyrself on July 10, 2017, 07:10:10 pm
Good time to analyze which witnesses were quick to react with this situation.
Title: Re: BitShares unplanned maintenance
Post by: yvv on July 10, 2017, 07:20:10 pm
Good news. The issue was handled pretty well.
Title: Re: BitShares unplanned maintenance
Post by: fav on July 10, 2017, 08:01:34 pm
Good time to analyze which witnesses were quick to react with this situation.

(https://i.imgur.com/btzT0JI.png)

I know xeldal, lafona and sahkan are updating. no idea about the others
Title: Re: BitShares unplanned maintenance
Post by: R on July 10, 2017, 09:24:36 pm
Good time to analyze which witnesses were quick to react with this situation.

(https://i.imgur.com/btzT0JI.png)

I know xeldal, lafona and sahkan are updating. no idea about the others
Surely their 'missed blocks' statistic is going to go through the roof soon? Might spur a shift in voting intent!
Title: Re: BitShares unplanned maintenance
Post by: karnal on July 11, 2017, 08:57:15 am
Has anyone written a technical overview of what went wrong and what the fix was?
Title: Re: BitShares unplanned maintenance
Post by: Chronos on July 11, 2017, 03:41:10 pm
Has anyone written a technical overview of what went wrong and what the fix was?
I'd definitely be interested in more details, as well. Bitshares outages have seen so rare since 2.0 was released. This was a major event!
Title: Re: BitShares unplanned maintenance
Post by: xeroc on July 11, 2017, 03:47:44 pm
I wrote an incident report .. waiting for the lawyer to approve the stuff that comes with that report :D