BitShares Forum

Main => General Discussion => Topic started by: alt on January 04, 2016, 12:30:11 am

Title: network down??
Post by: alt on January 04, 2016, 12:30:11 am
all my 4 nodes froze at block 2359544
Code: [Select]
1191310ms th_a       application.cpp:496           handle_block         ] Got block: #2359544 time: 2016-01-04T00:19:51 latency: 317 ms from: roadscape  irreversible: 2359526 (-18)
Title: Re: network down??
Post by: clayop on January 04, 2016, 12:43:03 am
Same here
Title: Re: network down??
Post by: puppies on January 04, 2016, 12:43:58 am
Same
Title: Re: network down??
Post by: wackou on January 04, 2016, 12:45:38 am
Same
Title: Re: network down??
Post by: cube on January 04, 2016, 12:50:35 am
all nodes stuck.
Title: Re: network down??
Post by: abit on January 04, 2016, 12:51:18 am
Code: [Select]
2016-01-04T00:19:51 th_a:invoke handle_block         handle_block ] Got block: #2359544 time: 2016-01-04T00:19:51 latency: 184 ms from
: roadscape  irreversible: 2359526 (-18)                  application.cpp:496
2016-01-04T00:20:06 th_a:invoke handle_transaction   handle_transaction ] Got 1 transactions from network                       applic
ation.cpp:545
2016-01-04T00:20:10 th_a:invoke handle_transaction   handle_transaction ] Got 1 transactions from network                       applic
ation.cpp:545
2016-01-04T00:20:47 th_a:invoke handle_transaction   handle_transaction ] Got 1 transactions from network                       applic
ation.cpp:545
2016-01-04T00:21:00 th_a:Witness Block Production          _push_block ] Failed to push new block:
10 assert_exception: Assert Exception
call.debt > 0 && call.collateral > 0 && settle.balance.amount > 0:
    {}
    th_a  db_market.cpp:236 match

    {"call":{"id":"1.8.1318","borrower":"1.2.97590","collateral":3104433333,"debt":50000,"call_price":{"base":{"amount":3104433333,"asset_id":"1.3.0"},"quote":{"amount":87500,"asset_id":"1.3.106"}}},"settle":{"owner":"1.2.95758","balance":{"amount":0,"asset_id":"1.3.106"},"settlement_date":"2016-01-04T00:19:54"},"match_price":{"base":{"amount":0,"asset_id":"1.3.106"},"quote":{"amount":0,"asset_id":"1.3.0"}},"max_settlement":{"amount":1375203,"asset_id":"1.3.106"}}
    th_a  db_market.cpp:261 match

    {"next_block.block_num()":2359545}
    th_a  db_block.cpp:536 _apply_block                 db_block.cpp:195

Title: Re: network down??
Post by: abit on January 04, 2016, 12:53:28 am
Caused by a "0 amount" settlement in GOLD

Requested in block 2330819: http://cryptofresh.com/b/2330819

Code: [Select]
get_settle_orders GOLD 100
[{
    "owner": "1.2.95758",
    "balance": {
      "amount": 0,
      "asset_id": "1.3.106"
    },
    "settlement_date": "2016-01-04T00:19:54"
  }
]
Title: Re: network down??
Post by: tonyk on January 04, 2016, 12:54:19 am
A settlement in GOLD

Nice!
Title: Re: network down??
Post by: Stan on January 04, 2016, 12:59:27 am
BM is on it.
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 01:30:05 am
I should have a patch ready soon. 
Title: Re: network down??
Post by: abit on January 04, 2016, 01:32:08 am
I should have a patch ready soon.
Thanks. Instructions needed.
Title: Re: network down??
Post by: tonyk on January 04, 2016, 01:35:56 am
BM is on it.

how we gonna handle this hard fork?

Who is gonna write the worker proposal?


I suggest a 2BTS/day worker....


also a good time to make bug fixing a profitable business (I think) ...How about a parallel 250K/mo. bug fixing worker for 14.7% of the time dedicated to such jobs?
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 01:42:33 am
Any witnesses willing to help try an untested patch can update to the latest master on github.   

I am not a witness so I cannot verify if this fixes block production or not.  You may have to specify a checkpoint. 

Title: Re: network down??
Post by: abit on January 04, 2016, 01:46:04 am
Any witnesses willing to help try an untested patch can update to the latest master on github.   

I am not a witness so I cannot verify if this fixes block production or not.  You may have to specify a checkpoint.
Building.
More info about this "specify a checkpoint", please add this to the witness_node start command:
Code: [Select]
--checkpoint '[2359545,"002400f965e4ad8b494d48dfb8d311d59cd771bc"]'
Title: Re: network down??
Post by: Bhuz on January 04, 2016, 01:48:21 am
Building too
Title: Re: network down??
Post by: abit on January 04, 2016, 02:10:25 am
New block produced:
Code: [Select]
locked >>> get_block 2359545
get_block 2359545
{
  "previous": "002400f85a7d07a722f801b839c75f89a4f55e67",
  "timestamp": "2016-01-04T02:08:30",
  "witness": "1.6.35",
  "transaction_merkle_root": "0000000000000000000000000000000000000000",
  "extensions": [],
  "witness_signature": "1f43e7663aa4f9f204506a7e29ae2c13b4a2d064444f635c644cd451f6fba132622a79c39a9785624b3953dff0eac5853c2f18195622b63861d188279161d44667",
  "transactions": [],
  "block_id": "002400f965e4ad8b494d48dfb8d311d59cd771bc",
  "signing_key": "BTS83K3qNNZBRLWMa6pXcwnfMJqC9J46ZpMFEYDKUDCNy2BgNrWth",
  "transaction_ids": []
}


please add this to the witness_node start command:
Code: [Select]
--checkpoint '[2359545,"002400f965e4ad8b494d48dfb8d311d59cd771bc"]'
Title: Re: network down??
Post by: abit on January 04, 2016, 02:28:58 am
Witnesses:

* If you want to follow my chain, please add my seed node 114.92.254.159:62015. and
 add this to the witness_node start command:
Code: [Select]
--checkpoint '[2359545,"002400f965e4ad8b494d48dfb8d311d59cd771bc"]' -s "114.92.254.159:62015"
(maybe the checkpoint is not needed here)

* If you want to start your new chain, please add this parameter to witness_node startup script:

Code: [Select]
--enable-stale-production --checkpoint '[2359545,"0000000000000000000000000000000000000000"]'
Title: Re: network down??
Post by: twitter on January 04, 2016, 03:11:40 am
not sure why ? i am getting "Segmentation fault (core dumped)" after starting node
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 03:28:56 am
not sure why ? i am getting "Segmentation fault (core dumped)" after starting node

Run in GDB and post backtrace.
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 03:30:06 am
We have 24% of the witnesses producing blocks in sync now.   The rest should update asap.
Title: Re: network down??
Post by: ffwong on January 04, 2016, 03:40:32 am
dragonball syncing
But have no enough vote to produce block 
Title: Re: network down??
Post by: taoljj on January 04, 2016, 03:53:26 am
delegate.taolje  is up.
Title: Re: network down??
Post by: Spectral on January 04, 2016, 04:05:00 am
witness spectral is updated, producing blocks and feed
seed node is up on 46.226.12.230:1984
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 04:10:37 am
57.8125% participation and rising..

Proxies please support delegated-proof-of-steak and clayop because they are quality witnesses who got up to date faster than everyone else.  A quick vote could get us up to 67% participation in a timely manner.
Title: Re: network down??
Post by: twitter on January 04, 2016, 04:12:09 am
“bue” is re-indexing +5% :) :)
Title: Re: network down??
Post by: clayop on January 04, 2016, 04:21:05 am
57.8125% participation and rising..

Proxies please support delegated-proof-of-steak and clayop because they are quality witnesses who got up to date faster than everyone else.  A quick vote could get us up to 67% participation in a timely manner.

Thanks. "delegate-clayop"
Title: Re: network down??
Post by: Harvey on January 04, 2016, 04:21:17 am
'harvey-xts' is running and needs vote back in service.

Code: [Select]
unlocked >>> get_witness harvey-xts
get_witness harvey-xts
{
  "id": "1.6.23",
  "witness_account": "1.2.3284",
  "last_aslot": 2015005,
  "signing_key": "BTS7V2jfYZEToTjGFndctgTPipMuMcobqzwsd8RQv77c5Vzcexgt6",
  "pay_vb": "1.13.49",
  "vote_id": "1:33",
  "total_votes": "15878702827678",
  "url": "-",
  "total_missed": 115,
  "last_confirmed_block_num": 2005541
}
Title: Re: network down??
Post by: dichalcog3nid3 on January 04, 2016, 04:28:54 am
w.node b33lz38v8 is fully synced.

edit: If we had more witnesses voted in, we would have reached 67% participation more quickly. i encourage proxies to vote in those members who are actively in sync with the network, but not producing blocks due to lack of votes. this suggests that it would be healthy for the network to have more active witnesses http://cryptofresh.com/witnesses (http://cryptofresh.com/witnesses)
Title: Re: network down??
Post by: BunkerChainLabs-DataSecurityNode on January 04, 2016, 04:44:28 am
DataSecurityNode is running with feeds updating.

Thanks to BM who responded immediately to the issue once it was discovered and resolved it so quickly.

Thanks to all the other delegates who had to work late into the night to get their witnesses up and going.
Title: Re: network down??
Post by: twitter on January 04, 2016, 05:14:06 am
currently only 14 active witnesses .... 7 out of  21 are dead for 4 hours ....


bue is up    ...please vote bue back to service



57.8125% participation and rising..

Proxies please support delegated-proof-of-steak and clayop because they are quality witnesses who got up to date faster than everyone else.  A quick vote could get us up to 67% participation in a timely manner.
Title: Re: network down??
Post by: cube on January 04, 2016, 05:21:40 am
bitcube node is up.  Working on seed node next.

Code: [Select]
"1120411ms th_a       application.cpp:820           get_blockchain_synop ] synopsis: ["002400f7711cb44e5d23c06550d38dca546d53f2","002400f85a7d07a722f801b839c75f89a4f55e67"]
1125000ms th_a       witness.cpp:185               block_production_loo ] Generated block #2360867 with timestamp 2016-01-04T05:18:45 at time 2016-01-04T05:18:45
1128339ms th_a       application.cpp:500           handle_block         ] Got block: #2360868 time: 2016-01-04T05:18:48 latency: 344 ms from: xeldal  irreversible: 2360849 (-19)
1"

"head_block_num": 2360866,
  "head_block_id": "0024062220ab51ad1ab52197380aa54b10e4a73e",
  "head_block_age": "1 second old",
  "next_maintenance_time": "41 minutes in the future",
  "chain_id": "4018d7844c78f6a6c41c6a552b898022310fc5dec06da467ee7905a8dad512c8",
  "participation": "67.18750000000000000",


Edit: all nodes updated.
Title: Re: network down??
Post by: puppies on January 04, 2016, 06:07:36 am
All of my nodes are updated and running.  Should be producing feeds as well.  I'm gonna get some sleep.  Good night everyone.
Title: Re: network down??
Post by: roadscape on January 04, 2016, 06:20:30 am
Witness roadscape and cryptofresh.com are back online.
Title: Re: network down??
Post by: spartako on January 04, 2016, 08:40:51 am
Witness spartako and telegram spartako_bot back online
Title: Re: network down??
Post by: wackou on January 04, 2016, 12:09:13 pm
witness wackou back online
Title: Re: network down??
Post by: pc on January 04, 2016, 12:54:37 pm
Witness cyrano updated and producing blocks again.
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 02:14:43 pm
Thank you everyone who helped get the witnesses back up in a timely manner.
Title: Re: network down??
Post by: twitter on January 04, 2016, 02:15:01 pm
not sure why ? i am getting "Segmentation fault (core dumped)" after starting node

Run in GDB and post backtrace.
It was due to my script pointed to old version client...

BM, my witness bue is back to work and need your votes to back to service[emoji1]
Title: Re: network down??
Post by: spartako on January 04, 2016, 02:21:06 pm
Also witness spartako is back to work but need BM votes to back in the active witness list    :D
Title: Re: network down??
Post by: bytemaster on January 04, 2016, 02:27:19 pm
I will vote the rest of you back in later today.  I hope everyone who was voted out knows it was nothing personal, I was just attempting to get to 67% participation before more drastic measures were required.
Title: Re: network down??
Post by: twitter on January 04, 2016, 03:24:46 pm
thanks for your note....
I was wondering why last night there were 7 out of 21 not producing blocks but my node(bue) was out . I guess those 7 do not need your votes and still be able to stay in the active witnesses list  .....  :P :)

I will vote the rest of you back in later today.  I hope everyone who was voted out knows it was nothing personal, I was just attempting to get to 67% participation before more drastic measures were required.