Author Topic: BTS seed node is decling while STEEM seed node is rising  (Read 3300 times)

0 Members and 1 Guest are viewing this topic.

Offline abit

  • Committee member
  • Hero Member
  • *
  • Posts: 4664
    • View Profile
    • Abit's Hive Blog
  • BitShares: abit
  • GitHub: abitmore

Quote
Also, while switching all the nodes to use p2p port 1776 this weekend I had several seeds get stuck for some unknown reason. They've been solidly online for 2 days now, but nothing has changed so I suspect future failures will occur again.
My guess is that there is somethign weird happending after a few weeks
of runtime. And we haven't seen this kind of behavior in STEEM because
their have been plenty of hardforks and restarts.

My node had been getting stuck every day in last week. I added some logging to the code and now waiting for next stuck so I can know more about the reason, but it hasn't been stuck for 2 days.. that sucks. Once we caught the bug, it's perhaps easy to fix though -- just catch one more exception, perhaps.

There is a bug identified earlier by @abit.  Perhaps abit can share more on is latest finding.
BitShares committee member: abit
BitShares witness: in.abit

Offline cube

  • Hero Member
  • *****
  • Posts: 1404
  • Bit by bit, we will get there!
    • View Profile
  • BitShares: bitcube
There is a bug identified earlier by @abit.  Perhaps abit can share more on is latest finding.
ID: bitcube
bitcube is a dedicated witness and committe member. Please vote for bitcube.

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12922
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
will running multiple instance of STEEM,MUSE,BTS on a same VPS cause that bug ?
only if you are using the same port and have too little RAM

What do you base that on xeroc? Sounds like you've spent some time troubleshooting this problem, is that correct?

Well, statement number one: "same ports" is actually quite obvious
because you can only connect to either of the seed nodes and not all of
them. The second statement is more like an estimate. If you run a
BitShares seed/full node you need some RAM. Some more RAM for MUSE and
yet more RAM for STEEM.
If your machine has 32gigs of RAM that should do the trick, otherwise
you will end up with an unresponsive machine.

To clearify, I do not run a public seed node but multiple full nodes on
several machines

Quote
Actually from recent experience and observation the one factor that triggers this bug the most is request volume. Before I changed the port I was averaging around 50 connections pretty consistently on all of my nodes. Now it's up to 80 - 90. All but one of my servers are running low RAM, but even with the increased connections RAM use has remained flat without an increase.
RAM depends in fact on the use of BitShares. For instance open orders
are kept in RAM. Once the load in the DEX picks up, so will the
requirments for RAM.

Quote
Also, while switching all the nodes to use p2p port 1776 this weekend I had several seeds get stuck for some unknown reason. They've been solidly online for 2 days now, but nothing has changed so I suspect future failures will occur again.
My guess is that there is somethign weird happending after a few weeks
of runtime. And we haven't seen this kind of behavior in STEEM because
their have been plenty of hardforks and restarts.

Quote
I added abit's simple bash script to my list of monit tasks so I'll be notified if any of the nodes get stuck. Wackou's seed monitor is also useful for anyone that wants to see the status of all seed nodes. I haven't upgraded my nodes to his latest release of bts_tools yet, but I will soon.
+5%!

Offline Thom

will running multiple instance of STEEM,MUSE,BTS on a same VPS cause that bug ?
only if you are using the same port and have too little RAM

What do you base that on xeroc? Sounds like you've spent some time troubleshooting this problem, is that correct?

Actually from recent experience and observation the one factor that triggers this bug the most is request volume. Before I changed the port I was averaging around 50 connections pretty consistently on all of my nodes. Now it's up to 80 - 90. All but one of my servers are running low RAM, but even with the increased connections RAM use has remained flat without an increase.

Also, while switching all the nodes to use p2p port 1776 this weekend I had several seeds get stuck for some unknown reason. They've been solidly online for 2 days now, but nothing has changed so I suspect future failures will occur again.

I added abit's simple bash script to my list of monit tasks so I'll be notified if any of the nodes get stuck. Wackou's seed monitor is also useful for anyone that wants to see the status of all seed nodes. I haven't upgraded my nodes to his latest release of bts_tools yet, but I will soon.
Injustice anywhere is a threat to justice everywhere - MLK |  Verbaltech2 Witness Reports: https://bitsharestalk.org/index.php/topic,23902.0.html

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12922
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
will running multiple instance of STEEM,MUSE,BTS on a same VPS cause that bug ?
only if you are using the same port and have too little RAM

Offline btswildpig

  • Hero Member
  • *****
  • Posts: 1424
    • View Profile
All the developers moved to steem?
you better follow the discussion and notice that there is a bug currently searched for in both, the steem backend node and bitshares that makes the nodes crash at some point ..
it just hasn't happend to the small steem network yet ..

//edit: BTS nodes are not run by the developers but by individuals!

will running multiple instance of STEEM,MUSE,BTS on a same VPS cause that bug ?
这个是私人账号,表达的一切言论均不代表任何团队和任何人。This is my personal account , anything I said with this account will be my opinion alone and has nothing to do with any group.

Offline xeroc

  • Board Moderator
  • Hero Member
  • *****
  • Posts: 12922
  • ChainSquad GmbH
    • View Profile
    • ChainSquad GmbH
  • BitShares: xeroc
  • GitHub: xeroc
All the developers moved to steem?
you better follow the discussion and notice that there is a bug currently searched for in both, the steem backend node and bitshares that makes the nodes crash at some point ..
it just hasn't happend to the small steem network yet ..

//edit: BTS nodes are not run by the developers but by individuals!

Offline ebit

  • Committee member
  • Hero Member
  • *
  • Posts: 1905
    • View Profile
  • BitShares: ebit
telegram:ebit521
https://weibo.com/ebiter

Offline sudo

  • Hero Member
  • *****
  • Posts: 2255
    • View Profile
  • BitShares: ags

Offline freedom

  • Sr. Member
  • ****
  • Posts: 303
    • View Profile
All the developers moved to steem?


« Last Edit: May 10, 2016, 07:24:45 am by freedom »