Author Topic: 0.5.1 and 0.5.3 are unusable for me  (Read 1796 times)

0 Members and 1 Guest are viewing this topic.

Offline santaclause102

  • Hero Member
  • *****
  • Posts: 2486
    • View Profile
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #15 on: January 25, 2015, 02:31:43 pm »
Is there a version atm that will work reliably with 2 GB of RAM? The latest one I had was .4.7.2 but 0.5 was a required update...

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #16 on: January 25, 2015, 07:56:11 pm »
I think I found the cause of the performance problems.

First, I want to correct what I said earlier. Whenever the wallet is closed and once the blockchain has synced to the present, there appears to be no lag in the CLI client. It maintains the sync as verified by get_info, I can type in commands normally, and I can run blockchain_* commands without any issue.

However, once I unlock the wallet, the performance problems occur. There is that terrible typing lag in the CLI client, RPC calls have huge latencies, and my HDD indicator light goes crazy. Looking at iotop I see the main IO user on top is [jbd2/dm-2-8] while normally with the wallet closed it is bitshares_client.

I ran strace on one of the bitshares_client threads and I see a lot of write followed by fdatasync system calls (constantly alternating). There is a continuous stream of these happening at a fast frequency (faster than one pair per second). By cross referencing the fd that it is writing and fdatasync-ing using lsof, I find that the program is frequently writing and flushing to "wallets/default/000707.log". It makes no sense to me why there would be any need to write or flush data to the wallets database when there is no user action to cause changes (such as changing votes or making transactions).

Edit:
By the way, I then mounted a tmpfs (meaning the entire filesystem is held in RAM) as the wallets directory and copied the contents of the original wallet over to that directory. I then ran the client and unlocked the wallet and everything ran very smoothly. Of course, the problem with that is that it is not actually committing any of the changes to the wallet database so if I restarted, all the changes to my transaction history, vote changes, etc. would be lost.

Also, I found it ridiculously easy to corrupt the wallet database during this process (but I was careful to make lots of backups of the folder at every step). I guess I can't make a copy of the wallets folder while it is unlocked and expect to use it later...
« Last Edit: January 25, 2015, 08:17:49 pm by arhag »

Offline vikram

Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #17 on: January 25, 2015, 08:15:42 pm »
I think I found the cause of the performance problems.

First, I want to correct what I said earlier. Whenever the wallet is closed and once the blockchain has synced to the present, there appears to be no lag in the CLI client. It maintains the sync as verified by get_info, I can type in commands normally, and I can run blockchain_* commands without any issue.

However, once I unlock the wallet, the performance problems occur. There is that terrible typing lag in the CLI client, RPC calls have huge latencies, and my HDD indicator light goes crazy. Looking at iotop I see the main IO user on top is [jbd2/dm-2-8] while normally with the wallet closed it is bitshares_client.

I ran strace on one of the bitshares_client threads and I see a lot of write followed by fdatasync system calls (constantly alternating). There is a continuous stream of these happening at a fast frequency (faster than one pair per second). By cross referencing the fd that it is writing and fdatasync-ing using lsof, I find that the program is frequently writing and flushing to "wallets/default/000707.log". It makes no sense to me why there would be any need to write or flush data to the wallets database when there is no user action to cause changes (such as changing votes or making transactions).

Ah, this is good news. I am aware of this particular problem and fixed it as part of https://github.com/BitShares/bitshares/issues/1192 which should be in 0.6.x. Can you confirm by testing a build of the commit I mentioned in my previous post?

Good job tracking down the problem.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #18 on: January 25, 2015, 08:22:30 pm »
Ah, this is good news. I am aware of this particular problem and fixed it as part of https://github.com/BitShares/bitshares/issues/1192 which should be in 0.6.x. Can you confirm by testing a build of the commit I mentioned in my previous post?

Good job tracking down the problem.

Glad to hear the problem is potentially solved. I'll try the build later when I get the chance and give you my feedback. I first want to create a new (throwaway) wallet with 0.5.3 and make sure the same performance problems exist, and then copy that wallet over to the client running the build you are talking about and see if it fixes the problem (I don't want to use my real wallet on an unofficial build).

Offline vikram

Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #19 on: January 25, 2015, 08:36:43 pm »
Ah, this is good news. I am aware of this particular problem and fixed it as part of https://github.com/BitShares/bitshares/issues/1192 which should be in 0.6.x. Can you confirm by testing a build of the commit I mentioned in my previous post?

Good job tracking down the problem.

Glad to hear the problem is potentially solved. I'll try the build later when I get the chance and give you my feedback. I first want to create a new (throwaway) wallet with 0.5.3 and make sure the same performance problems exist, and then copy that wallet over to the client running the build you are talking about and see if it fixes the problem (I don't want to use my real wallet on an unofficial build).

Be aware that the problem may only become noticeably bad when you have many registered accounts (as contacts or otherwise) in your wallet.

Offline vikram

Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #20 on: February 07, 2015, 10:46:37 pm »
I have tried both building 0.5.3 myself and using the pre-compiled 0.5.1 binaries. The results are the same: the clients are totally unusable.

The CLI client of both versions still has the unbearable typing lag I mentioned last time. Running it with --server and using the web wallet doesn't help. It is very slow. The RPC POST requests have an average latency of around 2 minutes and some of them timeout with an error!

The Qt client isn't better. The 0.5.3 Qt client is very slow and laggy and sometimes won't ever load pages like the market. In fact it completely froze my Ubuntu system (hard reset required) when I tried to load the market in an attempt to cover a short! The performance of the 0.5.1 Qt client was actually pretty decent, but I quickly realized the only reason that was the case was because the blockchain wasn't syncing at all beyond the point left by the CLI client (a problem I mentioned last time with v0.4.27.2), meaning its completely worthless.

So basically there is no properly functioning client for me at the moment. I only have 4 GB of RAM on the laptop so I didn't expect super snappy performance with the full client, but this is just ridiculous. It has to be some bug in the client right? Or is my computer somehow messed up?

Please let me know if these performance problems are still present in 0.6.0.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #21 on: February 08, 2015, 10:59:42 am »
Please let me know if these performance problems are still present in 0.6.0.

Those particular performance problems were solved with 0.6.0. I discussed it in more detail here. Thanks.

Offline Catchfire

  • Jr. Member
  • **
  • Posts: 29
    • View Profile
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #22 on: February 08, 2015, 01:35:43 pm »
I have tried both building 0.5.3 myself and using the pre-compiled 0.5.1 binaries. The results are the same: the clients are totally unusable.

The CLI client of both versions still has the unbearable typing lag I mentioned last time. Running it with --server and using the web wallet doesn't help. It is very slow. The RPC POST requests have an average latency of around 2 minutes and some of them timeout with an error!

The Qt client isn't better. The 0.5.3 Qt client is very slow and laggy and sometimes won't ever load pages like the market. In fact it completely froze my Ubuntu system (hard reset required) when I tried to load the market in an attempt to cover a short! The performance of the 0.5.1 Qt client was actually pretty decent, but I quickly realized the only reason that was the case was because the blockchain wasn't syncing at all beyond the point left by the CLI client (a problem I mentioned last time with v0.4.27.2), meaning its completely worthless.

So basically there is no properly functioning client for me at the moment. I only have 4 GB of RAM on the laptop so I didn't expect super snappy performance with the full client, but this is just ridiculous. It has to be some bug in the client right? Or is my computer somehow messed up?

Had 4G ram when I first downloaded v0.4.27.2.  Very slow if it came up at all. After expanding to 8G, operation has been seamless even with upgrades...now running 0.5.3.  Excuse me if I'm re-stating old material, there just seems to be many problems stated here that increased ram will take care of.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #23 on: February 08, 2015, 03:39:52 pm »
Had 4G ram when I first downloaded v0.4.27.2.  Very slow if it came up at all. After expanding to 8G, operation has been seamless even with upgrades...now running 0.5.3.  Excuse me if I'm re-stating old material, there just seems to be many problems stated here that increased ram will take care of.

I'm aware that increasing RAM will make things better (and so will using an SSD instead of a HDD), but that isn't really a solution.

The big concern is whether the RAM requirements are scaling with time/blocks or just scaling with user adoption. If it is the former, then that is a serious problem (and a bug because there is no reason why it needs to be) because it will eventually be too much to handle for even serious servers (well I suppose perhaps not if we assume Moore's law to continue to hold since: given any k > 0, e^x > k*x for x >> 1). But my guess is that it is the latter.

Even then, from what I have seen the RAM demands of bitshares_client grow unnecessarily with the time it has been running (when syncing the blockchain to the present). Restarting the client and letting it resume sync from where it last started seems to make things better (for a time). This is a performance bug that should eventually be fixed (I can completely understand if it isn't high priority though). As far as I can tell it only affects the client when the sync it is trying to catch up with the present block, and not when it has already synced and is simply running and maintaining the sync.

Obviously, even if these performance bugs were fixed, it is clear that the memory, CPU, and IO demands are never going to be trivial (especially as user adoption increases and transaction volume increases). So eventually very few people will be running full clients. And that is where a fully functional lightweight client comes in (it also is a necessity for mobile devices, which eventually should have the same functionality as a client running on a laptop/desktop). My hope is that the performance of the BitShares full client can be kept reasonable enough for long enough so that we do not need to exclude those with computers with 4 GB of RAM and HDD from running the full client, at least until we have a feature complete lightweight client (that means voting and exchange functionality as well). Furthermore, I seriously hope the performance demand of the BitShares full client doesn't exceed the capabilities of computers with 8 GB of RAM and SSD (and even more preferably 4 GB of RAM and HDD) until we have a much better lightweight validation/security model than just trusting the light wallet server (I have discussed what I would expect here).

Offline neo1344

  • Full Member
  • ***
  • Posts: 89
    • View Profile
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #24 on: February 11, 2015, 08:54:17 pm »
can you please provide me  the link for latest QT for W7,l was away for a while and still running 0.4.27.2 dont connect,
Thanks

Offline vikram

Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #25 on: February 11, 2015, 08:57:37 pm »
can you please provide me  the link for latest QT for W7,l was away for a while and still running 0.4.27.2 dont connect,
Thanks

https://bitshares.org/resources/downloads

Offline neo1344

  • Full Member
  • ***
  • Posts: 89
    • View Profile
Re: 0.5.1 and 0.5.3 are unusable for me
« Reply #26 on: February 11, 2015, 09:03:45 pm »
can you please provide me  the link for latest QT for W7,l was away for a while and still running 0.4.27.2 dont connect,
Thanks

https://bitshares.org/resources/downloads

Many thanks