Author Topic: 0.5.1 and 0.5.3 are unusable for me  (Read 6253 times)

0 Members and 1 Guest are viewing this topic.

Offline neo1344

  • Full Member
  • ***
  • Posts: 89
    • View Profile
can you please provide me  the link for latest QT for W7,l was away for a while and still running 0.4.27.2 dont connect,
Thanks

https://bitshares.org/resources/downloads

Many thanks

Offline vikram

can you please provide me  the link for latest QT for W7,l was away for a while and still running 0.4.27.2 dont connect,
Thanks

https://bitshares.org/resources/downloads

Offline neo1344

  • Full Member
  • ***
  • Posts: 89
    • View Profile
can you please provide me  the link for latest QT for W7,l was away for a while and still running 0.4.27.2 dont connect,
Thanks

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Had 4G ram when I first downloaded v0.4.27.2.  Very slow if it came up at all. After expanding to 8G, operation has been seamless even with upgrades...now running 0.5.3.  Excuse me if I'm re-stating old material, there just seems to be many problems stated here that increased ram will take care of.

I'm aware that increasing RAM will make things better (and so will using an SSD instead of a HDD), but that isn't really a solution.

The big concern is whether the RAM requirements are scaling with time/blocks or just scaling with user adoption. If it is the former, then that is a serious problem (and a bug because there is no reason why it needs to be) because it will eventually be too much to handle for even serious servers (well I suppose perhaps not if we assume Moore's law to continue to hold since: given any k > 0, e^x > k*x for x >> 1). But my guess is that it is the latter.

Even then, from what I have seen the RAM demands of bitshares_client grow unnecessarily with the time it has been running (when syncing the blockchain to the present). Restarting the client and letting it resume sync from where it last started seems to make things better (for a time). This is a performance bug that should eventually be fixed (I can completely understand if it isn't high priority though). As far as I can tell it only affects the client when the sync it is trying to catch up with the present block, and not when it has already synced and is simply running and maintaining the sync.

Obviously, even if these performance bugs were fixed, it is clear that the memory, CPU, and IO demands are never going to be trivial (especially as user adoption increases and transaction volume increases). So eventually very few people will be running full clients. And that is where a fully functional lightweight client comes in (it also is a necessity for mobile devices, which eventually should have the same functionality as a client running on a laptop/desktop). My hope is that the performance of the BitShares full client can be kept reasonable enough for long enough so that we do not need to exclude those with computers with 4 GB of RAM and HDD from running the full client, at least until we have a feature complete lightweight client (that means voting and exchange functionality as well). Furthermore, I seriously hope the performance demand of the BitShares full client doesn't exceed the capabilities of computers with 8 GB of RAM and SSD (and even more preferably 4 GB of RAM and HDD) until we have a much better lightweight validation/security model than just trusting the light wallet server (I have discussed what I would expect here).

Offline Catchfire

  • Jr. Member
  • **
  • Posts: 29
    • View Profile
I have tried both building 0.5.3 myself and using the pre-compiled 0.5.1 binaries. The results are the same: the clients are totally unusable.

The CLI client of both versions still has the unbearable typing lag I mentioned last time. Running it with --server and using the web wallet doesn't help. It is very slow. The RPC POST requests have an average latency of around 2 minutes and some of them timeout with an error!

The Qt client isn't better. The 0.5.3 Qt client is very slow and laggy and sometimes won't ever load pages like the market. In fact it completely froze my Ubuntu system (hard reset required) when I tried to load the market in an attempt to cover a short! The performance of the 0.5.1 Qt client was actually pretty decent, but I quickly realized the only reason that was the case was because the blockchain wasn't syncing at all beyond the point left by the CLI client (a problem I mentioned last time with v0.4.27.2), meaning its completely worthless.

So basically there is no properly functioning client for me at the moment. I only have 4 GB of RAM on the laptop so I didn't expect super snappy performance with the full client, but this is just ridiculous. It has to be some bug in the client right? Or is my computer somehow messed up?

Had 4G ram when I first downloaded v0.4.27.2.  Very slow if it came up at all. After expanding to 8G, operation has been seamless even with upgrades...now running 0.5.3.  Excuse me if I'm re-stating old material, there just seems to be many problems stated here that increased ram will take care of.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Please let me know if these performance problems are still present in 0.6.0.

Those particular performance problems were solved with 0.6.0. I discussed it in more detail here. Thanks.

Offline vikram

I have tried both building 0.5.3 myself and using the pre-compiled 0.5.1 binaries. The results are the same: the clients are totally unusable.

The CLI client of both versions still has the unbearable typing lag I mentioned last time. Running it with --server and using the web wallet doesn't help. It is very slow. The RPC POST requests have an average latency of around 2 minutes and some of them timeout with an error!

The Qt client isn't better. The 0.5.3 Qt client is very slow and laggy and sometimes won't ever load pages like the market. In fact it completely froze my Ubuntu system (hard reset required) when I tried to load the market in an attempt to cover a short! The performance of the 0.5.1 Qt client was actually pretty decent, but I quickly realized the only reason that was the case was because the blockchain wasn't syncing at all beyond the point left by the CLI client (a problem I mentioned last time with v0.4.27.2), meaning its completely worthless.

So basically there is no properly functioning client for me at the moment. I only have 4 GB of RAM on the laptop so I didn't expect super snappy performance with the full client, but this is just ridiculous. It has to be some bug in the client right? Or is my computer somehow messed up?

Please let me know if these performance problems are still present in 0.6.0.

Offline vikram

Ah, this is good news. I am aware of this particular problem and fixed it as part of https://github.com/BitShares/bitshares/issues/1192 which should be in 0.6.x. Can you confirm by testing a build of the commit I mentioned in my previous post?

Good job tracking down the problem.

Glad to hear the problem is potentially solved. I'll try the build later when I get the chance and give you my feedback. I first want to create a new (throwaway) wallet with 0.5.3 and make sure the same performance problems exist, and then copy that wallet over to the client running the build you are talking about and see if it fixes the problem (I don't want to use my real wallet on an unofficial build).

Be aware that the problem may only become noticeably bad when you have many registered accounts (as contacts or otherwise) in your wallet.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Ah, this is good news. I am aware of this particular problem and fixed it as part of https://github.com/BitShares/bitshares/issues/1192 which should be in 0.6.x. Can you confirm by testing a build of the commit I mentioned in my previous post?

Good job tracking down the problem.

Glad to hear the problem is potentially solved. I'll try the build later when I get the chance and give you my feedback. I first want to create a new (throwaway) wallet with 0.5.3 and make sure the same performance problems exist, and then copy that wallet over to the client running the build you are talking about and see if it fixes the problem (I don't want to use my real wallet on an unofficial build).

Offline vikram

I think I found the cause of the performance problems.

First, I want to correct what I said earlier. Whenever the wallet is closed and once the blockchain has synced to the present, there appears to be no lag in the CLI client. It maintains the sync as verified by get_info, I can type in commands normally, and I can run blockchain_* commands without any issue.

However, once I unlock the wallet, the performance problems occur. There is that terrible typing lag in the CLI client, RPC calls have huge latencies, and my HDD indicator light goes crazy. Looking at iotop I see the main IO user on top is [jbd2/dm-2-8] while normally with the wallet closed it is bitshares_client.

I ran strace on one of the bitshares_client threads and I see a lot of write followed by fdatasync system calls (constantly alternating). There is a continuous stream of these happening at a fast frequency (faster than one pair per second). By cross referencing the fd that it is writing and fdatasync-ing using lsof, I find that the program is frequently writing and flushing to "wallets/default/000707.log". It makes no sense to me why there would be any need to write or flush data to the wallets database when there is no user action to cause changes (such as changing votes or making transactions).

Ah, this is good news. I am aware of this particular problem and fixed it as part of https://github.com/BitShares/bitshares/issues/1192 which should be in 0.6.x. Can you confirm by testing a build of the commit I mentioned in my previous post?

Good job tracking down the problem.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
I think I found the cause of the performance problems.

First, I want to correct what I said earlier. Whenever the wallet is closed and once the blockchain has synced to the present, there appears to be no lag in the CLI client. It maintains the sync as verified by get_info, I can type in commands normally, and I can run blockchain_* commands without any issue.

However, once I unlock the wallet, the performance problems occur. There is that terrible typing lag in the CLI client, RPC calls have huge latencies, and my HDD indicator light goes crazy. Looking at iotop I see the main IO user on top is [jbd2/dm-2-8] while normally with the wallet closed it is bitshares_client.

I ran strace on one of the bitshares_client threads and I see a lot of write followed by fdatasync system calls (constantly alternating). There is a continuous stream of these happening at a fast frequency (faster than one pair per second). By cross referencing the fd that it is writing and fdatasync-ing using lsof, I find that the program is frequently writing and flushing to "wallets/default/000707.log". It makes no sense to me why there would be any need to write or flush data to the wallets database when there is no user action to cause changes (such as changing votes or making transactions).

Edit:
By the way, I then mounted a tmpfs (meaning the entire filesystem is held in RAM) as the wallets directory and copied the contents of the original wallet over to that directory. I then ran the client and unlocked the wallet and everything ran very smoothly. Of course, the problem with that is that it is not actually committing any of the changes to the wallet database so if I restarted, all the changes to my transaction history, vote changes, etc. would be lost.

Also, I found it ridiculously easy to corrupt the wallet database during this process (but I was careful to make lots of backups of the folder at every step). I guess I can't make a copy of the wallets folder while it is unlocked and expect to use it later...
« Last Edit: January 25, 2015, 08:17:49 pm by arhag »

Offline santaclause102

  • Hero Member
  • *****
  • Posts: 2486
    • View Profile
Is there a version atm that will work reliably with 2 GB of RAM? The latest one I had was .4.7.2 but 0.5 was a required update...

Offline vikram

My guess is this has to do with the fact that the DVS blockchain is currently very small and new and so the OS just can keep it all in the cache in the RAM rather than going crazy with the HDD seeks.

This is likely a large contributing factor. I have also made some additional optimizations to both syncing and wallet scanning in 0.6.x, which will hopefully help you in BTS. Given the problems in this thread I will continue looking for more low-hanging fruit in terms of removing excessive disk accesses, etc.

Actually, since you have been testing different builds, maybe you also want to try a CLI build of commit 1100744c23571392d14d270d6a69541f0174aa05 (current HEAD of bitshares branch) and see if there is any performance difference. That has all the latest code merged in as of Friday night, and should still be syncing successfully. It's of course not an official version so do not trust funds to it.
« Last Edit: January 24, 2015, 11:39:55 pm by vikram »

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Do we know for sure if it's due to HDD seeking (https://bitsharestalk.org/index.php?topic=12887.msg177979#msg177979 )? Or could it possibly be for example the networking threads stealing all CPU? Is there any way you might be able to profile the process to get more info (for example Mac has an easy profile button in task manager)?

Umm, not sure, I'll see what I can do.

But I should add. I just compiled and ran DevShares 0.6.1. The CLI client had very slight lag during the blockchain sync. Then once the sync was complete it is running very smoothly. The HDD light indicator is not constantly running while running the DVS client unlike the BTS client (and this is also reflected by the lower IORR and IOWR values in htop for the devshares_client threads). My guess is this has to do with the fact that the DVS blockchain is currently very small and new and so the OS just can keep it all in the cache in the RAM rather than going crazy with the HDD seeks.

Offline vikram

The CLI client of both versions still has the unbearable typing lag I mentioned last time. Running it with --server and using the web wallet doesn't help. It is very slow. The RPC POST requests have an average latency of around 2 minutes and some of them timeout with an error!

Do you still have the lag in CLI when no wallet is open? Is it equally laggy during syncing and after synced?

I still have lag whether the wallet is open or not. The blockchain syncing and indexing (not scanning) is what is causing the performance problems and lags. I believe it is a little better after reaching the present block and maintaining sync compared to catching up with the sync, but it may just be my imagination. The point is that the unbearable lag is still present even when the blockchain is never more than 0-20 seconds behind the present block (meaning "after sync").

Do we know for sure if it's due to HDD seeking (https://bitsharestalk.org/index.php?topic=12887.msg177979#msg177979 )? Or could it possibly be for example the networking threads stealing all CPU? Is there any way you might be able to profile the process to get more info (for example Mac has an easy profile button in task manager)?