BitShares Forum
Main => General Discussion => Topic started by: monsterer on May 22, 2015, 11:03:27 am
-
The high memory usage is probably a huge factor in the bad performance of the bitshares client. Why does it need so much RAM compared to bitcoind?
-
The high memory usage is probably a huge factor in the bad performance of the bitshares client. Why does it need so much RAM compared to bitcoind?
It uses alot more indices than bitcoind ...
Recall that the BitShares client has a BUILT-IN block explorer!
-
It uses alot more indices than bitcoind ...
Recall that the BitShares client has a BUILT-IN block explorer!
Why do the indices need to be RAM resident?
-
It uses alot more indices than bitcoind ...
Recall that the BitShares client has a BUILT-IN block explorer!
Why do the indices need to be RAM resident?
The bottle neck for BitShares is Disk IO. BitShares uses a lot of RAM in an attempt to minimize Disk IO.
Bitcoin has just a few LevelDB databases, BitShares has 2 dozen. Each LevelDB has its own cache.
This is all old news anyway, soon BitShares will be using less RAM than your web browser.
-
The bottle neck for BitShares is Disk IO. BitShares uses a lot of RAM in an attempt to minimize Disk IO.
Bitcoin has just a few LevelDB databases, BitShares has 2 dozen. Each LevelDB has its own cache.
This is all old news anyway, soon BitShares will be using less RAM than your web browser.
That bottle neck is probably the true bottle neck only for machines which aren't forced to swap the caches out to disk due to 'lower' real RAM than your development machines.
However, I am glad to hear you're already on it :)
-
The bottle neck for BitShares is Disk IO. BitShares uses a lot of RAM in an attempt to minimize Disk IO.
Bitcoin has just a few LevelDB databases, BitShares has 2 dozen. Each LevelDB has its own cache.
This is all old news anyway, soon BitShares will be using less RAM than your web browser.
That bottle neck is probably the true bottle neck only for machines which aren't forced to swap the caches out to disk due to 'lower' real RAM than your development machines.
However, I am glad to hear you're already on it :)
Even with 16 GB of RAM disk io is the bottleneck.
-
Even with 16 GB of RAM disk io is the bottleneck.
My point was that with substantially less RAM than your development machines have (for example 2GB on a win7 machine), plain disk IO is unlikely to be the bottleneck... I'd expect it to be related to the swapping necessary to maintain the vast array of caches you mentioned.
-
It uses alot more indices than bitcoind ...
Recall that the BitShares client has a BUILT-IN block explorer!
Why do the indices need to be RAM resident?
The bottle neck for BitShares is Disk IO. BitShares uses a lot of RAM in an attempt to minimize Disk IO.
Bitcoin has just a few LevelDB databases, BitShares has 2 dozen. Each LevelDB has its own cache.
This is all old news anyway, soon BitShares will be using less RAM than your web browser.
Oh, no, my web browser use 7 GB of RAM, let's say that "BitShares will be using less RAM than build-in calc" :)
-
This is all old news anyway, soon BitShares will be using less RAM than your web browser.
Really BM, now you mention it? I literally bought 8GB of RAM to be able to run the bitshares client without killing a few programs & swapping madly.
Seriously.
;D
-
This is all old news anyway, soon BitShares will be using less RAM than your web browser.
Don't know why, but I am loving to see things like that :) +5%
-
I'm intensely curious to know what choices have been made here.
Possibilities I thought off include consolidating all the leveldb databases by prefixing the leveldb keys with namespaces.
Or alternatively using a memory-mapped data-structure (btree) implementation.
-
I'm intensely curious to know what choices have been made here.
Possibilities I thought off include consolidating all the leveldb databases by prefixing the leveldb keys with namespaces.
Or alternatively using a memory-mapped data-structure (btree) implementation.
If disk io is the issue then neither of your suggestions would work.
Sent from my iPhone using Tapatalk
-
I'm intensely curious to know what choices have been made here.
Possibilities I thought off include consolidating all the leveldb databases by prefixing the leveldb keys with namespaces.
Or alternatively using a memory-mapped data-structure (btree) implementation.
1 massive level DB database would still have a random access pattern and make every lookup take longer (the cost per lookup grows with log(n) the number of items).
A memory mapped btree still requires disk IO (unless it is all cached). Representing the data in a form that is suitable for storing on disk also slows down access.
We load everything into memory on startup and keep them in boost::multi_index_container which allow both hashed and sorted indexes on multiple fields. So lookup is constant time and updates involve mere pointer manipulation. It takes mere seconds to load everything from disk to memory and then to save it again. For now we are loading and saving to/from a Level DB database, but in the future it will probably just be a set of flat files. Depending upon the type, we can get away with a "stack" based structure in a flat file.
-
When talking about disk IO bottleneck, we always check I and O (reads and writes) separately. We can't avoid IO, we just reduce the frequency, at lease reduce random accessing.
There are still rooms available for performance optimizing with the use of leveldb, E.G.
* The optional FilterPolicy mechanism can be used to reduce the number of disk reads substantially.
* Using less quantity of level DB's will hopefully reduce competing on writes/compacts.
* Better design of keys (of values)
* Compressing and caching
Anyway, if we've already chosen another way, please go ahead.
-
Ugh.
$ ps -xv | grep bitshares
2797 pts/6 Sl+ 18:01 4478 33385 1064798 955000 46.4 bitshares_client --server --httpport 9989
$ ps -xv | grep bitcoind
6659 ? Ssl 112:09 1230 4509 742338 595268 28.9 bitcoind
Both are accepting a maximum of 8 connections + both are not accepting incoming connections*
Bitshares also runs an http daemon (sure, it's technically nothing) but an extra 400~ MB in ram is acceptable (swap is set to 0 in this machine)
* - Don't worry, I have a different machine for seeding Bitcoin and Bitshares. I'm doing my part ;)
--
Oh, if you're interested what the "research" machine's stats are:
$ ps -xv | grep bitshares
2341 pts/1 Sl+ 6:02 24446 34648 2913743 1160992 14.1 bitshares_client --statistics-enabled --server --httpport 9989
$ ps -xv | grep bitcoind
11852 ? SLsl 71:12 5032 4865 2020482 892552 10.8 bitcoind
* - This still isn't the seed machine. Both instances are full indexing nodes. Bitshares also runs the http daemon. Both are connected to 8 peers. No incoming connections.
--
Oh! I forgot to add, bitcoind listens as an RPC server in both instances.
-
1 massive level DB database would still have a random access pattern and make every lookup take longer (the cost per lookup grows with log(n) the number of items).
A memory mapped btree still requires disk IO (unless it is all cached). Representing the data in a form that is suitable for storing on disk also slows down access.
We load everything into memory on startup and keep them in boost::multi_index_container which allow both hashed and sorted indexes on multiple fields. So lookup is constant time and updates involve mere pointer manipulation. It takes mere seconds to load everything from disk to memory and then to save it again. For now we are loading and saving to/from a Level DB database, but in the future it will probably just be a set of flat files. Depending upon the type, we can get away with a "stack" based structure in a flat file.
Thanks. That sounds like a really good simplification and with a great performance payoff. I played with indexing bitcoin transactions using in-memory structures, and found it really quick (eg. 90 mins for utxo set 1.7 GB ram).