At the moment I believe that there is no point running yam M7j on non-AES-NI configs - I expect it may be slower even than original miner, because of non-AES-NI codepath did not get enough attention from me. Probably for next versions.
As of less memory usage, I am working on that. Major part of the optimization was to reduce data exchange and contention between cores, that's why separate buffer per thread used. Reverting back to old schema is possible in theory, but it is complete rethink of the optimizations made and needs time. You can imagne even this set of optimizations took long time to get implemented.
As of miner behavior under low RAM situations, there is absolutely no special handling for that except for some null pointer checks. At first, applcation performance will degrade due to swapping (if you have one configured - I don't) and if there is not enough RAM even in case of swapping, it will (may) just crash, but will do so at very early stage.
If you are not getting hashrate, then system runs extremely slow due to swapping (=> more RAM or less threads), inter-socket data exchange over QPI (=> numactl or proper affinity setup) or no AES-NI. There is an ART parameter (Average Round Time in milliseconds) that shall be less than 60000 in regular cases. 45000 is typical, less than 30000 is fast.
If you reduced number of threads due to RAM constrains, consider using av=2 or av=3 to gain from non-HT algo variations.
yvg1900
P.S. At the end, it is MemoryCoin
It is supposed to consume A LOT of memory for PoW