I think you're mixing up two things here.
One thing is the issue with margin calls eating up limit orders before they can match the orders sitting on the book. We're investigating that.
The issue with whaleshares you are reporting is unrelated to that. Whaleshares is a UIA, not a smartcoin. Margin calls do not happen there. What I suspect is the problem there is rounding issues. Whaleshares can only be traded in whole units, which means that you will see huge rounding errors when trading only 2 units, for example.
Regarding the margin calls, I have created a unit test that reproduces the issue.
https://github.com/bitshares/bitshares-core/pull/341/commits/aa60533269d7ca5e534bedb8a63ac4742d50164cThe test also demonstrates that the issue appears only in a limited price range, i. e. between the call price of a margin call and the MSSP.
Your suggested fix simply lets the margin call match the order at the price of highest buy in the book. I think that's reasonable. I would still prefer a slightly modified version that uses the MSSP if the highest buy is higher than the MSSP. Resolving margin calls is important to the health of the bitasset, and margin calls only execute up to the MSSP.