If you did just restart the client, then it reindexed the entire blockchain, and you haven't restarted since, then that might be the issue, as reindexing seems to use a lot of memory that might not be released afterwards (or there's not enough memory pressure on your system to reclaim it).
Just make sure you restart at least once after having reindexed and you should see normal memory usage.