So you don't really need an escrow, because if you default on your loan it will damage your reputation in the community.
What is to prevent someone from building up a lot of trust over the course of years, and then, just stealing lots of money? I suppose the amounts could never be high enough to make scheme worthwhile?
Nothing; this type of fraud is endemic among immature P2P lending communities.
There are mature P2P lending communities that manage reputation fraud through identity verification, self-insuring and loan structuring: Lending Club, Lending Tree, Zopa etc.
Crypto P2P is trying to emulate the fiat P2P systems with mixed success: BTCJam, BitBond, BitLendingClub (ha!).
P2P lending requires both escrow and reputation to function efficiently.