1. Is there an option to cancel order?
In the UI, click on "Account" (tab at the top), then on "Open orders" (navbar on the left side). This should show your open orders, with a button to cancel each one.
2. When i exchange my BTS to BTC, where will the BTC will go to, i mean what wallet do i need to download?
When you trade on the DEX, whatever you buy stays in your bitshares wallet. If you buy OPEN.BTC, you can withdraw them to a bitcoin address of your choice.
3, What's bitUSD in the exchange?
It's a SmartCoin that is loosely pegged to its fiat counterpart. There are other SmartCoins as well, for example bitBTC tracks the value of actual BTC.
4. Is there already a fee for cancelled order?
Cancelling an order costs a very small fee (sub-cent?). If you cancel an order that has not been partially filled, the fee for placing that order will be returned to you, so effectively cancelling an unfilled order will return 90% of the original fee.