When a transaction contains a fill-or-kill order that is not filled, the whole transaction is rolled back. It does not become part of the blockchain and expires eventually. In that case you don't pay any fees, obviously. As if the transaction never existed.