Author Topic: Allow market/limit orders by having DEX execution require 2 confirmations?  (Read 3114 times)

0 Members and 1 Guest are viewing this topic.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
I like the consensus this thread is reaching: enabling market/limit orders and protecting against front running by broadcasting encrypted transactions meant for the next couple delegates in the round along with statistical analysis to keep delegates honest.

What are bytemaster's thoughts on this?

Offline Rune

  • Hero Member
  • *****
  • Posts: 1120
    • View Profile
we must implement the market order option, otherwise we will not have luck over competitors...  I am not sure that BM is convinced about that and that is the problem, not the difficulty of the solution...

Sent from my ALCATEL ONE TOUCH 997D

I don't think it's that urgent. The current system still works fine. It's only when we get to very high volume and liquidity that having to match orders individually begins to become annoying.

I think sending transactions to "the next n delegates only" like arhag suggested works great. The amount of delegates you send it to can vary based on your fees.

What do you do if you're at the end of the delegate round? I guess if you want to really be safe you simply wait 1-2 blocks.

It does create the issue that cross-asset transactions will not be detected by the receiver until they're included in a block. So if you send bitEUR to someone who accepts bitUSD then the transaction won't immediately show up in their POS system like it currently does with bitcoin.

Offline liondani

  • Hero Member
  • *****
  • Posts: 3737
  • Inch by inch, play by play
    • View Profile
    • My detailed info
  • BitShares: liondani
  • GitHub: liondani
we must implement the market order option, otherwise we will not have luck over competitors...  I am not sure that BM is convinced about that and that is the problem, not the difficulty of the solution...

Sent from my ALCATEL ONE TOUCH 997D


Offline Markus

  • Sr. Member
  • ****
  • Posts: 366
    • View Profile
Do we really need market order functionality for two linked trades?

How about:
1) The client searches all open limit orders (like the ripple path-finder) and determines the best combination of orders to move from Asset1 -> BitUSD -> Asset2.
2) The client then files a fill-or-kill order combination (one or more limit order(s) Asset1 -> BitUSD plus one or more limit order(s) BitUSD -> Asset2).

This order combination could only be executed in total or not at all and would have a fairly short expiry (maybe one minute) so that it could be altered and reposted if unsuccessful.

Isn't this exactly what we have been discussing? The problem isn't what I call the atomic chained market orders, but rather the existence of limit orders rather than the current WYAIWYG (what-you-ask-is-what-you-get) orders in BitShares. With a naive implementation of limit orders, you open the system up for front running attacks.

Now perhaps you didn't mean to say "limit orders" but rather WYAIWYG orders. In that case it solves the front running issue, but I worry that the fill-or-kill combined order you described is technically challenging to implement. I wouldn't want a chained order that sits in the order book because that complicates the matching logic. I want these chained orders to either fully get executed within a block (if the total conversion cost is less than the specified limit) or not execute at all.

Yes, I meant to say the current "WYAIWYG" type of order (i.e. a two-sided limit), not the one-sided limit orders common on most other exchanges.
If you combine fill-or-kill with immediate-or-cancel (only gets matched with existing orders at least one block old - no sitting in the orderbook for extended time) it should work.

I believe statistics are a very good tool to uncover systematic delegate front-running so I don't see that as a major issue.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
It's not impossible to detect - all you do is some simple statistical analysis on the expected average savings on market orders verses WYAIWYG compared with the practice and do that for each delegate. Outliers should indicate any delegate who is front running.

BTW, I think in transparent forging, only the next delegate in line for block production can see unconfirmed transactions rather than every delegate.

I don't think you should give the plain-text transaction to only the very next delegate because they might miss the block or your transaction may not reach them in time. But there is no need to give it to all 101 active delegates either. Perhaps making it available to the next two block signing delegates would be enough.

Even though either one of those two delegates could have done the front running, since the order is randomized every round, over time the statistical analysis should still point out the single delegate who produces the blocks in which there tends to be mysterious last second transactions that reduce the savings from market orders (relative to a WYAIWYG order set at the price limit) expected based on the state of the order book one block prior.

Offline monsterer

This I like. It didn't occur to me that we could prevent anyone other than delegates from seeing new transactions. The big problem is that if a delegate decides to frontrun, it might never be discovered. You could have a very trusted delegate doing frontrunning the entire time, and it would be pretty much impossible for you to ever discover this as long as they are good enough at hiding it.

However since all delegates have access to the new transactions they might be able to run analysis on each other, and concealed frontrunning would still be too risky for a relatively little reward.

It's not impossible to detect - all you do is some simple statistical analysis on the expected average savings on market orders verses WYAIWYG compared with the practice and do that for each delegate. Outliers should indicate any delegate who is front running.

BTW, I think in transparent forging, only the next delegate in line for block production can see unconfirmed transactions rather than every delegate.
My opinions do not represent those of metaexchange unless explicitly stated.
https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

Offline Rune

  • Hero Member
  • *****
  • Posts: 1120
    • View Profile
You're already trusting the delegates, so why not trust them not to front run the orders?

In that case you could upgrade from WYAIWYG if you could use something like transparent forging from NXT http://www.thenxtwiki.org/wiki/Transparent_Forging so that no one but the block producing delegate can see unconfirmed transactions.

This I like. It didn't occur to me that we could prevent anyone other than delegates from seeing new transactions. The big problem is that if a delegate decides to frontrun, it might never be discovered. You could have a very trusted delegate doing frontrunning the entire time, and it would be pretty much impossible for you to ever discover this as long as they are good enough at hiding it.

However since all delegates have access to the new transactions they might be able to run analysis on each other, and concealed frontrunning would still be too risky for a relatively little reward.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
You're already trusting the delegates, so why not trust them not to front run the orders?

In that case you could upgrade from WYAIWYG if you could use something like transparent forging from NXT http://www.thenxtwiki.org/wiki/Transparent_Forging so that no one but the block producing delegate can see unconfirmed transactions.

The link you provided doesn't explain how to prevent anyone other than the block producers from seeing the unconfirmed transaction. Of course that isn't hard to figure out for DPOS. Just encrypt the transaction before broadcasting with a random secret and include up to 101 (for each of the active delegates) encrypted copies of the secret (each one encrypted with a key only the corresponding active delegate could derive) along with the broadcast data. However, in theory this means that broadcast data can become useless after approximately 16 minutes since in theory all of the delegates in the top 101 can change. But that isn't a big issue since the expiration time of these types of orders should be on the order of minutes anyway (you don't want an old order to be active after the market has significantly changed over time).

This limits the potential set of front running attackers to only the 101 active delegates (practically, it limits it to just the next two or three delegates who will be signing blocks).

I would still personally prefer the two-stage order process for extra protection, at least as an option for the paranoid who are afraid the delegates are giving them a worse deal than they could get. The people who want the best deal on their order can wait a minute. The ones who want their orders to go through in 10 seconds can expose themselves to the risk that the delegates front run their orders.
Edit: Never mind. If the 10 s market orders coexist with the 1 minute one, then there is no front run protection for the 1 minute order, so they become useless. So we would either need to force all market orders to be delayed by a minute, or no one can have front run protection from the delegates.
« Last Edit: December 29, 2014, 09:58:07 pm by arhag »

Offline monsterer

You're already trusting the delegates, so why not trust them not to front run the orders?

In that case you could upgrade from WYAIWYG if you could use something like transparent forging from NXT http://www.thenxtwiki.org/wiki/Transparent_Forging so that no one but the block producing delegate can see unconfirmed transactions.
My opinions do not represent those of metaexchange unless explicitly stated.
https://metaexchange.info | Bitcoin<->Altcoin exchange | Instant | Safe | Low spreads

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Do we really need market order functionality for two linked trades?

How about:
1) The client searches all open limit orders (like the ripple path-finder) and determines the best combination of orders to move from Asset1 -> BitUSD -> Asset2.
2) The client then files a fill-or-kill order combination (one or more limit order(s) Asset1 -> BitUSD plus one or more limit order(s) BitUSD -> Asset2).

This order combination could only be executed in total or not at all and would have a fairly short expiry (maybe one minute) so that it could be altered and reposted if unsuccessful.

Isn't this exactly what we have been discussing? The problem isn't what I call the atomic chained market orders, but rather the existence of limit orders rather than the current WYAIWYG (what-you-ask-is-what-you-get) orders in BitShares. With a naive implementation of limit orders, you open the system up for front running attacks.

Now perhaps you didn't mean to say "limit orders" but rather WYAIWYG orders. In that case it solves the front running issue, but I worry that the fill-or-kill combined order you described is technically challenging to implement. I wouldn't want a chained order that sits in the order book because that complicates the matching logic. I want these chained orders to either fully get executed within a block (if the total conversion cost is less than the specified limit) or not execute at all.
« Last Edit: December 29, 2014, 07:54:14 pm by arhag »

Offline Rune

  • Hero Member
  • *****
  • Posts: 1120
    • View Profile
What do you guys think of this idea? https://bitsharestalk.org/index.php?topic=12748.0

Was based on the discussion here, thinking about how we could improve the liquidity of the DEX.

I actually really love price feeds but I think it would be amazing if we could remove them in a way that keeps the peg as safe as now.

Offline Markus

  • Sr. Member
  • ****
  • Posts: 366
    • View Profile
Do we really need market order functionality for two linked trades?

How about:
1) The client searches all open limit orders (like the ripple path-finder) and determines the best combination of orders to move from Asset1 -> BitUSD -> Asset2.
2) The client then files a fill-or-kill order combination (one or more limit order(s) Asset1 -> BitUSD plus one or more limit order(s) BitUSD -> Asset2).

This order combination could only be executed in total or not at all and would have a fairly short expiry (maybe one minute) so that it could be altered and reposted if unsuccessful.

Offline Rune

  • Hero Member
  • *****
  • Posts: 1120
    • View Profile
The problem with the second solution is that trades have to be faster than 30-50 seconds, I think. At 10 seconds we have a really sweet spot IMO, it's definitely fast enough for traders to take seriously. It's unfortunatey there isn't an obviously elegant solution to this. Maybe creating bids should be free from extra fees to create more bid side liquidity. If liquidity was in huge walls then having the WYSIWYG bid type isn't too bad, even for cross transfers (they won't be atomically linked, but close if there are obvious walls). Maybe they can even still be paid yield in order to incentivize providing liquidity. or even all yield could go as a payment to market makers. BTS shares shouldn't be allowed to vote while they are bids, though, to ensure they can't be suddenly changed in an attack. Cancelling and buying into an order should cost a fee, to prevent spam.

Offline arhag

  • Hero Member
  • *****
  • Posts: 1214
    • View Profile
    • My posts on Steem
  • BitShares: arhag
  • GitHub: arhag
Unfortunately we can't do the same with bitshares because we have to use a special type of WYSIWYG order on our DEX in order to prevent frontrunning by rogue delegates or relay nodes. Someone suggested that DEX orders could execute after 2 confirmations instead of executing in the first block they are included in. Then a hash of the prior block can be used to order them randomly, ensuring that frontrunning won't automatically be profitable. This would make exchange transactions take 20 seconds, but would also allow for the highly desirable automated IOU -> bitUSD -> IOU transactions that ripple can do.

Yes, this is one of the strategies I discussed in this post:
Assuming we don't want to give up on limit orders to prevent front running (as BitShares currently does), there are a few solutions to this problem. One is to just include these market orders in one ledger close but only act on them in the very next ledger close. The order in which the market orders of the previous ledger are processed depends on the ledger hash of the previous closed ledger, for example lexicographical ordering based on hash(hash(transaction) || hash(previous ledger)). Since no one can know for sure what that hash of the closed ledger will be until the ledger has actually closed, no one can know for sure what order the submitted market orders will be processed, and thus there is a chance that the front running will not lead to guaranteed profits.

But as I mentioned it is still not great because the front runner knows the transaction to construct that let's them win 50% of the time. To defend against front running better, we could set the system up so that it is too late for the anyone else to do anything to front run by the time they get the information needed to construct the front running transaction. I discussed that here:
I wonder whether or not the "you only get the price you ask for" model is the best way to fight against front running. Why not just have exchange orders (other than cancelling existing orders or covering shorts) be a two stage transaction? The first stage requires submitting a commitment of your order to the blockchain which expires in 50 seconds but requires you to wait 30 seconds before the second stage is valid. The second stage requires submitting the actual order within the appropriate time interval. By the time someone sees the actual order, submits a new commitment, and gets the second stage of their front running order accepted into the blockchain, the original order has either already been accepted (and potentially matched) or it has not and has in fact become invalid. If it did become invalid, the original order submitter then has the ability to reevaluate the order book and decide whether to resubmit the order or modify it. In that case, the person attempting to front run will not necessarily get guaranteed profits.


I would like to see BitShares support limit orders and market orders while still defending against front running. With that in place we can then support the atomic chained market orders (explained at the end of this post) that allow one to atomically pay in one asset with another asset (like Ripple).
« Last Edit: December 27, 2014, 08:52:51 pm by arhag »

Offline Rune

  • Hero Member
  • *****
  • Posts: 1120
    • View Profile
One of the things that really impressed me from the ripple discussions was how awesome the ripple path-finding algorithm is, as it basically allows you to make transactions with input of any currency and output of any other currency. Then the rippleDEX automatically does all the conversions in a single transaction ensuring you get the cheapest exchange rate possible, and can send USD that seamlessly arrives as JPY just 5 seconds later.

Unfortunately we can't do the same with bitshares because we have to use a special type of WYSIWYG order on our DEX in order to prevent frontrunning by rogue delegates or relay nodes. Someone suggested that DEX orders could execute after 2 confirmations instead of executing in the first block they are included in. Then a hash of the prior block can be used to order them randomly, ensuring that frontrunning won't automatically be profitable. This would make exchange transactions take 20 seconds, but would also allow for the highly desirable automated IOU -> bitUSD -> IOU transactions that ripple can do.

If this isn't optimal I was wondering if it's simply possible perform frontrunning analysis of delegates to ensure we only vote for honest delegates and pass transactions to honest relay nodes? I can't really figure out how this should be done in practice, but maybe it will overlap with the web of trust (Passing transactions to highly trusted nodes reduces risk of front running)?

I'm really excited by all the possibilities of the bitDEX no matter what we end up going with!