I guess it all comes down to two things, how fast of blocks you need for a DEX, and how many confirms retailers will require for transactions. Bologniex requires 10 for a deposit right now, I think. If all retailers required 10 confirms, anything over 3 second blocks would not be very feasible for physical, in-store use at something like a grocery store. If they only require 1 or 2, that part doesn't matter too much and you're back to the DEX issue.
I'm not sure how fast of blocks you need on a DEX for centralized exchanges to not take all their business away, but I'm feeling like the upper limit might be something like 5 seconds.