Percentage based transfer fee [BSIP10] implemented

[merivercap]

Also for other types of transactions such as premium names & UIA's I assume we can keep the discounts only for lifetime and annual members?

[merivercap]

Thanks @jakub and @abit for the response.  I think @bitcrab is very close to being in favor. 

I just wanted to confirm that we can create the following scenario with BSIP10.  I understand the proposal doesn't determine fees, but I just wanted to get confirmation the modes below are possible if the committee decides:

Mode A: 1 cent per transaction to the network (no incentive for referral program or membership)
Mode B: 1% per transaction, 1 cent minimum fee (example max. $1/$10/$20),  lifetime/annual members: 1 cent
Mode C: flat 20 cents per transaction, lifetime/annual members: 1 cent

In all three modes 1 cent goes to the network for each transaction so it's fair and the network doesn't care which mode anyone selects.  Mode B & C adds an extra fee layer that all goes to referrals.

@abit mentioned this:
"I'd like to emphasize again that fee rates can only be set by the committee, and issuers can only choose among the fee modes.

I'm willing to implement per-asset fee rates, however it would be funded via FBA, so won't come in the soon future."

I assume issuers are dependent on the committee, but eventually issuers should have flexibility to determine fee rates.  Is it too complicated to add that feature in BSIP10?  Do you have a rough estimate of the cost just as an FYI?  Also wouldn't it make sense as just another worker proposal instead of an FBA.  I think FBAs are for income generating features.

Since it seems the committee determines the fee it would be hard to accommodate all of CNY/USD/Eur for mode B.   We have to assume Mode B & C is for US/Eur Smartcoins and Mode A is for CNY Smartcoins.  Privatized Smartcoins, UIA's, FBA's would get to choose any mode, but can only use the fee schedule the committee determines for now.  Is that correct?

[jakub]

What is the timeline for this to be available?

We've made an estimate of 12 weeks after the proposal goes through.
Might be sooner if the work goes smoothly - it's hard to predict those things.

Currently the net vote for this worker is negative, which means the stake holders don't want this feature. So it won't come anytime soon. https://cryptofresh.com/workers

I believe currently it's only bitcrab actively voting against (not neutral but against) and, as his proxy has quite a lot of power (over 80m), we are currently losing the net vote.
http://cryptofresh.com/u/bitcrab

[abit]

Great job @abit!
Same to you @jakub!
Good suggestion on the GUI solution @roadscape!

I like the idea.  I'm strongly leaning towards percentage based fees because:
1) it will lower fees for new users who will probably test out smaller amounts in the beginning
2) might be slightly easier to explain
3) higher potential referral income for businesses and affiliates
4) it shouldn't make a noticeable difference to high risk merchants who accept payments and who subsidize fees.

Question:
-We can switch back to the original fee structure anytime right?

The issuers can switch at anytime.

-What is the timeline for this to be available?

Thx!

See https://bitsharestalk.org/index.php/topic,21230.0.html .
Currently the net vote for this worker is negative, which means the stake holders don't want this feature. So it won't come anytime soon. https://cryptofresh.com/workers

[merivercap]

Great job @abit!
Same to you @jakub!
Good suggestion on the GUI solution @roadscape!

I like the idea.  I'm strongly leaning towards percentage based fees because:
1) it will lower fees for new users who will probably test out smaller amounts in the beginning
2) might be slightly easier to explain
3) higher potential referral income for businesses and affiliates
4) it shouldn't make a noticeable difference to high risk merchants who accept payments and who subsidize fees.

Question:
-We can switch back to the original fee structure anytime right?
-What is the timeline for this to be available?

Thx!

[jakub]

We've made an official worker proposal announcement:
https://bitsharestalk.org/index.php/topic,21230.0.html

If you think percentage-based transfer fees make sense, please support us by your shareholder votes.

The main credit goes to @abit - I'm just trying to do my best to support his skills and extract knowledge from him and make good use of it.

[abit]

Will the recent issue related to an expiring order with 0-fee result in an issue with percentage based transfer fees?

I have no idea.. looks irrelevant.

[xeroc]

Will the recent issue related to an expiring order with 0-fee result in an issue with percentage based transfer fees?

[abit]

@abit ah.. yes, sorry, wrong quote.. I'll try again:

I wanted to offer a simpler approach to the issue of CER changing mid-transaction.

The problem: if you pay fees with a UIA who's CER happens to drop before your tx is processed, then your transaction will be rejected.

To avoid rejection of transactions, we can have the GUI slightly overpay UIA fees.

How much you overpay can be a setting in the wallet. The default could be 1%.

It will prevent this problem 99.9% of the time.

If you want to save a few cents at the risk of rejected transactions, you could set it to 0%.

I like this approach because it doesn't require changes to witness_node or the protocol. (afaik)
You can already overpay in the CLI.. so I think essentially it can be used as a form of 'rejection insurance' as a GUI feature too. Wouldn't this solve it?

Yes, it's a good idea, simple and effective. Thanks!

[roadscape]

@abit ah.. yes, sorry, wrong quote.. I'll try again:

I wanted to offer a simpler approach to the issue of CER changing mid-transaction.

The problem: if you pay fees with a UIA who's CER happens to drop before your tx is processed, then your transaction will be rejected.

To avoid rejection of transactions, we can have the GUI slightly overpay UIA fees.

How much you overpay can be a setting in the wallet. The default could be 1%.

It will prevent this problem 99.9% of the time.

If you want to save a few cents at the risk of rejected transactions, you could set it to 0%.

I like this approach because it doesn't require changes to witness_node or the protocol. (afaik)
You can already overpay in the CLI.. so I think essentially it can be used as a form of 'rejection insurance' as a GUI feature too. Wouldn't this solve it?

[abit]

[KNOWN ISSUES/LIMITATIONS]
* Unable to apply percentage base fee mode to BTS

Please be aware of this^ limitation.

I wish we had some choice regarding this, nevertheless I think percentage-based fee mode for BTS is not crucial - there could be a flat transfer fee for BTS which is kept quite low.

As I said.. the simplest solution would be for the GUI to slightly overpay UIA fees.

How much you overpay can be a setting in the wallet. The default could be 1%.

It will prevent this problem 99.9% of the time.

If you want to save a few cents at the risk of rejected transactions, you could set it to 0%.

@roadscape Maybe you quoted something wrong? "Unable to apply percentage base fee mode to BTS" is because  BTS's issuer is null-account, the committee is unable to change BTS's options. Unless we add a dedicated API for doing that.

[roadscape]

[KNOWN ISSUES/LIMITATIONS]
* Unable to apply percentage base fee mode to BTS

Please be aware of this^ limitation.

I wish we had some choice regarding this, nevertheless I think percentage-based fee mode for BTS is not crucial - there could be a flat transfer fee for BTS which is kept quite low.

As I said.. the simplest solution would be for the GUI to slightly overpay UIA fees.

How much you overpay can be a setting in the wallet. The default could be 1%.

It will prevent this problem 99.9% of the time.

If you want to save a few cents at the risk of rejected transactions, you could set it to 0%.

[abit]

I don't foresee this needing a lot of GUI work, a day at most (a day being 8 hours). As long as there's a flag on the asset that says what type of fee to use, and only transfers are affected, it should be trivial to add a switch to whichever transfer operation type that asset uses.

Fees need a little bit of work but not a whole lot, assuming the percent fee structure is easily available either in the asset object or in global parameters. We estimate fees using the global parameters and the operation type, so that would need to be extended to handle percent fees but that's not hard. And final operation fees are fetched from the witness node so that should not require any work on the GUI.

@svk Imo here is a list of work to be done in GUI:
* add an selection box on the "create asset" page and "update asset" page, so the issuer can set fee mode easily
* show transfer fee mode on asset details page, and maybe somewhere related
* use new operation to do all transfers (although the original operation can still be used on assets with flat mode)
* extend fee estimation to handle percent fees.
* the fee schedule page

Did I miss something?

I think Jakub will provide a document about data structure changes and API changes. Or you can dig into my code (after CNX reviewed)

Thanks for support!

OK.

How is the fee mode switched? Does it use the existing flags and permissions structure?

A new field in "extensions". Please check the examples in OP for reference (haven't been reviewed so far, so don't rely on it right now).
No permission involved so far.
Please let me know if I missed something. Thanks.

@svk more work involved:
* since we changed fee allocation algorithm, need to adjust "Fees and cashback" section of membership page, and maybe description of "Fee Division", and maybe some related pages.

[abit]

According to the code, CER is a member variable of asset_update_operation, so it's required when creating such operation. For whatever reason, it has to be valid. And the asset will be updated according to the operation when the operation executes. It's current behavior/feature.

Maybe we can change the logic here:
* don't force CER to be valid when updating an asset
* if it's valid, change CER of the asset as the operation requested
* if it's invalid, don't change CER of the asset

I'll check the code to figure out whether it's possible to make it this way. Or maybe need input from CNX about why it's designed like this and whether it's proper to change the behavior.

If it's OK to change the behavior, since it's not included in BSIP10, we may need to add it. Wish it's not too complex so that I needn't to charge more for developing this feature.

I understand the above but still I don't see the reason why you need to store the CER value in the committee proposal instead of retrieving its current value directly from the asset whenever it's needed to calculate a transfer fee.

In other words, why the algorithm cannot work like this?
(1) A new transfer for asset X is about to be sent,
(2) Retrieve from the blockchain the current value of CER for asset X,
(3) Use this value to calculate the transfer fee,
(4) Charge the sender with this fee.

Is there some ambiguity about what the value of CER for asset X is at any given point in time?

Well, the committee proposal is to "enable percentage based fee" for a committee-owned asset for example bitUSD. The real operation in the committee proposal is "asset_update_operation". CER is REQUIRED to be included in "asset_update_operation", which is current feature/behavior/limitation. In short, you have to manually set/change CER to enable percentage based fee. I proposed to change this behavior in last post, but I won't make this change unless it's included in BSIP.

Once percentage based fee is enabled for bitUSD, when a transfer is requested by a user, the fee would be calculated according to the algorithm you write above.

Anyway, after manually set CER with asset_update_operation, CER would be corrected maybe after one price feed, so perhaps it's not a big deal to leave the feature unchanged.

OK, I agree that it's not a big deal at this stage.
If it becomes problematic, we can address it in the future.

I pushed a patch to the test branch of my repo, now it's possible to update an asset without changing CER.
OP updated accordingly with a new example at the end.

[abit]

One scenario need to be taken into consideration:
Since CERs of smart coins change frequently, there may be race conditions when a transfer operation and a price feed operation appear at a same block.
Usually a user need some seconds to confirm her transaction, or say, after the "Please confirm the transaction" page is shown, before the user clicks 'OK', if CER changed, the transaction would probably fail due to insufficient fee provided. It will cause negative user experience.

Imo this should be included in BSIP.

I have an idea to solve this issue (sure it will make the whole thing more complicated):
* For smart coins, show a message "you need to confirm in 60 seconds" on the "please confirm" page, if the user failed to confirm in 60 seconds, disable the "OK" button.
* Maintain the "worst" CERs of every smart coins in last 3 minutes somewhere in witness_node. The fee is only required to be not lower than the lowest one.

Thoughts?
@jakub @svk

@abit @svk

Maybe there is a third option:
In case CER changes before the user has a chance to confirm the transfer, just charge the user more (or less) than the amount displayed on the confirmation screen. This way it will only fail in one situation which is quite rare: if the user does not have sufficient funds in their account to cover the unexpected increase in the transfer fee.

This is obviously not perfect (as occasionally there will be a discrepancy between the transfer fee displayed on the confirmation screen and the actual fee charged) but I think it's the best solution available from the UX perspective.
I think people are usually not bothered much about the exact value of the transfer fee as long as it is within some acceptable limits.

Sorry, we're unable to do it. Fee is a field in the transaction/operation which is signed by the user. We can't modify it, nor deduct even one Satoshi more than the amount in the transaction/operation.

Another option would be deduct some fee from the fee pool, if the issuer approves. But it's hard to judge why the fee provided by the user is insufficient -- due to CER change or something else (maybe attack).

Aren't we talking about a scenario where the CER changes after the user initially submits the transaction, but BEFORE they confirm it?  If so, then can't we present them with a new confirmation which includes the updated fee and a message explaining why the fee is slightly higher than expected?

Even if the UI calculate fee again, it still has chance to fail (sure chance is lower).

I think we should consider it seriously. If we can't make it reliable enough, perhaps it's acceptable for a PC GUI user to try again, but it may be terrible for 3rd party applications which transfer tokens automatically, or for a mobile user with a slow internet connection that unable to try again very soon.

In short, try to solve the problem at the witness_node level if can't solve at protocol level.

Another thing, if the committee decided to apply percentage fee to smart coins and/or BTS, we need to notify our partners (for example exchanges) in advance, so that they have time to adjust their applications if the applications aren't compatible with percentage fee.