BlockPay v1.5.4 released. Loyalty Points calculations updated, more connectivity improvements, minor UI improvements:
https://play.google.com/store/apps/details?id=de.bitsharesmunich.blockpay

Quote from: kenCode on February 03, 2017, 07:10:32 pm

[As for raising Witness pay.. I am against it:
 
If there are Witnesses running a profitable business in expensive places like Canada and America, then it becomes very obvious that a Witness node can be run pretty much anywhere else on Earth at a profit too. Hence, no payraise is needed for Witnesses at this time.
 
If your business cannot turn a profit, that should not be the Stakeholder's problem.

You're kidding right? You seriously think the profits returned from a witness constitutes a business?
...
There is little profit.

For some people, running a Witness node is a business, and might even be their sole source of income. I am also trying to introduce people to Bitshares just like you. Referral program, run a Witness node, trading on the DEx, building a profitable company on the platform, etc.
 
If a business' profits are not large enough, then that business must find a way to make that venture more profitable. As for Witness nodes, there are awesome server providers in Slovakia, Poland, Brasil, Mexico and everywhere in between, so just shop around a bit. Make the node more profitable.
 
Paying our Witness node operators more, will not make them any more productive or efficient.
Please don't make their profitability issues the Bitshares Stakeholder's problem, that is not fair.

Tons of updates this week!
https://github.com/kenCode-de?tab=repositories
 
BlockPay related:
 
This week's work was mainly concentrated around one main task: which was to enable the sending of the final Loyalty Points to the customer. This feature is only available for mobile wallets that support the graphenej library (https://github.com/kenCode-de/graphenej), because from all other forms of payment (like the Jaxx wallet) we can't assess who the customer is so that we can send them any Loyalty Points (hint hint!).
 
Anyways, in order for us to be able to detect the customer identity, it was required to expand the incoming funds detection mechanism. As it was being done, the app would just look for objects of type "account_balance_object" (Object id 2.5.x) from the subscription feed. But this object of course, only informs us about the value update of a given user's balance. No meaningful information about the origin of those funds is provided. To obtain this information, it was required that we looked also for "transaction_object" (Object id 2.7.x).
 
While introducing support for this additional object, it became clear that we needed a way to perform a sort of selective parsing of the data coming from the subscription feed.
 
The idea here is that because the stream of subscription objects contain a whole range of different objects, many of which we might not be interested in, instead of wasting CPU resources parsing ALL of them, it would be better to just look at their specific object ids, and then decide whether or not to perform a full object deserialization. This way we would basically be performing a very minimal deserialization on all objects, and then decide whether or not perform a full deserialization depending on whether we have at least one listener registered.
 
Most of this code was added to our graphenej library and introduced into the BlockPay app. And the work regarding this specific feature can be found here:
https://github.com/kenCode-de/blockpay-s/commit/da95c58ea3338dbe1e02fc0f97ad9abb096dc692
 
After successfully detecting incoming transactions and identifying the customer's bts account, some calculations are needed in order to decide how much (n) of the selected Loyalty Points tokens that we need to send to the customer. Most of the code for this feature was already present, but some corner cases had to be taken into consideration. For example, resetting the Loyalty Points "ratio" (ie: send n OBITS to customer after they have spent at least $n) back to zero, if the merchant decides to change his desired Loyalty Points token. This avoids the possibility of sending the wrong amount of Loyalty Points tokens to the customer if the merchant was to make that change in the settings screen.
 
Finally, the graphenej transaction broadcasting mechanism was used in place of the old one that relied on server components. Bandwidth can be very expensive, so less trips to the server is a good thing. This work can be found here:
https://github.com/kenCode-de/blockpay-s/commit/f9488b30bad2b8536be0a3fd3fc9d7cea54e78d5
 
Also there was still some Loyalty Points UIA list-duplication issues that were solved here:
https://github.com/kenCode-de/blockpay-s/commit/cb127778bca4e81af14f96dad289d5378acece85
 
BTS was added as an entry to the "Desired Smartcoin" list here:
https://github.com/kenCode-de/blockpay-s/commit/923c257475030b095ced564f3f7f99ead623c492
 
A bug that displayed unsupported currencies was fixed here:
https://github.com/kenCode-de/blockpay-s/commit/11e9ff0912dc7760f01d3cf2666b2d1670fb362e
 
Also the initial work to add support for all fiat currencies that currently have an associated Smartcoin was started, but since it was not yet finished, I will publish its commits here next week.
 
Once the liquidity issues are solved and the Bridge fully supports it, we will be adding support for additional Smartcoins such as bitGBP, bitGold, bitSilver, bitMXN, etc soon too.
 
C-IPFS related:
 
C-IPFS is extremely important because it will serve as a Core for most of our apps, a more secure way of acquiring app assets, faster (and even offline) app asset downloading, as well as an automation mechanism for secure data backups. We made tons of great improvements this week that will see daylight by end of next week. We are now connecting to remote IPFS swarm peers, and beginning encryption negotiations. This step is crucial that it be right, as it simply won't work otherwise.
 
We found a few hiccups in a past merge of c-ipfs and c-libp2p so this has been fixed in this latest commit:
https://github.com/kenCode-de/c-ipfs/commit/be4bee3119e4746aa68064b9de73b2a0533450ab
 
We began the buildout of secio, which is the encrypted way that ipfs swarm peers talk to each other:
https://github.com/kenCode-de/c-libp2p/commit/6d9a9e0e70377f1a74515b2ac807cdb504eb32bf
https://github.com/kenCode-de/c-libp2p/commit/513b778561cf773eb541ecf2d95c1a43acaf01eb
https://github.com/kenCode-de/c-libp2p/commit/29e1a0c31b3a78a82058226b080f777dd6e3ed70
 
To make secio work, you first have to connect with multistream. So here is the multistream implementation:
https://github.com/kenCode-de/c-libp2p/commit/d091a29b193a654f3b18e44da7b11139f3b4fa26
https://github.com/kenCode-de/c-libp2p/commit/6b24f0685503649e011c70f1950b998a2210efbb
 
There were a few longstanding incompatibilities with the GO version and C version, due to the way the peer ID was generated. Some adjustments were made and now the C version handles keys just like the GO version:
https://github.com/kenCode-de/c-libp2p/commit/94566ade6989ad144e34072249a4dc64c104eaf7
https://github.com/kenCode-de/c-libp2p/commit/5666a8a2ef8cbd85911f063681cee35444f81307
https://github.com/kenCode-de/c-libp2p/commit/6d5f7410c6c55f4d131500bc2179bfcb07584b89
 
More c-ipfs and c-libp2p commits:
https://github.com/kenCode-de/c-ipfs/commit/2431aba246d52604ff8dce9848868cce910396db
https://github.com/kenCode-de/c-ipfs/commit/15352732596e6098b2c9d7adf6966d61efd677b5
 
Implemented off-line (incomplete, need to save in local db) routing, is the most basic routing protocol, it doesn't need to communicate with other nodes, but proves that just as in the Go implementation the same calls in other modules can be made without knowing which protocol the structure has, will also be used as a skeleton for all other protocols, and we will be able to use multiple protocols or just those that are loaded.
 
Next week we will finish up with the c-ipfs encryption and p2p connectivity features. The first formal public Release of C-IPFS will then be posted here:
https://github.com/kenCode-de/c-ipfs/releases
 
Android Smartcoins Wallet:
 
Smartcoins Wallet v1.5.8 was released today:
https://play.google.com/store/apps/details?id=de.bitsharesmunich.smartcoinswallet

• This version includes UI fixes, tighter integration with the new graphenej library and better performance.
• Added support to pt-BR and pt-PT as distinct versions of pt which involved not only change the xml strings, but also a similar treatment the app already does for Chinese variations at SettingActivity and Helper class.
• Changed the PIN dialog to be auto-submitted.
• Previous PIN was hardcoded to 6 digits. It is now 6 to "unlimited" digits. Next week we are also adding a 10sec timer, so that if you switch apps or something you should not have to retype your PIN again. I found that kind of annoying.
• Currently all dialogs of the app needed a refactor, because it is important to do dialog dismiss when the Activity they belong to pause or stop. Otherwise this may cause a memory leak. So we have to keep track of which dialog is open when the app is closed for example, dismiss the dialogs, and show them again, if they were visible before the termination. Memory leaks suck, so we wanted to add this support before that could ever happen.
• Minor Changes: updated EULA, fixed some variables syntax, updated app version to v1.5.8, updated the 3min "Auto-close" feature so that it is enabled by default (in v1.5.9), updated LTM and "Delete Contact" dialogs to remove weird whitespace, and also updated/fixed some language translations (english and portuguese)

Known issues: I still have to clean up the eReceipts layout, and make the Transactions display work in real-time again. Will fix those things over the next couple days and release another update as v1.5.9. As always, never keep more funds in a wallet than you can afford to lose.
 
The relevant commits and open source for this work are here:
https://github.com/kenCode-de/smartcoins-wallet
 
Stealth related:
 
Libsnark integration (for our new "trustless" setup algo) nearing completion:
https://github.com/kenCode-de/graphene/commit/efea55d15e0b6a5adbc3fe3764cba024c10d99c9
https://github.com/kenCode-de/libsnark/commit/69c380bfe7cc183b7e7eba580bb962dc3e2f90ee
 
Gadget and zero-knowledge proof fixes:
https://github.com/kenCode-de/graphene/commit/82166971f909860fd2adbc198280de42fb61ec90
 
By tuesday of this coming week we will begin the libsnark integration with bitshares-core (graphene) and then we can start the connections of the api to the UI work we have been doing. Stealth transactions are gonna really blow the doors off of the competition. A Bitshares Stealth transaction means "unknown" sent n "unknown" to "unknown". True privacy.
 
Alfredo Garcia, Bitshares Core Dev update:
 
Alfredo has been kicking butt on the github issues, adding the new streamlined asset count, and holders code, fixing the indentation issues with ElMato and nathan, and starting on the "websocket spamming too much data" issue that even bytemaster wanted to fix months ago. That old issue should be fixed by end of next week as well. That issue is a big one too, because if you ever want to use a graphene app over mobile, you could be stuck with consuming around 200K per block and bandwidth is very expensive so this one is marked as High Priority.
 
https://github.com/bitshares/bitshares-core/pull/226
https://github.com/bitshares/bitshares-core/pull/229
PR 229 solves issue 101: https://github.com/bitshares/bitshares-core/issues/101
 
Chris4210 has updated the roadmap for Alfredo too so that I can keep his mountain of work rolling and never stops. I see lots of great stuff coming to Bitshares again.

BlockPay v1.5.2 released. More "Desired Smartcoin" choices being added, UI updates, Loyalty Points upgraded to the new graphenej library, more speed improvements. This version also includes all of the updates from our recent security audit. Since most of the underlying code has been upgraded now, it kind of messed up the UI a bit, so we are making it look nicer over the next few days and will release that update as v1.5.3.
https://play.google.com/store/apps/details?id=de.bitsharesmunich.blockpay

Tons of updates this week!
https://github.com/kenCode-de?tab=repositories
 
BlockPay related:
 
BlockPay v1.5.1 has finally been published on the google play store, just got a bunch of UI work we want to do to it now and some other minor things that still need the graphenej integration (Loyalty Points for instance). Once Alfredo's latest graphene updates get committed by nathan, then BlockPay will run much faster and initial setup sync's up a lot faster then too.
 
We have totally removed all reliance on the cryptofresh api and wrote our own api's. I am not a fan of our apps having to rely on third-parties. Our recent security audits played a big part in this and have now all been upgraded successfully.
 
Today we are moving the Loyalty Points feature onto our new graphenej library, tomorrow we are adding direct DEx support, Sunday and Monday we are cleaning up the UI of the eReceipts, Tuesday and Wednesday we are moving the Transactions code onto graphenej as well and cleaning up the UI there, on Thursday we will start the redesign of the settings screen and initial app setup process so that merchant's can understand it better. As you add features to software, it can turn the UI into a cluster fuck so this redesign is imperative IMO.
 
I will try to upload BlockPay v1.5.2 for you guys by tomorrow (saturday) with some of the stuff mentioned above, especially UI work and the Loyalty Points feature.
 
C-IPFS related:
 
We're adding some features to the c-libp2p repo, and c-libp2p-routing at moment:
https://github.com/kenCode-de/c-libp2p/commit/ee6049804a86cf7f2f85bb0b96cb4e046e01328e
https://github.com/kenCode-de/c-libp2p/commit/6a30c46492101b164e91c8ff1f1ea923bc2135a1
https://github.com/kenCode-de/c-libp2p/commit/f2031179a962495836c453c02593dc2e924dca9e
https://github.com/kenCode-de/c-libp2p/commit/0ca303bb1c11695b6d965fac9a13723072309a4c
 
We also added code for message signing:
https://github.com/kenCode-de/c-libp2p/commit/9a7c494436c486498a8fc520cbe13f3a361f9e6c
https://github.com/kenCode-de/c-libp2p/commit/d13a47d7d5c997892ebf34b148a905824119addd
 
Need to complete the Kademlia DHT port, IPNS Publish feature, and implement Swarm so that we can implement the daemon for the Linux and Mac public Release which will be published as v1.0.0 in the next week or so here:
https://github.com/kenCode-de/c-ipfs/releases
 
Android Smartcoins Wallet:
 
All security audits have been completed and integrated successfully. Our new graphenej lib can be audited here too if you like:
https://github.com/kenCode-de/graphenej
 
This week we are doing more UI work, adding direct DEx support into graphenej, and working on the additional chain support.
 
Stealth related:
 
More testing completed of our new "trustless" algo, will reveal more of that for you guys and on our testnet soon. Proof and verification implementation,  libsnark gadgets for notes, commitments and joinsplit transaction work:
https://github.com/kenCode-de/graphene/commit/3159c26ee3511788ee0ffd5e1e2a6246dd18232f
 
Updated to the new bitshares-ui repo, added underlying functionality for wallets to be able to keep records of Stealth accounts. Not tested yet due to a bug in their latest bitshares-ui which doesn't allow account creation from LTM accounts.
 
Github work: https://github.com/kenCode-de/graphene-ui/tree/graphene-ui-dev-v3-BITSHARES_UPDATE
 
UI work has resumed and we will have more screenshots for you soon. Tonight we will begin the initial connections of the UI to graphene.
 
Alfredo Garcia, Bitshares Core Dev update:
 
As ordered, Alfredo began work on Monday, January 23rd. I got him rolling on outstanding github issues and graphene additions already:
 
1) Install and test asset api to the new bitshares-core (https://github.com/bitshares/bitshares-core/) project from the old cryptonomex (https://github.com/cryptonomex/graphene/) repo. They have differences that were addressed to make the asset api work and the first function get_asset_holders_count.
Pull request: https://github.com/bitshares/bitshares-core/pull/220
 
2) add new function get_all_asset_holders that returns a vector of assets_ids:holder count.
Commit: https://github.com/bitshares/bitshares-core/pull/220/commits/0316affea7219da51cae49d27a92d1f133f63834
 
3) corrected indents as per nathan request, changed counter loop function to use distance for efficiency.
Commits:
https://github.com/bitshares/bitshares-core/pull/220/commits/6a0f41ffc2be9fc1d9824cf98c17f4629484b534
https://github.com/bitshares/bitshares-core/pull/220/commits/b50bba13c8dddbe7879b7b5888341bcc1b7a506e
https://github.com/bitshares/bitshares-core/pull/220/commits/b6fbc97d3a172522533356b31c9e595e85116cf2
 
4) github issue: https://github.com/bitshares/bitshares-core/issues/105
Created a public function that does the same as get_account_history_public but can be called without authentication.
Commit: https://github.com/bitshares/bitshares-core/commit/7872a2f51b8fbeda0db5b6df737c78be764ee658
 
At least 3 more github Issues will be fixed and PR'd early next week along with updated bitshares-core (graphene) code which will fix the "websocket spamming too much data" issue once and for all. There have been quite a few requests for a mobile-friendly version of our DEx and better bandwidth utilization for all mobile clients and wallets. Right now, each block consumes about 200K of your data plan, so once Alfredo fixes that too, we no longer have to worry about data consumption.
 
Keep an eye on the google play store, I will post quite a few more updates for our apps this week.

No, it's two different use cases and UI's entirely. Echo will use all of the core components that we have been building, but also include voice, video, chat and things that the Smartcoins Wallet just doesn't need.

Tons of updates this week!
https://github.com/kenCode-de?tab=repositories
 
C-IPFS and BlockPay related:
 
We are just finishing up the c-libp2p and c-ipfs-routing components this week (eliminating reliance on http, ssl and dns is no easy task) and will have the first formal Release of C-IPFS for Linux and Mac published here by friday, january 27th:
https://github.com/kenCode-de/c-ipfs/releases
 
So, with Storage and Routing completed and the first formal public Release of C-IPFS by end of next week, we can finally begin on the "2.0" architecture for BlockPay 2.0 as well as the Smartcoins Wallet 2.0 by end of next week. The mobile wallet will get a new brand too, still tossing around different name ideas... Smarty, Cirrus, Toke, Chainz, ..if you have ideas for the rebranding of Smartcoins Wallet, let me know. It will support 7 different chains and their tokens/assets. It also has a new material design, better UI/UX. Anyway, let me know..
 
Routing commits:
https://github.com/kenCode-de/c-ipfs/commit/7d3418e9c715e70d86c81452639a5ef239a22f31
https://github.com/kenCode-de/c-libp2p/commit/3b301f823ad3993d25d92fdd3905bdb9b2409487
https://github.com/kenCode-de/c-libp2p/commit/e3fc5f640953ef23eb28ccffd086c05e2e7947c2
 
Since all of the Core components for 2.0 are near completion now, the upgrade procedure will not be that difficult, since the other libraries have already been published for you as well (like graphenej). Wallets developers should love that lib, it makes it insanely easy to make many different types of graphene-connected mobile wallets now for iOS, Android, Windows Phones, etc. I would love to see Jaxx add support for the Graphene chain now.....
 
Android Smartcoins Wallet and security audits:
 
Security audits have been completed and incorporated into our new graphenej library:
https://github.com/kenCode-de/graphenej
 
Smartcoins Wallet has been upgraded to the new architecture and has been published to google play. Lots of UI fixes (especially in the Transactions and Balances display) to come this week and will be published by end of next week as well.
 
During this process, we are also upgrading the architecture of BlockPay so that it runs a hell of a lot faster and stronger too. I have a debug apk you can play with if you like, but only bitUSD has been done so far. In the next few days I will publish it to google play once we have all the smartcoins working with this new lib (with direct DEx support in graphenej).
 
Upgrading the architecture of BlockPay to graphenej:
A huge waste of user's data transfer and battery usage was detected and fixed. This happened because even after the user leaves the app, the android system keeps its process running and its activities and resources are not automatically discarded, but instead kept around as the user might come back. The websocket connection to the witness node in this case was left open and receiving large amounts of data. This is also an issue that affected the Smartcoins Wallet so we fixed that as well.
 
Another outstanding issue with both apps is that once subscribed to events from the witness node, all sorts of events get broadcasted, essentially spamming the apps with unwanted notifications. This is is why this fix however important, is not reducing all the unused data transfer, just the unused data transfer that used to take place after the user left the app. Since apps can keep running in background for an unlimited amount of time, this was an important reduction of something that otherwise might have turned into an unbounded amount of traffic.
 
Of course this is not fixing the remaining issue yet with the unwanted notifications while the user is actively using the app, something that might be even more pronounced in the Smartcoins Wallet, since it might be consuming the limited 3G data transfer quota of the user. But in order to fully solve this problem, a modification in the witness code is required (it was not fully fixed in this old Issue: https://github.com/cryptonomex/graphene/issues/540).
 
Some of the relevant work for this Witness code update can be found at this commit: 3b081c428ec406de46e0f568484d8afeb91dd4d3
 
A new table called "balances" is used to keep track of the user's balances for every asset. The table is also dynamically updated from the notifications received via the websocket connection. Commit: 2f63eeb1d4b2cc417d1e81bfe00ba9cf6f90cdc7
 
The assets table has been updated in order to also track the information about the number of holders that each asset has. This is then used to filter the list of UIA and smartcoins, which was featuring a veeeery long list, including many inactive assets. The holders information is obtained from the cryptofresh API, but we now plan to request it directly from the witness node (commit: 5cd0c65d676274b51a88c37001700c8bbb6a5942) so that our software ultimately no longer relies on third parties.
 
A new table called "bridge_rates" is now used to keep track of not only the transfer rates between any two cryptos that the Bridge offers support for, but also the supported trading pairs. Commit: 706231122c0d2f0fba0938db20e0b8c40913962d
 
Aditional support for smartcoins and bitcoin input has been extended to all other cryptos supported by the bridge. The conversions can then be classified into 3 distinct categories:

• Bitshares assets (no need for a Bridge)
• Bridge-supported using a single conversion
• Bridge-supported using a 2-step conversion

Up till last week, we already had support for conversions 1 and 2. The third case is required because for some cryptocurrencies, there are no direct trading pairs with the user's desired smartcoin. For these cases, which amount to the large majority of pairs, we need to ask for an initial conversion to BTS, and only then convert that BTS into the desired smartcoin.
 
The ideal solution would be to just use the DEx for this second conversion, but since the graphenej library is currently lacking support for trade operations, we decided to just use the bridge a second time for now. In the short term however, the direct DEx support will be a priority. This work can be mostly found here: 66125491fb2eec6fb2df337666f8735b07512198
 
After a conversion has been initiated, the expected balance change is calculated and the app signals the success of the transaction only after the desired balance change has been detected. This verification was not being done before and thus it was very easy to scam the merchant by sending any amount of BTS. This has been fixed here: ec8e2d681ed3eeda4d481e68d07a39f6160fa347
 
A number of other bug fixes and small improvements have also been performed on the apps. An outstanding issue was also detected with the bitEUR conversion. Since the Blocktrades bridge doesn't support direct trading between BTS and bitEUR, the 2-step conversion strategy described above is not enough to obtain bitEUR. So for now the app is not supporting bitEUR as a final smartcoin because of this bridge limitation. This is another important reason to add the direct DEx support this coming week.
 
Stealth related:
 
Regarding our new "trustless" setup algo, the original zcash multiparty trust setup was written in Rust. So, we are converting the Rust to C++ for direct graphene integration. The core functionality of this algo will be done in just a few more days, and then the interfaces and objects can be finished up this coming week for all of the UI connections. I also want this packaged up so that other graphene-based wallets can easily take advantage of Stealth, so modularity has been a high priority.
 
Automated Stealth transaction backups to c-ipfs can be started by end of next week too since the first formal Release of c-ipfs will be published then:
https://github.com/kenCode-de/c-ipfs/releases
 
Additional work being performed:
Code cleanup, join split implementation, verification, proofs, libsnark integration, commits:
https://github.com/kenCode-de/graphene/commit/849c808ebabebd388641daf8abf75d7b37b78355
https://github.com/kenCode-de/graphene/commit/0983477201821f3856871e09202eb9f8130f4d5c
https://github.com/kenCode-de/graphene/commit/0e70a69c87364107fb51818fb716fc00565097c5
 
Graphene-UI work can now resume on friday, january 27th so that Stealth actually looks nice too. A simple, logical UI/UX for this is imperative.
 
Alfredo Garcia, Bitshares Core Dev update:
 
As I stated in the Worker Proposal, I will be posting updates on his work here each week too.
He is voted in now and his 6-month job technically starts on Monday, January 23rd.
 
I already got him rolling though on "get_asset_holders_count" and "get_all_assets_holder_count". This is a bolt-on for @ElMato (Matias') asset api. It takes as parameter an asset id just as "get_asset_holders" original function created by him but it returns just an integer with the number of asset holders.
 
So, using it with wscat: > {"id":1, "method":"call", "params":[2,"get_asset_holders_count",["1.3.0"]]}
< {"id":1,"result":0}
> {"id":1, "method":"call", "params":[2,"get_asset_holders_count",["1.3.0"]]}
< {"id":1,"result":1}
> {"id":1, "method":"call", "params":[2,"get_asset_holders_count",["1.3.0"]]}
< {"id":1,"result":2}

The first call returns 0 and there was no user created in the node, we had an additional window running with the client wallet. The second call is with user nathan created as the cookbook (https://github.com/cryptonomex/graphene/wiki/CLI-Wallet-Cookbook) returns 1. The third call is with user nathan and following the cookbook there is an additional user called "my-account" where we transfer some of the nathan funds. This call returns 2. Nice and clean, minimal bytes transmitted.
 
By the way, the 7 chains that the mobile wallet will support will require me to add one more expert, so I'm getting him up to speed this week too. Our 2.0 apps are gonna seriously kick ass.