Worker Proposal - Alfredo Garcia - Core Developer for the second half of 2017

[oxarbitrage]

https://github.com/bitshares/bitshares-core/tree/develop

if you check the commits: https://github.com/bitshares/bitshares-core/commits/develop the last commit in master is from July 10(on page 4 currently). everything from there up to today is what will be included into the next bitshares-core release.

[Brekyrself]

Where is the develop branch located on github so we can see the commits etc..?

Thanks again for your hard work making BTS better!

[oxarbitrage]

Thanks for the update. 

-Any idea how 3rd parties are notified about changes such as issue 411/412?  This could be a reason why Bittrex delisted BTS?
-Will you come up with a release schedule such as bi-weekly or monthly to provide new versions with all these fixes?

For the first point the fix will be included in the next release @pc is preparing among all the other fixes/changes that were done since the last release. There is a witnesses telegram channel, i am not member of it as i am not a witness but pretty sure the p2p issue is being discussed there. I honestly have no idea at this moment why bittrex delisted bts, not even know if it was due a technical aspect or something else.

In regards to a release schedule it actually depends on how much code was inserted and what is the importance of changes to make a release. In the case of the current release we are working on, it toke a lot more time than preferred but the reason for that is that there is a hardfork. Releases with hardfork are harder to implement than releases without it.
Please note that in the past we were merging all new code directly to master branch, this changed and now all the new code go into a develop branch, when the develop branch haves enough or significant changes they get merged to master with a release.
This new extra step is better for all as the develop branch haves more time to be tested and less bugs will be introduced to master but it haves the problem of giving an impression to outsiders that no commit was done since the last release long time ago. The extra step of the develop branch is here to stay but merges to master should be ideally more often(like around 1 a month). Exact estimates on when a new release will be out is not a very good strategy, if there are not enough changes there is no point on making a release just to be on a expected timeframe.

Hope it helps clarify something

[Brekyrself]

Thanks for the update. 

-Any idea how 3rd parties are notified about changes such as issue 411/412?  This could be a reason why Bittrex delisted BTS?
-Will you come up with a release schedule such as bi-weekly or monthly to provide new versions with all these fixes?

[oxarbitrage]

I am probably missing stuff, need to go back to reports every 1 week instead of 2!

bitshares-core:

- https://github.com/bitshares/bitshares-core/issues/411 - @dnotestein pointed us to a few commits made to the steem p2p code that will improve the operation of bitshares node. @xeroc made a cherrypick of the steem code(https://github.com/bitshares/bitshares-core/pull/412) that saved a lot of time than making this changes manually. Tested a full resync and the code worked fine so the solution was marged and the issue closed.

- https://github.com/bitshares/bitshares-core/pull/406 - code was made to fix issue https://github.com/bitshares/bitshares-core/issues/265 - need testing before merging and issue closing.

- https://github.com/bitshares/bitshares-core/pull/415 - We want to reduce ram in new installations without interfacing with existing business. @xeroc made some changes, i reviewed and approved but then @pc found some problems so we reverted back until this is fixed.

- @pc found some problem with a unit test of account_history when making the last testing for the next release. We fixed the issue here: https://github.com/bitshares/bitshares-core/pull/420 - merged.

- I started to work on a plugin that will allow to reduce full node ram usage by storing the history of accounts in the database(elasticsearch). At the same time it will solve several relational issues in searching in the account history in an efficient way. This is a big and time consuming task, more updates in this post on the next report.

explorer:

explorer is going slow but good, as cryptofresh has been working fine after the server change i dont have the preassure of being the only working explorer so i am focusing on having more stats suggested by the community i nthe form of issues.

- almost all the changes requested by @tbone were made to the assets page. part of the discussion can be found in the assets page issue at : http://bitshares-explorer.io/#/assets
closing the issue by now.

- https://github.com/oxarbitrage/bitshares-explorer/issues/11 changes were made to the individual asset page as: http://bitshares-explorer.io/#/assets/1.3.1152 closed the issue.

- https://github.com/oxarbitrage/bitshares-explorer/issues/7 fixed as part of the assets page work. closed.

- https://github.com/oxarbitrage/bitshares-explorer/issues/9 working on account page issues, not moved to production yet but can see progress in the develop version: http://bitshares-explorer.io:9009/#/accounts/1.2.121

[oxarbitrage]

Adding a report now with the work from the last 1 week and a half. I am posting it today as i am going out for a small trip tomorrow and i'll be back Tuesday next week. I am going to Buenos Aires for family issues but also going to meet @ElMato and i have the luck that this Saturday there is an EOS event, ill be there

bitshares-core:

- https://github.com/bitshares/bitshares-core/issues/108 -> closing UI related issue after asking @svk and @wmbutler , old issue, not a problem anymore.
- https://github.com/bitshares/bitshares-core/issues/403 -> assigned myself the issue, made some initial research but the bulk of the work will be done next week to change index.
- https://github.com/bitshares/bitshares-core/issues/75 -> located documentation confirming the operation will delete and not refuse to sign, also found line where proposal is deleted from the blockchain. @abit mentioned another issue with the implementation but will be discussed in  new issue, closing this one.
- https://github.com/bitshares/bitshares-core/issues/409 -> helping with build issues in ubuntu 17.10.
- https://github.com/bitshares/bitshares-core/issues/265 -> added pull request for the issue at: https://github.com/bitshares/bitshares-core/pull/406 -> under review by  the team now.
- merged seed addition https://github.com/bitshares/bitshares-core/pull/400/files
- merged documentation change by @abit https://github.com/bitshares/bitshares-core/pull/401
- https://github.com/bitshares/bitshares-core/issues/390 ->  closed issue

- https://github.com/bitshares/bitshares-core/pull/405 -> new pull request was made to resolve the issue of the get account history by date and trying to build a framework to solve other issues that require similar actions. we are unsure on how we will proceed, options haves been presented and it is in discussion with the bitshares dev community.

explorer:

- big improvements were made to the assets page with the guide of @tbone https://github.com/bitshares/bitshares-core/issues/390
- https://github.com/oxarbitrage/bitshares-explorer/issues/3 -> done and closed issue.

- explorer server was hacked this week, mainly because i have other sites, old sites in the same server. i had a hard time to restore it.  @wmbutler has sent me an instance to run the explorer, and it will be moved there next week. thank you very much @wmbutler !

[pc]

Thanks Alfredo! You help with the release preparations, and your testing have been very valuable! Good Job!

[oxarbitrage]

folks, 2 weeks has passed since my last report, here is a summary of the last work done.

bitshares core:

all the pull requests with hardfork were reviewed and merged to the develop branch. @pc later updated the testnet based on this branch.

- https://github.com/bitshares/bitshares-core/pull/386 - merged.
- https://github.com/bitshares/bitshares-core/pull/369 -  merged.
- https://github.com/bitshares/bitshares-core/pull/348 - merged.
- https://github.com/bitshares/bitshares-core/pull/340 -  merged.

https://github.com/bitshares/bitshares-core/issues/390

sent link to new documentation of memory reduction techniques and closed the issue.

https://github.com/bitshares/bitshares-core/pull/391

a few more changes were made to the documentation to reflect memory size.  merged pull.

https://github.com/bitshares/bitshares-core/pull/339

after some additional testing the pull by peter conrad was merged.

https://github.com/bitshares/bitshares-core/pull/346

requested @xeroc to fix the conflicts, now pull under review by me for a possible merge into the develop , can be a good starting point to develop plugins for newcomers.

https://github.com/bitshares/bitshares-core/pull/392

reviewed and merge this small fixes and cleanup by peter conrad.

https://github.com/bitshares/bitshares-core/issues/395

assistance provided for build issues, problem was fixed, closed the issue.

https://github.com/bitshares/bitshares-core/issues/396

removed header in pull request: https://github.com/bitshares/bitshares-core/pull/397 merged and closed issue.

https://github.com/bitshares/bitshares-core/pull/398

tested and merged patch by peter.

https://github.com/bitshares/bitshares-core/pull/379

a new solution has been proposed in this pull request to access some needed blockchain data without spending any additional ram. this was a very big research work. i can say that i spent almost the last entire week working on this solution that i hope can be approved by the other devs as it can bring additional features to our api calls not currently possible. i am not really sure of the future of this pull request(if it will be accepted or not) but what learned in the process is invaluable.

note will upload the proposed solution tomorrow morning as i found a small bug i want to fix before pulling. will update here when it is up.

https://github.com/bitshares/bitshares-core/issues/355

this was also a time consuming task. i was able to confirm bitshares-core full project compiles in a 32 bits ubuntu system even if in production this is pretty much useless it can be of interest for private blockchains or testnets. closed the issue.

explorer:

the main issues the explorer was having was the possibility to browse all user history, this has been added and it is fully supported with the help of a full node provided by @xeroc. the other main problem was that the explorer service was crashing with high traffic, this was fixed, it can still be improved further but it is working better now.

issues are open for the explorer at https://github.com/oxarbitrage/bitshares-explorer/issues and will be added to this report as thye get resolved.

i plan to spend significant time this week in the explorer to keep improving it as it is gaining traffic and it is having great feedback. now that the hardfork code was merged in the core and that very time consuming advances had been made at pull 379 the idea is to have more time for it this week.

[oxarbitrage]

Thanks for the consistent updates.  Any opinion on this feature being discussed for a future core project? https://github.com/bitshares/bitshares-ui/issues/356

you can submit the issue to the bitshares-core at: https://github.com/bitshares/bitshares-core/issues and we will consider it however as it needs a hardfork it is going to be hard to see it done in a short period of time. please note we are after a hardfork now and this is not something that happen very often. @svk comments it is also big work, that decrease even further the possibilities of materializing it as it will need to be done throw a bsip(https://github.com/bitshares/bsips). still, if you can convince an important number of people(shareholders) that this is important then it will eventually be done.

[Brekyrself]

Thanks for the consistent updates.  Any opinion on this feature being discussed for a future core project? https://github.com/bitshares/bitshares-ui/issues/356

[oxarbitrage]

bitshares-core:

- https://github.com/bitshares/bitshares-core/issues/358 - work in progress here, need changes: https://github.com/bitshares/bitshares-core/pull/379
- https://github.com/bitshares/bitshares-core/issues/388 - samples and links provided. closed issue.

hardfork reviews:

we are in the process of merging code for 4 hardforks to the develop branch, 1 main and 3 smaller bug fixes.

- https://github.com/bitshares/bitshares-core/pull/386 - tested, reviewed and approved.
- https://github.com/bitshares/bitshares-core/pull/369 - tested, reviewed and approved.
- https://github.com/bitshares/bitshares-core/pull/348 - tested, reviewed and approved.
- https://github.com/bitshares/bitshares-core/pull/340 - this is the main hardfork, is under review and testing now.

please note that hardfork will be also tested in a testnet before production release.

explorer:

released first production version at :

http://bitshares-explorer.io

this week efforts in the explorer will be to display all operation history in the accounts page, something the community is needing since a while.

documentation:

- https://github.com/bitshares/bitshares-core/issues/378 - wiki page created https://github.com/bitshares/bitshares-core/wiki/Memory-reduction-for-nodes , sent pull request to link it at main README: https://github.com/bitshares/bitshares-core/pull/391 and https://github.com/bitshares/docs.bitshares.org/pull/17

[oxarbitrage]

Hello shareholders, it has been 2 weeks since last report. I realized today that last week i didn't made it, was and still very busy working but that's no excuse, i should write it weekly, is even easier for me.

My apologies for that and i will try to don't do the same again. Anyways, the report today includes the last 15 days of work by this worker.

Bitshares Core Tasks:

- https://github.com/bitshares/bitshares-core/issues/358 -> work in progress, have a solution, will pull tomorrow.

- https://github.com/bitshares/bitshares-core/issues/355 -> i have an old machine 32 bits to test compilation, i remember i tested it a long time ago and didn't worked for me but don't remember the reason.

- https://github.com/bitshares/bitshares-core/issues/361 -> closed.

- https://github.com/bitshares/bitshares-core/pull/352 -> made requested changes, merged.

- https://github.com/bitshares/bitshares-core/issues/367 -> opened issue about error caused by one of the last merges. @pc identified the cause of the problem and submitted a patch: https://github.com/bitshares/bitshares-core/pull/368 -> works good now, merged patch.

- https://github.com/bitshares/bitshares-core/issues/295 Closed. Made additional changes to core to solve this issue, this are at pull: https://github.com/bitshares/bitshares-core/pull/364
Issue was opened in the UI to make a warning when subscriptions to accounts its passing 100: https://github.com/bitshares/bitshares-ui/issues/299 -> issue was closed by @svk as this will be handled different in the upcoming GUI release.

- https://github.com/bitshares/bitshares-core/pull/373 -> adding 2 new calls to work with transactions hashes from the api. first call was easy but the second i want to make need more bigger changes. work in progress, i have a working prototype, new commits will be added to introduce the second call in the next days.

- https://github.com/bitshares/bitshares-core/issues/375 -> unrelated, replied and closed.

- https://github.com/bitshares/bitshares-core/pull/345 -> tested and merged.

reviewing now:

- https://github.com/bitshares/bitshares-core/pull/376 -> testing.
- https://github.com/bitshares/bitshares-core/pull/341 -> testing, the log message is fine to identify the problem, the tests cases are a bit unclear still. more review needed before merging, i have to check with @alexpmorris.
- https://github.com/bitshares/bitshares-core/pull/348 (HF) -> testing hardfork with the debug node.
- https://github.com/bitshares/bitshares-core/pull/369 (HF) -> testing hardfork with the debug node.

Explorer:

Today it is the deadline of a very rough estimate i made for the explorer of 1 month, plan was to have all the TODO tasks of the explorer finished:

https://github.com/oxarbitrage/bitshares-explorer/blob/master/Roadmap1.md

https://github.com/oxarbitrage/bitshares-explorer/blob/master/TODO.md

The TODO is right now at 90%. The reason it is not at 100% today are a few:

- there is also a TODO 2 where some tasks were made: https://github.com/oxarbitrage/bitshares-explorer/blob/master/TODO2.md
- the estimate was rough and some tasks required more time.
- some tasks not included in TODO1 and TODO2 had to be done.

The remaining 10% of the TODO1 is just cleanup, performance and code disclosure.

All the features and sections of the explorer are working*, i encourage to visit one more time: http://bitshares-explorer.io:9009/
Performance will increase as i remove all the crap and package it for distribution, it haves some problems but it is fully functional.

* Only section not working is transactions section(http://bitshares-explorer.io:9009/#/txs) but that is not part of TODO1.Trying to first add some code to the core in order to achieve transactions section and search.

As it was previously mentioned, the project is 2 repos.

https://github.com/oxarbitrage/bitshares-python-api-backend -> api python backend serving the explorer website only at the moment but can serve other applications. Public and fully updated.

https://github.com/oxarbitrage/bitshares-explorer

To release the code of the explorer part, with all the cleaning and stuff i have to do, i think i am going to need 1 week more.

Others:

Upticks:

https://github.com/xeroc/python-bitshares/issues/10

Docs.Bitshares.Org:

https://github.com/bitshares/docs.bitshares.org/pull/16

Bitshares.GitHub.IO:

to be merged after next release of bitshares UI:
https://github.com/bitshares/bitshares.github.io/pull/108

SonarCloud:

https://sonarcloud.io/dashboard?id=BitShares-develop%3Adevelop

taking the opportunity to mention that @pc added our develop bitshares-core branch, this will allow to improve code quality. For a huge project like bitshares the code quality ratings are pretty impressive.

Until next week, thanks for taking the time of reading

[oxarbitrage]

Hello all, here is the tasks this worker made on the last week until Sunday 13-08.

- https://github.com/bitshares/bitshares-core/pull/334
conflicts resolved and merged. related issue was closed https://github.com/bitshares/bitshares-core/issues/298

- https://github.com/bitshares/bitshares-core/pull/352
some additional calls for workers and committee members were added to the node, needed by the explorer and useful(i hope) for some others. there are a few changes that will need to be done to merge them to bitshares.

- https://github.com/bitshares/bitshares-core/pull/336
merged pull to remove msg from witness node by peter conrad.

- https://github.com/bitshares/bitshares-core/pull/337
merged hardfork cleanup by peter conrad.

- created milestone for next hardforks: https://github.com/bitshares/bitshares-core/milestones

- explorer: i was expecting to be at 80% of the TODO but i am actually at 73%(https://github.com/oxarbitrage/bitshares-explorer/blob/master/TODO.md). A bit behind the expected, i plan now to be at 90% from the next report and very close to release.

[oxarbitrage]

Hello bitshares community, here is a list of work done by this worker in the last week.

bitshares core:

- https://github.com/bitshares/bitshares-core/issues/329
added side1 and side2 account ids of the trade at pull : https://github.com/bitshares/bitshares-core/pull/344 - merged and closed the issue.

- https://github.com/bitshares/bitshares-core/issues/168
@takaaki7 submitted pull request to fix this issue. pull is at: https://github.com/bitshares/bitshares-core/pull/347 i tested and merged the pull and closed the issue.

special thanks to @takaaki7 who started to collaborate in the bitshares-core development in a good way.

- https://github.com/bitshares/bitshares-core/pull/339
@pc has submitted a pull request that will reduce the "unclean shutdowns" and full replay restart in  nodes by trying to save and use the most data possible from existing chain and log. this is being fully tested and improved, can read more in the pull conversation. we expect to merge a final solution very soon.

- https://github.com/bitshares/bitshares-core/pull/288
after fully testing all the aspects of this pull from @pc we decided with @abit it was ready enough  to merge so we did it. this new code allows to start nodes with a --plugin parameter that will load the witness with only the required combinations of plugs.

explorer:

i am putting more effort on the explorer v1 as some shareholders requested more speed on the project due to repeated problems with cryptofresh. current TODO is at 60% done, new plan is to be at 80% for the next report in 1 week. please check the TODO online file to see progress of version 1:

https://github.com/oxarbitrage/bitshares-explorer/blob/master/TODO.md

additions and suggestions for v2 will be considered and added to v2 file: https://github.com/oxarbitrage/bitshares-explorer/blob/master/TODO2.md

[oxarbitrage]

trying to put more effort to the explorer to have v1 faster as i am aware of the repeated problems of cryptofresh in the last time.