Main > Stakeholder Proposals

[WorkerProposal]Data solution:data structured,flexible API and fast-built method

(1/5) > >>

R:
RE: ZeroMQ implementation - https://github.com/oxarbitrage/bitshares-core/tree/zeromq/libraries/plugins/zeromq

Could produce a ZeroMQ client which streams to postgresql 👍

R:

--- Quote from: vianull ---1) Develop a Postgresql-based plugin. We are going to build a structured on-chain data storage  using Postgresql.
--- End quote ---

I like the idea of developing new plugins for exporting the data from the bitshares full node, the more plugins the merrier in my opinion. If you could split the development of the plugin from the other two goals to reduce the cost of the worker proposal then I'd support this WP.

None of your images are loading for me, could you upload them to an alternative image server please?

I'd be very interested in a plugin for Google's BigQuery, streaming directly into a table for running REST API based queries on Google's servers.

oxarbitrage:
Please don't get me wrong, i do think is a good idea to have ways to visualize the data with good graphics, API calls and others.

However, the worker do not consider there is already a framework available to do it, can be communication problems, last changes to make it all possible are relatively new, etc.

Another issue i have with this worker is the price, even if you have to make it all from scratch i think the worker price is at least 3 times above common sense.

I will be happy to support a worker that consider this 2 points.

Digital Lucifer:

--- Quote from: vianull on November 06, 2018, 02:17:39 pm ---
--- Quote from: oxarbitrage on November 05, 2018, 06:16:54 pm ---In general i don't agree with this worker, here are some reasons:

- Cost. It cost almost 300k usd(288000 usd to be exact to do it).
- Limited. It is database specific, a new worker may came in saying mongodb or whatever is better for their specific needs.
- Closed to participate. No one except the bts.ai team can participate in the development.
- Closed source. It don't says anything that all the work will be open source and MIT license. Will bts.ai be open source and inside the bitshares organization ?
- Reinvent the wheel. The Elasticsearch plugin is working great and have all the data needed, the synchronization time is of 20 hours according to a last report, all the data inside operations is structured and available. That cost 0 to bitshares as it is already done. Doing the same from scratch using another database by a new team is IMHO a waste of resources. The core team, with the accumulated experience can do a postgres plugin in extremely reduced time if that is what the community needs. Also, the core team can pay a team or individual to do the plugin as plugins are core work and will need review and approval from core team members.
- Benefit. Besides having some better visualizations of some data which i think is important i don't see any other real benefit of the proposal.

In my opinion i will like to see some day a general worker that will do this and others in a bounty style, most of the API links mentioned are not being developed because there is  no funding to get developers on board. There is already a ruby project for bitshares at https://github.com/MatzFan/bitshares-ruby not being improved because of lack of funding, among many other dead projects.

I think that bitshares needs a worker proposal similar to the core worker where teams and individuals can participate in the development of different tools of the bitshares ecosystem.

It honestly looks to me like reinventing the wheel after all the work it has been done in this particular regard, discard everything and start from scratch instead of build on top of previous tools to save time, resources and advance.

Just my personal opinion, i don't have any voting power or influence to decide what is accepted or not.

--- End quote ---


The important thing is NOT PostgreSQL or ES or Mongo or anything . What we care most about is how to implement the requirment .

such as:
1) How to draw charts using specified data in a certain period, e.g. the average price of a market pair, the issue/burn number of an asset, and the feed price of a smart asset.
All these stuffs are unstructured by ES currently, which take some problems to our implements.
2) Bitshares is an exchange such that there may be more professional approach to the storage and analysis the transaction data in exchange viewpoint.

As mentioned in this worker, we will open source under MIT. If this worker is validated, then all future data APIs on bts.ai will be open source, which has been part of this worker.

--- End quote ---

2) BitShares is a Blockchain who have much brighter and wider future than just an Exchange.

vianull:
Thank you for all your suggestions . I think we need to reassess our proposal and think it through.

Navigation

[0] Message Index

[#] Next page