BitShares Forum
Main => General Discussion => Topic started by: wackou on October 27, 2015, 09:41:41 pm
-
For those that don't know the bts_tools package, the best description can probably be given with screenshots (a picture is worth a thousand words!): http://bts-tools.readthedocs.org/en/latest/monitor.html#screenshots
-------------------------------------
A bit later than what I originally hoped for, here comes the bts_tools package ported to support graphene clients, in this case BitShares 2.0.
Given the new asynchronous model of communication via websockets, I had to refactor more than originally planned, but the bulk of it is now done, and I learnt quite a great deal in the process (wanted to look into async programming in python for some time now, and this was the perfect occasion!). Thanks go to Xeroc for writing his python-graphenelib module, that was extremely helpful in getting me bootstrapped and understanding how it all worked.
Please bear in mind that there are probably still a few things not working, or not working properly. The basic functionality is there, though:
- cpu/ram/network connections graph
- feeds publishing, using either a time interval (as before) or a time slot (as currently setup in the telegram chat)
- some other plugins: wallet_state, voted_in, online, network_connections
notably missing:
- detection of blocks missed
So for the non-faint of heart that don't mind a few quirks, you can install the tools as usual with:
$ pip3 install bts_tools
and you should get version >= 0.3. The documentation, although not updated yet, should still mostly apply to everything: http://bts-tools.readthedocs.org/
What you can do then:
- bts2 build [tag]: builds the latest version, or a given version
- bts2 run: runs the witness client
- bts2 run_cli: runs the cli wallet
- bts2 monitor: launch the monitoring tools
Things to watch out for in this release:
- the config file is still sensitive, ie: you need to configure everything properly, there is very little checking. The default config.yaml file comes with the minimum required fields you need to configure to get something working, edit at will. If you had a previous config.yaml file, it is highly recommended to delete it beforehand.
- in particular, you need to setup correctly an api_access.json file (as described here: https://github.com/BitShares/bitshares-2#accessing-restricted-apis - the saltpass.py script can be found here: https://github.com/BitShares/bitshares-2/blob/bitshares/programs%2Fwitness_node%2Fsaltpass.py) and configure your config.ini file to use it. This is needed to get access to restricted APIs of the client such as the network one.
What I plan to do next is to iron out the support, port the remaining monitoring plugins and stabilize the code base. Support for Muse clients should also come very soon :)
Let me know what you think!
-
Great work! Very excited! Much Wow!
-
Very cool
-
Congratulations! You've persistently improved these tools starting from ground level. Good Job!
-
+5%
-
For those that don't know the bts_tools package, the best description can probably be given with screenshots (a picture is worth a thousand words!): http://bts-tools.readthedocs.org/en/latest/monitor.html#screenshots
Let me know what you think!
The screenshots are outdated. The dates are in 2014 and the terms used are for BTS1. Could you please put up screenshots for BTS2?
(http://graphene.cubeconnex.com/download/bts_tools_screenshot2.png)
-
Nice! +5%
-
The screenshots are outdated. The dates are in 2014 and the terms used are for BTS1. Could you please put up screenshots for BTS2?
I haven't gotten around to making new screenshots yet, was planning to do it after fixing everything. Currently the screenshots I could take don't show everything or some wrong fields, and I prefer to leave old screenshots for now than put broken ones. They would be very similar in spirit, though, as the UI hasn't changed much since bts1.
-
The screenshots are outdated. The dates are in 2014 and the terms used are for BTS1. Could you please put up screenshots for BTS2?
I haven't gotten around to making new screenshots yet, was planning to do it after fixing everything. Currently the screenshots I could take don't show everything or some wrong fields, and I prefer to leave old screenshots for now than put broken ones. They would be very similar in spirit, though, as the UI hasn't changed much since bts1.
Looking forward to the new fields being ready. It would be a great tool to use.
-
Released 0.3.1 which fixes a few things, has supports for Muse (should mostly work, but not extensively tested), and has enhanced feeds support. From the changelog:
* support for Muse clients
* better feeds script. Process is now the following:
- get the BTS/BTC valuation from Poloniex, CCEDK, Bter, Btc38 (configurable)
- get the BTC/USD valuation from BitcoinAverage, with fallback on Bitfinex and Bitstamp
- get the BTS valuation in other fiat currencies using Yahoo forex rates
- get market indices using Yahoo, Google, Bloomberg (configurable)
* can specify 'boost_root' option in build environment in config.yaml
* minor bugfixes everywhere
When using multiple feed providers, price is always weighted by the volume of the exchange. Screenshots coming soon now, once I fix feeds display (in info screen) and "blocks produced / blocks missed" (always at 0 for now)
-
Released 0.3.2 as an interim release to fix the CNY feed publishing. Price is now taken from pure BTS/CNY markets on Btc38, Bter and Yunbi, weighed by volume. There are (a lot!) more fixes in master, but those will have to wait for the next release of the BitShares backend.
-
Released 0.3.2 as an interim release to fix the CNY feed publishing. Price is now taken from pure BTS/CNY markets on Btc38, Bter and Yunbi, weighed by volume. There are (a lot!) more fixes in master, but those will have to wait for the next release of the BitShares backend.
Bter do not trade bts 2.0
Please do not include Bter until it will honor bts 2.0
-
Released 0.3.2 as an interim release to fix the CNY feed publishing. Price is now taken from pure BTS/CNY markets on Btc38, Bter and Yunbi, weighed by volume. There are (a lot!) more fixes in master, but those will have to wait for the next release of the BitShares backend.
Bter do not trade bts 2.0
Please do not include Bter until it will honor bts 2.0
oh, I didn't know they hadn't upgraded yet... Anyone using my script can remove it themselves by not including it in the list of active feed providers in the config.yaml file. I will remove it from the default config for next version.
-
I just released 0.3.3, which fixes quite a new things overall and general stability should be much better now. There are still some minor features missing, but in general this should work as well as it did for BitShares 0.9.x.
note: this requires the latest bitshares client to work (v2.0.151209)
0.3.3 (2015-12-10)
------------------
* NOTE: requires the latest version (v2.0.151209) of the BitShares witness client
* better integration with the websocket event loop
* network views available again
* overall lots of minor fixes and general stability improvements
-
0.3.4 has a few fixes, is more robust when checking feeds on the chinese exchanges (btc38, yunbi), publishes the "correct" price for BTS/CNY and also publishes feed price for the TCNY asset with correct parameters.
0.3.4 (2015-12-21)
------------------
* added support for publishing TCNY feed; generally more robust feeds fetching/publishing
* renamed 'bts2' commandline tool to 'bts'. 'bts2' still working for convenience, old 'bts' is still available as 'bts1'.
* change in the config.yaml format: https://github.com/wackou/bts_tools/commit/24e962820775a8a23e0b45d26c501aa7e723ff64
-
0.3.4 has a few fixes, is more robust when checking feeds on the chinese exchanges (btc38, yunbi), publishes the "correct" price for BTS/CNY and also publishes feed price for the TCNY asset with correct parameters.
0.3.4 (2015-12-21)
------------------
* added support for publishing TCNY feed; generally more robust feeds fetching/publishing
* renamed 'bts2' commandline tool to 'bts'. 'bts2' still working for convenience, old 'bts' is still available as 'bts1'.
* change in the config.yaml format: https://github.com/wackou/bts_tools/commit/24e962820775a8a23e0b45d26c501aa7e723ff64
Greak work wackou, I will upgrade my nodes later tonday. Hopefully this will eliminate the divide by zero errors I've been seeing with feed publishing.
-
+5% good to see things starting to decentralized!
Have you considered putting up a worker or some other means to monetize your efforts?
-
Hopefully this will eliminate the divide by zero errors I've been seeing with feed publishing.
You might get it once when you start the bts_tools, but once it's been running for a bit you shouldn't see those anymore, unless both yunbi and btc38 go down for an extended period of time. I've been running it myself for 2 days without issues.
-
+5% good to see things starting to decentralized!
Have you considered putting up a worker or some other means to monetize your efforts?
The initial idea was to partner with Thom to implement the "old" backbone proposal (https://bitsharestalk.org/index.php/topic,13837.msg230185.html#msg230185), but this has been put on hold for a while until I complete the port to graphene. It is mostly finished, but in the meantime I had some ideas to refactor a bit the code, and I'd really like something really stable before implementing any new features, in order to know I can safely rely on the foundations. I estimate this might take another month or two.
I also thought about putting a worker proposal just to work on the tools, but the last few months have been really demanding in real life, and it's kinda hard to plan ahead as I have little free time. Given that I'm not sure to be able to put in the time required to follow the plan outlined in a worker proposal, I think I'm going to skip it for now and work when I can, without pressure or accountability (it's a pastime then, not a job :) ). I might (and probably will) reconsider it in the future, but I guess this will come after the backbone project. I do have lots of ideas, though ;D
-
In last nights turmoil to restore network operation, the monitor on one of the seed nodes was not granted access to the network API. No restart of the witness, cli wallet or monitor process was able to resolve the issue.
I double checked the password in the config.yaml file was correct. What solved the problem was a reboot of the OS and starting everything up from scratch. Why that affected the api security scheme I have no clue. Obviously the python code was not the issue since a reboot was all it took to restore proper operation of the API.
It's interesting that only 1 of the API's were affected. Could that indicate a bug in either the witness or cli?
Actually I may have ruled out the python code prematurely. It is just as much involved as the other elements and none were changed. I have no basis for pointing the finger at any one of the 3 elements.
-
Released 0.4, here's the changelog:
0.4 (2016-05-01)
----------------
* API CHANGE: complete rework of the configuration system, please delete
your old config.yaml file if you have any
* added full support for Steem, including feed publishing
* added feed publishing for BTS assets: TUSD, CASH.USD, CASH.BTC, ALTCAP
* added monitoring plugin that checks on the amount of free disk space
* fixed missed block notification for Graphene clients
* logs are now also present as rotating logfiles in the ~/.bts_tools folder
I posted a more lengthy post on Steem here: https://steemit.com/steem/@wackou/btstools-release-0
Hope you like it!
-
Released 0.4, here's the changelog:
0.4 (2016-05-01)
----------------
* API CHANGE: complete rework of the configuration system, please delete
your old config.yaml file if you have any
* added full support for Steem, including feed publishing
* added feed publishing for BTS assets: TUSD, CASH.USD, CASH.BTC, ALTCAP
* added monitoring plugin that checks on the amount of free disk space
* fixed missed block notification for Graphene clients
* logs are now also present as rotating logfiles in the ~/.bts_tools folder
I posted a more lengthy post on Steem here: https://steemit.com/steem/@wackou/btstools-release-0
Hope you like it!
Excellent! Good to have you back working on bts_tools Nico!