BitShares Forum
Main => Stakeholder Proposals => Topic started by: wackou on October 10, 2014, 08:50:31 am
-
tl;dr delegate bids are described at http://digitalgaia.io
Hi all,
I have been a C++/Python linux developer for the last ~10 years, and more recently on OSX, which is why I started contributing to BitShares by doing the initial work for packaging Keyhotee on OSX, then fixed a few small things to be able to compile the bitshares toolkit natively using clang on OSX (instead of macports gcc), and participated in nearly all the dry runs. I am not so active on the forums (for lack of time, and knowledge about economics, although I'm learning ;) ) but I try to read and stay up-to-date as much as possible, as well as report bugs whenever I can on github.
As I started running a delegate since the launch of BitShares in July, I realized I needed some tools to be able to monitor my delegate easily, so I started to work on the bitshares_delegate_tools, which you can find here:
https://github.com/wackou/bitshares_delegate_tools
(there are a couple of screenshots on the github readme)
It then occurred to me that if each delegate were to be doing something more for BitShares, on top of running the delegate client, then this could become a fundamental feature of BitShares DACs as it would create an entire ecosystem of competent, dedicated and diverse set of people, all working towards the same goal of enhancing the network. I believe that creativity comes as sparks of ideas, from diverse and unexpected places in the world, when unfettered minds dream of making a better world :)
This is why I'd like to present the concept of Digital Gaia, where I propose 2 delegate bids:
- wackou.digitalgaia, which is used to fund my development of the bitshares_delegate_tools
- backbone.digitalgaia, which is used to fund specialized nodes useful for the health of the network
The website describing the delegate bids more in details can be found at http://digitalgaia.io
Hoping you like what I'm proposing, and thank you for reading (and voting for me!)
Cheers,
Wackou.
-
I just looked through your website, very nicely presented. If you don't mind I might create something similar for my delegates, I also like how you've split the delegates to fund different activities, I might do that as well!
Will change my votes around for you next time I vote :)
-
+5%
here is information i found usefull http://digitalgaia.io/backbone.html http://digitalgaia.io/wackou.html
-
Sure, go ahead, take anything you like from the web, I plan to put it under Creative Commons and release the source code for the site on github too, as soon as I find the time (this weekend hopefully)
-
Sure, go ahead, take anything you like from the web, I plan to put it under Creative Commons and release the source code for the site on github too, as soon as I find the time (this weekend hopefully)
Awesome, I might wait for you to release it on Github then, was thinking it would be great to have this kind of template for delegates, I'll happily contribute to that.
-
I like you backbone concept ... wanted to implemented it pretty much the same way .. just don't have enough machines :(
-
I like you backbone concept ... wanted to implemented it pretty much the same way .. just don't have enough machines :(
We could cooperate on this. I have sufficient amount of hardware and reliable internet connections.
@wackou my idea for immortal.bitdelegate significantly overlaps with your backbone delegate.
Do you want to create a list of "backbone" nodes and make all of them connected to each other?
I have already listed my seed node and it is included in the BitsharesX client but I have other nodes that "hide" the delegates. Similar to this proposal: https://bitsharestalk.org/index.php?topic=5249.0 (https://bitsharestalk.org/index.php?topic=5249.0)
We should link most of these nodes.
What do you think?
-
We could cooperate on this. I have sufficient amount of hardware and reliable internet connections.
Thanks for the offer .. I will certainly come back to you on that issue
Do you want to create a list of "backbone" nodes and make all of them connected to each other?
I have already listed my seed node and it is included in the BitsharesX client but I have other nodes that "hide" the delegates. We should link most of these nodes.
What do you think?
I also need nodes to hide my delegates .. and I'd love to have some mixture system of VPN and MESH... so that we have a network of (almost) directly connected delegates but private IPs so that even delegates don't know any true IPs ..
or maybe you just connect your 'hiding' nodes to each other and your delegate only interacts with hiding nodes YOU control
-
@xeroc nothing prevents you to link your "hiding"/"relay" nodes with everyone else's including seed nodes. Your delegates will be still safe.
@wackou how do you plan to implement this:
"[TODO] automatic failover agent that kicks in whenever a delegate fails to produce a block, or looks like it will be unable to produce its next block"
There were few solutions for "backup" delegates that caused multiple blocks signing. There are a lot of caveats in this task.
-
There were few solutions for "backup" delegates that caused multiple blocks signing. There are a lot of caveats in this task.
@wacku: Additionally, the common community consensus on this issue is that missing blocks is by far less dangerous than signing two block which produces forks...
-
@emski, @xeroc: about the backbone delegates, I think that ideally they should not all be managed by a single entity (me, or whoever else) for obvious decentralization reasons, similarly to why they shouldn't be run on the same VPS or in the same geographical locations. The more diverse, the better. (@emski: I did see your post yesterday, but considering the time I invested in making the webpage--not my biggest skill--I decided to keep it anyway :D)
However, it is a good idea to have a public place to list all of those seed nodes/backbone nodes, so that they can either be included by default in the client, or that people can easily add them manually. I can offer to host such a list on my server, or maybe we could code it and then have svk host it on bitsharesblocks (excellent job on that btw, svk, bitsharesblocks seriously rocks!). The list would be updated in realtime and show the status (online/offline) of each node.
In the concept of backbone nodes they also need to be connected in a fully-connected graph, so having a central place listing those nodes helps the backbone nodes to know who their peer nodes are and maintain live connections to most of them.
or maybe you just connect your 'hiding' nodes to each other and your delegate only interacts with hiding nodes YOU control
I thought about this, but it somehow feels fragile to me, as if your relay nodes are down for whatever reasons then your delegate becomes isolated. Unless you have a lot of them for a single delegate, in which case it seems a bit wasteful. I believe that it should be possible to create a network which has a topology that allows both efficiency and security for the delegates, in a public way. My backbone node is a proposal towards that goal, but we can probably do better.
@emski: I remember your post about ddos prevention, I had a similar wish to hide the delegates at the time and it inspired me to keep thinking in that direction, as I see this as something that should be (relatively) easily achievable, and would provide perfect protection.
-
@wackou how do you plan to implement this:
"[TODO] automatic failover agent that kicks in whenever a delegate fails to produce a block, or looks like it will be unable to produce its next block"
There were few solutions for "backup" delegates that caused multiple blocks signing. There are a lot of caveats in this task.
This one is tricky indeed, hence the reason it is still TODO and not DONE yet ;D I see at least 2 cases here:
- the main client crashes: this is easy to monitor with a high degree of certainty and launch the client again -> shouldn't be too much of a problem
- the network connection is lost / seems lost: this is the tricky one (IMO), as the monitoring could be run on a different machine which would see the main one as down, and decide to bring up the backup node, when in fact the main node did produce its block and it arrived a bit later. I think that by having delegates connected to the backbone nodes would help mitigate this issue a bit, but I can't think of a completely foolproof solution for now.
I have a crazy idea, though, that just occurred to me: if you have 3 instances of the delegate running, all with block production off. 20 seconds before signing a block, those 3 instances need to arrive at a consensus on which one is going to sign the next block, by becoming a "leader" (like in this presentation of the raft algorithm: http://thesecretlivesofdata.com/raft/). If a node receives at least 1 ACK from another node, then it knows that it can produce the block as itself + another node have agreed that it's going to produce the next block. In effect, this acts as a 2-of-3 multisig, and provides some redundancy. There might be better solutions, but even though the automatic backup node is not an easy problem, I believe it should be possible to solve in a nice enough way.
-
2 out of 3 method should work. However it requires some additional resources.
Crashing is OK. However there were some cases of "freezing" that cannot be detected by the watchdog (you'll need to monitor for missed blocks and then restart the client).
I think for now the missing blocks notifications are sufficient. It is good to have alternatives though.
-
yes, I also do believe that missed blocks are not such a big deal. If you miss one, then the delegate after you will pick up the transactions you couldn't sign and pocket the transactions fees for itself. So instead of being validated in avg 5 seconds, a block might take 10s. Still leagues ahead of the competition! Obviously, it shouldn't become systematic for a delegate to miss block (or for 50 delegates to start missing blocks at the same time, ie: during an upgrade), but one missed block from time to time should go completely unnoticed by the users of the network.
So my next plans for the bitshares_delegate_tools is not to focus on this automatic backup agent, but rather on:
- enhance the way I publish feeds, or use your script or xeroc's version
- adapt the tools to also be able to build the DNS client (and setup one for me!)
- I'd like to focus a bit also on collaborating with people like you and setting up the backbone, as I believe it is a low-hanging fruit that can reap big rewards for the whole network
-
I put the source code for the website on github, you can grab it at https://github.com/wackou/digitalgaia
-
I put the source code for the website on github, you can grab it at https://github.com/wackou/digitalgaia
Nice, will check it out tomorrow!
-
great work +5%
-
bytemaster should vote you up!
-
Thanks! I already have an active delegate (wackou-delegate) but it's my old one which I'd like to phase out in favor of wackou.digitalgaia, I imagine BM hasn't switched his votes yet (or hasn't seen this post yet)