Here are our comments:
Several critical reports have been submitted and fixed through HackTheDex, proof for its value and necessity.
In terms of Owasp vulnerability ratings, HTD auditors have not yet rated any publicly disclosed vulnerabilities as critical, so this is an inaccurate claim to make.
Refined the wording. This was not meant to reference the OWASP scale and was merely subjective.
The proposal will use allocated funds to reward those that step forward with exploits, relative to the overall risk assessment of the exploit.
Do you want vulnerability disclosure reports, or are you requesting the full development of weaponized exploits?
By the owasp rating guide, two vulnerability factors (https://www.owasp.org/index.php/OWASP_Risk_Rating_Methodology#Vulnerability_Factors) incentivize the creation of automated metasploit modules.
All rewards are marked as "up to", and here is a quote from the proposal
To qualify for higher rewards, the submission should include everything
the auditing panel would need to reproduce and verify the vulnerability.
The submission should also be well-prepared and of high quality so that
it can be shared with the community as part of our transparency process.
We see your criticism in the process, and we will attempt to mitigate it. The new worker proposal will be using a new set of collaborative tools for auditors that will make the process clearer, and ultimately more transparent (the new tools would allow e.g. public fetching of the ticket status). It is our duty to make sure all auditors are fully aware of the OWASP factors/rating system and we will re-iterate on that.
How the vulnerability is scored, and the methods used to determine the score, are at the sole discretion of the panel chosen to audit the report
Since you're using the owasp vulnerability rating system, there shouldn't be undisclosed deviation from this scoring mechanism. I get not disclosing details before they're mitigated, but this comes off as a clause to avoid reporters successfully contesting low vulnerability ratings.
This is somewhat true as it is meant to give the final say to the audit panel. Several factors go into that calculation that would require full disclosure of reports, e.g.
- quality of report
- extent of attack vector provided, analysis and deduction of possible exploits
- completeness of report (hidden attack vectors that were found in review)
Disclosing that information would not be recommendable (at least we thought that) due to security concerns, and as such the calculation of reward may not be transparently comprehensible, not to the public, and in cases not even towards the reporter itself (at least not without NDA). We take it as our homework to research how other bounty programs handle such sensitive cases.
- The turnaround time from submitting report to being in receipt of reward is multiple months. It's discouraging to wait months with an unknown payout status; if there was a faster turnaround time then researchers would be more confident in dedicating time towards HTD.
- Reports which aren't vulnerabilities don't get responses & you can't check on progress of any submitted reports without manual telegram/email communications every couple weeks. Rejection emails would be great.
Certainly one of the biggest reasons for the change of collaborative tools, and as such I expect this to be no longer an issue (response on all submissions, status updates and finalized reports within weeks not months)
- There has been no news update since late July 2018 nor changes to the website aside from populated reports/leaderboard/receipts. The organizer of the first HTD WP left the role & this role was filled by another community member without a blog update. What happened to Matt?
The dayjob dilemma appeared. Both Blockchain Projects and Ryan Fox are full-time working in the blockchain space.
- One of my submissions broke the submission form, resulting in a partial report submission. Bug report for your bug report.
The submit form has been changed, it is now using local email client and highlights the submit email as well.
- Final scores for vulnerabilities are not disclosed, only the grade category for Likelihood, Impact & Severity. This further distances the researcher from contesting low rewards. A simple final_owasp_rating:reward calculation would make researchers reward expectations more realistic months before final payout. The full disclosure of vulnerability details (inc final score) is an industry standard practiced by nvd.nist.gov , cvedetails.com and vuldb.com (etc..) which HTD aught to replicate so as to be more transparent.
Agreed, we will discuss how to refine the process in terms of disclosing the detailed OWASP scores.
- It's the researcher's responsibility to report the vulnerability to nist/cvedetails/vuldb; HTD should take the initiative to boast about solved vulnerabilities to potentially attract veteran security researchers (and potentially devs/investors). Bitcoin has vulnerability reports on these websites.
Expect an update on that after we have discussed this internally.
- Report 20180918A (reflected XSS vuln) has a high impact, whereas 20180728A (stored XSS vuln) and 20180801A (stored XSS vuln) have Medium impacts. Regardless of how they trigger (stored/reflected) the end result is identical (XSS attack). I believe that 20180728A and 20180801A aught to have been higher ranked & rewarded.
Once the new tools are setup we could reexamine those reports, no promises though. Please ping us then.
I love the HTD WP concept & don't hold any malice against anyone nor intend to grief/troll, I wish to simply improve the HTD process. I'll be voting to support this WP and hope that the above information helps.
And thank you for that!
Best regards,
Blockchain Projects BV
Stefan Schießl
Topic: [Worker Renewal] HackTheDEX.io -- a BitShares Bug Bounty Program (2019) (Read 3360 times)
