Author [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] [EN] [ZH] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] Topic: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser  (Read 5299 times)

0 Members and 1 Guest are viewing this topic.

Offline mint chocolate chip

Re: [ANN][AGS] Open Source Angelshares-Blockchain-Parser
« Reply #15 on: January 03, 2014, 10:46:30 AM »
Really nice, easy to understand the daily breakdown looking at this, I'll surely be revisiting it over the next couple hundred days

Offline 5chdn

  • Sr. Member
  • ****
  • Posts: 487
  • i wonder how many chars i can put in this field 50
    • View Profile
    • Votesapp
  • GitHub: 5chdn
Re: [ANN][AGS] Open Source Angelshares-Blockchain-Parser
« Reply #16 on: January 03, 2014, 11:02:00 AM »
You can use command line args to start parsing at a certain day. To get today's stats only, just run it with:

Code: [Select]
$ ruby pts_chain.rb 38069

Offline stuartcharles

  • Sr. Member
  • ****
  • Posts: 281
    • View Profile
Re: [ANN][AGS] Open Source Angelshares-Blockchain-Parser
« Reply #17 on: January 03, 2014, 11:30:35 AM »
great work
« Last Edit: January 03, 2014, 11:33:27 AM by stuartcharles »

Offline wasthatawolf

  • Full Member
  • ***
  • Posts: 188
    • View Profile
  • BTS: loon3
Re: [ANN][AGS] Open Source Angelshares-Blockchain-Parser
« Reply #18 on: January 04, 2014, 06:08:42 PM »
With the help of Bootstrap, I hacked together a page using PHP to beautify donshoe's parsed txt files.  Still a work in progress.

http://joelooney.org/ags/




Offline 5chdn

  • Sr. Member
  • ****
  • Posts: 487
  • i wonder how many chars i can put in this field 50
    • View Profile
    • Votesapp
  • GitHub: 5chdn
Re: [ANN][AGS] Open Source Angelshares-Blockchain-Parser
« Reply #19 on: January 04, 2014, 07:27:50 PM »
With the help of Bootstrap, I hacked together a page using PHP to beautify donshoe's parsed txt files.  Still a work in progress.

http://joelooney.org/ags/

Nice, pls check if this helps you:

UPDATE: CLEAN CSV FILES HERE:
http://q39.protoshar.es/ags/4/pts.csv.txt
http://q39.protoshar.es/ags/4/btc.csv.txt

These files are improved.
« Last Edit: January 11, 2014, 04:10:24 PM by donschoe »

Offline wasthatawolf

  • Full Member
  • ***
  • Posts: 188
    • View Profile
  • BTS: loon3
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #20 on: January 04, 2014, 08:14:31 PM »
I ended up using the daily summary data break as a checkpoint for collecting data for each day.  I'm not sure how much faster the page would load at this point if I used the cleaner txt file.

The several second load time is likely due to parsing the entire txt file for both btc and pts each time.

What I think I'll do to speed it up is store daily summary information (top deposit, number of deposits, sum of deposits) in a MySQL database.  Then, all I would need is a csv file with the current period data for each blockchain.

Great work by the way.  Thanks for parsing the two blockchains and posting the live data.

Offline boombastic

  • Sr. Member
  • ****
  • Posts: 250
    • View Profile
    • AngelShares Explorer
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #21 on: January 04, 2014, 09:50:17 PM »
I've put up a frond-end based on donschoe's daily parsed data input.  You can check it out at:

http://agsexplorer.com/

I was trying to build on live blockchains, but since I am new working with blockchain, still have some issues to solve.  donschoe's data is good enough for now.  Thanks.

There's one question, the time in the txt file looks not consistent with the same transaction found on blockchained.info. Isn't the time should be equal to the block in which the transaction is included?
http://bitshares.dacplay.org/r/boombastic
Support My Witness: mr.agsexplorer
BTC: 1Bb6V1UStz45QMAamjaX8rDjsnQnBpHvE8

Offline 5chdn

  • Sr. Member
  • ****
  • Posts: 487
  • i wonder how many chars i can put in this field 50
    • View Profile
    • Votesapp
  • GitHub: 5chdn
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #22 on: January 04, 2014, 10:07:32 PM »
I've put up a frond-end based on donschoe's daily parsed data input.  You can check it out at:

http://agsexplorer.com/

I was trying to build on live blockchains, but since I am new working with blockchain, still have some issues to solve.  donschoe's data is good enough for now.  Thanks.

There's one question, the time in the txt file looks not consistent with the same transaction found on blockchained.info. Isn't the time should be equal to the block in which the transaction is included?

This looks damn awesome!

I'm using the timestamp of the block where the transaction is picked up in. Maybe the time of the transaction can be issued earlier. I'm not sure - does that matter?

Please rethink using the v0.2 csv files. They are basicly the same but I will shut down v0.1 API soon as they are not accurate enough!

I ended up using the daily summary data break as a checkpoint for collecting data for each day.  I'm not sure how much faster the page would load at this point if I used the cleaner txt file.

The several second load time is likely due to parsing the entire txt file for both btc and pts each time.

What I think I'll do to speed it up is store daily summary information (top deposit, number of deposits, sum of deposits) in a MySQL database.  Then, all I would need is a csv file with the current period data for each blockchain.

Great work by the way.  Thanks for parsing the two blockchains and posting the live data.

You also may want to upgrade to the v0.2 API which has more accurate results. For daily statistics you can use the last line of each day.
« Last Edit: January 04, 2014, 10:10:12 PM by donschoe »

Offline boombastic

  • Sr. Member
  • ****
  • Posts: 250
    • View Profile
    • AngelShares Explorer
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #23 on: January 04, 2014, 10:10:59 PM »
Thanks, will check out v2 tomorrow. Gotta go to bed now, it's already morning here in Shanghai.


Sent from my iPhone using Tapatalk
http://bitshares.dacplay.org/r/boombastic
Support My Witness: mr.agsexplorer
BTC: 1Bb6V1UStz45QMAamjaX8rDjsnQnBpHvE8

Offline wasthatawolf

  • Full Member
  • ***
  • Posts: 188
    • View Profile
  • BTS: loon3
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #24 on: January 04, 2014, 10:50:32 PM »
Do you plan to shut down the v0.1 API once the bitcoin blockchain is finished parsing in v0.2?
« Last Edit: January 04, 2014, 11:10:00 PM by wasthatawolf »

Offline 5chdn

  • Sr. Member
  • ****
  • Posts: 487
  • i wonder how many chars i can put in this field 50
    • View Profile
    • Votesapp
  • GitHub: 5chdn
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #25 on: January 04, 2014, 11:48:32 PM »
No, i will keep running the v0.1 untill my own services are migrated. but just be safe and upgrade soon.

Offline boombastic

  • Sr. Member
  • ****
  • Posts: 250
    • View Profile
    • AngelShares Explorer
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #26 on: January 05, 2014, 06:23:26 PM »
hi donschoe,

I've upgraded to v2 api and found some problems.

The subtotal seems to be incorrectly calculated.  It seems it only happen when several donation transactions happen within the same blocks.

Here are some snippets of 01-15's data

Quote
"278645";"2014-01-05 01:09:04 UTC";"1M16SzSwY9RXpxadEyQ6vdjnNUntk4iogu";"0.12345";"0.22445";"22276.67631989"
"278645";"2014-01-05 01:09:04 UTC";"1765s6HMMane9jMf74Nsop2eBCXbtQUWCv";"0.01313365";"0.72445";"6901.78756298"
"278645";"2014-01-05 01:09:04 UTC";"16PtgDmLb8gixJZKAzgXa8PWeyos9n4vur";"0.48686635";"0.72445";"6901.78756298"

You will see the second line's total is incorrect, the same pattern repeats and therefore the last line total is untrustful.  Please verify.

Another question, if one donation transaction contains multiple inputs, which input address will you credit as donation credit? inputs[0]?
http://bitshares.dacplay.org/r/boombastic
Support My Witness: mr.agsexplorer
BTC: 1Bb6V1UStz45QMAamjaX8rDjsnQnBpHvE8

Offline 5chdn

  • Sr. Member
  • ****
  • Posts: 487
  • i wonder how many chars i can put in this field 50
    • View Profile
    • Votesapp
  • GitHub: 5chdn
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #27 on: January 05, 2014, 08:28:47 PM »
@boombastic, i can answer all your questions at once:

if you see the same sum value, its a single transaction. its not desired but no big issue. will fix that in future releases. edit: fixed.

multiple input addresses are all displayed by the script. there is a discussion goin on here you might wish to join https://bitsharestalk.org/index.php?topic=2036.0
« Last Edit: January 05, 2014, 08:36:25 PM by donschoe »

Offline boombastic

  • Sr. Member
  • ****
  • Posts: 250
    • View Profile
    • AngelShares Explorer
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #28 on: January 05, 2014, 08:53:16 PM »
Thanks @donschoe, I will follow that post.


Sent from my iPhone using Tapatalk
http://bitshares.dacplay.org/r/boombastic
Support My Witness: mr.agsexplorer
BTC: 1Bb6V1UStz45QMAamjaX8rDjsnQnBpHvE8

Offline wasthatawolf

  • Full Member
  • ***
  • Posts: 188
    • View Profile
  • BTS: loon3
Re: [ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser
« Reply #29 on: January 05, 2014, 11:05:30 PM »
Updated with v0.2 data - Angelshares Summary Data

http://joelooney.org/ags/

Source: http://pastebin.com/w7quHJeM

Decided to keep it simple.  PHP only, this should be a good solution for anyone that wants to run the code and view Angelshares summary data locally with no javascript.

 

Google+