[ANN][AGS-PRASER] Open Source Angelshares-Blockchain-Parser

[5chdn]

The AGS parser demo site does not appear to be updating at this moment.

The timestamp of the last BTC donation - 2014-02-05 05:53:09 UTC
The timestamp of the last PTS donation - 2014-02-05 07:27:13 UTC

I have personally made both BTC and PTS donations since the timestamp of the most recent parsed transaction, so I would expect the timestamps to be more recent.

The machine running these example APIs died on feb/05. I will setup a better machine soon and publish it here. Until then you have to live with the other sites available. Sorry for this

[Evan]

The AGS parser demo site does not appear to be updating at this moment.

The timestamp of the last BTC donation - 2014-02-05 05:53:09 UTC
The timestamp of the last PTS donation - 2014-02-05 07:27:13 UTC

I have personally made both BTC and PTS donations since the timestamp of the most recent parsed transaction, so I would expect the timestamps to be more recent.

[5chdn]

Three mugs O' beer finally sent

Thanks earthbound

[wasthatawolf]

Thanks!

[earthbound]

Three mugs O' beer finally sent

[earthbound]

Okay, no, really, seriously, I'll get to those paltry donations When I do, I'll edit this post to say: [Date blah blah blah--sent.]

It's just that I've been forced to re-download the Bitcoin and Protoshares blockchains, so that I can't send any transactions from my personal wallets, yet. Ah, snap! Maybe I can make use of the git repository donschoe posted . . .

[5chdn]

I'm using git to backup the blockchains. http://git.jdqirc.net/donschoe/ags-datafeed

Updating the chain every hour, you can use the raw feed:
http://git.jdqirc.net/donschoe/ags-datafeed/raw/master/4/pts.csv.txt
http://git.jdqirc.net/donschoe/ags-datafeed/raw/master/4/btc.csv.txt
http://git.jdqirc.net/donschoe/ags-datafeed/raw/master/3/balances.json

I hope in future I dont have to reparse the blockchain so often!

[5chdn]

Seems to be an issue with the BTC API v0.2, a lot of blank space and the earliest donation shown is at 2014-01-07 12:29:52 UTC

Not sure what happened. Better upgrade to v0.3 immidiatly:
<removed>

I will keep v0.1 running until v0.2 is fixed.


A bug was introduced in v0.3, upgrade to v0.4 immidiatly.

http://q39.protoshar.es/ags/4/pts.csv.txt
http://q39.protoshar.es/ags/4/btc.csv.txt
http://q39.protoshar.es/ags/3/balances.json

I promise not to change the API unless there are bugs found. This v0.4 will be final. I made some in-depth checks on that one, this should be really, really accurate now!

[wasthatawolf]

Seems to be an issue with the BTC API v0.2, a lot of blank space and the earliest donation shown is at 2014-01-07 12:29:52 UTC

[5chdn]

Can this script be modified to generate a list of all unspent outputs necessary for building a genesis block?  I believe there is another bounty where such a modified script would be helpful to win.

Not sure what you mean by "unspent outputs"? It lists all holding addresses and their connected AGS balance, I thought that was the info needed to build a genesis block, but I never did that before, just let me know what else you need, the script is easily modified.

This was directed more toward not honoring AGS but honoring PTS in a new blockchain.  If there was a website that provided the current unspent balance of every output in the blockchain then someone wanting to build a new genesis block would have a very easy task.  Namely, import the table as a single send to many transaction.

Ah no that's not so easy. Got to think about that. How did FreeTrade do it?

[bytemaster]

Can this script be modified to generate a list of all unspent outputs necessary for building a genesis block?  I believe there is another bounty where such a modified script would be helpful to win.

Not sure what you mean by "unspent outputs"? It lists all holding addresses and their connected AGS balance, I thought that was the info needed to build a genesis block, but I never did that before, just let me know what else you need, the script is easily modified.

This was directed more toward not honoring AGS but honoring PTS in a new blockchain.  If there was a website that provided the current unspent balance of every output in the blockchain then someone wanting to build a new genesis block would have a very easy task.  Namely, import the table as a single send to many transaction. 

[5chdn]

will release api version 0.3 tomorrow and shut down 0.1 as soon as possible. upgrade your applications 

[5chdn]

Can this script be modified to generate a list of all unspent outputs necessary for building a genesis block?  I believe there is another bounty where such a modified script would be helpful to win.

Not sure what you mean by "unspent outputs"? It lists all holding addresses and their connected AGS balance, I thought that was the info needed to build a genesis block, but I never did that before, just let me know what else you need, the script is easily modified.

[bytemaster]

Can this script be modified to generate a list of all unspent outputs necessary for building a genesis block?  I believe there is another bounty where such a modified script would be helpful to win.

[earthbound]

Sorry to say they may be paltry tips, but I'll get to them . . .

[5chdn]

Updated OP, added new feature.

[boombastic]

Yo @earthbound, a tip would be nice.

http://agsexplorer.com

More features will be added.

BTC: 1Bb6V1UStz45QMAamjaX8rDjsnQnBpHvE8
PTS: PgULYtxcDJM7SKABfECipFiSjrmtjiPQd9

[wasthatawolf]

If you found my PHP front end and source code useful (http://joelooney.org/ags/), you can tip me here (no pressure  )...

BTC:  16TsCgrKwQ3efzw6LkNuBHmvP6aJy21HGq
PTS:  PYDhxRmdd8YQdYLD757s7robuvafFdMARi

Thanks!

Note: this code will show the previous period's data until BTC and PTS donations are made in the current period.

[earthbound]

Y'all who've made pretty and more detailed front ends for this, I'd like to tip you (in addition to donschoe).

Where should I send the tips?

donschoe, if they provide addresses, will you please update your OP with info to tip to them as well?

One other thing: it'd be nice if these listed the time zone and time at which a donation period resets (for example, in donschoe's version, this could be given in parenthesis after the time countdown). I forget when that daily reset is . . . is it midnight Greenwich Mean Time?

Thanks!

[wasthatawolf]

Updated with v0.2 data - Angelshares Summary Data

http://joelooney.org/ags/

Source: http://pastebin.com/w7quHJeM

Decided to keep it simple.  PHP only, this should be a good solution for anyone that wants to run the code and view Angelshares summary data locally with no javascript.

[boombastic]

Thanks @donschoe, I will follow that post.


Sent from my iPhone using Tapatalk

[5chdn]

@boombastic, i can answer all your questions at once:

if you see the same sum value, its a single transaction. its not desired but no big issue. will fix that in future releases. edit: fixed.

multiple input addresses are all displayed by the script. there is a discussion goin on here you might wish to join https://bitsharestalk.org/index.php?topic=2036.0

[boombastic]

hi donschoe,

I've upgraded to v2 api and found some problems.

The subtotal seems to be incorrectly calculated.  It seems it only happen when several donation transactions happen within the same blocks.

Here are some snippets of 01-15's data

"278645";"2014-01-05 01:09:04 UTC";"1M16SzSwY9RXpxadEyQ6vdjnNUntk4iogu";"0.12345";"0.22445";"22276.67631989"
"278645";"2014-01-05 01:09:04 UTC";"1765s6HMMane9jMf74Nsop2eBCXbtQUWCv";"0.01313365";"0.72445";"6901.78756298"
"278645";"2014-01-05 01:09:04 UTC";"16PtgDmLb8gixJZKAzgXa8PWeyos9n4vur";"0.48686635";"0.72445";"6901.78756298"

You will see the second line's total is incorrect, the same pattern repeats and therefore the last line total is untrustful.  Please verify.

Another question, if one donation transaction contains multiple inputs, which input address will you credit as donation credit? inputs[0]?

[5chdn]

No, i will keep running the v0.1 untill my own services are migrated. but just be safe and upgrade soon.

[wasthatawolf]

Do you plan to shut down the v0.1 API once the bitcoin blockchain is finished parsing in v0.2?

[boombastic]

Thanks, will check out v2 tomorrow. Gotta go to bed now, it's already morning here in Shanghai.


Sent from my iPhone using Tapatalk

[5chdn]

I've put up a frond-end based on donschoe's daily parsed data input.  You can check it out at:

http://agsexplorer.com/

I was trying to build on live blockchains, but since I am new working with blockchain, still have some issues to solve.  donschoe's data is good enough for now.  Thanks.

There's one question, the time in the txt file looks not consistent with the same transaction found on blockchained.info. Isn't the time should be equal to the block in which the transaction is included?

This looks damn awesome!

I'm using the timestamp of the block where the transaction is picked up in. Maybe the time of the transaction can be issued earlier. I'm not sure - does that matter?

Please rethink using the v0.2 csv files. They are basicly the same but I will shut down v0.1 API soon as they are not accurate enough!

I ended up using the daily summary data break as a checkpoint for collecting data for each day.  I'm not sure how much faster the page would load at this point if I used the cleaner txt file.

The several second load time is likely due to parsing the entire txt file for both btc and pts each time.

What I think I'll do to speed it up is store daily summary information (top deposit, number of deposits, sum of deposits) in a MySQL database.  Then, all I would need is a csv file with the current period data for each blockchain.

Great work by the way.  Thanks for parsing the two blockchains and posting the live data.

You also may want to upgrade to the v0.2 API which has more accurate results. For daily statistics you can use the last line of each day.

[boombastic]

I've put up a frond-end based on donschoe's daily parsed data input.  You can check it out at:

http://agsexplorer.com/

I was trying to build on live blockchains, but since I am new working with blockchain, still have some issues to solve.  donschoe's data is good enough for now.  Thanks.

There's one question, the time in the txt file looks not consistent with the same transaction found on blockchained.info. Isn't the time should be equal to the block in which the transaction is included?

[wasthatawolf]

I ended up using the daily summary data break as a checkpoint for collecting data for each day.  I'm not sure how much faster the page would load at this point if I used the cleaner txt file.

The several second load time is likely due to parsing the entire txt file for both btc and pts each time.

What I think I'll do to speed it up is store daily summary information (top deposit, number of deposits, sum of deposits) in a MySQL database.  Then, all I would need is a csv file with the current period data for each blockchain.

Great work by the way.  Thanks for parsing the two blockchains and posting the live data.

[5chdn]

With the help of Bootstrap, I hacked together a page using PHP to beautify donshoe's parsed txt files.  Still a work in progress.

http://joelooney.org/ags/

Nice, pls check if this helps you:

UPDATE: CLEAN CSV FILES HERE:
http://q39.protoshar.es/ags/4/pts.csv.txt
http://q39.protoshar.es/ags/4/btc.csv.txt

These files are improved.

[wasthatawolf]

With the help of Bootstrap, I hacked together a page using PHP to beautify donshoe's parsed txt files.  Still a work in progress.

http://joelooney.org/ags/

[stuartcharles]

great work

[5chdn]

You can use command line args to start parsing at a certain day. To get today's stats only, just run it with:

Code: [Select]

$ ruby pts_chain.rb 38069

[mint chocolate chip]

Really nice, easy to understand the daily breakdown looking at this, I'll surely be revisiting it over the next couple hundred days

[5chdn]

Thanks for all the donations!

And now vote me for MMC executive!

[cass]

https://coinplorer.com/PTS/Transactions/bb2c83d49d66f253a949b902bf3ea885ff25c325fb309cf4e175a9a466fc880a

Donations sent - Danke Donschoe für deinen Einsatz thx for your efforts to the community

Cass

[aasl]

nice work!

[5chdn]

Who can I send a 1 PTS tip to for this bot?

amazing work ... thx donschoe pls provide tip jar

As mentioned in the first post, the donation addresses are: 1Bzc7PatbRzXz6EAmvSuBuoWED96qy3zgc or PcDLYukq5RtKyRCeC1Gv5VhAJh88ykzfka 

Thanks! If you have any feature requests let me know!

[cass]

amazing work ... thx donschoe pls provide tip jar

[bytemaster]

Who can I send a 1 PTS tip to for this bot?

[bytemaster]

The bot on IRC now tells us the value of AGS:

21:34:30     donschoe | .ags pts
21:34:30   +sternburg | Angleshares value: 2.247 AGS/PTS = 8.97 USD. Total donations today: 2225.249 PTS.

what channel is this bot in?

on freenode:

Code: [Select]

    bot.rooms.new("#memorycoin").join
    bot.rooms.new("#extasie").join
    bot.rooms.new("##mmc-spam").join
    bot.rooms.new("#protoshares").join
    bot.rooms.new("#bitsharestalk").join
    bot.rooms.new("##pts-spam").join
    bot.rooms.new("##primecoin").join
    bot.rooms.new("#beeeeer.org").join
    bot.rooms.new("##xpm-spam").join

Awesome work guys!

[5chdn]

The bot on IRC now tells us the value of AGS:

21:34:30     donschoe | .ags pts
21:34:30   +sternburg | Angleshares value: 2.247 AGS/PTS = 8.97 USD. Total donations today: 2225.249 PTS.

what channel is this bot in?

on freenode:

Code: [Select]

    bot.rooms.new("#memorycoin").join
    bot.rooms.new("#extasie").join
    bot.rooms.new("##mmc-spam").join
    bot.rooms.new("#protoshares").join
    bot.rooms.new("#bitsharestalk").join
    bot.rooms.new("##pts-spam").join
    bot.rooms.new("##primecoin").join
    bot.rooms.new("#beeeeer.org").join
    bot.rooms.new("##xpm-spam").join

[toast]

Is the BTC parser updating properly?  I sent BTC to the AGS address yesterday, but the last entry on the BTC parser is dated 2013-12-29.

Edit:  I see it's updating now.

Yes the BTC chain is still being parsed. The bitcoin network is huge and thats why it will take some hours to catch up!

The bot on IRC now tells us the value of AGS:

21:34:30     donschoe | .ags pts
21:34:30   +sternburg | Angleshares value: 2.247 AGS/PTS = 8.97 USD. Total donations today: 2225.249 PTS.

what channel is this bot in?

[5chdn]

Is the BTC parser updating properly?  I sent BTC to the AGS address yesterday, but the last entry on the BTC parser is dated 2013-12-29.

Edit:  I see it's updating now.

Yes the BTC chain is still being parsed. The bitcoin network is huge and thats why it will take some hours to catch up!

The bot on IRC now tells us the value of AGS:

21:34:30     donschoe | .ags pts
21:34:30   +sternburg | Angleshares value: 2.247 AGS/PTS = 8.97 USD. Total donations today: 2225.249 PTS.

[seraphim]

Wow, great!
We definitely needed that!

[Evan]

Is the BTC parser updating properly?  I sent BTC to the AGS address yesterday, but the last entry on the BTC parser is dated 2013-12-29.

Edit:  I see it's updating now.

[5chdn]

Thanks for posting!
I see in the code it should reset for a new day but
http://q39.protoshar.es/ags/4/pts.txt
didn't output what the code should. Don't know if it's a bug or you're running different code, just wanted to point it out

Thanks for your feedback!

The code will reset every day. Note that all payments from 2013 are counted to Jan/01/2014. Thats why you dont see any reset yet.

[maqifrnswa]

Thanks for posting!
I see in the code it should reset for a new day but
http://q39.qhor.net/ags/pts.txt
didn't output what the code should. Don't know if it's a bug or you're running different code, just wanted to point it out

[5chdn]

I've written a ruby parser which displays AGS donations in realtime by parsing the BTC and PTS blockchains.

It's open source and available on github. https://github.com/donSchoe/ags-parser

See realtime parsing on my demo site  (v0.4):
http://q39.protoshar.es/ags/4/pts.csv.txt
http://q39.protoshar.es/ags/4/btc.csv.txt
http://q39.protoshar.es/ags/3/balances.json

Using git to backup the chain every hour: http://git.jdqirc.net/donschoe/ags-datafeed
http://git.jdqirc.net/donschoe/ags-datafeed/raw/master/4/pts.csv.txt
http://git.jdqirc.net/donschoe/ags-datafeed/raw/master/4/btc.csv.txt
http://git.jdqirc.net/donschoe/ags-datafeed/raw/master/3/balances.json

You can use it to create some nice angelshares sites and push the community forward. I'm no frontend guy, but would be nice to see this in action. The text version is enough for my personal needs
It's the first time I tried to programmatically parse a blockchain, damn nice experience! But dont expect magic and be aware of bugs!

Report them to me or on github or use this testing thread: https://bitsharestalk.org/index.php?topic=1989.0

If you like it, feel free to send some Coins my way:
BTC: 1Bzc7PatbRzXz6EAmvSuBuoWED96qy3zgc
PTS: PcDLYukq5RtKyRCeC1Gv5VhAJh88ykzfka

Cheers!


Urgent: Upgrade to v0.4 immidiatly: v0.3 is bugged, v0.1 API is inaccurate and will be disabled.


Overview: Sites using my API so far, donate some some coins too, if you like them:

Yo @earthbound, a tip would be nice.

http://agsexplorer.com

More features will be added.

BTC: 1Bb6V1UStz45QMAamjaX8rDjsnQnBpHvE8
PTS: PgULYtxcDJM7SKABfECipFiSjrmtjiPQd9

If you found my PHP front end and source code useful (http://joelooney.org/ags/), you can tip me here (no pressure  )...

BTC:  16TsCgrKwQ3efzw6LkNuBHmvP6aJy21HGq
PTS:  PYDhxRmdd8YQdYLD757s7robuvafFdMARi

Thanks!

Note: this code will show the previous period's data until BTC and PTS donations are made in the current period.

Update: removed links to older versions.