BitShares Forum
Main => General Discussion => Topic started by: biophil on January 25, 2018, 08:11:16 pm
-
I'm trying to compile a good record of my Bitshares activity, and the web wallet doesn't give you much history. Does anybody have a nice CSV history service or anything? Or a better web interface that lets you download history?
-
you can see all your(and others) full account history in the open explorer by going to the account page like http://open-explorer.io/#/accounts/biophil in the full account history section.
this uses a connection to a full traditional node.
then you can use also the csv exporter located here: http://open-explorer.io/bitshares-account-exporter/
and use your account id(1.2.1361), start date need to be something before 2015-10 and end date now to get all the history for your account.
the account exporter is not very friendly in its output right now but it will get all the operations made in a period of time, some improvements will came out soon to make it compatible with for example cointracking.info by exporting only the fill orders in a friendly format.
the exporter is connected to an elasticsearch bitshares node, this node is having some problems in the last few days as the server where it is located cant handle it anymore so it can be slow to export or down when you try. if this is the case let me know and ill send you the csv as i already downloaded it for your account now that it is working.
the elasticsearch node is going to be moved to a better server i already have in my power and the migration will be done between tomorrow and this weekend. this will allow to have a more stable elasticsearch node that will be able to bring richer features to the open-explorer and other applications.
hope it helps.
-
Thanks for the pointer. I tried the CSV exporter, and it looks like something is wrong -- it hangs for a while (30 seconds?) and then gives me a CSV that only has column headers but no actual data. Nice explorer, by the way. I only just learned about it today; has it been operating a while?
I'll PM you my email address to send me what you exported!
-
It has 30 sec timeout hard coded. So you better retrieve your history in chunks, which can be processed in less than 30 sec.
-
It has 30 sec timeout hard coded. So you better retrieve your history in chunks, which can be processed in less than 30 sec.
I'll try that more deliberately, but I tried something like that already and didn't get anything. But I'll give it another go.
-
it's the server issue, i am configuring new server right now. will put the chain to sync in a bit and in some hours i will have you guys try the exporter again, everything should work fine.
-
Ok guys, sorry the delay. I had moved elasticsearch to a new node and now it is fully in sync and working. I changed the exporter to point to the new place, please try again:
http://open-explorer.io/bitshares-account-exporter/
I also increased the timeout to 60 seconds so big accounts can be downloaded without the need to go over it in chunks.
I confused at first with your account, thought it was biophil but i see now it is http://open-explorer.io/#/accounts/zebulon with 3782 operations.
I was able to download your account history now, sending the csv by email too but you can go ahead and try to download from the online tool.
thanks for the patience.
-
Hello. I'm also trying to download all of my transactions in order to reconcile my account. I followed the above steps and it appeared to work. The issue is the CSV file has plenty of data, but not the simple data I'm looking for - such as fill price and amount of bts purchased.
Appreciate any help!
-
Hello. I'm also trying to download all of my transactions in order to reconcile my account. I followed the above steps and it appeared to work. The issue is the CSV file has plenty of data, but not the simple data I'm looking for - such as fill price and amount of bts purchased.
Appreciate any help!
I'll be writing some Python tools to parse the CSV file into human-readable data, if that's any help. I'll come back here with a github link when/if I get the tools working in a publicly accessible way.
-
Ok guys, sorry the delay. I had moved elasticsearch to a new node and now it is fully in sync and working. I changed the exporter to point to the new place, please try again:
http://open-explorer.io/bitshares-account-exporter/
I also increased the timeout to 60 seconds so big accounts can be downloaded without the need to go over it in chunks.
I confused at first with your account, thought it was biophil but i see now it is http://open-explorer.io/#/accounts/zebulon with 3782 operations.
I was able to download your account history now, sending the csv by email too but you can go ahead and try to download from the online tool.
thanks for the patience.
Fantastic, looks like the tool works fine now, and I also got the CSV by email. Thanks very much!
-
Hello. I'm also trying to download all of my transactions in order to reconcile my account. I followed the above steps and it appeared to work. The issue is the CSV file has plenty of data, but not the simple data I'm looking for - such as fill price and amount of bts purchased.
Appreciate any help!
I'll be writing some Python tools to parse the CSV file into human-readable data, if that's any help. I'll come back here with a github link when/if I get the tools working in a publicly accessible way.
that will be awesome, i plan to improve it a bit but don't have the time so any help is very appreciated.
-
Looking forward to the update. Thanks a lot!
-
Just wondering if there has been some progress made on the transaction history?
-
Don't get me wrong, I'm a huge bitshares fan. However, getting a complete an accurate account history should be an automatic for an exchange!
-
OK.Ok.Ok
So I've spent 10 hours going over the csv script to figure out my transactions. Was making fabulous progress until I realized there appears to be no difference between a buy or sell order. So frustrating!! Can you tell the difference?? The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts.
Please help - I'm dying - just figuratively of course!
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.49894451","account_id":"1.2.493907","pays":{"amount":66903,"asset_id":"1.3.121"},"receives":{"amount":1728763,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":1935000,"asset_id":"1.3.121"},"quote":{"amount":50000000,"asset_id":"1.3.0"}},"is_maker":true}]
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.47254046","account_id":"1.2.493907","pays":{"amount":1126,"asset_id":"1.3.121"},"receives":{"amount":19263,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":2750000,"asset_id":"1.3.121"},"quote":{"amount":47009350,"asset_id":"1.3.0"}},"is_maker":true}]
-
OK.Ok.Ok
So I've spent 10 hours going over the csv script to figure out my transactions. Was making fabulous progress until I realized there appears to be no difference between a buy or sell order. So frustrating!! Can you tell the difference?? The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts.
Please help - I'm dying - just figuratively of course!
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.49894451","account_id":"1.2.493907","pays":{"amount":66903,"asset_id":"1.3.121"},"receives":{"amount":1728763,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":1935000,"asset_id":"1.3.121"},"quote":{"amount":50000000,"asset_id":"1.3.0"}},"is_maker":true}]
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.47254046","account_id":"1.2.493907","pays":{"amount":1126,"asset_id":"1.3.121"},"receives":{"amount":19263,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":2750000,"asset_id":"1.3.121"},"quote":{"amount":47009350,"asset_id":"1.3.0"}},"is_maker":true}]
Both above are of same side, paid 1.3.121 which is bitUSD and received 1.3.0 which is BTS.
Take a look at other records.
-
I guess I'm the only one that feels an accurate transaction history should be one of the first priorities for an exchange??
-
Where exactly do you think our history is incomplete or inaccurate?
-
OK.Ok.Ok
So I've spent 10 hours going over the csv script to figure out my transactions. Was making fabulous progress until I realized there appears to be no difference between a buy or sell order. So frustrating!! Can you tell the difference?? The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts.
Please help - I'm dying - just figuratively of course!
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.49894451","account_id":"1.2.493907","pays":{"amount":66903,"asset_id":"1.3.121"},"receives":{"amount":1728763,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":1935000,"asset_id":"1.3.121"},"quote":{"amount":50000000,"asset_id":"1.3.0"}},"is_maker":true}]
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.47254046","account_id":"1.2.493907","pays":{"amount":1126,"asset_id":"1.3.121"},"receives":{"amount":19263,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":2750000,"asset_id":"1.3.121"},"quote":{"amount":47009350,"asset_id":"1.3.0"}},"is_maker":true}]
Both above are of same side, paid 1.3.121 which is bitUSD and received 1.3.0 which is BTS.
Take a look at other records.
That is my entire point Abit - they were NOT the same side! The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts. However the code makes them both look identical...and therefore impossible to reconcile from the csv script data.
-
Where exactly do you think our history is incomplete or inaccurate?
Let me give you a clear example.
Code from CSV
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.50478545","account_id":"1.2.493907","pays":{"amount":2335000,"asset_id":"1.3.121"},"receives":{"amount":50000000,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":2335000,"asset_id":"1.3.121"},"quote":{"amount":50000000,"asset_id":"1.3.0"}},"is_maker":true}]
Shows I paid 233.5 BitUS to buy 500 bts, but in fact I did the exact opposite which is was copied from "My Trades" on the main BTS trading screen which only lasts for 2 weeks.
s 0.467 500 233.5 2018-01-26 1:12
This is the accurate trade - where I sold 500 BTS at a price of .467 to receive 233.5 BitUS. The only difference is the green (buy) or the red (sell) from "My Trades" which is not translated in the CSV script code.
I now hope this is clear.
Greatly appreciate your input and assistance on this. I just want an accurate picture for tax purposes.
-
OK.Ok.Ok
So I've spent 10 hours going over the csv script to figure out my transactions. Was making fabulous progress until I realized there appears to be no difference between a buy or sell order. So frustrating!! Can you tell the difference?? The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts.
Please help - I'm dying - just figuratively of course!
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.49894451","account_id":"1.2.493907","pays":{"amount":66903,"asset_id":"1.3.121"},"receives":{"amount":1728763,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":1935000,"asset_id":"1.3.121"},"quote":{"amount":50000000,"asset_id":"1.3.0"}},"is_maker":true}]
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.47254046","account_id":"1.2.493907","pays":{"amount":1126,"asset_id":"1.3.121"},"receives":{"amount":19263,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":2750000,"asset_id":"1.3.121"},"quote":{"amount":47009350,"asset_id":"1.3.0"}},"is_maker":true}]
Both above are of same side, paid 1.3.121 which is bitUSD and received 1.3.0 which is BTS.
Take a look at other records.
That is my entire point Abit - they were NOT the same side! The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts. However the code makes them both look identical...and therefore impossible to reconcile from the csv script data.
I think you misunderstood. They ARE both sells or both buys. Paste more records here or somewhere else (e.g. pastebin), I can help find the records on the other side for you.
Perhaps there is a bug in the csv exporter.
Bottom line, accurate full history are on the chain.
I do agree that we do need good tools to export the histories, it's important (we do have many important things to be done). Perhaps you can setup a service for it and earn some money.
-
I am also interested in improving the export function and UI.
Any suggestion which tools to use?
-
I am trying to view my account activity
When I go to http://open-explorer.io/#/accounts/darkf0rce, I can see the first page of my activity but when I click on any other page, it comes back with a blank (it shows me total of 42 pages of activity)
Any suggestions?
-
I am trying to view my account activity
When I go to http://open-explorer.io/#/accounts/darkf0rce, I can see the first page of my activity but when I click on any other page, it comes back with a blank (it shows me total of 42 pages of activity)
Any suggestions?
seems there is a problem with the pagination . ill fix during this weekend among other things in the explorer. if you can pls submit an issue here as a reminder https://github.com/oxarbitrage/open-explorer/issues
thank you very much.
-
fixed, thanks https://github.com/oxarbitrage/open-explorer/issues/28
-
OK.Ok.Ok
So I've spent 10 hours going over the csv script to figure out my transactions. Was making fabulous progress until I realized there appears to be no difference between a buy or sell order. So frustrating!! Can you tell the difference?? The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts.
Please help - I'm dying - just figuratively of course!
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.49894451","account_id":"1.2.493907","pays":{"amount":66903,"asset_id":"1.3.121"},"receives":{"amount":1728763,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":1935000,"asset_id":"1.3.121"},"quote":{"amount":50000000,"asset_id":"1.3.0"}},"is_maker":true}]
[4,{"fee":{"amount":0,"asset_id":"1.3.0"},"order_id":"1.7.47254046","account_id":"1.2.493907","pays":{"amount":1126,"asset_id":"1.3.121"},"receives":{"amount":19263,"asset_id":"1.3.0"},"fill_price":{"base":{"amount":2750000,"asset_id":"1.3.121"},"quote":{"amount":47009350,"asset_id":"1.3.0"}},"is_maker":true}]
Both above are of same side, paid 1.3.121 which is bitUSD and received 1.3.0 which is BTS.
Take a look at other records.
That is my entire point Abit - they were NOT the same side! The top script is a sell from bts to bitUS. The bottom script is a buy from bitUS to bts. However the code makes them both look identical...and therefore impossible to reconcile from the csv script data.
I think you misunderstood. They ARE both sells or both buys. Paste more records here or somewhere else (e.g. pastebin), I can help find the records on the other side for you.
Perhaps there is a bug in the csv exporter.
Bottom line, accurate full history are on the chain.
I do agree that we do need good tools to export the histories, it's important (we do have many important things to be done). Perhaps you can setup a service for it and earn some money.
Hello Abit,
I've been doing more 'detective' work on the csv exporter. It seems recently (early February) that the exporter is correctly identifying the “pays” asset and the “receive” asset. However, the exporter misses a few actual transactions – about 10%!!
For instance, I purchased some BTS on March 6 (see below) but these transactions do not even appear on the CSV exporter data! I have found other missing data points as well.
PRICE BTS B DATE
0.180000 400.00000 72.0000 03/6 12:22:42
0.190000 250.00000 47.5000 03/6 04:39:30
A couple of other questions
1. Where would I find a list of assets?
For instance - asset_id":"1.3.121 is bitUSD
2. How do I get on the BTS Chain? I’ve tried the URL below – but I almost always get a message that something went wrong – problem with a node?
https://www.cryptofresh.com/
3. The ‘operations’ section from the CSV exporter has 6 values 0-5
I believe they correspond to the following. Am I accurate?
0 – a transfer of some sort- it appears anyone can transfer a ‘new’ asset to a bts account holder
1 – limit order
2 – cancel order
3 – Unsure
4 – Executed order
5 – Start of a new account
You asked me if I would be interested in trying to help out with this service? I certainly would! I’m not much of a programmer at all, but I will put in the time if you will direct me…
Thanks again for your assistance. I want BTS to succeed because I believe it levels the playing field for everyone and I feel Stan Larimer’s character and integrity is first rate!
-
let me answer this @Spady007
in regards to the exporter, thanks for the notification. the elasticsearch server that the bitshares exporter uses to get data is at full disk so some of the last transactions are not getting inserted. i am requesting more disk at the server, then i need to replay the chain and will be working again, i'll update here.
1- for a list of assets if cryptofresh is down you can use http://open-explorer.io/#/assets(not complete list) or if you know the asset name or the id you can do http://open-explorer.io/#/assets/1.3.3027 or if you know the name do http://open-explorer.io/#/assets/NEXTCOIN
2- there are some problems pretty often with cryptofresh but there is a private project, we cant do anything for it. as an alternative you may use other explorers like the built in explorer of the reference wallet https://wallet.bitshares.org/#/explorer/blocks, the already mentioned open explorer(http://open-explorer.io/#/dashboard) or a new explorer project(https://bts.ai/?locale=en).
3- a list of operation number - operation name can be found at: http://open-explorer.io/#/fees
-
the exporter problem was fixed as more hd was added to the elasticsearch server.
please retry: http://open-explorer.io/bitshares-account-exporter/index.php
account ID: 1.2.493907
start date: 2018-03-01 09:00:22
end date: 2018-03-22 09:55:22
account page: http://open-explorer.io/#/accounts/1.2.493907
-
Thanks Oxarbitrage! I will update my transactions. Hoping Bitshares 3.0 when released will be more user friendly and have a full (at least 1 year) transaction list. I figured my transactions using CSV, but quite an arduous process!!
-
Thank You oxarbitrage Your answer solve my Problem.
-
I'm trying to determine how to compute the price paid based on the fill price and quote amount. Can anyone help?
[4,
{
"fee":{"amount":2000,"asset_id":"OPEN.EOS"},
"order_id":"1.7.62838114",
"account_id":"1.2.2503",
"pays":{"amount":4700001,"asset_id":"BTS"},
"receives":{"amount":1000000,"asset_id":"OPEN.EOS"},
"fill_price":{"base":{"amount":157093302,"asset_id":"OPEN.EOS"},
"quote":{"amount":738338519,"asset_id":"BTS"}},
"is_maker":false}]
-
there is a new exporter from @svk that i will check to see if we can merge the 2 together to have full account report with cointracking.info format:
https://github.com/svk31/bitshares-report
-
you can see all your(and others) full account history in the open explorer by going to the account page like http://open-explorer.io/#/accounts/biophil in the full account history section.
this uses a connection to a full traditional node.
then you can use also the csv exporter located here: http://open-explorer.io/bitshares-account-exporter/
and use your account id(1.2.1361), start date need to be something before 2015-10 and end date now to get all the history for your account.
the account exporter is not very friendly in its output right now but it will get all the operations made in a period of time, some improvements will came out soon to make it compatible with for example cointracking.info by exporting only the fill orders in a friendly format.
the exporter is connected to an elasticsearch bitshares node, this node is having some problems in the last few days as the server where it is located cant handle it anymore so it can be slow to export or down when you try. if this is the case let me know and ill send you the csv as i already downloaded it for your account now that it is working.
the elasticsearch node is going to be moved to a better server i already have in my power and the migration will be done between tomorrow and this weekend. this will allow to have a more stable elasticsearch node that will be able to bring richer features to the open-explorer and other applications.
hope it helps.
Thank You oxarbitrage your post help me a lot.
-
Thanks for the pointer. I tried the CSV exporter, and it looks like something is wrong -- it hangs for a while (30 seconds?) and then gives me a CSV that only has column headers but no actual data. Nice explorer, by the way. I only just learned about it today; has it been operating a while?
-
just tried and it is working fine. make sure you use your account id(1.2.X) and not your name. should work.