Bin backup format

[svk]

Deposit keys are just addresses for withdrawal of OPEN.X assets, blocktrades has made use of the database to store them.

[bilthon]

Well, since writing the original post I did find out some more about the format being used here. As you said the "linked_accounts"  seems to be an array of accounts that the keys in either the "wallet" or "private_keys" happen to control. The idea I had that these could be an array of accounts controlling another account doesn't seem to be the correct one.

Still even though I managed to create a backup file that the web wallet will successfully import and list, it is not yet working. Mainly because it doesn't look like the web wallet is being able to actually read the private keys from the file. I'm not sure where the problem might be, and I do recognize there are a couple of fields missing in my backup file, but none of these ("password_pubkey" and "brainkey_pubkey") should be needed in order to retrieve the private keys.

Here's what I'm creating so far:

Code: [Select]

{
  "linked_accounts": [
    {
      "chainId": "4018d7844c78f6a6c41c6a552b898022310fc5dec06da467ee7905a8dad512c8",
      "name": "bilthon-5"
    }
  ],
  "private_keys": [
    {
      "brainkey_sequence": 0,
      "encrypted_key": "81e68ca372d1c7596bbf0e462992cbb84f35cb62b3a615d1bb5b8d7af082672029f9ad7f4c273ac21ed5436bec08bd30",
      "id": 0,
      "pubkey": "BTS8DuGHXpHYedq7qhT65BEEdQPvLT8nxZ862Hf8NgvSZUMuwUFkn"
    }
  ],
  "wallet": [
    {
      "backup_date": "2017-02-19T22:49:01",
      "brainkey_backup_date": "2017-02-19T22:49:01",
      "brainkey_sequence": 0,
      "chain_id": "4018d7844c78f6a6c41c6a552b898022310fc5dec06da467ee7905a8dad512c8",
      "created": "2017-02-19T22:49:01",
      "encrypted_brainkey": "f22f5c4bbae9620a11bdc3794dec73ad9ef2c16c3a822cd7f3e5851e34623ee1085291f65a88dd7a714bffadcd81ab9ddabcc1c35871f5795cb68d79f628e503395ac8a657f38d063144b703b46ab50d604679eada05f1a31696066025fb50b4",
      "encryption_key": "ae00c24d22b507893841ff67770976b84bbd97259814b92641ad7caf2d37730d0861a3561cd5590d92d0759596f428a2",
      "id": "bilthon-5",
      "last_modified": "2017-02-19T22:49:01",
      "public_name": "bilthon-5"
    }
  ]
}
And to add more information to the discussion, here's the decrypted and decompressed contents of the backup file the web wallet is generating.

Code: [Select]

{
  "linked_accounts": [
    {
      "name": "bilthon83",
      "chainId": "4018d7844c78f6a6c41c6a552b898022310fc5dec06da467ee7905a8dad512c8"
    }
  ],
  "private_keys": [
    {
      "brainkey_sequence": 0,
      "encrypted_key": "19a787f3c6c1c848e20f9e69a233c7d7d44db44170ed539e1d2349378698e4ea226958a887927f81e56fe491599db2be",
      "id": 1,
      "pubkey": "BTS5WMHRj65FK3KyRpS3wJp5iC1WWnScHunRLGoCMM4A2NCLDKy6i"
    },
    {
      "brainkey_sequence": 1,
      "encrypted_key": "9e3f5a5484bfb4563fe6177a93994f6412d96f956b29f9a86a743fcdfe209a2d1a6748bbaab55ef5efed8d376b2a326c",
      "id": 2,
      "pubkey": "BTS59LjmiQ3hh9ZMK5gYyj7EjFvuf3YJFCbU5pEQ1gzatBsPycwoJ"
    }
  ],
  "wallet": [
    {
      "public_name": "default",
      "password_pubkey": "BTS7bq4tVD5CLBTPowT4WJJHmSh8YumfoDYMiZHiCzNWQMq8DcGFX",
      "encryption_key": "3fe90371fa1d8306a12fa2b96e14424650a4747c05d118f534f5aa28342e24d6b2c7d3d7f6123fd5b49d0f57758c49fc",
      "encrypted_brainkey": "90d601625c32118e16ad87721593ecfb82c6afe01f4424f1aae640d5c2202cb8051921b9e1499104e8c55b6f11452484c05fcc3f143f7fd91c5aed96982a5fed491dde5f30f2144d22d23eb242c3df0d6168ea1382186154c011a588244599da07151862a5f5a0cb9aec5dad0a08c583",
      "brainkey_pubkey": "BTS4xgzQXHK96ojeTHWCNGZpERCL8wi359y7KgiR2EmUfp9KdHsGm",
      "brainkey_sequence": 2,
      "brainkey_backup_date": null,
      "created": "2016-07-22T17:48:18.639Z",
      "last_modified": "2017-02-19T21:58:17.560Z",
      "chain_id": "4018d7844c78f6a6c41c6a552b898022310fc5dec06da467ee7905a8dad512c8",
      "id": "default",
      "backup_date": "2017-02-19T22:35:40.599Z",
      "deposit_keys": {
        "openledger": {
          "[1][bilthon83][agrs][open.agrs]": [
            {
              "address": "13Kevab5ZKVYCAfbHbSaEaAUDCW4vwXgEw",
              "memo": null
            },
            {
              "address": "1HmaSdGSAkwJ9ajPe1cVybzhHrZm14GGVy",
              "memo": null
            },
            {
              "address": "1D5tKoMeBS82d2QvM8nzwXzoJnGpFMLLPH",
              "memo": null
            }
          ],
          "[1][bilthon83][btc][open.btc]": [
            {
              "address": "1NY76B7Ames7sLw2wp3HwM3nCJR3o9E3oE",
              "memo": null
            },
            {
              "address": "1AZqqYVirxpRSJJgZGzUGimk9QdEzZwEpj",
              "memo": null
            },
            {
              "address": "1Mzcxk3bVThTWR7ywDEHzJujEAMdGYrPMm",
              "memo": null
            }
          ]
        }
      }
    }
  ]
}

As you can see, here we not only have the 2 previously mentioned fields in the wallet object ("password_pubkey" and "brainkey_pubkey") but also this "deposit_keys" with a bunch of BTC addresses (?!). This just added more questions to my head.

Anyone can help answer some of them?

Best regards
Nelson R. Pérez

[svk]

My guess is a lot of the things that seem redundant deal with complex wallets with many accounts, and especially the accounts imported from BTS 1 where you would have private keys that did not directly correspond to an account for example.

Linked accounts are a way to track accounts added manually via private key iirc.

The private keys array can contain keys added manually that do not belong to the brain key of the wallet. When you load such a wallet the GUI makes an api call to find out which of those private keys have accounts associated with them, and it also tracks the accounts that don't in order to not look them up more than once.

[bilthon]

I just wrote some code that decrypts and uncompresses the .bin file resulting from the backup procedure available at the Open Ledger, and have some comments and questions regarding this specific topic.

I would appreciate if other more informed parties can engage in a discussion here, and maybe that could help me expand the documentation about this feature, since I feel that it is somewhat lacking in that aspect.

First lets take a look at the resulting JSON string:

Code: [Select]

{
  "wallet": [
    {
      "public_name": "bilthon-25",
      "password_pubkey": "BTS7bq4tVD5CLBTPowT4WJJHmSh8YumfoDYMiZHiCzNWQMq8DcGFX",
      "encryption_key": "9f95ca531a6750f3749ea8a2b70202bd59a9a051875608c04237cab3fda7f0c31129297a36af9c46f83a68713389a83d",
      "encrypted_brainkey": "b71760d18703a7bba4c2525277dfedb9042aab58c71190deeb05bc90f2097bf634bc0533ac375395cfd0263cd3614abf22f90e53e7e7b88ca811f2b614ff053f60e9ec842bdd034adaa5f795e25c9615a6d9a60c73f03dc101c97bf7877d7897",
      "brainkey_pubkey": "BTS5hfdU3ihm8wDPYuavWrGFopGySMcgEewmCDvTGPV29o14mtp7p",
      "brainkey_sequence": 1,
      "brainkey_backup_date": "2017-02-14T21:07:19.819Z",
      "created": "2017-02-14T21:07:20.193Z",
      "last_modified": "2017-02-14T21:07:20.699Z",
      "chain_id": "4018d7844c78f6a6c41c6a552b898022310fc5dec06da467ee7905a8dad512c8",
      "id": "bilthon-25",
      "backup_date": "2017-02-14T21:08:56.382Z"
    }
  ],
  "private_keys": [
    {
      "encrypted_key": "2ed26a282ae4ed06b594c91a352036d79a35c6c1a24d9d42523a74b5edde2c0f71155097cc9f6306a92a25ed2b846b83",
      "pubkey": "BTS5ynSurmrfWj1HrF81wxaLuAtTbp4yxWDsZzc7b93XPEK4aiR1i",
      "brainkey_sequence": 0,
      "id": 1
    }
  ],
  "linked_accounts": []
}
All the sensitive information here is encrypted, so I guess there's no harm in publishing it here. And the referring account is just a test one with not much funds in it anyways.

1- My first observation regarding this object is the replication of data in both the "public_name" and "id" fields. They are always the same. No matter what I do, I always seem to be getting the name of my wallet there. Which in this case just happened to be the same as my account name, but that is not necessarily always the case. Anyways I don't see the point of having 2 fields which always hold the same data.

2- The previous remak actually leads me to the second point, which is that it would be really useful to have the name of the account a key is referring to also stated here. I do recognize that the authority scheme can be more complex than just 1 key per account, and maybe this information was not included because of the desire  to just keep things simple. But I don't see why the JSON object could not be expanded in order to accomodate a current snapshot of an account's authorities for instance.

3- There's also an issue with the "brainkey_sequence" field. I noticed it is being 1. It turns out that this account specifically was created outside of the web wallet with a key derivated using the sequence value of zero. This is suggesting me that the this field is not actually being dynamically generated. Or at least not for imported accounts. The "brainkey_sequence" was assumed to be 1 here, when in fact it was 0, and thus the information provided here is wrong.

4- Also there's the "private_keys" array. This field can also seem kind of redundant, since the information contained in the brainkey and brainkey_sequence can be very well be used to derive the encrypted key at the field called "encrypted_key".  I say kind of, because I acknowledge situations where we might have just the private key and not an associated brain key, and we might be interested in store that into the bin backup. But my recommendation would be to use this field only for those specific cases.

5- And now my final point has to do with the "linked_accounts" field. This is a question actually (remember I told you I had one? . I modified bilthon-25 to be controlled by bilthon-24, assuming that this would make the "linked_accounts" array have at least one element. But nothing happened here. So I'm in the dark with respect of what this field is actually supposed to have. Does anyone have any idea?

Best regards
Nelson R. Pérez