One thing that came to mind we should discuss is the list of sources witnesses should use during our coordinated experimentation.
Looking over Zapata's script I see it supports a rather large list of feed sources, some require an API key and others don't. I would be surprised if every witness has all API keys for the sources that require them.
If all witnesses are not using the same list of feed sources it introduces an uncontrolled an unnecessary variable that needs to be understood so it doesn't adversely affect the goal of finding optimal feed and algorithm parameters.
So lets determine which feed sources we should use for each BitAsset we'll be optimizing the feeds for. Once that is known witnesses can obtain the necessary API keys for those sources. Waiting for the day of testing to do this is unwise. Witnesses need to prepare, and be ready when experimentation starts which means having the necessary API keys for the feed sources we agree to use.
If a fee is required to obtain any API keys we'll use it or the requirements for a source are beyond what a witness is willing / able to pay, perhaps some witnesses could share API keys to use while testing.
Another consideration is experimentation might reveal weaknesses of certain sources and we might decide to change sources during experimentation.
Feed sources are only one aspect of things we should coordinate among witnesses. Feed publish frequency is another. In the early days of graphene witnesses picked a time slot (0 - 59 minutes past the hour) to publish feeds to assure someone was always publishing a feed at almost anytime. Is the timing each witness publishes feeds important? Could it affect the algorithm?
I hope to see more people and witnesses participate in discussing these things here, otherwise I feel experimentation will be chaotic and hard to follow, and I'll have less confidence the algorithm was actually "engineered".
Lets start a list of feed sources we will use for BitUSD. Which sources are witnesses currently using?
Although I have temporarily disabled feeds for BitUSD, here are the sources I was using for that:
Binance, Bitfinex, BitStamp, CoinMarketCap, CoinCap, AEX, ZB, Livecoin
Those are listed in the config.yaml for bts_tools. The list of possible feed providers in bts_tools (list of code modules):
aex.py, bitfinex.py, bloomberg.py, coincap.py, fixer.py, poloniex.py, uphold.py,
binance.py, bitstamp.py, btc38.py, coinmarketcap.py, yunbi.py, bitcoinaverage.py,
bittrex.py, bter.py, currencylayer.py, livecoin.py, quandl.py, zb.py
bts_tools will be irrelevant for experimentation, but I wanted to kick off discussion of feed sources by describing what I'm using.
I have API keys for bitcoinaverage.py, currencylayer.py and quandl.py.