Programmatically extending a datafeed

This issue has had me stumped for at 2+ weeks and I've decided to reach out for help.

A common question I see about backtrader is how to extend a datafeed to include additional columns in the input data file (e.g. indicators calculated outside of backtrader). This has been thoroughly documented and a good example is shown in pandas-data-optix.py which is partially shown below:

What if we didn't know the names of the additional columns (or there were hundreds of columns that are unique to each data file) and want to automatically add every column not in the standard list (below)?

The heavy use of Metaclasses is beyond my knowledge level at the moment and using the debugger to follow how the PandasDataOptix class is created step-by-step is difficult.

The issue is that the lines, params, and datafields variables must be populated when the PandasDataOptix class is created. The MetaBase.__call__() method calls Lines.__init__() which cycles through the supplied lines and creates a LineBuffer for each. Since this all happens before PandasDataOptix.__init__() no arguments can be passed to the class (such as lines, params, and datafields).

I got it working by manually re-running several *._derive() methods to overwrite some of the PandasDataOptix class attributes after it was created but this is very hacky and surely not the best way:

You are absolutely correct! Despite hours of searching for how to pass arguments to a class as it's created I never came across this alternative method for creating a class dynamically. It works perfectly.

You have taught me so much about Python just from snooping through backtrader code. Thank you Daniel!

In case anyone else finds this useful here is the data-pandas-optix.py file (backtrader/samples/data-pandas) with all parameters defined externally from the PandasData class. The original class creation is included but commented out.