I have an intresset in control theory - optimal control. But unfortunately all control theories are for linear models. That's become a very difficult issue when to implement a linear controller for a nonlinear system. It works, but it requries lots of tuning and taking account to exceptions.
I have done a library that can take account on limitation in the system. It's called Adaptive Model Predictive Control. The library is written in 100% C code and can be applied on all embedded systems. It's just C code that does math stuff.
I have now plants to extend CControl so the library can train a deep neural network for image classification, e.g robotic view. Also I want to use deep neural network for reinforcement learning, e.g controller for heavy nonlinear system that not even a transfer function/state space model can be described in.
CControl have Recursive Least Squares, but that algorithm is only ment for system that can be described onto a transfer function and the system have slow change in its parameters. Not complex and fast.
So my question for you is:
What type of math do I need to use for training a deep neural network?
I have done some research by my self and I know that I need to use backpropagation with an optimization algorithm. But I can only do backpropagation for 1-single layer with one perceptron. I need to do backpropgagation with linear algebra style. Also with an optimization algorithm as well.
So how can I use backpropagation with linear algebra and what optimization algorithm should I use? Notice that the optimization algorithm need to be adaptive and I cannot have an algorthm that requries lots of knowledge about the system, because I don't have any knowledge about my data.
$\endgroup$ 5 Reset to default
