Reflecting to optimise | Magnus Ross
- This is nothing to be proud of, but I have never really studied optimisation in depth.
- Oh sure, I know my Adam from my AdaGrad and I even used L-BFGS one time, but when people start talking about dual spaces and convergence for L∞L^\infty continuous functions, I tend to glaze over a bit.
- For some reason I think in my head a lot of it seemed a bit old fashioned, whatever that means; why do I need to learn about optimising constrained convex functions in the gradient-descent-for-everything-wait-actually-just-use-a-pre-trained-model era?
Unverified
- This is nothing to be proud of, but I have never really studied optimisation in depth.
- Oh sure, I know my Adam from my AdaGrad and I even used L-BFGS one time, but when people start talking about dual spaces and convergence for L∞L^\infty continuous functions, I tend to glaze over a bit.
- For some reason I think in my head a lot of it seemed a bit old fashioned, whatever that means; why do I need to learn about optimising constrained convex functions in the gradient-descent-for-everything-wait-actually-just-use-a-pre-trained-model era?
Sources: Github