Optimization Approach to a Block of Layers and Derivative Free Optimization

Abstract

I present a new view to the output activations of a block of layers in deep neural networks. In particular, we view the output activation of a linear operator, convolutional or fully connected, followed by a non-linearity, and followed by another linear operator as an approximate solution to a certain convex optimization problem. We show that replacing layers with optimization layers as solvers improve performance in different settings.

Adel Bibi

Senior Researcher in Machine Learning and R&D Distinguished Advisor

My research interests include machine learning, computer vision, and optimization.