TIdDLe Forum

Rufin VanRullen

Administrator

Registered: 2019-03-08

Posts: 18

Website

ANITI seminar: Mathieu Chalvidal, @LAAS-CNRS, Friday October 30th @3pm

Mathieu Chalvidal, ANITI PhD student in the chairs of Thomas Serre and Rufin VanRullen, will present his recent work on:

"A control perspective on parameterization in infinite-depth neural networks''.
Abstract: The perspective of artificial neural modules as mathematical functions describing a vector field over a topological activity space opens interesting perspectives for deep learning: This view interprets the sequential computation of stacked or recurrent neural modules in traditional deep neural networks as the discrete evolution of a dynamical system, such that a particular parametrization of the module gives rise to particular trajectories of hidden state/activity tensor given their initial point (input stimuli). Recently, new approaches have developed further this view suggesting to solve the ordinary differential equation (Neural ODE) associated with the said module function, such that model depth corresponds to integration over time. This interpretation culminates in the assimilation of neural networks as particular flows. Despite their elegant formulation and lightweight memory cost, neural ordinary differential equations (NODEs) suffer from known representatio!
nal limitations. In particular, the single flow learned by NODEs cannot express all homeomorphisms from a given data space to itself, and their static weight parametrization restricts the type of functions they can learn compared to discrete architectures with layer-dependent weights. Here, we describe a new module called neurally- controlled ODE (N-CODE) designed to improve the expressivity of NODEs. The parameters of N-CODE modules are dynamic variables governed by a trainable map from initial or current activation state, resulting in forms of open-loop and closed-loop control, respectively.

Anyone is welcome to join this ANITI seminar in-person at the LAAS (just say you come for the ANITI talk). The seminar will also be accessible through the link:: https://seminar.laas.fr/b/gui-lyf-mp4-6v7 (no need for a password,  just identify yourself with a name or pseudo to enter the virtual room).

Rufin VanRullen

Administrator

Registered: 2019-03-08

Posts: 18

Website

Re: ANITI seminar: Mathieu Chalvidal, @LAAS-CNRS, Friday October 30th @3pm

[lockdown update]:
This talk will still happen, but 100% virtual. Obviously, do not go to LAAS for this talk. The link is still valid: https://seminar.laas.fr/b/gui-lyf-mp4-6v7
See you all at 3pm!