Yann LeCun’s Deep Learning Course at CDS


Description

This course concerns the latest techniques in deep learning and representation learning, focusing on supervised and unsupervised deep learning, embedding methods, metric learning, convolutional and recurrent nets, with applications to computer vision, natural language understanding, and speech recognition. The prerequisites include: DS-GA 1001 Intro to Data Science or a graduate-level machine learning course.


DS-GA 1008 · SPRING 2021 

Instructors​: Lectures – Yann LeCun | Practicum – Alfredo Canziani
Lectures​: ​Mondays, 9:30 – 11:30am EST, Zoom
Practica: ​Tuesdays, 9:30 – 10:30am EST
Forum​: ​r/NYU_DeepLearning
Discord: ​NYU DL
Material: 2021 Repo

Please note we’re officially supporting direct communication with students taking this course online via our Reddit and Discord platforms.


2021 edition disclaimer

Check the repo’s README.md and learn about:

  • Content new organisation
  • The semester’s second half intellectual dilemma
  • This semester repository
  • Previous releases

Lectures

Most of the lectures, labs, and notebooks are similar to the previous edition, nevertheless, some are brand new. I will try to make clear which is which.

Legend: ? slides, ? notes, ? Jupyter notebook, ? YouTube video.


Translations

?? English | ?? French

If you’re interested in assisting the Deep Learning team with translation, please contact Alfredo Canziani at canziani@nyu.edu.


Theme 1: Introduction


Theme 2: Parameters sharing


Theme 3: Energy based models, foundations


Theme 4: Energy based models, advanced


Theme 5: Associative memories


Theme 6: Graphs


Theme 7: Control

  1. Planning and control ? ?
  2. The Truck Backer-Upper ? ? ?
  3. Prediction and Planning Under Uncertainty ? ?

Theme 8: Optimisation

  • Optimisation (I) [from last year] ? ?
  • Optimisation (II) ? ? ?

Miscellaneous



DS-GA 1008 · SPRING 2020 · CDS

Instructors​: Lectures – Yann LeCun | Practicum – Alfredo Canziani
Lectures​: ​Mondays, 16:55 – 18:35
Practica: ​Tuesdays, 19:10 – 20:00
Material​: Google Drive, Notebooks
NYU Deep Learning Reddit


Lectures

Legend: ? slides, ? Jupyter notebook, ? YouTube video.


Translations

?? English | ?? Arabic | ?? Bengali, Bangla | ?? Chinese | ?? French | ?? Hungarian | ?? Italian | ?? Japanese | ?? Korean | ?? Persian | ?? Portuguese | ?? Russian | ?? Turkish | ?? Serbian | ?? Spanish | ?? Vietnamese


*Contribution Instructions


Week Format Title Resources
1 Lecture 1.1. Motivation of Deep Learning, and Its History and Inspiration ?️ ?
1.2. Evolution and Uses of CNNs and Why Deep Learning?
Practicum 1.3. Problem Motivation, Linear Algebra, and Visualization ? ? ?
2 Lecture 2.1. Introduction to Gradient Descent and Backpropagation Algorithm ?️ ?
2.2. Computing gradients for NN modules and Practical tricks for Back Propagation
Practicum 2.3. Artificial neural networks (ANNs) ? ? ? ?
3 Lecture 3.1. Visualization of neural networks parameter transformation and fundamental concepts of convolution ?️ ?
3.2. ConvNet Evolutions, Architectures, Implementation Details and Advantages
Practicum 3.3. Properties of natural signals ? ? ?
4 Practicum 4.1. Linear Algebra and Convolutions ? ?
5 Lecture 5.1. Optimisation Techniques I ?️ ?
5.2. Optimisation Techniques II
Practicum 5.3. Understanding convolutions and automatic differentiation engine ? ? ?
6 Lecture 6.1. Applications of Convolutional Network ?️ ?️ ?
6.2. RNNs, GRUs, LSTMs, Attention, Seq2Seq, and Memory Networks
Practicum 6.3. Architecture of RNN and LSTM Model ? ? ?️ ?
7 Lecture 7.1. Energy-Based Models ?️ ?
7.2. SSL, EBM with details and examples
Practicum 7.3. Introduction to autoencoders ?️ ? ?
8 Lecture 8.1. Contrastive Methods in Energy-Based Models ?️ ?
8.2. Regularized Latent Variable Energy Based Models
Practicum 8.3. Generative Models – Variational Autoencoders ?️ ? ?
9 Lecture 9.1. Discriminative Recurrent Sparse Auto-Encoder and Group Sparsity ?️ ?
9.2. World Models and Generative Adversarial Networks
Practicum 9.3. Generative Adversarial Networks ?️ ? ?
10 Lecture 10.1. Self-Supervised Learning – Pretext Tasks ?️ ?
10.2. Self-Supervised Learning – ClusterFit and PIRL
Practicum 10.3. The Truck Backer-Upper ?️ ? ?
11 Lecture 11.1. Activation and loss functions (part 1) ?️ ?️ ?️ ?
11.2. Loss Functions (cont.) and Loss Functions for Energy Based Models
Practicum 11.3. Prediction and Policy learning Under Uncertainty (PPUU) ?️ ? ?
12 Lecture 12.1. Deep Learning for NLP ?️ ?
12.2. Decoding Language Models
Practicum 12.3. Attention and the Transformer ?️ ? ?
13 Lecture 13.1. Graph Convolutional Networks I ?️ ?
13.2. Graph Convolutional Networks II
Practicum 13.3. Graph Convolutional Networks III ?️ ? ?
14 Lecture 14.1. Deep Learning for Structured Prediction ?️ ?
14.2. Graphical Energy-based Methods
Practicum 14.3. Overfitting and regularization ?️ ? ?️ ? ?
15 Practicum 15.1. Inference for Latent-Variable Energy-Based Models (EBMs) ?️ ?
15.2. Training Latent-Variable Energy-Based Models (EBMs) ?️ ?

People


Yann LeCun

Role: Instructor
About: Silver Professor of Computer Science and Data Science, NYU and Turing Award Winner
Contact: yann@cs.nyu.edu

Alfredo Canziani

Role: Practicum Instructor
About: Assistant Professor/Faculty Fellow (2020 – 2022) of Computer Science
Contact: canziani@nyu.edu

Mark Goldstein

Role: Assistant
About: PhD student in CS at NYU
Contact: goldstein@nyu.edu

Zeming Lin

Role: Webmaster
About: PhD student in CS at NYU
Contact: zl2799@nyu.edu



— Yann Creative Commons License
Deep Learning by New York University, Yann LeCun, Alfredo Canziani is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Based on a work at https://cds.nyu.edu/deep-learning/