Programme outline
Learning objectives
- Manipulate different types of data, such as images, videos, etc.
- Implement neural network models and layers matching the type of data that the model is supposed to process.
- Obtain an overview of more advanced computer vision models, such as generative AI (artificial intelligence) and adversarial machine learning.
Day 1
- Introduction of course, reminders of the previous deep learning course
- Bringing all students up to speed on the content of the previous course, on which this course will build up. Project announcements.
- An introduction to computer vision and image processing
- The image datatype and the problem of using linear layers on images. The convolution operation and its use for image processing.
- The convolution operation and the Conv2D layer
- Implementation of the convolution operation with its different flavours. Creating a Conv2d layer in PyTorch, using it in our first Convolutional Neural Network, using it on the MNIST dataset.
- State of the Art CV models and their important lessons
- Overview of the state-of-the-art models in CV and the important lessons and layers they brought to the world of computer vision: Dropout, BatchNorm, LayerNorm, Residuals, etc.
Day 2
- Adversarial machine learning
- The ideas and lessons of adversarial machine learning, key concepts and objectives of AML.
- Gradient-based adversarial machine learning
- Discussing attack methods using gradients of the model against itself. Defense mechanisms to make our computer vision models more robust to attacks.
- About sequential data and the video data type
- Discussing the sequential datasets, their uses in data science and the challenge they pose to our deep learning models. A first attempt at building a deep learning model for sequential data.
- Recurrent neural networks and vision transformer networks
- A primer on GRUs and LSTMs and a brief introduction to vision transformers
Day 3
- Introduction to generative models
- From autoencoders to generative adversarial networks.
- Advanced generative adversarial networks and their ethical implications
- Discussing the advanced implementations of generative adversarial networks, followed by a brief discussion about ethics and generative AI.
- Project consultations (optional)
Mode of assessment
- Class participation, 10%
- Quizzes, 30%
- Project, 60%