Federal University of Rio Grande do Norte

Technology Center

Graduate Program in Electrical and Computer Engineering

Department of Computer Engineering and Automation

EEC2318 Machine Learning

References

• 📚 Godoy, Daniel. Deep Learning with PyTorch - Step by Step. [Link]
• 📚 Tam, Adrian. Deep Learning with PyTorch. [Link]
• 📚 Cristina, Stefania; Saeed, Mehreen. Building Transformer with Attention. [Link]
• 📚 Huyen, Chip. Designing Machine Learning Systems. [Link]

Week 01: Course Outline

• Detailed breakdown of the course structure and content, exploring various aspects and applications of Machine Learning.
• Motivation, Syllabus, and other issues.

Week 02: Fundamentals of Machine Learning

• Fundamentals and first steps about the study of Machine Learning models.

Week 03: Visualizing Gradient Descent

• Effective Theory Approach in Deep Learning: big-picture idea.
• Visualizing Gradient Descent: The lesson walks through the process of implementing gradient descent in a linear regression model, covering: Initialization and training of neural networks, viewing them as a set of parameters and functions. Detailed steps of gradient descent, including initializing parameters, computing predictions, computing loss, computing gradients, and updating parameters. The significance of learning rates and their influence on the gradient descent process.
• Gradient Descent with PyTorch
• Critical Evaluation and Key Takeaways

Week 04: Rethinking the training loop: a simple classification problem

• Rethinking the training loop:
  • build a function to perform training steps, implement our own dataset class, use data loaders to generate mini-batches
  • build a function to perform mini-batch gradient descent, evaluate our model
  • save / checkpoint our model to disk
  • load our model from disk to resume training or to deploy
• Going Classy:
  • define a class to handle model training
  • implement the constructor method
  • understand the difference between public, protected, and private methods of a class
  • integrate the code we’ve developed so far into our class
  • instantiate our class and use it to run a classy pipeline
• A simple classification problem:
  • build a model for binary classification
  • understand the concept of logits and how it is related to probabilities
  • use binary cross-entropy loss to train a model
  • use the loss function to handle imbalanced datasets
  • understand the concepts of decision boundary and separability

Week 05: Machine Learning and Computer Vision - Part I

• From a shallow to a deeep-ish clasification model:
  • data generation for image classification
  • transformations using torchvision
  • dataset preparation techniques
  • building and training logistic regression and deep neural network models using PyTorch
  • focusing on various activation functions like Sigmoid, Tanh, and ReLU