FYS-8601 Self-Supervised Learning - 5 stp

This course will provide both a basic understanding of the techniques used in self-supervised learning and the very recent developments of the field. The course will cover self-supervised learning in the context of important data domains such as image, text, and time series data, and give the students a deeper understanding of the theory that underpins current self-supervised methods. The course will consist of 5 days of teaching with both lectures and practical components.

Core concepts

• History of self-supervised learning.
• Motivation and fundamental concepts.
• Data augmentation, encoders, projections head, and loss functions.

Self-supervised learning in computer vision

• Contrastive methods.
• Non-contrastive methods.
• Clustering-based methods.
• Masking-based methods.

Self-supervised learning in NLP

• SSL for text modelling.
• Multi-modal SSL: Language & Vision.

Self-supervised learning in time series analysis

• Noise-based methods.
• Temporal-based methods.

Advanced self-supervised learning theory

• Role of projection head.
• Effect of data augmentation.
• XAI in representation learning.
• Loss functions analysis.

Relevance of course in program of study : Learning a meaningful representation from data without human supervision is a well established problem in machine learning. However, recent works on self-supervised representation learning have achieved impressive performance on a wide range of different tasks involving data types such as images, time series, and text. In some cases, the performance in these unsupervised method can even rival their supervised counterparts. Self-supervised approaches learn representations in a similar manner as in supervised learning, but creates the labels from unlabeled datasets. How to produce the labels from unlabeled datasets and how to formulate a self-supervised loss are key components in self-supervised learning.

PhD students or holders of a Norwegian master´s degree of five years (300 ECTS) or 3 (180 ECTS) + 2 years (120 ECTS) or equivalent may be admitted. PhD students must upload a document from their university stating that there are registered PhD students. This group of applicants does not have to prove English proficiency and are exempt from semester fee. Holders of a Master´s degree must upload a Master´s Diploma with Diploma Supplement / English PhD students at UiT The Arctic University of Norway register for the course through StudentWeb. External applicants apply for admission through SøknadsWeb. All external applicants have to attach a confirmation of their status as a PhD student from their home institution. Students who hold a Master of Science degree, but are not yet enrolled as a PhD-student have to attach a copy of their master's degree diploma. These students are also required to pay the semester fee.

Recommended prerequisites: Programming skills in python and hands on knowledge of python programming for deep learning.

Application code: 9304, application deadline: May 1st

The course is limited to 40 places. Qualified applicants are ranked on the basis of a lottery if there are more applicants than available places.

Knowledge - The student is able to

• describe advanced self-supervised techniques
• Desribe the role of domain-knowledge in self-supervised learning.
• describe the development of self-supervised learning
• discuss recent developments in the field
• discuss advanced self-supervised learning in an applied setting

Skills - The student is able to

• explain the fundamental ideas behind self-supervised learning
• apply the learned material to new applications or problem settings
• use self-supervised learning for research and industrial settings using software libraries such as e.g. Pytorch or TensorFlow
• make appropriate method and architecture choices for a given application or problem setting

General competence - The student is able to

• give an interpretation of recent developments and provide an intuition of the open questions in the field of self-supervised learning
• Show an understanding of why self-supervised learning has shown great improvements over the last couple of years

Lectures: 20 hours

Self-study sessions: 40 hours

Project work: spread over 8 weeks - net time 50 hours

Hands-on sessions: 10 hours

Net effort (~120 hours)

The date for the exam can be changed. The final date will be announced at your faculty early in May and early in November.