Type of course

The course can be taken as a singular course. Registration is open for UiT students and members of NORA Research School and Digital Life Norway Research School. It will be conducted as a concentrated course in the style of summer- and winter school courses conducted under NORA/DLN, etc.

NOTE: First lecture will be in the beginning of May and will be given digitally.

Admission requirements

Bachelor's degree of 180 ECTS or equivalent. The bachelor’s degree could be within several subject areas for example in computer science, mathematics, statistics, physics ect.

Higher level of English language proficiency must be documented, see here for more information of approved documentation of this requirement.

Recommended prerequisites: Programming skills in python and / or INF-1400. Hands on knowledge of python programming for deep learning.

Application code: 9302

Application deadline: March 15th, 2023.

The course is limited to 20 places. Qualified applicants are ranked on the basis of a lottery if there are more applicants than available places.

Course content

This course will consider different topics of importance regarding interpretable deep learning, equipping the students with knowledge of approaches that can be used to explain deep learning, and deep learning approaches that are more explainable than others. In addition, the students will receive practical skills of applying selected approaches for explaining deep learning, which will equip the students with practical skills of adapting to the rapid pace of technology development in the field of explainable artificial intelligence / interpretable deep learning.

Introductory concepts (3 hours)

• Explainability and interpretability
• Black box models versus explainable models
• Knowledge versus performance, need of explainability
• Example case(s)

Model-agnostic approaches (6 hours)

• Knowledge abstraction and encoding
• Interpretation and visualization of abstract encoding - concepts and techniques, visual explanation
• Feature importance and feature interaction
• Counterfactual methods

Neural networks and explainability (6 hours)

• Network visualization and neuron interaction
• Tracing and explainable backpropagation
• Depth of network, and abstraction
• Class activation maps
• Saliency and attention models
• Fuzzy neural networks - type 1 and 2

Extensive lab work, self-exercises and groups are planned for competence development is also included.

Relevance of course in program of study: Artificial intelligence (AI) and deep learning approaches are often considered as black boxes, i.e. as a type of algorithms that accomplish learning tasks but cannot explain how. However, as AI/deep learning is increasingly absorbed as adopted for accomplishing cognitive tasks for human beings, it is becoming important that the deep learning models are understandable by humans, such that artificial and human intelligence can co-exist and collaborate. In critical tasks such as deriving, from given data, a correct medical diagnosis and prognosis, collaboration between artificial and human intelligence in imperative so that the suggestions or decision from artificial intelligence are both more accurate and more trustworthy.

Recommended prerequisites

Objectives of the course

A general educational aim of the course will be to equip students with knowledge and skills regarding interpretable artificial intelligence/deep learning, for considering explainable approaches for solving a neural learning problem as well as for developing an explanation for an existing knowledge model developed by a black box approach. This will enable the students to understand, work with, and solve deep learning tasks with a balance of explainability and accuracy, as needed.

An brief introduction to explainable / interpretable deep learning is being offered in this course. This will fill the knowledge gap for those who want to learn more about deep learning and develop trustworthy, reliable deep learning models. Recognizing the significance of interpretable models for computationally-intensive deep learning architectures, as well as the analysis and comprehension of complex biological applications, and the need for cross-disciplinary collaborations in future biotech, medicine, and AI.

Knowledge - The student

• understands the need for explainability and the basic concepts underlying the ability or disability of a black-box model being interpretable.
• learns about the knowledge versus performance trade-off and examples to build an interpretable system.
• learns the various model-agnostic approach for interpretability and the underlying concept of knowledge encoding and abstractions in neural networks, which differs from the traditional knowledge encoding in classical machine learning systems.
• knows a variety of explainable approaches of deep learning
• knows tools and visualization techniques used for making black-box models explainable
• comprehends the neural network and its complexity, learning about tools to explain the abstraction of deep networks.
• learns about fuzzy learning in neural network and its basics in making a system more explainable.

Skills - The student can

• apply explainable approaches for problem solving
• compute feature importance and counterfactual explanation
• visualize knowledge encoding in the model
• compute class activation and saliency maps and popular visualization techniques.
• compare different deep learning approaches for explainability and performance

General competence - The student has developed

• the confidence of assessing the explainability / interpretability of a deep learning model
• the confidence and skills of making a knowledge model explainable
• the confidence and skills of making choices and discussing solutions for interpretable deep learning
• enough background and confidence to discuss, assess, and potentially contribute to advances and newest developments in explainable/interpretable deep learning
• perspectives about the value, ethics, and compromises of explainable/interpretable deep learning.

Language of instruction and examination

Teaching methods

Lectures: 15 hours

Self-study session: 30 hours

Project work: spread over 8 weeks - net time 50 hours

Group / Self-work session: 20 hours

Hands-on session: 3 hours

Project consultation session: 4 hours

Net effort (~125 hours)

Note! First lecture will be in the beginning of May and will be given digitally.