Seniorforsker
Martin Jullum
- Avdeling Statistisk modellering og maskinlæring
- Telefonnummer +47 22 85 26 08
- E-post Jullum@nr.stage.dekodes.no
Prosjekter
Publikasjoner
- 88 publikasjoner funnet
Jullum, Martin og Aas, Kjersti. (2026).
Seminar: Datadrevet antihvitvasking og svindeldeteksjon. Norsk Regnesentral
NVA
Annen presentasjon
Jullum, Martin. (2026).
shapr – Conditional Shapley Value Explanation in R and Python. Epidemiology and Data Science department, Amsterdam University Medical Centers
NVA
Vitenskapelig foredrag
Kapar, Jan; Koenen, Niklas og Jullum, Martin. (2026).
What’s Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models.
Vis sammendrag
Abstract Evaluating synthetic tabular data is challenging, since they can differ from the real data in so many ways. There exist numerous metrics of synthetic data quality, ranging from statistical distances to predictive performance, often providing conflicting results. Moreover, they fail to explain or pinpoint the specific weaknesses in the synthetic data. To address this, we apply explainable AI (XAI) techniques to a binary detection classifier trained to distinguish real from synthetic data. While the classifier identifies distributional differences, XAI concepts such as feature importance and feature effects, analyzed through methods like permutation feature importance, partial dependence plots, Shapley values and counterfactual explanations, reveal why synthetic data are distinguishable, highlighting inconsistencies, unrealistic dependencies, or missing patterns. This interpretability increases transparency in synthetic data evaluation and provides deeper insights beyond conventional metrics, helping diagnose and improve synthetic data quality. We apply our approach to two tabular datasets and generative models, showing that it uncovers issues overlooked by standard evaluation techniques.
Jullum, Martin. (2025).
Local Model-Agnostic Methods in Explainable AI -- Brief overview + a bit of Shapley values. University of Oslo
NVA
Vitenskapelig foredrag
Jullum, Martin; Kolstø, Johannes Voll og Lenkoski, Alex. (2025).
Usikkerhetsmodellering av spotprisprognoser – fase 1.
NVA
Rapport
Johannessen, Fredrik og Jullum, Martin. (2025).
Finding money launderers using heterogeneous graph neural networks.
Vis sammendrag
The finance industry depends on effective anti-money laundering (AML) systems to ensure compliance and maintain operational efficiency. However, existing AML systems, which are predominantly rule-based, frequently struggle to detect money laundering accurately. In particular, their inability to learn from historical data and properly account for diverse
customer behavior is problematic. Also accounting for the vast amounts of transactional data generated daily, this challenge calls for big data analytics and advanced machine learning techniques. In line with this, the present paper explores a graph neural network (GNN) approach, a state-ofthe-art machine learning technique, to identify money laundering activities within a large heterogeneous network constructed from real-world bank transactions and business role data from DNB, Norway’s largest bank. To this end, we extend the (homogeneous) Message Passing Neural Network (MPNN) architecture to operate on a heterogeneous graph, and demonstrate its strong performance in detecting money laundering activities. We showcase the suitability of utilizing GNN methodology to improve electronic surveillance systems for detecting money laundering, thereby contributing a pioneering approach to AML through the application of advanced data science techniques. To the best of our knowledge, this is the first publication applying heterogeneous GNNs for AML purposes with a large real-world heterogeneous network.
Breivik, Olav Nikolai; Skaug, Hans Julius; Jullum, Martin og Biuw, Martin. (2025).
Spatial Variation on Multiple Scales in Line Transect Data; the Case of Antarctic Fin Whales.
Olsen, Lars Henry Berge og Jullum, Martin. (2025).
Improving the Weighting Strategy in KernelSHAP.
Vis sammendrag
In Explainable AI (XAI), Shapley values are a popular model-agnostic framework for explaining predictions made by complex machine learning models. The computation of Shapley values requires estimating non-trivial contribution functions representing predictions with only a subset of the features present. As the number of these terms grows exponentially with the number of features, computational costs escalate rapidly, creating a pressing need for efficient and accurate approximation methods. For tabular data, the framework is considered the state-of-the-art model-agnostic approximation framework. approximates the Shapley values using a weighted sample of the contribution functions for different feature subsets. We propose a novel modification of which replaces the stochastic weights with deterministic ones to reduce the variance of the resulting Shapley value approximations. This may also be combined with our simple, yet effective modification to the variant implemented in the popular Python library. Additionally, we provide an overview of established methods. Numerical experiments demonstrate that our methods can reduce the required number of contribution function evaluations by 5% to 50 % while preserving the same accuracy of the approximated Shapley values – essentially reducing the running time by up to 50%. These computational advancements push the boundaries of the feature dimensionality and number of predictions that can be accurately explained with Shapley values within a feasible runtime.
Jullum, Martin og Olsen, Lars Henry Berge. (2025).
Improving the weighting strategy in KernelSHAP. University of Munich
Jullum, Martin. (2025).
Introduction to Local Model-Agnostic methods in XAI. Bjørnar Tessem
NVA
Faglig foredrag
Løland, Anders; Jullum, Martin; Ingebrigtsen, Didrik Sten og Gjuvsland, Elin Ruhlin. (2025).
Hva skjer når avfall møter algoritmer?
Vis sammendrag
Vi snakker med Didrik Sten Ingebrigtsen (medgründer og "Research Lead" i Sensorita) og Martin Jullum (seniorforsker på NR) om hvorfor Sensorita finnes og hvorfor avfallsbransjen ikke er så digital i dag. Men mest snakker vi om hva som skjer i prosjektet "RewaCC" – som på norsk kan skrives ut til "Komplette digitale tvillinger av avfallscontainere".
RewaCC er et såkalt Innovasjonsprosjekt i Næringslivet (IPN) og finansiert av Norges Forskningsråd. I prosjektet jobber NRs forskere med kompanse på signalbehandling, maskinlæring og statistisk modellering sammen med Sensorita for å utnytte radardata for å automatisk finne ut hva slags materiale en avfallscontainer er fylt med. RewaCC og Sensorita kan på sikt endre hele avfallsbransjen.
Sannsynligvis VIKTIG er en serie om KI og digitalisering produsert av Norsk Regnesentral, Anders Løland er programleder. Produsert av Elin Ruhlin Gjuvsland.
Pirbhulal, Sandeep; Abie, Habtamu; Jullum, Martin; Nielsen, Didrik og Løland, Anders. (2025).
AI/ML for 5G and Beyond Cybersecurity.
Vis sammendrag
The advancements in communication technology (5G and beyond) and global connectivity Internet of Things (IoT) also come with new security problems that will need to be addressed in the next few years. The threats and vulnerabilities introduced by AI/ML based 5G and beyond IoT systems need to be investigated to avoid the amplification of attack vectors on AI/ML. AI/ML techniques are playing a vital role in numerous applications of cybersecurity. Despite the ongoing success, there are significant challenges in ensuring the trustworthiness of AI/ML systems. However, further research is needed to define what is considered an AI/ML threat and how it differs from threats to traditional systems, as currently there is no common understanding of what constitutes an attack on AI/ML based systems, nor how it might be created, hosted and propagated [ETSI, 2020]. Therefore, there is a need for studying the AI/ML approach to ensure safe and secure development, deployment, and operation of AI/ML based 5G and beyond IoT systems. For 5G and beyond, it is essential to continuously monitor and analyze any changing environment in real-time to identify and reduce intentional and unintentional risks. In this study, we will review the role of the AI/ML technique for 5G and beyond security. Furthermore, we will provide our perspective for predicting and mitigating 5G and beyond security using AI/ML techniques
Redelmeier, Annabelle Alice; Jullum, Martin; Aas, Kjersti og Løland, Anders. (2024).
MCCE: Monte Carlo sampling of valid and realistic counterfactual explanations for tabular data.
Vis sammendrag
We introduce MCCE: Monte Carlo sampling of valid and realistic Counterfactual Explanations for tabular data, a novel counterfactual explanation method that generates on-manifold, actionable and valid counterfactuals by modeling the joint distribution of the mutable features given the immutable features and the decision. Unlike other on-manifold methods that tend to rely on variational autoencoders and have strict prediction model and data requirements, MCCE handles any type of prediction model and categorical features with more than two levels. MCCE first models the joint distribution of the features and the decision with an autoregressive generative model where the conditionals are estimated using decision trees. Then, it samples a large set of observations from this model, and finally, it removes the samples that do not obey certain criteria. We compare MCCE with a range of state-of-the-art on-manifold counterfactual methods using four well-known data sets and show that MCCE outperforms these methods on all common performance metrics and speed. In particular, including the decision in the modeling process improves the efficiency of the method substantially.
Olsen, Lars Henry Berge; Glad, Ingrid Kristine; Jullum, Martin og Aas, Kjersti. (2024).
A comparative study of methods for estimating model-agnostic Shapley value explanations.
Vis sammendrag
Shapley values originated in cooperative game theory but are extensively used today as a model-agnostic explanation framework to explain predictions made by complex machine learning models in the industry and academia. There are several algorithmic approaches for computing different versions of Shapley value explanations. Here, we consider Shapley values incorporating feature dependencies, referred to as conditional Shapley values, for predictive models fitted to tabular data. Estimating precise conditional Shapley values is difficult as they require the estimation of non-trivial conditional expectations. In this article, we develop new methods, extend earlier proposed approaches, and systematize the new refined and existing methods into different method classes for comparison and evaluation. The method classes use either Monte Carlo integration or regression to model the conditional expectations. We conduct extensive simulation studies to evaluate how precisely the different method classes estimate the conditional expectations, and thereby the conditional Shapley values, for different setups. We also apply the methods to several real-world data experiments and provide recommendations for when to use the different method classes and approaches. Roughly speaking, we recommend using parametric methods when we can specify the data distribution almost correctly, as they generally produce the most accurate Shapley value explanations. When the distribution is unknown, both generative methods and regression models with a similar form as the underlying predictive model are good and stable options. Regression-based methods are often slow to train but quickly produce the Shapley value explanations once trained. The vice versa is true for Monte Carlo-based methods, making the different methods appropriate in different practical situations.
Jullum, Martin og Aas, Kjersti. (2024).
Statistiske metoder versus maskinlæringsmetoder. Den Norske Aktuarforening
NVA
Faglig foredrag
Jullum, Martin; Aase, Frida Svendal og Aas, Kjersti. (2024).
More effective computation of Shapley values. Universitetet i Tromsø
NVA
Vitenskapelig foredrag
Breivik, Olav Nikolai og Jullum, Martin. (2024).
Leveraging Norwegian Data to Improve Danish Insurance Risk Models.
NVA
Rapport
Jullum, Martin og Aas, Kjersti. (2024).
Finetuning credit scoring ensemble models for FundingPartner.
NVA
Rapport
Løland, Anders og Jullum, Martin. (2024).
Hvorfor er maskinlæring nødvendig i kampen mot hvitvasking?
NVA
Programdeltagelse
Jullum, Martin; Aas, Kjersti og Løland, Anders. (2024).
Introduction to the 1st Oslo Invitational Workshop on Model-Agnostic Explainable AI. Norwegian Computing Center, BigInsight, Integreat
NVA
Faglig foredrag
Jullum, Martin; Aas, Kjersti; Løland, Anders; Aase, Frida Svendal og Olsen, Lars Henry Berge. (2024).
On conditional Shapley values for prediction explanation - Adaptive & variance stabilizing estimation with KernelSHAP. University of Bremen
NVA
poster
Jullum, Martin; Aas, Kjersti; Aase, Frida Svendal og Løland, Anders. (2024).
Recent computational advances in Shapley values based prediction explanation. Norwegian Statistical Association
NVA
Vitenskapelig foredrag
Jullum, Martin. (2024).
How to navigate in the Explainable AI jungle. BI Norwegian Business School
NVA
Faglig foredrag
Jullum, Martin. (2024).
Forskning på ML for hvitvaskingsdeteksjon. Norsk Regensentral
NVA
Faglig foredrag
Jullum, Martin; Løland, Anders; Prabhu, Robindra og Sjødin, Jacob. (2024).
eXplego: An XAI-method selection tool. Norwegian Computing Center, BigInsight, Integreat
NVA
Faglig foredrag
Abidi, Osama; Hubin, Aliaksandr; Frausig, Jesper og Jullum, Martin. (2023).
Using Graph Bayesian Neural Networks for fraud pattern detection and classification from bank transactions data.
Vis sammendrag
The thesis aims to conduct an investigation into money laundering networks by synthesizing many earlier methods. The methodology begins by training a Bayesian graph neural network on a data set from DNB to learn the relationship between the money laundering status of a transacting party and its features and network properties. One can then get node embeddings by extracting the activation values of each prediction at the last hidden layer in the model. This will hopefully yield informative node embeddings that one could then visualize along with their predicted class and associated uncertainty by applying dimensionality reduction through Principal Component Analysis. If there are nodes that have the same predicted class and magnitude of uncertainty that cluster together, one could try retrieving their associated networks and investigate the patterns.
Jullum, Martin. (2023).
Forsker tror ikke strømregningen blir mye lavere med ny støtteordning.
NVA
Intervju
Vis sammendrag
Forsker tror ikke strømregningen blir mye lavere med ny støtteordning: – Det er ikke så mye i kroner
Strømregningen blir neppe særlig mye lavere med regjeringens nye strømstøtteordning, ifølge seniorforsker Martin Jullum i Norsk Regnesentral. Men den blir mer forutsigbar.
Jullum, Martin; Sjødin, Jacob; Prabhu, Robindra og Løland, Anders. (2023).
eXplego: An interactive Tool that Helps you Select Appropriate XAI-methods for your Explainability Needs.
Vis sammendrag
The growing demand for transparency, interpretability, and explainability of machine learning models and AI systems has fueled the development of methods aimed at understanding the properties and behavior of such models (XAI). Since different methods answer different explainability questions, it is crucial to understand the kind of explanation the different XAI-methods provide, and in what situations they should be used. We introduce eXplego, an interactive tree-structured tool designed to assist users in selecting the most suitable XAI method for their use case. eXplego prompts users to answer questions regarding the type of explanation they seek, guiding them along the branches of the decision
tree for further inquiries. After 2-5 questions, the tree reaches one of its leaves to suggest an XAI method aligned with the user’s explainability need. The tool also provides helpful practical examples, simplified descriptions of the suggested method’s functionality and interpretability, points to consider when using the method, and links to the paper introducing the method, additional resources, and software implementations. The tool is developed from an in-depth study to discern the characteristics of the most prominent methods and the nature of the explanations they provide. We believe eXplego will help streamline the process of XAI method selection and contribute to the practical implementation of XAI in
various domains. The tool is available at explego.nr.no.
Tjøstheim, Dag Bjarne; Jullum, Martin og Løland, Anders. (2023).
Some recent trends in embeddings of time series and dynamic networks.
Vis sammendrag
We give a review of some recent developments in embeddings of time series and dynamic networks. We start out with traditional principal components and then look at extensions to dynamic factor models for time series. Unlike principal components for time series, the literature on time-varying nonlinear embedding is rather sparse. The most promising approaches in the literature is neural network based, and has recently performed well in forecasting competitions. We also touch on different forms of dynamics in topological data analysis (TDA). The last part of the article deals with embedding of dynamic networks, where we believe there is a gap between available theory and the behavior of most real world networks. We illustrate our review with two simulated examples. Throughout the review, we highlight differences between the static and dynamic case, and point to several open problems in the dynamic case.
Jullum, Martin. (2023).
Et forslag til strømstøtte basert på timespriser.
Vis sammendrag
Jeg foreslår at strømstøtten blir timebasert, med en støtte på 90 prosent av spotpris over cirka 87 øre. Den hindrer både gratisstrøm, negative priser og ekstrempriser og gir samme kostnad for staten.
Engebretsen, Solveig; Jullum, Martin og Løland, Anders. (2023).
Introduksjon til sentrale metoder i statistisk modellering og maskinlæring. Forsvarets forskningsinstitutt
NVA
Faglig foredrag
Tjøstheim, Dag Bjarne; Jullum, Martin og Løland, Anders. (2023).
Statistical Embedding: Beyond Principal Components.
Vis sammendrag
There has been an intense recent activity in embedding of very high-dimensional and nonlinear data structures, much of it in the data science and machine learning literature. We survey this activity in four parts. In the first part, we cover nonlinear methods such as principal curves, multidimensional scaling, local linear methods, ISOMAP, graph-based methods and diffusion mapping, kernel based methods and random projections. The second part is concerned with topological embedding methods, in particular mapping topological properties into persistence diagrams and the Mapper algorithm. Another type of data sets with a tremendous growth is very high-dimensional network data. The task considered in part three is how to embed such data in a vector space of moderate dimension to make the data amenable to traditional techniques such as cluster and classification techniques. Arguably, this is the part where the contrast between algorithmic machine learning methods and statistical modeling, represented by the so-called stochastic block model, is at its greatest. In the paper, we discuss the pros and cons for the two approaches. The final part of the survey deals with embedding in
R2, that is, visualization. Three methods are presented: t-SNE, UMAP and LargeVis based on methods in parts one, two and three, respectively. The methods are illustrated and compared on two simulated data sets; one consisting of a triplet of noisy Ranunculoid curves, and one consisting of networks of increasing complexity generated with stochastic block models and with two types of nodes.
Jullum, Martin. (2023).
Why AI needs maths and stats -- lessons from working in a CS field. Department of Mathematics, University of Oslo
NVA
Faglig foredrag
Jullum, Martin. (2023).
ML in AML - Machine Learning for Anti Money Laundering. Oslo Machine Learning Meetup
NVA
Faglig foredrag
Jullum, Martin; Aas, Kjersti; Løland, Anders og Redelmeier, Annabelle Alice. (2023).
A ridiculously simple approach to counterfactual explanations. NTNU Trondheim
NVA
Vitenskapelig foredrag
Jullum, Martin; Aas, Kjersti; Løland, Anders og Redelmeier, Annabelle Alice. (2023).
A ridiculously simple approach to counterfactual explanations. LMU Munich
NVA
Vitenskapelig foredrag
Pirbhulal, Sandeep; Abie, Habtamu; Jullum, Martin; Nielsen, Didrik og Løland, Anders. (2022).
AI/ML for 5G and Beyond Cybersecurity.
NVA
Rapport
Olsen, Lars Henry Berge; Glad, Ingrid Kristine; Jullum, Martin og Aas, Kjersti. (2022).
Using Shapley Values and Variational Autoencoders to Explain Predictive Models with Dependent Mixed Features.
Vis sammendrag
Shapley values are today extensively used as a model-agnostic explanation framework to explain complex predictive machine learning models. Shapley values have desirable theoretical properties and a sound mathematical foundation in the field of cooperative game theory. Precise Shapley value estimates for dependent data rely on accurate modeling of the dependencies between all feature combinations. In this paper, we use a variational autoencoder with arbitrary conditioning (VAEAC) to model all feature dependencies simultaneously. We demonstrate through comprehensive simulation studies that our VAEAC approach to Shapley value estimation outperforms the state-of-the-art methods for a wide range of settings for both continuous and mixed dependent features. For high-dimensional settings, our VAEAC approach with a non-uniform masking scheme significantly outperforms competing methods. Finally, we apply our VAEAC approach to estimate Shapley value explanations for the Abalone data set from the UCI Machine Learning Repository.
Jullum, Martin. (2022).
Prediction Explanation with Shapley values. CLArg Group, Department of Computing, Imperial College London
NVA
Vitenskapelig foredrag
Tjøstheim, Dag Bjarne; Jullum, Martin og Løland, Anders. (2022).
Statistical embedding: Beyond principal components. Matematisk institutt, Universitetet i Oslo
NVA
Faglig foredrag
Aas, Kjersti; Jullum, Martin og Løland, Anders. (2021).
Explaining individual predictions when features are dependent: More accurate approximations to Shapley values. International Joint Conferences on Artificial Intelligence
NVA
poster
Aas, Kjersti; Nagler, Thomas; Jullum, Martin og Løland, Anders. (2021).
Explaining predictive models using Shapley values and non-parametric vine copulas.
Vis sammendrag
In this paper the goal is to explain predictions from complex machine learning models. One method that has become very popular during the last few years is Shapley values. The original development of Shapley values for prediction explanation relied on the assumption that the features being described were independent. If the features in reality are dependent this may lead to incorrect explanations. Hence, there have recently been attempts of appropriately modelling/estimating the dependence between the features. Although the previously proposed methods clearly outperform the traditional approach assuming independence, they have their weaknesses. In this paper we propose two new approaches for modelling the dependence between the features. Both approaches are based on vine copulas, which are flexible tools for modelling multivariate non-Gaussian distributions able to characterise a wide range of complex dependencies. The performance of the proposed methods is evaluated on simulated data sets and a real data set. The experiments demonstrate that the vine copula approaches give more accurate approximations to the true Shapley values than their competitors.
Jullum, Martin; Redelmeier, Annabelle Alice og Aas, Kjersti. (2021).
groupShapley: Efficient prediction explanation with Shapley values for feature groups.
Jullum, Martin. (2021).
groupSHAP: Efficient Shapley value explanation through feature groups. SINTEF
NVA
poster
Främling, Kary; Westberg, Marcus; Jullum, Martin; Madhikermi, Manik og Malhi, Avleen Kaur. (2021).
Comparison of Contextual Importance and Utility with LIME and Shapley Values.
Vis sammendrag
Different explainable AI (XAI) methods are based on different notions of ‘ground truth’. In order to trust explanations of AI systems, the ground truth has to provide fidelity towards the actual behaviour of the AI system. An explanation that has poor fidelity towards the AI system’s actual behaviour can not be trusted no matter how convincing the explanations appear to be for the users. The Contextual Importance and Utility (CIU) method differs from currently popular outcome explanation methods such as Local Interpretable Model-agnostic Explanations (LIME) and Shapley values in several ways. Notably, CIU does not build any intermediate interpretable model like LIME, and it does not make any assumption regarding linearity or additivity of the feature importance. CIU also introduces the value utility notion and a definition of feature importance that is different from LIME and Shapley values. We argue that LIME and Shapley values actually estimate ‘influence’ (rather than ‘importance’), which combines importance and utility. The paper compares the three methods in terms of validity of their ground truth assumption and fidelity towards the underlying model through a series of benchmark tasks. The results confirm that LIME results tend not to be coherent nor stable. CIU and Shapley values give rather similar results when limiting explanations to ‘influence’. However, by separating ‘importance’ and ‘utility’ elements, CIU can provide more expressive and flexible explanations than LIME and Shapley values.
Jullum, Martin; Redelmeier, Annabelle Alice og Aas, Kjersti. (2021).
Efficient and simple prediction explanations with groupShapley: A practical perspective.
Vis sammendrag
Shapley values has established itself as one of the most appropriate and theoretically sound frameworks
for explaining predictions from complex machine learning models. The popularity of Shapley values
in the explanation setting is probably due to Shapley values’ unique theoretical properties. The main
drawback with Shapley values, however, is that the computational complexity grows exponentially in
the number of input features, making it unfeasible in many real world situations where there could be
hundreds or thousands of features. Furthermore, with many (dependent) features, presenting/visualizing
and interpreting the computed Shapley values also become challenging. The present paper introduces
and showcases a method that we call groupShapley. The idea of the method is to group features and then
compute and present Shapley values for these groups instead of for all individual features. Reducing
hundreds or thousands of features to half a dozen or so feature groups makes precise computations
practically feasible, and the presentation and knowledge extraction greatly simplified. We give practical
advice for using the approach and illustrate its usability in three different real world examples. The
examples vary in both data type (regular tabular data and time series), feature dimension (medium to
high), and application (insurance, genetics, and banking).
Jullum, Martin; Redelmeier, Annabelle Alice og Aas, Kjersti. (2021).
Efficient and simple prediction explanations with groupShapley: A practical perspective. XAI.it
NVA
Vitenskapelig foredrag
Jullum, Martin; Aas, Kjersti og Løland, Anders. (2021).
Explaining individual predictions when features are dependent: More accurate approximations to Shapley values. IJCAI
NVA
Vitenskapelig foredrag
Jullum, Martin; Redelmeier, Annabelle Alice og Aas, Kjersti. (2021).
Efficient Shapley value explanations through feature groups. The Arctic University of Norway
NVA
Vitenskapelig foredrag
Grotmol, Øyvind; Jullum, Martin; Aas, Kjersti og Scheuerer, Michael. (2021).
White paper on performance evaluation of volatility estimation
methods for Exabel.
NVA
Rapport
Grotmol, Øyvind; Scheuerer, Michael; Aas, Kjersti og Jullum, Martin. (2021).
Whitepaper on Exabel’s Factor Model.
NVA
Rapport
Aas, Kjersti; Jullum, Martin og Løland, Anders. (2021).
Explaining individual predictions when features are dependent: More accurate approximations to Shapley values.
Vis sammendrag
Explaining complex or seemingly simple machine learning models is an important practical problem. We want to explain individual predictions from such models by learning simple, interpretable explanations. Shapley value is a game theoretic concept that can be used for this purpose. The Shapley value framework has a series of desirable theoretical properties, and can in principle handle any predictive model. Kernel SHAP is a computationally efficient approximation to Shapley values in higher dimensions. Like several other existing methods, this approach assumes that the features are independent. Since Shapley values currently suffer from inclusion of unrealistic data instances when features are correlated, the explanations may be very misleading. This is the case even if a simple linear model is used for predictions. In this paper, we extend the Kernel SHAP method to handle dependent features. We provide several examples of linear and non-linear models with various degrees of feature dependence, where our method gives more accurate approximations to the true Shapley values.
Jullum, Martin. (2020).
How to open the black box – individual prediction explanation. Department of Mathematical Sciences, NTNU
NVA
Vitenskapelig foredrag
Vis sammendrag
Why did just you get a rejection on your loan application? Why is the price of your car insurance higher than that of your neighbor? More and more such decisions are made by complex statistical/machine learning models based on relevant data. Such (regression) models are often referred to as "black boxes" due to the difficulty of understanding how they work and produce different predictions. As these methods become increasingly important for individuals in our society, there is a clear need for methods which can help us understand their predictions, that is "open the black box". In this talk, I will motivate why this is useful and important. I will further discuss how Shapley values from game theory can be used as an explanation framework. To correctly explain the predictions, it is crucial to model the dependence between the covariates. I will exemplify this by showing that even a simple linear regression model is difficult to explain when the covariates are highly dependent. Finally, I will lay out recent work and methodology for modeling such dependence and how that leads to more accurate explanations through the Shapley value framework.
Jullum, Martin; Thorarinsdottir, Thordis Linda og Bachl, Fabian E.. (2020).
Estimating seal pup production in the Greenland Sea by using Bayesian hierarchical modeling.
Vis sammendrag
The Greenland Sea is an important breeding ground for harp and hooded seals. Estimates of annual seal pup production are critical factors in the estimation of abundance that is needed for management of the species. These estimates are usually based on counts from aerial photographic surveys. However, only a minor part of the whelping region can be photographed, because of its large extent. To estimate total seal pup production, we propose a Bayesian hierarchical modelling approach motivated by viewing the seal pup appearances as a realization of a log‐Gaussian Cox process by using covariate information from satellite imagery as a proxy for ice thickness. For inference, we utilize the stochastic partial differential equation module of the integrated nested Laplace approximation framework. In a case‐study using survey data from 2012, we compare our results with existing methodology in a comprehensive cross‐validation study. The results of the study indicate that our method improves local estimation performance, and that the increased uncertainty of prediction of our method is required to obtain calibrated count predictions. This suggests that the sampling density of the survey design may not be sufficient to obtain reliable estimates of seal pup production.
Jullum, Martin; Løland, Anders; Huseby, Ragnar Bang; Ånonsen, Geir og Lorentzen, Johannes P. (2020).
Detecting money laundering transactions with machine learning.
Vis sammendrag
Purpose
The purpose of this paper is to develop, describe and validate a machine learning model for prioritising which financial transactions should be manually investigated for potential money laundering. The model is applied to a large data set from Norway’s largest bank, DNB.
Design/methodology/approach
A supervised machine learning model is trained by using three types of historic data: “normal” legal transactions; those flagged as suspicious by the bank’s internal alert system; and potential money laundering cases reported to the authorities. The model is trained to predict the probability that a new transaction should be reported, using information such as background information about the sender/receiver, their earlier behaviour and their transaction history.
Findings
The paper demonstrates that the common approach of not using non-reported alerts (i.e. transactions that are investigated but not reported) in the training of the model can lead to sub-optimal results. The same applies to the use of normal (un-investigated) transactions. Our developed method outperforms the bank’s current approach in terms of a fair measure of performance.
Originality/value
This research study is one of very few published anti-money laundering (AML) models for suspicious transactions that have been applied to a realistically sized data set. The paper also presents a new performance measure specifically tailored to compare the proposed method to the bank’s existing AML system.
Redelmeier, Annabelle Alice; Jullum, Martin og Aas, Kjersti. (2020).
Explaining Predictive Models with Mixed Features Using Shapley Values and Conditional Inference Trees.
Vis sammendrag
It is becoming increasingly important to explain complex, black-box machine learning models. Although there is an expanding literature on this topic, Shapley values stand out as a sound method to explain predictions from any type of machine learning model. The original development of Shapley values for prediction explanation relied on the assumption that the features being described were independent. This methodology was then extended to explain dependent features with an underlying continuous distribution. In this paper, we propose a method to explain mixed (i.e. continuous, discrete, ordinal, and categorical) dependent features by modeling the dependence structure of the features using conditional inference trees. We demonstrate our proposed method against the current industry standards in various simulation studies and find that our method often outperforms the other approaches. Finally, we apply our method to a real financial data set used in the 2018 FICO Explainable Machine Learning Challenge and show how our explanations compare to the FICO challenge Recognition Award winning team.
Otneim, Håkon; Jullum, Martin og Tjøstheim, Dag Bjarne. (2020).
Pairwise local Fisher and naive Bayes: Improving two standard discriminants.
Vis sammendrag
The Fisher discriminant is probably the best known likelihood discriminant for continuous data. Another benchmark discriminant is the naive Bayes, which is based on marginals only. In this paper we extend both discriminants by modeling dependence between pairs of variables. In the continuous case this is done by local Gaussian versions of the Fisher discriminant. In the discrete case the naive Bayes is extended by taking geometric averages of pairwise joint probabilities. We also indicate how the two approaches can be combined for mixed continuous and discrete data. The new discriminants show promising results in a number of simulation experiments and real data illustrations.
Sellereite, Nikolai og Jullum, Martin. (2020).
shapr: An R-package for explaining machine learning models with dependence-aware Shapley values.
Vis sammendrag
A common task within machine learning is to train a model to predict an unknown outcome
(response variable) based on a set of known input variables/features. When using such models
for real life applications, it is often crucial to understand why a certain set of features lead to
a specific prediction. Most machine learning models are, however, complicated and hard to
understand, so that they are often viewed as “black-boxes”, that produce some output from
some input.
Shapley values (Shapley, 1953) is a concept from cooperative game theory used to distribute
fairly a joint payoff among the cooperating players. Štrumbelj & Kononenko (2010) and later
Lundberg & Lee (2017) proposed to use the Shapley value framework to explain predictions by
distributing the prediction value on the input features. Established methods and implementations for explaining predictions with Shapley values like Shapley Sampling Values (Štrumbelj
& Kononenko, 2014), SHAP/Kernel SHAP (Lundberg & Lee, 2017), and to some extent
TreeSHAP/TreeExplainer (Lundberg et al., 2020; Lundberg, Erion, & Lee, 2018), assume
that the features are independent when approximating the Shapley values. The R-package
shapr, however, implements the methodology proposed by Aas, Jullum, & Løland (2019),
where predictions are explained while accounting for the dependence between the features,
resulting in significantly more accurate approximations to the Shapley values.
Jullum, Martin. (2020).
Investigating mesh-based approximation methods for the normalization constant in the log Gaussian Cox process likelihood.
Vis sammendrag
The log Gaussian Cox process (LGCP) is a frequently applied method for modeling point pattern data. The normalization constant of the LGCP likelihood involves an integral over a latent field. That integral is computationally expensive, making it troublesome to perform inference with standard methods. The so‐called stochastic partial differential equation–integrated nested Laplace approximation (SPDE‐INLA) framework enables fast approximate inference for a range of hierarchical models, where a key component is to approximate the latent field by a triangulated mesh. Recent research has made it possible to fit LGCP models with this framework using an approximate integration method to compute the integral. We carefully describe several alternative variants of that approximate integration method and derive an analytical formula for the integral in question, which actually is exact under the triangular mesh assumption used by SPDE‐INLA. We compare the different integration strategies through a comprehensive simulation study and find that the analytical formula is often more accurate, but not always. Among the approximate integration methods, we recommend a simple extension to a method implemented in an R‐package for fitting LGCP models.
Jullum, Martin; Aas, Kjersti og Løland, Anders. (2019).
Opening the black box -- individual prediction explanation. Big Insight
NVA
Vitenskapelig foredrag
Jullum, Martin; Aas, Kjersti og Løland, Anders. (2019).
How to open the black box -- Individual prediction explanation. Norsk Statistisk Forening
NVA
Vitenskapelig foredrag
Jullum, Martin og Bolstad, Lars Erik. (2019).
Mindre rutinearbeid med maskinlæring -- Automatisk deteksjon av hvitvasking. Den norske dataforening
NVA
Faglig foredrag
Vis sammendrag
Globalt svindles det hvert år trolig for flere billioner amerikanske dollar bare gjennom hvitvasking. Dagens systemer for å forsøke å avdekke hvitvasking består i stor grad av regelbaserte alarmsystemer, etterfulgt av en møysommelig manuell analyse av alarmene. Dagens framgangsmåte er arbeidskrevende og ikke optimal for å avdekke nye hvitvaskingsmønstre. Med maskinlæring, gode data og skreddersøm kan vi automatisere og effektivisere hvitvaskingsjakten.
Kom og hør den enkle hemmeligheten som gir vår metode et forsprang til alternative metoder.
Redelmeier, Annabelle Alice; Aas, Kjersti; Jullum, Martin og Løland, Anders. (2019).
Shapley explanations using conditional inference trees.
NVA
Rapport
Jullum, Martin; Løland, Anders; Huseby, Ragnar Bang; Ånonsen, Geir og Lorentzen, Johannes P. (2018).
Detecting money laundering transactions -- which transactions should we learn from?
NVA
Rapport
Vis sammendrag
We develop, describe and validate a machine learning model for prioritising which financial transactions should be manually investigated for potential money laundering. The model is applied to a large data set from Norway's largest bank, DNB. This is one of very few published anti-money laundering models for suspicious transactions that has been trained and validated on a realistically sized data set. We demonstrate that the common approach of not utilising non-reported alerts, i.e.~transactions that are investigated but not reported, in the training of the model can lead to sub-optimal results. We also demonstrate the benefit of including normal (uninvestigated) transactions in the training of the model, and study whether explicitly modelling the probability that a transaction is a normal transaction can improve the money laundering detection rate. Therefore, we present a new performance measure for comparing our method to the existing anti-money laundering system in the bank. Using this performance measure, we clearly outperform the bank's current approach.
Løland, Anders; Jullum, Martin og Huseby, Ragnar Bang. (2018).
Detecting money laundering transactions – two stories. DNB
NVA
Faglig foredrag
Jullum, Martin; Thorarinsdottir, Thordis Linda og Bachl, Fabian. (2018).
Estimating seal pup production in the Greenland Sea using Bayesian hierarchical modeling.
Vis sammendrag
The Greenland Sea is an important breeding ground for harp seals (Pagophilus groenlandicus) and hooded seals (Cystophora cristata). An estimate of the annual seal pup
production is a critical factor in the abundance estimation needed for management of the species. Estimates of seal pup production are usually based on counts from aerial photographic surveys. However, due to the large extent of typical whelping regions, only a minor part of the complete area can be photographed. To estimate the total seal pup production, we propose a Bayesian hierarchical modelling approach motivated by viewing
the seal pup appearances as a realization of a log-Gaussian Cox process using covariate information from satellite imagery as a proxy for ice-thickness. For inference, we utilize the spatial partial differential equation (SPDE) module of the integrated nested Laplace
approximation (INLA) framework. In a case study using survey data from 2012, we compare our results with existing methodology in a comprehensive cross-validation study. The new proposed method improves local estimation performance and more accurately addresses the associated uncertainty.
Jullum, Martin og Hjort, Nils Lid. (2018).
Parametric or nonparametric, that’s the question.
NVA
Vitenskapelig foredrag
Thorarinsdottir, Thordis Linda; Jullum, Martin og Guttorp, Peter. (2017).
Bayesian modelling of cluster point process models.
NVA
Vitenskapelig foredrag
Holden, Lars; Jullum, Martin og Sandve, Geir Kjetil. (2017).
Statistical modeling of repertoire overlap in entire sampling spaces.
Vis sammendrag
We analyze the distribution of T-cell clonotypes in a compartment like blood based on samples. In particular, we study how the distribution of clonotype frequencies changes between different samples. We consider this as a sampling problem and formulate the problem as a generalization of the classical statistical problem of comparing samples from an urn. Due to the low sampling size compared to the number of different clonotypes in the entire sampling space, the classical methodology that works directly with clonotype frequencies in samples is not suited. We approach this challenge by representing other properties of the sample. Our re-representation allows for easy sampling model fitting and testing under natural model conditions. Although we here focus on the application on clonotypes, the new methodology generalizes seamlessly to other applications.
Jullum, Martin; Thorarinsdottir, Thordis Linda og Bachl, Fabian. (2017).
Estimating the seal pup abundance in the Greenland Sea with Bayesian hierarchical modeling. Norsk statistisk forening
NVA
Vitenskapelig foredrag
Jullum, Martin. (2017).
A focused model selection criterion for selecting among
parametric and nonparametric models. Focustat group, UiO
NVA
Vitenskapelig foredrag
Aas, Kjersti; Jullum, Martin og Neef, Linda Reiersølmoen. (2017).
Maskinlæring for vurdering av forsikringsrisiko.
NVA
Rapport
Kolbjørnsen, Odd; Buland, Arild; Hauge, Ragnar; Røe, Per; Jullum, Martin; Metcalfe, Richard William og Skjæveland, Øyvind. (2016).
Bayesian AVO inversion to rock properties using a local neighborhood in a spatial prior model.
Vis sammendrag
The spatial structure of the subsurface is an important factor when interpreting seismic data. The Bayesian methodology is a valuable tool for integrating these spatial relations in the inversion process as it merges the information together and assesses the uncertainty of the model. In the everyday use of the Bayesian methodology, however, the computational cost is a challenge. We describe a new approach that utilizes a local neighborhood to include the spatial constraints and assess the uncertainties in the inversion using fast and parallelizable computations. The approach is applicable for both discrete lithology-fluid prediction and estimation of rock properties, such as porosity and saturation.
Jullum, Martin; Thorarinsdottir, Thordis Linda og Bachl, Fabian. (2016).
Estimating seal pup abundance with LGCP. NTNU
NVA
Vitenskapelig foredrag
Jullum, Martin; Hjort, Nils Lid og Kolbjørnsen, Odd. (2016).
New focused approaches to topics within model selection and approximate Bayesian inversion.
Jullum, Martin. (2016).
FIC with a nonparametric candidate
– a new strategy for FIC construction. Focustat group, UiO
NVA
Vitenskapelig foredrag
Jullum, Martin. (2016).
New focused approaches to topics within model selection and approximate Bayesian inversion. Matematisk institutt, UiO
NVA
Faglig foredrag
Jullum, Martin og Kolbjørnsen, Odd. (2015).
A Gaussian-based framework for local Bayesian inversion of geophysical data to rock properties.
Jullum, Martin og Kolbjørnsen, Odd. (2015).
An Approximate Bayesian Inversion Framework based on Local-gaussian Likelihoods. EAGE
NVA
Vitenskapelig foredrag
Jullum, Martin og Kolbjørnsen, Odd. (2015).
An Approximate Bayesian Inversion Framework based on Local-Gaussian Likelihoods.
Jullum, Martin. (2013).
Parametric or Nonparametric: The Focused Information Criterion Approach (+ Approximate Bayesian Inference). Statistics for Innovation
NVA
Vitenskapelig foredrag
Storvik, Bård og Jullum, Martin. (2011).
Strekker Laban seg litt lengre?
NVA
Populærvitenskapelig artikkel