Jarle Hamar Reksten

Chain-of-Thought (CoT) prompting has emerged as a simple yet powerful strategy to elicit structured reasoning in large language models. This study investigates how CoT prompting influences reasoning behavior and task performance of large/vision–language models (LLMs/VLMs) applied to Earth Observation (EO). We compare an EO-specialized model (Falcon) with three general-purpose models (LLaVA, LLaVA-CoT, and o3) across two datasets: RSVQAxBEN, a large open EO benchmark, and a proprietary aerial dataset from Narvik. Experiments contrast baseline and CoT-style prompts to assess both factual accuracy and reasoning quality, complemented by an LLM-as-judge evaluation. Results show that CoT prompting benefits only the large-scale o3 model, while smaller or mid-scale models experience degraded accuracy—confirming that effective reasoning is an emergent property of scale. CoT adds transparency by revealing how models reason, though its outputs can still be partly opaque due to safety or internal constraints. On Narvik, o3 generalizes well to unseen EO data, but CoT prompting does not improve quantitative accuracy. These findings suggest that CoT currently offers greater value for interpretability than for performance. Future work should explore inference-time perception–reasoning strategies—where an EO model like Falcon provides scene-level facts that guide o3’s reasoning—to improve both trustworthiness and accuracy without retraining.

This report details the data analysis products developed within the SeaBee project, a national infrastructure for drone-based services for use in coastal and aquatic research, mapping and monitoring of habitats, animal communities, and anthropogenic impacts. We present an advanced, automated data analysis pipeline that leverages deep learning for two primary tasks: pixel-wise thematic mapping of coastal habitats and object detection for counting wildlife. The pipeline utilizes models such as U-Net and Faster R-CNN to process high-resolution drone imagery (RGB, MSI, and HSI) and incorporates a novel hierarchical classification structure for habitat mapping and a robust method for detecting out-of-distribution (OOD) samples. We demonstrate the pipeline's pixel-wise mapping effectiveness through extensive experiments at three diverse Norwegian coastal sites—Remøy, Vega, and Ølbergholmen—achieving high accuracy in mapping complex habitats like kelp forests and various substrate types. Furthermore, the object detection framework shows strong performance in the automated counting and classification of 11 seabird species and coastal seals, offering a significant improvement in efficiency over traditional survey methods. The results confirm that the SeaBee pipeline is a powerful, scalable tool for environmental research and management, though we also discuss challenges such as data imbalance and model generalizability that will inform future work. This research is funded by the Research Council of Norway, project ID #296478, to the Norwegian Infrastructure for drone-based research, mapping, and monitoring in the coastal zone (SeaBee).

Foundation models (FMs) are transforming the field of Artificial Intelligence (AI) by learning inherent information from vast amounts of unlabeled data, enabling adaptation to numerous applications. Their integration into the Earth Observation (EO) ecosystem promises to revolutionize the information value chain, impacting industry, research, and science. However, EO applications present unique challenges, including the diverse needs for detail or rapid processing, and the variety of data value and sensor characteristics. Models must handle data from multiple sensors at varying ground sampling distances (GSD). Vision Transformers (ViT), often trained using self-supervised learning (SSL), form the backbone of many modern FMs by learning from complex data patterns without explicit supervision. We introduce FM4CS, a versatile foundation model specifically designed for climate and society EO applications. FM4CS aims to address the aforementioned challenges by supporting four different Sentinel sensors: Sentinel-1 SAR, Sentinel-2 MSI, Sentinel-3 OLCI, and Sentinel-3 SLSTR. Inspired by approaches like USat for multi-sensor data handling at native resolutions and FlexiViT for operating across a wide range of patch sizes without retraining, FM4CS employs a single ViT architecture. The model utilizes individual patch embedding layers for each sensor channel, allowing flexibility in processing subsets of spectral bands. It adapts the number of patches per band based on GSD and uses spectral group pooling to manage token sequence length. To accommodate flexible patch sizes, FM4CS incorporates a training procedure where patch size is randomized, allowing the model to adapt dynamically at inference time. For handling the need for positional information of the patches across different sensors, resolutions and patch sizes, FM4CS adapts the 2D ALiBi (Attention by Linear Bias) relative positional encoding scheme. The pre-training dataset for FM4CS is curated to ensure diversity. Instead of stacking small image crops, data is sampled using the Sentinel-2 tiling grid, co-locating Sentinel-1, Sentinel-2, and Sentinel-3 imagery for given locations and time intervals. A stratified sampling approach, based on k-means clustering of ESA WorldCover maps and Sentinel-2 RGB composites, is used to capture the diversity of global land cover and address imbalances. Oceanic data sampling incorporates shipping traffic density, oil and gas installations, and areas with higher probabilities of sea ice and icebergs. The dataset also includes ERA5-Land variables to facilitate multi-modal pretext tasks, leveraging daily statistics for variables such as soil moisture, temperature, and snow cover. FM4CS is trained using several SSL tasks. These include pixel-level input band reconstruction, similar to masked image modeling, where a lightweight ViT decoder reconstructs masked tokens. Additionally, the model predicts existing maps such as ESA WorldCover from Sentinel-1/2 data, and other land cover maps (ESA GlobCover, MOD12Q1) from the Sentinel-3 sensors using a cross-entropy loss. Image-level tasks involve the prediction of ERA5 variables, latitude, longitude and data acquisition month.

To leverage Earth observation (EO) data for large scale analysis, automatic methods is a prerequisite. Since 2012, deep learning (DL) models have brought about a revolutionary change in the analysis of image data and are currently considered state-of-the-art for a broad spectrum of EO tasks. However, a bottleneck with supervised DL models is that they often require a vast amount of labelled data to be trained, and the research community has therefore started to explore alternatives to supervised learning. During the last years, foundation models (FM) signify a change of thinking in computer vision. FMs are trained on a vast volume of unlabeled data and can identify complex patterns due to their large-scale learning capabilities. Typically, an additional head or decoder (small network) is added to the FM, which is trained and adapted to various use-cases by means of a small amount of labelled data. FM have also started to be explored for EO applications, however, current EO-based FMs are limited in terms of handling different modalities with large differences in resolution. Modern FMs are often based on transformers and are trained using self-supervised learning (SSL). There are several SSL schemes in place, including masked autoencoders (MAE) where we mask part of the input data and learn the model to predict the masked data. This is not useful by itself, but the model learns compressed representation of the data, which can be leveraged in downstream applications. This potentially makes the FM more useful than models trained on a limited set of labeled data. The Norwegian Computing Center and UiT – The Arctic University of Norway are in collaboration with user partners Romanian National Meteorological Administration, Danish Meteorological Institute, Polar View and Norwegian Water Resources and Energy Directorate developing a multi-modal FM. The FM is designed to process data from the satellites Sentinel-1 SAR, Sentinel-2 and Sentinel-3 OLCI and SLSTR. The FM is based on vison transformers (ViT) but utilizing the same principle as the USat approach to handle the different resolutions between the modalities. The training of the FM is based on the MAE approach, and to ensure that the SSL work efficiently, we have developed a smart sampling scheme during that provides relevant and diverse training data. In addition to SSL, we have also created a learning task in regressing climate variables from the ERA5 dataset. To train the FM, over 20 TB of Sentinel data was collected and processed using the LUMI supercomputer. The multi-modal FM is demonstrated on the following use-cases: mapping of snow, flood zone mapping, mapping and monitoring of sea ice, iceberg detection, early draught warning and mapping of wetlands. The resolution of the target use-case products are vastly different, e.g. for snow mapping we aim for a ground sampling distance (GSD) of 250m whereas for flood zone mapping we aim for a GSD of 10m. We have therefore trained two versions of the FM: one aiming for high-resolution products with GSD between 10 – 60m, and one aiming for low-resolutions products with GSD above 100m. The downstream tasks are implemented using the open-source framework TerraTorch, which is a flexible fine-tuning framework for geospatial FMs. TerraTorch supports common fine-tuning tasks such as image segmentation and pixel-wise regression along with a selection of task-specific decoder heads.

To leverage Earth observation (EO) data for large scale analysis, automatic methods is a prerequisite. Since 2012, deep learning (DL) models have brought about a revolutionary change in the analysis of image data and are currently considered state-of-the-art for a broad spectrum of EO tasks. However, a bottleneck with supervised DL models is that they often require a vast amount of labelled data to be trained, and the research community has therefore started to explore alternatives to supervised learning. Inspired by the progress in large language models, foundation models (FM) are now being applied extensively in computer vision. FMs are trained on a vast volume of unlabeled data and can identify complex patterns due to their large-scale learning capabilities. Typically, an additional head or decoder (small network) is added to the FM, which is trained and adapted to various use-cases by means of a small amount of labelled data. FM have also started to be explored for EO applications, however, current EO-based FMs are limited in terms of handling different modalities with large differences in resolution. Modern FMs are often based on transformers and are trained using self-supervised learning (SSL). There are several SSL schemes in place, including masked autoencoders (MAE) where we mask part of the input data and learn the model to predict the masked data. This is not useful by itself, but the model learns compressed representation of the data, which can be leveraged in downstream applications. This potentially makes the FM more useful than models trained on a limited set of labeled data. The Norwegian Computing Center and UiT – The Arctic University of Norway are in collaboration with user partners Romanian National Meteorological Administration, Danish Meteorological Institute, Polar View and Norwegian Water Resources and Energy Directorate developing a multi-modal FM. The FM is designed to process data from the satellites Sentinel-1 SAR, Sentinel-2 and Sentinel-3 OLCI and SLSTR. The FM is based on vison transformers (ViT) but utilizing the same principle as the USat approach to handle the different resolutions between the modalities. The training of the FM is based on the MAE approach, and to ensure that the SSL work efficiently, we have developed a smart sampling scheme during that provides relevant and diverse training data. In addition to SSL, we have also created a learning task in regressing climate variables from the ERA5 dataset. To train the FM, over 20 TB of Sentinel data was collected and processed using the LUMI supercomputer. The multi-modal FM is demonstrated on the following use-cases: mapping of snow, flood zone mapping, mapping and monitoring of sea ice, iceberg detection, early draught warning and mapping of wetlands. The downstream tasks are implemented using the open-source framework TerraTorch, which is a flexible fine-tuning framework for geospatial FMs. The FM4CS model is one of the first to apply Sentinel-3 data, which makes it attractive for climate applications.

The existing cultural heritage mapping in Norway is incomplete. Some selected areas are mapped well, while the majority of areas only contain chance discoveries, often with bad positional accuracy. The goal of this research was to develop automated tools for improving the cultural heritage mapping in Norway, thus enabling detailed mapping of large areas within realistic budgets and time frames. The focus was on three types of cultural heritage that occur frequently in many types of Norwegian landscape: grave mounds, pitfall traps in deer hunting systems and charcoal kilns. A recent development in deep neural networks for object detection in natural images is the region-proposing convolutional neural network (R-CNN), which may also be used for cultural heritage detection in local relief model (LRM) visualizations of airborne laser scanning (ALS) data. Python code for ‘Simple Faster R-CNN’ was downloaded from Github. On 737 test images (16.6 km2) not seen during training, 87 % of the true cultural heritage objects were correctly identified, while 24 % of the predicted cultural heritage locations were false. However, all test images were small (150 m × 150 m) and contained at least one cultural heritage object, meaning that the false positive rate may be higher for an entire landscape. In Larvik municipality, Vestfold and Telemark County, on a 67 km2 area not seen during training, the R-CNN correctly identified 38 % of the true grave mounds, with 89 % false positives. On a 937 km2 area covering Øvre Eiker municipality, Viken County, the R-CNN correctly identified 90 % of the charcoal kilns, with 38 % false positives. In conclusion, we have demonstrated that Faster R-CNN is well suited for semi-automatic detection of cultural heritage objects such as charcoal kilns, grave mounds and pitfall traps in high resolution airborne lidar data. However, it is desirable to reduce the false positive rate in order to limit the amount of visual inspection needed when the method is applied to large areas for detailed archaeological mapping.

Traffic estimation from very-high-resolution remote-sensing imagery has received increasing interest during the last few years. In this article, we propose an automatic system for estimation of the annual average daily traffic (AADT) using very-high-resolution optical remote-sensing imagery of urban areas in combination with high-quality, but very spatially limited, ground-based measurements. The main part of the system is the vehicle detection, which is based on the deep learning object detection architecture mask region-based convolutional neural network (Mask R-CNN), modified with an image normalization strategy to make it more robust for test images of various conditions and the use of a precise road mask to assist the filtering of driving vehicles from parked ones. Furthermore, to include the high-quality ground-based measurements and to make the traffic estimates more consistent across neighbouring road links, we propose a graph smoothing strategy that utilizes the road network. The fully automatic processing chain has been validated on a set of aerial images covering the city of Narvik, Norway. The precision and recall rate of detecting driving vehicles was 0.74 and 0.66, respectively, and the AADT was estimated with a root mean squared error (RMSE) of 2279 and bias of −383. We conclude that separating driving vehicles from parked ones may be challenging if vehicles are parked along the roads and that for urban environment with short road links several remote-sensing images covering the road links at different time instances are necessary in order to benefit from the remote-sensing images.

After large flood incidents in Norway, The Norwegian Water Resources and Energy Directorate (NVE), has the responsibility for documenting the flooded areas. This has so far mainly been performed by utilising aerial images and visual interpretation. Satellite images are a valuable source of additional information as they are able to cover vast areas in each satellite pass. In this paper a fully automated system for detecting and delineating floods with the use of Synthetic Aperture Radar (SAR) images from the Sentinel-1 satellites is presented. In SAR images wet areas and water bodies usually show lower backscatter than dry areas. The flood detection system is thus based on comparing a reference image acquired before the flood with the flood event image. A Sentinel-1 training dataset has been obtained and manually annotated by NVE from three flood events in Norway. This training set has been used to train a random forest (RF) classifier, which outputs a score for each pixel in the SAR image. This score image is thresholded in order to obtain a crude flood detection. Unfortunately, changes in the backscatter may also be triggered by other events such as melting snow and harvested fields of crops. To mitigate such lookalikes, several techniques have been implemented and tested. This includes masking based on size, slope and height above nearest drainage (HAND). The experiments presented show that the system performance is very good. Of the 179 manually labelled flood objects, 168 are detected. The system is being applied operationally at NVE.

Airborne Laser Scanning (ALS) is a technique well-suited to creating Digital Terrain Models with the purpose of detecting cultural heritage, given that the cultural heritage manifests itself in the terrain model. This paper presents a pilot web portal for semi-automatic mapping of archaeological features in ALS data. The intended users are archaeologists in the county administrations in Norway. The pilot portal is already a useful tool for archaeologists in the participating pilot counties in Norway, and exposes the need for a national infrastructure for processing of ALS data. Automatic detection based on deep learning is successfully applied. Traditional pattern recognition methods are also included, but obtain high false positive rates and thus require more manual editing. The web portal supports the following types of cultural heritage: Grave mound, pitfall trap, charcoal burning pit, and charcoal kiln. The web portal is demonstrated with ALS data from three different locations in Norway.

Detection and characterization of avalanches are important for making avalanche inventories as well as for the management of emergency situations. In this paper we propose a scheme for automatic detection and mapping of avalanches in SAR images. The approach builds upon the hypothesis that compacted rough snow of an avalanche has very high backscatter intensity values compared to homogeneous snow cover and bare ground, and hence, by comparing the event image with a reference image we may detect and map avalanches in the scene. The proposed approach consists of two steps: (i) an initial detection of potential avalanche objects and, (ii) supervised classification of avalanche candidates using a random forest classifier. The approach is evaluated on a set of Radarsat-2 ultra-fine images, and the out-of-bag error rate is 6.4%. We conclude that an operational automatic algorithm may be feasible provided enough training data is available.

This project was started in 2002 with the overall aim of developing a cost-effective method for surveying and monitoring cultural heritage sites on a regional and national scale. The project focuses on the development of automated pattern recognition methods for detecting and locating cultural heritage sites. This note describes the achievements of the project during 1 March 2015 – 1 March 2016. The project is funded by the Norwegian Directorate for Cultural Heritage (Riksantikvaren). The project has seen two major breakthroughs in the current reporting period. Firstly, the use of deeplearning methodology is able to dramatically improve the detection performance of semi-automatic detection of charcoal kilns. An initial experiment indicates that 85% of the true charcoal kilns may be detected, with only 10% false positives. A similar approach may be used in semi-automatic detection of grave mounds. Secondly, the pilot portal has been scaled to be able to process entire airborne laser scanning datasets. The previous version was only able to process small test data sets of less than a few km². However, the new detection methods, based on deep learning, are not available from the pilot portal yet. A third major activity has been field verification of semi automatic detections. The main focus has been on charcoal kilns belonging to the ironwork in Lesja, Oppland County, resulting in a near complete mapping of the kilns and producing valuable cultural-historical information. Preliminary results from the development of the pilot portal and the use of deep learning for the detection of charcoal kilns will be presented at international conferences in March and April 2016. In conclusion, the project has now scientifically demonstrated that semi-automatic detection methods work when applied to airborne laser scanning (ALS) data. This is of great importance now that large amounts of ALS data are becoming available to heritage management because of the implementation of the National Detailed Height-model in Norway. Project funding for one more year is vital to implement deep learning-based methods in the pilot portal. This will turn the pilot portal into a workhorse for semi-automatic detection of archaeological structures from airborne laser scanning data, of great benefit for the archaeological use of ALS data