1. LLM in the loop: a framework for contextualizing counterfactual segment perturbations in point cloudsVeljka Kočić, Niko Lukač, Dzemail Rozajac, Stefan Schweng, Christoph Gollob, Arne Nothdurft, Karl Stampfer, Javier Del Ser, Andreas Holzinger, 2025, original scientific article Abstract: Point Cloud Data analysis has seen a major leap forward with the introduction of PointNet algorithms, revolutionizing how we process 3D environments. Yet, despite these advancements, key challenges remain, particularly in optimizing segment perturbations to influence model outcomes in a controlled and meaningful way. Traditional methods struggle to generate realistic and contextually appropriate perturbations, limiting their effectiveness in critical applications like autonomous systems and urban planning. This paper takes a bold step by integrating Large Language Models into the counterfactual reasoning process, unlocking a new level of automation and intelligence in segment perturbation. Our approach begins with semantic segmentation, after which LLMs intelligently select optimal replacement segments based on features such as class label, color, area, and height. By leveraging the reasoning capabilities of LLMs, we generate perturbations that are not only computationally efficient but also semantically meaningful. The proposed framework undergoes rigorous evaluation, combining human inspection of LLM-generated suggestions with quantitative analysis of semantic classification model performance across different LLM variants. By bridging the gap between geometric transformations and high-level semantic reasoning, this research redefines how we approach perturbation generation in Point Cloud Data analysis. The results pave the way for more interpretable, adaptable, and intelligent AI-driven solutions, bringing us closer to realworld applications where explainability and robustness are paramount.
Keywords: explainable AI, point cloud data, counterfactual reasoning, LiDAR, 3D point cloud data, interpretability, human-centered AI, large language models, K-nearest neighbors
Published in DKUM: 19.05.2025; Views: 0; Downloads: 3
 Full text (7,24 MB) |
2. Enhancing trust in automated 3D point cloud data interpretation through explainable counterfactualsAndreas Holzinger, Niko Lukač, Dzemail Rozajac, Emil Johnston, Veljka Kočić, Bernhard Hoerl, Christoph Gollob, Arne Nothdurft, Karl Stampfer, Stefan Schweng, Javier Del Ser, 2025, original scientific article Abstract: This paper introduces a novel framework for augmenting explainability in the interpretation of point cloud data by fusing expert knowledge with counterfactual reasoning. Given the complexity and voluminous nature of point cloud datasets, derived predominantly from LiDAR and 3D scanning technologies, achieving interpretability remains a significant challenge, particularly in smart cities, smart agriculture, and smart forestry. This research posits that integrating expert knowledge with counterfactual explanations – speculative scenarios illustrating how altering input data points could lead to different outcomes – can significantly reduce the opacity of deep learning models processing point cloud data. The proposed optimization-driven framework utilizes expert-informed ad-hoc perturbation techniques to generate meaningful counterfactual scenarios when employing state-of-the-art deep learning architectures. The optimization process minimizes a multi-criteria objective comprising counterfactual metrics such as similarity, validity, and sparsity, which are specifically tailored for point cloud datasets. These metrics provide a quantitative lens for evaluating the interpretability of the counterfactuals. Furthermore, the proposed framework allows for the definition of explicit interpretable counterfactual perturbations at its core, thereby involving the audience of the model in the counterfactual generation pipeline and ultimately, improving their overall trust in the process. Results demonstrate a notable improvement in both the interpretability of the model’s decisions and the actionable insights delivered to end-users. Additionally, the study explores the role of counterfactual reasoning, coupled with expert input, in enhancing trustworthiness and enabling human-in-the-loop decision-making processes. By bridging the gap between complex data interpretations and user comprehension, this research advances the field of explainable AI, contributing to the development of transparent, accountable, and human-centered artificial intelligence systems.
Keywords: explainable AI, point cloud data, counterfactual reasoning, information fusion, interpretability, human-centered AI
Published in DKUM: 06.03.2025; Views: 0; Downloads: 6
 Full text (186,97 KB) |
