| Title: | LLM in the loop: a framework for contextualizing counterfactual segment perturbations in point clouds |
|---|
| Authors: | ID Kočić, Veljka (Author)
ID Lukač, Niko (Author)
ID Rozajac, Dzemail (Author)
ID Schweng, Stefan (Author)
ID Gollob, Christoph (Author)
ID Nothdurft, Arne (Author)
ID Stampfer, Karl (Author)
ID Del Ser, Javier (Author)
ID Holzinger, Andreas (Author) |
|---|
| Files: |  LLM_in_the_Loop_A_Framework_for_Contextualizing_Counterfactual_Segment_Perturbations_in_Point_Clouds.pdf (7,24 MB)
MD5: 19B58061989BF92CB7C80EDBA041C477
|
|---|
| Language: | English |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | Point Cloud Data analysis has seen a major leap forward with the introduction of PointNet algorithms, revolutionizing how we process 3D environments. Yet, despite these advancements, key challenges remain, particularly in optimizing segment perturbations to influence model outcomes in a controlled and meaningful way. Traditional methods struggle to generate realistic and contextually appropriate perturbations, limiting their effectiveness in critical applications like autonomous systems and urban planning. This paper takes a bold step by integrating Large Language Models into the counterfactual reasoning process, unlocking a new level of automation and intelligence in segment perturbation. Our approach begins with semantic segmentation, after which LLMs intelligently select optimal replacement segments based on features such as class label, color, area, and height. By leveraging the reasoning capabilities of LLMs, we generate perturbations that are not only computationally efficient but also semantically meaningful. The proposed framework undergoes rigorous evaluation, combining human inspection of LLM-generated suggestions with quantitative analysis of semantic classification model performance across different LLM variants. By bridging the gap between geometric transformations and high-level semantic reasoning, this research redefines how we approach perturbation generation in Point Cloud Data analysis. The results pave the way for more interpretable, adaptable, and intelligent AI-driven solutions, bringing us closer to realworld applications where explainability and robustness are paramount. |
|---|
| Keywords: | explainable AI, point cloud data, counterfactual reasoning, LiDAR, 3D point cloud data, interpretability, human-centered AI, large language models, K-nearest neighbors |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Publication date: | 08.05.2025 |
|---|
| Publisher: | IEEE XPLORE |
|---|
| Year of publishing: | 2025 |
|---|
| Number of pages: | 17 str. |
|---|
| PID: | 20.500.12556/DKUM-92856  |
|---|
| UDC: | 004.7 |
|---|
| ISSN on article: | 2169-3536 |
|---|
| COBISS.SI-ID: | 236009987 |
|---|
| DOI: | 10.1109/ACCESS.2025.3568052 |
|---|
| Publication date in DKUM: | 19.05.2025 |
|---|
| Views: | 0 |
|---|
| Downloads: | 3 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
