Improving single image localization through domain adaptation and large kernel attention with synthetic data

authored by: Dansheng Yao, Hehua Zhu, Bangke Ren, Xiaoying Zhuang
Abstract: In the realm of digital twin technology, image localization emerges as a crucial aspect, particularly in the challenging domain of civil engineering construction. Unlike the data-rich environments typical of structure-from-motion (sfm) technologies, the construction phase of civil engineering projects often faces economic constraints that limit data collection. This results in sporadic and localized snapshots, rather than comprehensive spatial and temporal coverage of the entire scene. Such prevalent data sparsity poses significant challenges to achieving accurate image localization. Our research is tailored to address this specific challenge, focusing on single image localization in environments where data is inherently sparse. We introduce a multi-scale convolutional attention network, incorporating feature-fused adversarial components, to effectively navigate the complexities of sparse data typical in civil engineering construction sites. The network employs large kernel convolutions for refined channel and spatial attention, ensuring precise location information transmission, even in data-limited scenarios. This accuracy is further augmented by multi-scale convolutional layers and a multi-level discriminator network, aiming to minimize the domain shift between virtual and real-world imagery. Our approach was rigorously tested and subjected to ablation studies on two public datasets, confirming its efficacy. In indoor settings, we achieved a median localization accuracy of 1.12 m and 9.80°, and in outdoor environments, our best results were 3.69 m and 1.67°. These outcomes highlight the effectiveness of our method in addressing the unique challenges posed by data sparsity in civil engineering construction. We also investigated the impact of domain adaptation on localization accuracy across different feature levels, finding that its effect varies depending on the degree of alignment between virtual and real datasets. In conclusion, this study offers a significant contribution to image localization in digital twin technology, particularly in the challenging context of data-sparse civil engineering construction processes. It paves the way for future research in optimizing image localization techniques in similar sparse data environments.
Organisation(s): Institute of Photonics
External Organisation(s): Tongji University
Type: Article
Journal: Engineering Applications of Artificial Intelligence
Volume: 137
No. of pages: 18
ISSN: 0952-1976
Publication date: 11.2024
Publication status: Published
Peer reviewed: Yes
ASJC Scopus subject areas: Control and Systems Engineering, Artificial Intelligence, Electrical and Electronic Engineering
Electronic version(s): https://doi.org/10.1016/j.engappai.2024.108951 (Access: Closed)
: Details in the research portal "Research@Leibniz University"

BibTeX

@article{fcdcb700fec543e9983ccefc51dc033a,
title = "Improving single image localization through domain adaptation and large kernel attention with synthetic data",
abstract = "In the realm of digital twin technology, image localization emerges as a crucial aspect, particularly in the challenging domain of civil engineering construction. Unlike the data-rich environments typical of structure-from-motion (sfm) technologies, the construction phase of civil engineering projects often faces economic constraints that limit data collection. This results in sporadic and localized snapshots, rather than comprehensive spatial and temporal coverage of the entire scene. Such prevalent data sparsity poses significant challenges to achieving accurate image localization. Our research is tailored to address this specific challenge, focusing on single image localization in environments where data is inherently sparse. We introduce a multi-scale convolutional attention network, incorporating feature-fused adversarial components, to effectively navigate the complexities of sparse data typical in civil engineering construction sites. The network employs large kernel convolutions for refined channel and spatial attention, ensuring precise location information transmission, even in data-limited scenarios. This accuracy is further augmented by multi-scale convolutional layers and a multi-level discriminator network, aiming to minimize the domain shift between virtual and real-world imagery. Our approach was rigorously tested and subjected to ablation studies on two public datasets, confirming its efficacy. In indoor settings, we achieved a median localization accuracy of 1.12 m and 9.80°, and in outdoor environments, our best results were 3.69 m and 1.67°. These outcomes highlight the effectiveness of our method in addressing the unique challenges posed by data sparsity in civil engineering construction. We also investigated the impact of domain adaptation on localization accuracy across different feature levels, finding that its effect varies depending on the degree of alignment between virtual and real datasets. In conclusion, this study offers a significant contribution to image localization in digital twin technology, particularly in the challenging context of data-sparse civil engineering construction processes. It paves the way for future research in optimizing image localization techniques in similar sparse data environments.",
keywords = "3D model, Domain adaptation, Large kernel attention, Synthetic dataset, Visual localization",
author = "Dansheng Yao and Hehua Zhu and Bangke Ren and Xiaoying Zhuang",
note = "Publisher Copyright: {\textcopyright} 2024",
year = "2024",
month = nov,
doi = "10.1016/j.engappai.2024.108951",
language = "English",
volume = "137",
journal = "Engineering Applications of Artificial Intelligence",
issn = "0952-1976",
publisher = "Elsevier Ltd.",
}

Back