https://scholars.lib.ntu.edu.tw/handle/123456789/630225
標題: | Generating Construction Safety Observations via CLIP-Based Image-Language Embedding | 作者: | Tsai, Wei Lun JACOB JE-CHIAN LIN SHANG-HSIEN HSIEH |
關鍵字: | CLIP | Construction safety | Image captioning | Safety inspection | 公開日期: | 1-一月-2023 | 卷: | 13807 LNCS | 來源出版物: | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) | 摘要: | Safety inspections are standard practices to prevent accidents from happening on construction sites. Traditional workflows require an inspector to document the violations through photos and textual descriptions explaining the specific incident with the objects, actions, and context. However, the documentation process is time-consuming, and the content is inconsistent. The same violation could be captioned in various ways, making the safety analysis tricky. Research has investigated means to improve the documentation process efficiency through applications with standardized forms and develop language understanding models to analyze the safety reports. Nevertheless, it is still challenging to streamline the entire documentation process and accurately compile the reports into meaningful information. We propose an image-language embedding model that automatically generates textual safety observations through the Contrastive Language-Image Pre-trained (CLIP) fine-tuning and CLIP prefix captioning designed based on the construction safety context. CLIP can obtain the contrastive features to classify the safety attribute types for images, and CLIP prefix captioning generates the caption from the given safety attributes, images, and captions. The framework is evaluated through a construction safety report dataset and could create reasonable textual information for safety inspectors. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/630225 | ISBN: | 9783031250811 | ISSN: | 03029743 | DOI: | 10.1007/978-3-031-25082-8_24 |
顯示於: | 土木工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。