Yun LinWei-Chun GaoChu-Ping LinHsuan-Ju TsaiYi-Ju ChenYan-Fu Kuo2024-10-282024-10-282024https://www.scopus.com/record/display.uri?eid=2-s2.0-85206111089&origin=resultslisthttps://scholars.lib.ntu.edu.tw/handle/123456789/722505Tomato is one of commonly cultivated crops worldwide. The yield and quality of tomato are, however, significantly impacted by diseases, pests, and disorders (DPD). Most of these symptoms on tomato plants usually show up on leaves, which might include spots, yellowing, necrosis, and leaf distortion, and can be confounded and confusing at a certain level. Thus, correctly identifying the cause of a symptom is crucial for tomato management. Conventionally, the cause of a symptom was identified using naked-eye or microscopic examination by experienced farmers or experts, respectively. However, these approaches may be biased or time-consuming. By contrast, immediate actions may need to be taken in crop management. Thus, this study proposes to rapidly identify the causes of DPD for tomato in the fields using smartphones and convolutional neural networks (CNNs). In this study, approximately 11,000 images of tomato leaves with DPD symptoms were collected in the field. The causes of the symptoms were identified by experts in Taiwan Agricultural Research Institute. Four CNNs were trained to identify the causes of the symptoms using the images as the input. These four CNNs include (1) a tomato leaf verification model (LVM) to authenticate if a received image is a tomato leaf, (2) a leaflet and pinnate compound leaf classification model (LCM) to differentiate between leaflets and pinnate compound leaves, (3) a leaflet identification model (LIM) to distinguish 13 categories of DPD on leaflet, and (4) an abaxial surface classification model (BCM) to determine if a lesion is caused by leaf mold or new powdery mildew using images of leaf abaxial surfaces. The four trained CNNs were hosted on a cloud service. A chatbot controller was also built to manage the communications between users and CNNs, enabling users to send tomato leaf images taken in the field through instant messaging applications on their smartphones. The LVM, LCM, LIM, and BCM achieved an accuracy of 81.63%, an accuracy of 92.87%, an overall mean average precision of 88.20%, an accuracy of 96.10%, respectively. Clearly, the proposed approach simulated the logic of experts in tomato issue diagnosis and can assist in tomato cultivation management in the field rapidly.falseChatbotdeep learningsmartphonetomato leaflettomato pinnate compound leaf[SDGs]SDG2conference paper10.13031/aim.2024002562-s2.0-85206111089