近日，華中農業大學柑橘全程機械化平臺和人工智能與統計學習團隊以“Predicting and Visualizing Citrus Colour Transformation Using a Deep Mask-Guided Generative Network”為題在農藝學領域期刊Plant Phenomics發表研究論文。該研究通過生成式深度學習方法，實現了對柑橘果皮顏色變化的高精度可視化預測。
柑橘果皮顏色是果實發育的良好指標，因此監測和預測其顏色變化可以幫助農作物的管理和收獲進行決策。研究觀察了107個臍橙樣本，在其果皮顏色轉變期間采集了7535張柑橘圖像，構建了首個柑橘轉色數據集。研究提出了一種將視覺感知融入深度學習的網絡框架，包括分割網絡、生成網絡和感知損失網絡；生成網絡中的嵌入層將圖像特征和時間信息進行了有效融合，使得該網絡模型能夠根據輸入圖像和不同的時間間隔來生成果皮顏色轉變的預測圖像。
為了方便現實場景中的應用，研究團隊將該模型移植到了安卓設備APP上，通過手機相機拍攝柑橘圖像，并在APP中輸入感興趣的時間間隔即可完成預測。該研究成果預測柑橘果皮顏色的變化并使其可視化，為柑橘果園管理實踐提供有效幫助，并為其他水果作物的研究提供了借鑒和擴展的可能性。
我校理學院碩士研究生鮑澤韓和副教授李偉夫為本論文共同第一作者，工學院副研究員陳耀暉和海南大學生物醫學工程學院副教授肖馳為共同通訊作者。理學院教授陳洪和日本RIKEN研究所研究員Vijay John參與了論文的研究和指導工作。
【英文摘要】
Citrus rind colour is a good indicator of fruit development, and methods to monitor and predict colour transformation therefore help the decisions of crop management practices and harvest schedules. This work presents the complete workflow to predict and visualize citrus colour transformation in the orchard featuring high accuracy and fidelity. 107 sample Navel oranges were observed during the colour transformation period, resulting in a dataset containing 7535 citrus images. A framework is proposed that integrates visual saliency into deep learning, and it consists of a segmentation network, a deep mask-guided generative network, and a loss network with manually-designed loss functions. Moreover, the fusion of image features and temporal information enables one single model to predict the rind colour at different time intervals, thus effectively shrinking the number of model parameters. The semantic segmentation network of the framework achieves the mean intersection over union score of 0.9694, and the generative network obtains a peak signal-to-noise ratio of 30.01 and a mean local style loss score of 2.710, which indicate both high quality and similarity of the generated images and are also consistent with human perception. To ease the applications in the real world, the model is ported to an Android-based application for mobile devices. The methods can be readily expanded to other fruit crops with a colour transformation period. The dataset and the source code are publicly available at Github.
日期：2023-06-13
柑橘果皮顏色是果實發育的良好指標，因此監測和預測其顏色變化可以幫助農作物的管理和收獲進行決策。研究觀察了107個臍橙樣本，在其果皮顏色轉變期間采集了7535張柑橘圖像，構建了首個柑橘轉色數據集。研究提出了一種將視覺感知融入深度學習的網絡框架，包括分割網絡、生成網絡和感知損失網絡；生成網絡中的嵌入層將圖像特征和時間信息進行了有效融合，使得該網絡模型能夠根據輸入圖像和不同的時間間隔來生成果皮顏色轉變的預測圖像。
為了方便現實場景中的應用，研究團隊將該模型移植到了安卓設備APP上，通過手機相機拍攝柑橘圖像，并在APP中輸入感興趣的時間間隔即可完成預測。該研究成果預測柑橘果皮顏色的變化并使其可視化，為柑橘果園管理實踐提供有效幫助，并為其他水果作物的研究提供了借鑒和擴展的可能性。
我校理學院碩士研究生鮑澤韓和副教授李偉夫為本論文共同第一作者，工學院副研究員陳耀暉和海南大學生物醫學工程學院副教授肖馳為共同通訊作者。理學院教授陳洪和日本RIKEN研究所研究員Vijay John參與了論文的研究和指導工作。
【英文摘要】
Citrus rind colour is a good indicator of fruit development, and methods to monitor and predict colour transformation therefore help the decisions of crop management practices and harvest schedules. This work presents the complete workflow to predict and visualize citrus colour transformation in the orchard featuring high accuracy and fidelity. 107 sample Navel oranges were observed during the colour transformation period, resulting in a dataset containing 7535 citrus images. A framework is proposed that integrates visual saliency into deep learning, and it consists of a segmentation network, a deep mask-guided generative network, and a loss network with manually-designed loss functions. Moreover, the fusion of image features and temporal information enables one single model to predict the rind colour at different time intervals, thus effectively shrinking the number of model parameters. The semantic segmentation network of the framework achieves the mean intersection over union score of 0.9694, and the generative network obtains a peak signal-to-noise ratio of 30.01 and a mean local style loss score of 2.710, which indicate both high quality and similarity of the generated images and are also consistent with human perception. To ease the applications in the real world, the model is ported to an Android-based application for mobile devices. The methods can be readily expanded to other fruit crops with a colour transformation period. The dataset and the source code are publicly available at Github.
日期：2023-06-13
