en
×

分享给微信好友或者朋友圈

使用微信“扫一扫”功能。
作者简介:

庄志豪,男,硕士生,研究方向为电子信息、人工智能.2593852745@qq.com;

王敏(通信作者),女,博士,教授,研究方向为电子信息、人工智能.yu0801@163.com

中图分类号:V321.2+53;TP391.41

文献标识码:A

DOI:10.13878/j.cnki.jnuist.2022.05.008

参考文献 1
Tang Y Z,Yang P,Zhou Z M,et al.Improving cloud type classification of ground-based images using region covariance descriptors[J].Atmospheric Measurement Techniques,2021,14(1):737-747
参考文献 2
Zhang Y C.Calculation of radiative fluxes from the surface to top of atmosphere based on ISCCP and other global data sets:refinements of the radiative transfer model and the input data[J].Journal of Geophysical Research:Atmospheres,2004,109(D19):1121-1125
参考文献 3
Naud C M,Booth J F,del Genio A D.The relationship between boundary layer stability and cloud cover in the post-cold frontal region[J].Journal of Climate,2016,29(22):8129-8149
参考文献 4
Cui F,Ju R R,Ding Y Y,et al.Prediction of regional global horizontal irradiance combining ground-based cloud observation and numerical weather prediction[J].Advanced Materials Research,2014,1073/1074/1075/1076:388-394
参考文献 5
Yuan F,Lee Y H,Meng Y S.Comparison of radio-sounding profiles for cloud attenuation analysis in the tropical region[C]//2014 IEEE Antennas and Propagation Society International Symposium(APSURSI).July 6-11,2014,Memphis,TN,USA.IEEE,2014:259-260
参考文献 6
Phung V H,Rhee E J.A deep learning approach for classification of cloud image patches on small datasets[J].Journal of Information and Communication Convergence Engineering,2018,16(3):173-178
参考文献 7
Liu L,Sun X J,Chen F,et al.Cloud classification based on structure features of infrared images[J].Journal of Atmospheric and Oceanic Technology,2011,28(3):410-417
参考文献 8
Cheng H Y,Yu C C.Block-based cloud classification with statistical features and distribution of local texture features[J].Atmospheric Measurement Techniques,2015,8(3):1173-1182
参考文献 9
Hu J,Chen Z B,Yang M,et al.A multiscale fusion convolutional neural network for plant leaf recognition[J].IEEE Signal Processing Letters,2018,25(6):853-857
参考文献 10
Wu X P,Zhan C,Lai Y K,et al.IP102:a large-scale benchmark dataset for insect pest recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).June 15-20,2019,Long Beach,CA,USA.IEEE,2019:8779-8788
参考文献 11
Singh M,Glennen M.Automated ground-based cloud recognition[J].Pattern Analysis and Applications,2005,8(3):258-271
参考文献 12
Calbo J,Sabburg J.Feature extraction from whole-sky ground-based images for cloud-type recognition[J].Journal of Atmospheric and Oceanic Technology,2008,25(1):3-14
参考文献 13
Heinle A,Macke A,Srivastav A.Automatic cloud classification of whole sky images[J].Atmospheric Measurement Techniques,2010,3(3):557-567
参考文献 14
Kazantzidis A,Tzoumanikas P,Bais A F,et al.Cloud detection and classification with the use of whole-sky ground-based images[J].Atmospheric Research,2012,113:80-88
参考文献 15
Lu J Y,Wang Y Z,Zhu Y Q,et al.P_Segnet and NP_Segnet:new neural network architectures for cloud recognition of remote sensing images[J].IEEE Access,2019,7:87323-87333
参考文献 16
Ye L,Cao Z G,Xiao Y,et al.Supervised fine-grained cloud detection and recognition in whole-sky images[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(10):7972-7985
参考文献 17
Li L Y,Li X Y,Jiang L Y,et al.A review on deep learning techniques for cloud detection methodologies and challenges[J].Signal,Image and Video Processing,2021,15(7):1527-1535
参考文献 18
Liu S,Zhang Z,Mei X.Ground-based cloud classification using weighted local binary patterns[J].Journal of Applied Remote Sensing,2015,9(1):095062
参考文献 19
Li Q Y,Zhang Z,Lu W T,et al.From pixels to patches:a cloud classification method based on a bag of micro-structures[J].Atmospheric Measurement Techniques,2016,9(2):753-764
参考文献 20
Ricciardelli E,Romano F,Cuomo V.Physical and statistical approaches for cloud identification using Meteosat Second Generation-Spinning Enhanced Visible and Infrared Imager Data[J].Remote Sensing of Environment,2008,112(6):2741-2760
参考文献 21
Taravat A,Del Frate F,Cornaro C,et al.Neural networks and support vector machine algorithms for automatic cloud classification of whole-sky ground-based images[J].IEEE Geoscience and Remote Sensing Letters,2015,12(3):666-670
参考文献 22
Pfister G,McKenzie R L,Liley J B,et al.Cloud coverage based on all-sky imaging and its impact on surface solar irradiance[J].Journal of Applied Meteorology,2003,42(10):1421-1434
参考文献 23
Urquhart B,Kurtz B,Dahlin E,et al.Development of a sky imaging system for short-term solar power forecasting[J].Atmospheric Measurement Techniques,2015,8(2):875-890
参考文献 24
Peng Z Z,Yu D T,Huang D,et al.3D cloud detection and tracking system for solar forecast using multiple sky imagers[J].Solar Energy,2015,118:496-519
参考文献 25
Chu Y H,Pedro H T C,Nonnenmacher L,et al.A smart image-based cloud detection system for intrahour solar irradiance forecasts[J].Journal of Atmospheric and Oceanic Technology,2014,31(9):1995-2007
参考文献 26
Mantelli Neto S L,von Wangenheim A,Pereira E B,et al.The use of euclidean geometric distance on RGB color space for the classification of sky and cloud patterns[J].Journal of Atmospheric and Oceanic Technology,2010,27(9):1504-1517
参考文献 27
Jayadevan V T,Rodriguez J J,Cronin A D.A new contrast-enhancing feature for cloud detection in ground-based sky images[J].Journal of Atmospheric and Oceanic Technology,2015,32(2):209-219
参考文献 28
Nouri B,Kuhn P,Wilbert S,et al.Cloud height and tracking accuracy of three all sky imager systems for individual clouds[J].Solar Energy,2019,177:213-228
参考文献 29
Yang J,Min Q,Lu W,et al.An automated cloud detection method based on the green channel of total-sky visible images[J].Atmospheric Measurement Techniques,2015,8(11):4671-4679
参考文献 30
Zhuo W,Cao Z G,Xiao Y.Cloud classification of ground-based images using texture-structure features[J].Journal of Atmospheric and Oceanic Technology,2014,31(1):79-92
参考文献 31
Xia M,Lu W T,Yang J,et al.A hybrid method based on extreme learning machine and k-nearest neighbor for cloud classification of ground-based visible cloud image[J].Neurocomputing,2015,160:238-249
参考文献 32
Kliangsuwan T,Heednacram A.FFT features and hierarchical classification algorithms for cloud images[J].Engineering Applications of Artificial Intelligence,2018,76:40-54
参考文献 33
Hughes M,Hayes D.Automated detection of cloud and cloud shadow in single-date landsat imagery using neural networks and spatial post-processing[J].Remote Sensing,2014,6(6):4907-4926
参考文献 34
Howard L.On the modifications of clouds[M].London:Cambridge University Press,2011
参考文献 35
Buch K A,Sun C H.Cloud classification using whole-sky imager data[J].Ninth Symposium on Meteorological Observations & Instrumentation,1995,16(3):353-358
参考文献 36
Zhen Z,Wang F,Sun Y J,et al.SVM based cloud classification model using total sky images for PV power forecasting[C]//2015 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference(ISGT).February 18-20,2015,Washington,DC,USA.IEEE,2015:1-5
参考文献 37
Zhou Z C,Zhang F,Xiao H X,et al.A novel ground-based cloud image segmentation method by using deep transfer learning[J].IEEE Geoscience and Remote Sensing Letters,2618,PP(99):1-5
参考文献 38
Kristollari V,Karathanassi V.Convolutional neural networks for detecting challenging cases in cloud masking using Sentinel-2 imagery[C]//Eighth International Conference on Remote Sensing and Geoinformation of the Environment(RSCy2020),2020,1152:188-201
参考文献 39
Song Q Q,Cui Z H,Liu P.An efficient solution for semantic segmentation of three ground-based cloud datasets[J].Earth and Space Science,2020,7(4):e2019EA001040
参考文献 40
Dev S,Lee Y H,Winkler S.Categorization of cloud image patches using an improved texton-based approach[C]//2015 IEEE International Conference on Image Processing(ICIP).September 27-30,2015,Quebec City,QC,Canada.IEEE,2015:422-426
参考文献 41
Zhang J L,Liu P,Zhang F,et al.CloudNet:ground-based cloud classification with deep convolutional neural network[J].Geophysical Research Letters,2018,45(16):8665-8672
参考文献 42
Liu S,Li M,Zhang Z,et al.Multi-evidence and multi-modal fusion network for ground-based cloud recognition[J].Remote Sensing,2020,12(3):464-483
参考文献 43
Liu S,Duan L L,Zhang Z,et al.Ground-based remote sensing cloud classification via context graph attention network[J].IEEE Transactions on Geoscience and Remote Sensing,2021.DOI:10.1109/TGRS.2021.3063255
参考文献 44
Fang C Y,Jia K B,Liu P Y,et al.Research on cloud recognition technology based on transfer learning[C]//2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).November 18-21,2019,Lanzhou,China.IEEE,2019:791-796
参考文献 45
Gan J R,Lu W T,Li Q Y,et al.Cloud type classification of total-sky images using duplex norm-bounded sparse coding[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10(7):3360-3372
参考文献 46
Zhang J L,Liu P,Zhang F,et al.Ensemble meteorological cloud classification meets Internet of dependable and controllable things[J].IEEE Internet of Things Journal,2021,8(5):3323-3330
参考文献 47
Rahman M M,Shakeri M,Tiong S K,et al.Prospective methodologies in hybrid renewable energy systems for energy prediction using artificial neural networks[J].Sustainability,2021,13(4):2393-2420
参考文献 48
Frniak M,Markovic M,Kamencay P,et al.Vehicle classification based on FBG sensor arrays using neural networks[J].Sensors,2020,20(16):4472
参考文献 49
Ahmad J,Farman H,Jan Z.Deep learning methods and applications[M]//Deep learning:convergence to big data analytics.Singapore:Springer Singapore,2018:31-42
参考文献 50
Liu Z H,Zhou S D,Wang M,et al.Ground-based visible-light cloud image classification based on a convolutional neural network[C]//2019 4th International Conference on Information Systems and Computer Networks(ISCON).November 21-22,2019,Mathura,India.IEEE,2019:108-112
参考文献 51
Shi C Z,Wang C H,Wang Y,et al.Deep convolutional activations-based features for ground-based cloud classification[J].IEEE Geoscience and Remote Sensing Letters,2017,14(6):816-820
参考文献 52
Ye L,Cao Z G,Xiao Y.DeepCloud:ground-based cloud image categorization using deep convolutional features[J].IEEE Transactions on Geoscience and Remote Sensing,2017,55(10):5729-5740
参考文献 53
Liu S,Li M,Zhang Z,et al.Ground-based cloud classification using task-based graph convolutional network[J].Geophysical Research Letters,2020,47(5):e2020GL087338
参考文献 54
Phung V H,Rhee E J.A high-accuracy model average ensemble of convolutional neural networks for classification of cloud image patches on small datasets[J].Applied Sciences,2019,9(21):4500
参考文献 55
Liu S,Li M.Multimodal GAN for energy efficiency and cloud classification in Internet of Things[J].IEEE Internet of Things Journal,2019,6(4):6034-6041
参考文献 56
Zhuang F Z,Qi Z Y,Duan K Y,et al.A comprehensive survey on transfer learning[J].Proceedings of the IEEE,2021,109(1):43-76
参考文献 57
Fang C Y,Jia K B,Liu P Y,et al.Research on cloud recognition technology based on transfer learning[C]//2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).November 18-21,2019,Lanzhou,China.IEEE,2019:791-796
参考文献 58
Manzo M,Pellino S.Voting in transfer learning system for ground-based cloud classification[J].Machine Learning and Knowledge Extraction,2021,3(3):542-553
参考文献 59
Li G S.A novel computer-aided cloud type classification method based on convolutional neural network with squeeze-and-excitation[J].Journal of Physics:Conference Series,2021,1802(3):032051
参考文献 60
Deng J,Berg A,Satheesh S,et al.ImageNet large scale visual recognition challenge 2012(ILSVRC2012)[EB/OL].[2021-06-06].https://image-net.org/challenges/LSVRC/2012/
参考文献 61
Shin H C,Roth H R,Gao M C,et al.Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging,2016,35(5):1285-1298
参考文献 62
Ker J,Wang L P,Rao J,et al.Deep learning applications in medical image analysis[J].IEEE Access,2018,6:9375-9389
参考文献 63
Mikołajczyk A,Grochowski M.Data augmentation for improving deep learning in image classification problem[C]//2018 International Interdisciplinary PhD Workshop(IIPhDW).IEEE,2018:117-122
参考文献 64
Budhiman A,Suyanto S,Arifianto A.Melanoma cancer classification using ResNet with data augmentation[C]//2019 International Seminar on Research of Information Technology and Intelligent Systems(ISRITI).December 5-6,2019,Yogyakarta,Indonesia.IEEE,2019:17-20
参考文献 65
Liu X,Chi M M,Zhang Y F,et al.Classifying high resolution remote sensing images by fine-tuned VGG deep networks[C]//2018 IEEE International Geoscience and Remote Sensing Symposium.July 22-27,2018,Valencia,Spain.IEEE,2018:7137-7140
参考文献 66
Szegedy C,Ioffe S,Vanhoucke V,et al.Inception-v4,inception-ResNet and the impact of residual connections on learning[J].arXiv e-print,2016,arXiv:1602.07261
参考文献 67
Chollet F.Xception:deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).July 21-26,2017,Honolulu,HI,USA.IEEE,2017:1800-1807
参考文献 68
Zhu Y,Newsam S.DenseNet for dense flow[C]//2017 IEEE International Conference on Image Processing(ICIP).September 17-20,2017,Beijing,China.IEEE,2017:790-794
参考文献 69
Sandler M,Howard A,Zhu M L,et al.MobileNetV2:inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:4510-4520
参考文献 70
Wang M,Zhou S D,Yang Z,et al.CloudA:a ground-based cloud classification method with a convolutional neural network[J].Journal of Atmospheric and Oceanic Technology,2020,37(9):1661-1668
参考文献 71
Xie W Y,Liu D,Yang M,et al.SegCloud:a novel cloud image segmentation model using a deep convolutional neural network for ground-based all-sky-view camera observation[J].Atmospheric Measurement Techniques,2020,13(4):1953-1961
参考文献 72
Li M,Liu S,Zhang Z.Dual guided loss for ground-based cloud classification in weather station networks[J].IEEE Access,2019,7:63081-63088
参考文献 73
Luo Q X,Meng Y,Liu L,et al.Cloud classification of ground-based infrared images combining manifold and texture features[J].Atmospheric Measurement Techniques,2018,11(9):5351-5361
目录contents

    摘要

    深度学习的自动学习特征和精确的预测能力使其在地基云分类上获得成功,更复杂更优良的深度学习网络在地基云分类领域得到研究和应用.近两年来一些大规模地基云分类数据集被公布,但还没有文献对这些大数据集进行完整地介绍和使用.本文对深度学习地基云分类领域最新的研究进展进行了详细的概括,并介绍了最新发布的国际标准大规模数据集,最后对几种经典的卷积神经网络深度学习模型在地基云分类上的性能进行了评估,验证了卷积神经网络在地基云分类领域的适用性.

    Abstract

    With the rapid development of deep learning,its automatic learning characteristics and accurate prediction ability make it successful in ground-based cloud classification.More complex and better deep learning networks are applied and studied in the field of ground-based cloud classification.In the past two years,some large-scale ground-based cloud classification data sets have been published,yet there is no literature on fully introduction and use of these large data sets.Here,we list the lately issued massive data sets,then introduce the cloud classification technology,especially detail the latest research progress in deep learning-based cloud classification,and finally assess and compare several classic convolution neural network learning models on their ground-based cloud classification performance.The convolution neural network is verified to be effective in the field of ground-based cloud classification.

  • 0 引言

  • 云是大自然的产物,它由漂浮在空中的微小水滴或冰晶组成[1],这些物质组成的云层覆盖了大约68%的地球表面[2].对云及其特性的研究在许多应用中有着非常重要的作用,例如气候模拟、天气预报、气象学研究、太阳能生产和卫星通信等[3-6].云与水文循环也紧密相连,通过与太阳和陆地辐射的相互作用,影响局部和全球尺度的能量平衡.因为不同的云类型对地球的地表-大气系统有不同的辐射影响[7],所以对云类型的自动分类研究有着重要的意义.

  • 从图像中提取特征并基于图像特征执行分类是图像分类的主要手段[8].按照特征提取方式的不同,云图像分类主要有手工特征和深度学习特征两种表示方法[9],也即传统的机器学习方法和深度学习方法[10].

  • 传统方法主要使用各种特征提取手段来提取选择好的特征,然后用机器学习的方法进行分类.大多数研究者集中研究不同云类别对应的特征提取,以此达到对不同云的分类识别.例如:Singh等[11]评价了5种不同的云分类特征提取方法,即自相关矩阵、共现矩阵、边缘频率、Law's特征和原始长度,最后使用神经网络和K-最近邻方法对云进行分类;Calbo等[12]利用统计纹理特征(12个特征)、基于傅里叶频谱的模式特征(4个特征)和基于阈值图像的特征(6个特征)对8种预定义的天空条件进行了分类,但最终准确率仅有62%;Heinle等[13]提出了一种基于能量和熵等纹理特征的自动云分类算法,使用K-最近邻分类器对7种天空情况进行分类;Kazantzidis等[14]提出一种改进的K-最近邻算法,利用统计色彩和纹理特征,并考虑了云量、太阳天顶角和天空图像中雨滴的存在.虽然传统方法可以在不同程度上对不同类型的云进行分类,但它具有许多局限性:1)人工制作的特征依赖于专家的知识,且提取方法较为复杂[15];2)结果和效率高度依赖于训练样本和使用数据集的整体质量;3)特征提取和分类器是两个分割的环节,受到分类方法和特征提取模型的限制,识别能力不高[16].

  • 近年来随着人工智能的快速发展,深度学习在图像识别领域取得了成功,深度学习方法能够从图像数据中自动地提取特征,因此不需要任何特征提取方法[17].由于深度学习在图像识别领域的强大性能和优秀表现,学者们将深度学习应用到地基云分类领域并取得了更好的结果,尤其是最近大规模地基云数据集的出现使得更复杂的深度学习模型在该领域得到应用.

  • 本文首先介绍了国内外现有的标准地基云图数据集,然后针对云的特征和云的类别进行分析和讨论,接着介绍了最新的基于深度学习方法的地基云分类研究,并选择几种经典的卷积神经网络模型在地基云数据集上评估其性能,最后对基于地面的云分类的研究做出总结和展望.

  • 1 地基云图数据集

  • 地基云图数据集的缺乏是制约地基云分类发展的主要因素之一,直到近年来自动云观测技术的完善和使用深度学习制作数据集技术的出现,更大规模地基云的数据集才得以问世.本部分将从云图采集、云分类、公共数据集3个方面进行介绍.

  • 1.1 云图采集仪器

  • 如今,云观测的自动成像设备经过不断的发展已经愈发完善,与人类观测者相比,自动观测设备不仅节省了大量的人力物力,而且可以连续记录天空的原始信息,并储存图像供气象研究者使用[18].运用于云图像自动成像的采集方式通常有基于卫星(space-based satellite)云图成像、空基探空仪(air-based radiosonde)、基于地面(ground-based)云图成像3种.基于卫星的采集仪器能够由上而下地观测到大面积的云层[19],可以提供大规模的云信息,但其空间分辨率有限,无法描述局部的小尺度云特征,导致忽略一些小的云,例如低云或薄云由于与地表温度和亮度相似,经常会混淆[20].而空基探空仪成本大大高出基于地面的设备[21],故基于地面的成像设备成为研究人员的最佳选择.近些年来,用于获取全天图像的地面仪器数量有所增加[22],已经有一些地面成像设备被研发成功并应用于对云的研究中,例如由圣地亚哥加州大学斯克里普斯海洋学研究所开发的Whole-Sky Imager(WSI),它可以获取大角度的地面全天空图像[23],与卫星图像显示的是空间分辨率有限的大面积云层顶部不同,地面WSI显示的是局部地区云层底部,对于当地天气研究或太阳探测具有不可替代的作用[24-25].

  • 此外YanKee环境公司研发的Total-Sky Imager (TSI)也被经常使用,TSI可提供白天半球的彩色图像和一些其他关键的天空和气象信息[26-27].另一个常用的地面观测系统是日本通信实验室研发的All-Sky Imager(ASI),它以高时间和空间分辨率克服了卫星成像系统时空分辨率低的缺点[28].目前经过研究者的研究和创新,自动云观测设备已经趋于完善,这些地面观测设备大多使用红-绿-蓝(RGB)彩色图像来捕捉天空的状况[29],从这些设备提供的连续的全天空图像中,人们可以推断出云的宏观属性,例如云的高度、云的覆盖率、云的类型[30],这对自动分类的研究有非常大的意义.

  • 1.2 云分类标准

  • 研究人员针对云分类任务将云分为几类的问题也有所不同.传统上,云的分类以云的形状为基本要素,同时考虑云的形状成因和其内部的微观结构.气象学家一般将云分为3类,即卷云、积云和层云[31],根据云的透明度、扩散和排列,每一种云的类型都可以继续细分为许多种类[32].此外,对于每一种云类型,它可能会与附属的云及一些特性一起出现且会随着时间改变形状,转变为其他类型的云[33].正是由于这种性质,自动云分类任务充满了挑战和困难.更详细的,Haward将云分为高、中、低3族10属29类(分类标准见表1)[34].研究者大多将几个相似的、区别度很小的云类别组合成一个天空条件进行自动云分类任务,如积雨云和雨层云,2类云颜色相近且边缘都不清晰,相似度很高难以区分,将它们作为不同类别进行分类无疑增大了分类的挑战性,且2种云出现时都伴有降水,因此常常将这2种云组合成一类进行分类.此外,研究者分类的标准并不一致,Buch等[35]将地面云图分为5种不同的天空条件,Calbo等[12]为自动云分类定义了8种天空条件,Zhen等[36]将全天图像定义为4种天空条件.由此可见,研究者们在对云进行分类时标准不一,在地基云分类任务中将云分成什么类型需要研究者根据数据集样本的具体情况进行判断和验证.

  • 1.3 公共数据集

  • 随着地基云观测设备的日趋发展和完善,越来越多的云图数据被研究人员收集和标注.制作数据集对图片质量也有严格要求[37],需要挑选出噪声小、特征信息明显的云图像.此外,由于有些云的类别相似度较高,需要专家进行标注提高置信度,这使得制作地基云分类数据集充满了挑战[38].由于数据集制作困难,一直以来研究者们都使用小数据集进行地基云分类的研究,这毫无疑问极大地阻碍了地基云分类领域的发展.如今,深度学习的出现使得制作数据集变得简单[39].研究人员一般将采集来的云图使用深度学习模型进行预分类,然后对分类错误、分类置信度低的图片进行人工分类,这样大规模数据集的制作就不再复杂,地基云分类的大规模数据集近两年也相继出现.本节先介绍国内外常用的两种标准云图小数据集:SWIMCAT、CCSN,再对3种新发布的地基云分类大数据集MGCD、GCD(地基云数据集)、HBMCD进行介绍.各数据集样本云图如图1所示,各数据集总结见表2.

  • 表1 10种云属简要介绍[34]

  • Table1 Brief introduction of 10cloud categories

  • 1)SWIMCAT数据集

  • SWIMCAT数据集全称为新加坡全天空成像分类数据集,是由Dev等[40]使用广角高分辨率天空成像设备(WSI)收集和制作的新加坡全天成像地基云数据集,它将784张挑选好的云图分为晴空(clear sky)、混合云(patterned clouds)、厚黑云(thick dark clouds)、厚白云(thick white clouds)、薄云(veil clouds).

  • 2)CCSN数据集

  • CCSN[41]全称为卷云积云层云雨云数据集,将云基于气象标准分为10个类别,并首次将飞行轨迹(Contrails)纳入地基云分类任务(表2中简称Ct).该数据集包含了2 543幅云图像,是常用的SWIMCAT的3倍,但在深度学习环境中还是不能满足需求,而且其样本图像包含一些房屋树木等噪声,在学习云特征时容易造成干扰.

  • 3)MGCD数据集

  • MGCD[42]全称为多模态地基云数据集,是目前国内外第一个记录地基云图多模态信息的数据集.Liu等[42]在天津收集了超过22个月,包含8 000幅地基云样本和每个样本对应的多模态信息,即温度、湿度、压力和风速,由气象站收集并储存在一个向量中.8 000幅图片将10种云属组合成5种类型,加上晴空和混合云,共分为7种天空类型,其中云量低于10%的图片被定义为晴空(clear),含有不少于2种云的图片被定义为混合云(mixed).

  • 4)GCD数据集

  • GCD数据集[43]全称为地基云数据集,包括19 000幅地基云图片,与MGCD分类标准一样,分为7种天空条件.该数据集在中国10个省份收集超过1年,具有很大的多样性.除此之外,该数据集的规模远远超过之前的数据集,拓展了深度学习在地基云分类领域上的研究.

  • 5)HBMCD数据集

  • HBMCD数据集[44]全称为华云BJUT-MIP云图数据集,是由北京工业大学使用华云观测设备收集的.该数据集将10 942幅地基云图像分为11类,即10类标准云属和1类无云(No),是目前按照10类云属划分的最大的数据集.11类云图各类别样本量均匀,易于在该数据集上开展深度学习地基云分类的研究.

  • 表2 地基云分类数据集

  • Table2 Summary of ground-based cloud classification data sets

  • 图1 地基云分类数据集各类别样本图片

  • Fig.1 Various sample images of ground-based cloud classification data set

  • 传统方法使用图像处理层面的算法对地基云图像提取特征信息然后使用机器学习算法进行分类.由于公共数据集的缺乏,大多研究者使用小样本数据集,如SWIMCAT数据集,但其并不包含所有的云类别[45],而CCSN包含10个云属但图像中有一些房屋树木等干扰,且这些数据集样本量较少[46].随着深度学习在地基云分类领域的成功应用,大规模的数据集是深度学习成功的关键,使得人们对数据集的样本规模有了更多的要求.最近的GCD和HBMCD分别将云划分为7种天空条件和气象标准的10类云属,数据集规模较大,便于研究者使用深度学习开展地基云自动分类的研究[47].但面对层数比较深、结构更复杂的神经网络模型时,仍然会出现因数据量不足而导致的过拟合现象,因此还需要规模更大、分类置信度更高、干扰噪声更小、各类别样本数分布更均匀的数据集来推动地基云分类任务的研究和发展[48].

  • 2 深度学习地基云分类研究回顾

  • 深度学习在2012年以来得到了广泛应用.随着深度学习算法的发展,如线性整流单元(ReLU)和随机失活(DropOut)的出现,深度学习识别和预测的准确性和训练速度大大提高,与此同时计算机硬件如GPU性能的提升也是深度学习飞速发展的重要原因.如今,深度学习已被应用于图像识别、大数据处理、自动驾驶等多个领域[49].

  • 在云分类任务中,通过将深度学习和传统提取手工特征再分类的方法比较,可以发现深度学习可以使计算机自动地识别云的特征,减少人的参与过程[50].目前,已经有多种深度学习方法被应用于地基云分类任务上,并有良好的表现,在一些相似类别的区分任务上准确率有很大提升.

  • 由于卷积神经网络在图像识别领域出众的表现,一些研究者将其应用到地基云分类任务中,并评估卷积神经网络在地基云分类领域的适用性.Shi等[51]使用深度卷积激活的云图像的特征进行地基云识别,使用了5类784张图片和7类100张图片的不同数据集进行了实验,实验结果超过了大部分传统方法.Ye等[52]通过模式挖掘和选择策略从多个卷积层中采集局部视觉特征,然后使用Fisher向量进行编码,提出名为DeepCloud的新型特征提取方法对云进行识别,验证了深度卷积特征在云图像表示方面的优越性.Zhang等[41]提出了一个名为CloudNet的简单网络,将地基云图像分为11个类别,分类结果超过89%.随着CNN在地基云分类上展示出卓越的性能,其他一些网络也被应用在地基云分类上并有良好的表现.Liu等[53]将图卷积网络应用于云分类,提出一种基于任务的图卷积网络模型(Task-Based Graph Convolutional Network),在8 000张云图片中将云分成7类,最高准确率达到89.48%.Liu等[43]还提出一种上下文图注意网络(Context Graph Attention),在19 000张的云图数据集中获得84.30%的准确率.

  • 然而现有的大多数深度学习模型参数较多,需要大规模的数据集进行训练,但现有数据集大多规模较小,实验结果可能受到训练过拟合的干扰.针对这一问题,Phung等[54]提出一种在小规模数据集上的云分类方法,设计了一个简单的CNN模型,并应用两种正则化方法在SWIMCAT上获得了最大精度99.36%的结果.Liu等[55]将多模态用于云分类,提出多模态生成对抗网络(Mutimodel Gan),可以同时生成云图像和对应的多模态信息,提高分类网络的生成能力,使用Gan网络生成的云图信息可以补足数据集小而造成的过拟合问题.深度神经网络在训练前,首先会对网络中的参数初始化,使用随机初始化的方法充满着不确定性.针对这个问题,迁移学习有着优异的表现,其主要思想是将相关领域中学习到的知识或模式应用到其他领域[56].显然,使用预训练的网络比用随机初始化的权重从头开始训练一个新网络更容易、更快.迁移学习在地基云领域也有优秀的表现,如Fang等[57]使用迁移学习在地基云分类中获得96.55%的准确率,Manzo等[58]使用基于投票的迁移学习在多个数据集中都获得了较高的准确率,最高达99.98%.

  • 经典的CNN模型对图像识别具有普遍适用性,在超大规模数据集ImageNet上也能获得很高的准确率,而地基云图像作为天空中的自然图像,特征和颜色较为明显,相邻类别相似度较大,因此针对地基云识别提出对云分类更具鲁棒性的网络是必要的.研究人员对CNN进行了不同的改进,如Li等[59]神经网络SENet,增大有效特征图的权重,减小无效特征图的权重;Fang等[44]提出一种轻量级的卷积神经网络模型LCCNet,对地基云图有较好的表征能力和较低的运算复杂度,在11类云分类任务中达到97.25%的准确率;Zhang等[46]使用集成学习对多个卷积神经网络和残差神经网络进行了模型集成用于云分类,集成后的模型实现了较高的云分类精确率,且超过任何单一的深度学习模型.

  • 总体来说,在地基云分类任务中使用深度学习方法成为研究者的优先选择,其强大的识别性能和简便的实验操作促使研究者尝试更多类型的网络结构和深度学习算法来进行地基云分类,对于自动云识别系统畅想目标的实现更进了一步.

  • 3 基于CNN的地基云识别评估

  • 为对CNN在地基云识别上的表现做出详细的评估,本章先对CNN进行简单的介绍,然后选取目前性能最好的几种CNN网络分别在小数据集和大数据集上进行实验,最后对实验结果做出系统的评估.

  • 3.1 卷积神经网络

  • 卷积神经网络(CNN)[60]是深度学习的代表算法之一,可以学习云图像中传统方法难以学习和提取的复杂的边缘、纹理信息,CNN在大规模图像识别任务中已经获得了巨大的成功.CNN是一种包含卷积计算的深度前馈神经网络,它的卷积层通过卷积运算来代替网络中至少一层的矩阵线性运算,通过权重共享,减低了网络的复杂度[61].研究表明,低层CNN捕获图像的纹理信息[62],深层反映图像高级语义信息.根据CNN中的一系列卷积,逐层捕获相邻像素间的信息,可以确保CNN对地基云分类任务的准确性与鲁棒性.为此本文构建出一个简单的CNN网络,该网络由5个卷积层组成,每个卷积层有2个卷积层核进行卷积计算,每2个卷积层之间含有1个池化层.将1张地基云图像传输进该网络,得到其通过每一层卷积层后的特征图,如图2所示.从特征图可以观察到,前2层特征图片纹理较为明显,通过后面的卷积层后纹理特征逐渐模糊,呈现出语义的特征.目前CNN的深度模型经过不断的改良和创新,有越来越多的优秀模型被提出,这些模型在地基云分类领域被应用并获得了令人满意的效果.经典的CNN模型包括VGG、ResNet、DenseNet、Inception、Xception和MobileNet等,这些CNN模型都有各自的优点,在图像识别问题上获得了优良的成果.

  • 图2 地基云图传入CNN的特征图

  • Fig.2 Feature images of ground-based cloud by CNN

  • CNN的主要缺点是过多的参数在数据集不足的情况下会导致过拟合的发生,并且随着网络层数的加深,容易发生梯度消失现象使得模型难以优化且计算复杂度很高导致运算时间过长[63].经典的CNN模型针对这些问题也做出了良好的解答,其中深度残差网络ResNet[64]借用Highway Network的思想,允许一部分原始信息直接从输入层传递到最后一层,优化了深度网络层数太深而无法训练的问题.牛津大学研发的VGG模型[65]引入了非线性变换,降低了计算量,且通过Multi-Scale方法训练和预测,可以增加训练数据量,减少过拟合.Google研发的Inception[66]将较大的卷积层拆分成小的卷积层,这种分解对参数进行了解离,既减少了过拟合又加快了运算速度.Xception[67]在Inception上加以改进,使用深度可分离卷积(Depthwise Separable Convolution)以更少的参数来代替普通的3×3卷积核,在基本不增加模型复杂度的条件下优化了模型效果.DenseNet[68]采用网络更窄、参数更少的设计,使得梯度传递更加有效,更容易训练.MobileNet[69]是一种轻量级框架,在保证准确率的情况下,将参数压缩至1/7,参数量极小,运算速度也超过其他模型.

  • 3.2 实验设计与参数

  • 为了更进一步探究各种CNN模型在地基云分类上的表现,本文先是对上述的6种CNN模型及其变体共8种模型在小数据集SWIMCAT上进行了评估,实验的CNN模型分别为:VGG16、VGG19、ResNet50、ResNet101、InceptionV3、Xception、DenseNet201、MobileNet,它们也是目前在图像识别任务中表现最好的8种模型.同时为了分析CNN模型在大数据集中的表现,还选取了在上个实验中表现较好的其中4种CNN模型DenseNet201、ResNet50、InceptionV3、VGG16在大数据集GCD上进行评估.

  • 本次实验引入了迁移学习,迁移学习的优秀之处在于其使用了模型在更大的数据集如ImageNet上预训练的参数,大大加快了训练速度和准确率.为此以VGG网络为例,在同样的参数下让VGG分别在迁移学习和非迁移学习下训练,实验结果如图3所示.从实验结果可以看出非迁移学习训练了10个epoch准确率才收敛,且准确率仅能达到88%,而迁移学习收敛只用了2~3个epoch,并且准确率高达99.36%.

  • 本次实验在Ubuntu环境上采用Python语言和Tensorflow框架在Jupyter平台进行编程实现.由于SWIMCAT数据集较小,为对比训练时间,在型号inter-i7-11的CPU的运行,GCD数据集在型号GTX-2080Ti的GPU机器上运行,各个模型的准确率如图4和图5所示,模型总结如表3所示.

  • 两次实验所使用的参数基本相同,首先将数据集按照4∶1划分为训练集和验证集,将图片大小转化为CNN模型的输入尺寸大小,各模型输入尺寸在表3中可见,初始学习率(learning rate)为0.000 1,学习率衰减(decay rate)为0.96,训练周期(epoch)为20,使用交叉熵损失(cross entropy)来更新参数,批量大小(batch size)在SWIMCAT数据集上设为8,在GCD数据集上设为32.模型代码和预训练权重使用Tensorflow提供的网络模型及其在ImageNet数据集上预训练的数据进行实验.

  • 图3 迁移学习与非迁移学习对比

  • Fig.3 Comparison between transfer learning and non-transfer learning

  • 图4 8种CNN模型在SWIMCAT数据集上运行结果

  • Fig.4 Results of eight CNN models running on SWIMCAT data set

  • 图5 4种CNN模型在GCD数据集实验结果

  • Fig.5 Experimental results of four CNN models on GCD data set

  • 表3 8种卷积神经网络总结

  • Table3 Summary of 8convolutional neural networks

  • 3.3 实验结果评估

  • 为详细评估各CNN模型,除训练集和验证集准确率外,还构建出4种CNN模型在GCD数据集上的混淆矩阵(图6)和各分类的召回率(Recall)、F1值(表4).在SWIMCAT上由于样本量较少,模型表现较好,准确率大多在98%以上,即114张的验证集仅有2~3张预测错误,这种情况下混淆矩阵和召回率等不适合用于对8种网络进行评估.本节在8种模型的准确率、训练时间和参数等方面对模型进行评估.

  • 从实验结果中得出的结论如下:

  • 1)在小数据集上,使用卷积神经网络和迁移学习对地基云进行分类的结果已经超过大部分现有的传统方法,VGG16、ResNet50、ResNet101在验证集的最高准确率都达到了99.36%,其他几个模型的准确率也很接近,除MobileNet外均超过98%,这足以表明卷积神经网络在地基云分类任务中的适用性和优越性.

  • 2)本实验使用的迁移学习是在超大规模图像分类数据集ImageNet上预训练的权重,避免了深度学习随机初始化权重的随机性,使训练速度和效果显著提升.使用深度学习进行地基云分类的实验步骤十分简洁,不需要复杂的提取特征过程,这是因为卷积神经网络会自动地学习地基云图像的形状、纹理等特征[70],因此不需要对地基云图进行特征的提取,只需要搭建好模型加载预训练的权重,再将数据集进行训练即可[71].实验结果展现了迁移学习和CNN结合后的强大性能,从实验复杂性和识别结果精确性上均超过绝大多数算法.

  • 3)表3中的VGG参数量最大,2种模型参数均超过1亿,所以epoch训练时间也较长.准确率上VGG以99.36%在8种模型中最高,因为VGG将卷积层的卷积核参数改为统一的3×3,这种多次重复使用统一大小的卷积核的方法使得VGG可以提取更复杂、更具表达性的特征.DenseNet和ResNet也有良好的表现,准确率分别高达98.72%和99.36%,且损失曲线下降速度快.Inception和Xception准确率都达到了98.08%,这2个网络参数量和训练时间都非常相近,也表现出了对地基云识别的强大性能.MobileNet作为轻量级模型,参数量最小,仅有300多万,因此每轮训练时间也较短,达到验证集最高精确率迭代次数较多,但是在准确率上表现不佳,这可能是由于其参数过少,在学习特征时将云的部分特征遗失所致.总的来说,这8种网络性能最低的MobileNet在SWIMCAT数据集上准确率也达到了95.51%,这已经超过了许多的传统方法,准确率最高的VGG和ResNet达到了99.36%,这证明了CNN在小数据集地基云分类任务上发挥着出色的性能.

  • 4)在包含12 000张图像的GCD数据集上进行的实验可以发现DenseNet的准确率最高,达93.3%.且4种网络模型在GCD的表现相比SWIMCAT,无论是训练集还是验证集准确率都有降低,这是因为该数据集规模远远超过SWIMCAT数据集,且GCD将云图分为7类,包含了全部的10种云属.从混淆矩阵(图6)与召回率、F1值(表4)可以看出,4种模型预测的效果总体比较相近,其中层积云、积雨云2个种类的预测容易混淆,这2个类别的云召回率、F1值较其他类别而言偏低,层积云的召回率、F1值大多低于80%.通过查看层积云、积雨云的一些样本图片发现,这2个类别样本非常相似,云体灰暗,一些图片边缘不明显、难以区分,导致预测分类结果不高.其他类别的云召回率、F1值大多超过95%,可以应用于自动云分类任务中.这4种模型在超过1万张图片的数据集上准确率依然高于90%,远远高于许多传统方法.

  • 总体来看,卷积神经网络在地基云识别上的性能卓越,准确率、训练速度等超过传统方法,尤其是最近几个大数据集的诞生,使得更多的神经网络模型得以在地基云分类上使用.

  • 4 总结与展望

  • 本文主要针对最新的地基云分类研究进行介绍,从深度学习方法角度对近几年的研究进展进行了详细的概括.然后介绍了最新的几种云图大数据集,这些数据集允许更复杂的深度学习模型在地基云识别领域得到评估和应用,解决了之前因云图数据集缺乏而造成的困扰.最后使用迁移学习对卷积神经网络模型在地基云识别上的性能进行评估.评估时,先是在SWMICAT数据集上使用8种经典的卷积神经网络进行训练和预测,8种模型的性能都超过了绝大部分传统方法,再使用其中4种性能较好的模型在12 000张云图片的GCD数据集上进行评估,得出在大数据集上DenseNet201预测效果最好的结论.

  • 目前研究人员已经将许多新的模型应用于地基云分类领域[72],期待有更适合地基云分类任务的网络模型出现.除卷积神经网络外,生成对抗网络和注意力机制等都在图像识别中有良好的表现[73],期待有更多网络在地基云分类上得到应用.目前基于深度学习的地基云分类研究大多基于模型上的评估和验证,期待有更多更适合地基云分类的算法被提出,使得地基云分类在深度学习方法上得以发展和进步.

  • 5 数据可用性申明

  • 本文使用的所有模型、代码可以在https://gitee.com/zhuangzhihaoz/deeplearning_cloud上查阅.本文使用的SWIMCAT数据集和GCD数据集可以分别向其发布者请求获得:

  • 表4 4种CNN模型在GCD上的召回率和F1值

  • Table4 Recall rates and F1values of the four CNN models on GCD

  • 图6 4种CNN模型在GCD数据集上的混淆矩阵

  • Fig.6 Confusion matrices of four CNN models on GCD data set

  • SWIMCAT:http://vintage.winklerbros.net/swimcat.html

  • GCD:https://github.com/shuangliutjnu/TJNU-Ground-based-Cloud-Dataset

  • 参考文献

    • [1] Tang Y Z,Yang P,Zhou Z M,et al.Improving cloud type classification of ground-based images using region covariance descriptors[J].Atmospheric Measurement Techniques,2021,14(1):737-747

    • [2] Zhang Y C.Calculation of radiative fluxes from the surface to top of atmosphere based on ISCCP and other global data sets:refinements of the radiative transfer model and the input data[J].Journal of Geophysical Research:Atmospheres,2004,109(D19):1121-1125

    • [3] Naud C M,Booth J F,del Genio A D.The relationship between boundary layer stability and cloud cover in the post-cold frontal region[J].Journal of Climate,2016,29(22):8129-8149

    • [4] Cui F,Ju R R,Ding Y Y,et al.Prediction of regional global horizontal irradiance combining ground-based cloud observation and numerical weather prediction[J].Advanced Materials Research,2014,1073/1074/1075/1076:388-394

    • [5] Yuan F,Lee Y H,Meng Y S.Comparison of radio-sounding profiles for cloud attenuation analysis in the tropical region[C]//2014 IEEE Antennas and Propagation Society International Symposium(APSURSI).July 6-11,2014,Memphis,TN,USA.IEEE,2014:259-260

    • [6] Phung V H,Rhee E J.A deep learning approach for classification of cloud image patches on small datasets[J].Journal of Information and Communication Convergence Engineering,2018,16(3):173-178

    • [7] Liu L,Sun X J,Chen F,et al.Cloud classification based on structure features of infrared images[J].Journal of Atmospheric and Oceanic Technology,2011,28(3):410-417

    • [8] Cheng H Y,Yu C C.Block-based cloud classification with statistical features and distribution of local texture features[J].Atmospheric Measurement Techniques,2015,8(3):1173-1182

    • [9] Hu J,Chen Z B,Yang M,et al.A multiscale fusion convolutional neural network for plant leaf recognition[J].IEEE Signal Processing Letters,2018,25(6):853-857

    • [10] Wu X P,Zhan C,Lai Y K,et al.IP102:a large-scale benchmark dataset for insect pest recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).June 15-20,2019,Long Beach,CA,USA.IEEE,2019:8779-8788

    • [11] Singh M,Glennen M.Automated ground-based cloud recognition[J].Pattern Analysis and Applications,2005,8(3):258-271

    • [12] Calbo J,Sabburg J.Feature extraction from whole-sky ground-based images for cloud-type recognition[J].Journal of Atmospheric and Oceanic Technology,2008,25(1):3-14

    • [13] Heinle A,Macke A,Srivastav A.Automatic cloud classification of whole sky images[J].Atmospheric Measurement Techniques,2010,3(3):557-567

    • [14] Kazantzidis A,Tzoumanikas P,Bais A F,et al.Cloud detection and classification with the use of whole-sky ground-based images[J].Atmospheric Research,2012,113:80-88

    • [15] Lu J Y,Wang Y Z,Zhu Y Q,et al.P_Segnet and NP_Segnet:new neural network architectures for cloud recognition of remote sensing images[J].IEEE Access,2019,7:87323-87333

    • [16] Ye L,Cao Z G,Xiao Y,et al.Supervised fine-grained cloud detection and recognition in whole-sky images[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(10):7972-7985

    • [17] Li L Y,Li X Y,Jiang L Y,et al.A review on deep learning techniques for cloud detection methodologies and challenges[J].Signal,Image and Video Processing,2021,15(7):1527-1535

    • [18] Liu S,Zhang Z,Mei X.Ground-based cloud classification using weighted local binary patterns[J].Journal of Applied Remote Sensing,2015,9(1):095062

    • [19] Li Q Y,Zhang Z,Lu W T,et al.From pixels to patches:a cloud classification method based on a bag of micro-structures[J].Atmospheric Measurement Techniques,2016,9(2):753-764

    • [20] Ricciardelli E,Romano F,Cuomo V.Physical and statistical approaches for cloud identification using Meteosat Second Generation-Spinning Enhanced Visible and Infrared Imager Data[J].Remote Sensing of Environment,2008,112(6):2741-2760

    • [21] Taravat A,Del Frate F,Cornaro C,et al.Neural networks and support vector machine algorithms for automatic cloud classification of whole-sky ground-based images[J].IEEE Geoscience and Remote Sensing Letters,2015,12(3):666-670

    • [22] Pfister G,McKenzie R L,Liley J B,et al.Cloud coverage based on all-sky imaging and its impact on surface solar irradiance[J].Journal of Applied Meteorology,2003,42(10):1421-1434

    • [23] Urquhart B,Kurtz B,Dahlin E,et al.Development of a sky imaging system for short-term solar power forecasting[J].Atmospheric Measurement Techniques,2015,8(2):875-890

    • [24] Peng Z Z,Yu D T,Huang D,et al.3D cloud detection and tracking system for solar forecast using multiple sky imagers[J].Solar Energy,2015,118:496-519

    • [25] Chu Y H,Pedro H T C,Nonnenmacher L,et al.A smart image-based cloud detection system for intrahour solar irradiance forecasts[J].Journal of Atmospheric and Oceanic Technology,2014,31(9):1995-2007

    • [26] Mantelli Neto S L,von Wangenheim A,Pereira E B,et al.The use of euclidean geometric distance on RGB color space for the classification of sky and cloud patterns[J].Journal of Atmospheric and Oceanic Technology,2010,27(9):1504-1517

    • [27] Jayadevan V T,Rodriguez J J,Cronin A D.A new contrast-enhancing feature for cloud detection in ground-based sky images[J].Journal of Atmospheric and Oceanic Technology,2015,32(2):209-219

    • [28] Nouri B,Kuhn P,Wilbert S,et al.Cloud height and tracking accuracy of three all sky imager systems for individual clouds[J].Solar Energy,2019,177:213-228

    • [29] Yang J,Min Q,Lu W,et al.An automated cloud detection method based on the green channel of total-sky visible images[J].Atmospheric Measurement Techniques,2015,8(11):4671-4679

    • [30] Zhuo W,Cao Z G,Xiao Y.Cloud classification of ground-based images using texture-structure features[J].Journal of Atmospheric and Oceanic Technology,2014,31(1):79-92

    • [31] Xia M,Lu W T,Yang J,et al.A hybrid method based on extreme learning machine and k-nearest neighbor for cloud classification of ground-based visible cloud image[J].Neurocomputing,2015,160:238-249

    • [32] Kliangsuwan T,Heednacram A.FFT features and hierarchical classification algorithms for cloud images[J].Engineering Applications of Artificial Intelligence,2018,76:40-54

    • [33] Hughes M,Hayes D.Automated detection of cloud and cloud shadow in single-date landsat imagery using neural networks and spatial post-processing[J].Remote Sensing,2014,6(6):4907-4926

    • [34] Howard L.On the modifications of clouds[M].London:Cambridge University Press,2011

    • [35] Buch K A,Sun C H.Cloud classification using whole-sky imager data[J].Ninth Symposium on Meteorological Observations & Instrumentation,1995,16(3):353-358

    • [36] Zhen Z,Wang F,Sun Y J,et al.SVM based cloud classification model using total sky images for PV power forecasting[C]//2015 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference(ISGT).February 18-20,2015,Washington,DC,USA.IEEE,2015:1-5

    • [37] Zhou Z C,Zhang F,Xiao H X,et al.A novel ground-based cloud image segmentation method by using deep transfer learning[J].IEEE Geoscience and Remote Sensing Letters,2618,PP(99):1-5

    • [38] Kristollari V,Karathanassi V.Convolutional neural networks for detecting challenging cases in cloud masking using Sentinel-2 imagery[C]//Eighth International Conference on Remote Sensing and Geoinformation of the Environment(RSCy2020),2020,1152:188-201

    • [39] Song Q Q,Cui Z H,Liu P.An efficient solution for semantic segmentation of three ground-based cloud datasets[J].Earth and Space Science,2020,7(4):e2019EA001040

    • [40] Dev S,Lee Y H,Winkler S.Categorization of cloud image patches using an improved texton-based approach[C]//2015 IEEE International Conference on Image Processing(ICIP).September 27-30,2015,Quebec City,QC,Canada.IEEE,2015:422-426

    • [41] Zhang J L,Liu P,Zhang F,et al.CloudNet:ground-based cloud classification with deep convolutional neural network[J].Geophysical Research Letters,2018,45(16):8665-8672

    • [42] Liu S,Li M,Zhang Z,et al.Multi-evidence and multi-modal fusion network for ground-based cloud recognition[J].Remote Sensing,2020,12(3):464-483

    • [43] Liu S,Duan L L,Zhang Z,et al.Ground-based remote sensing cloud classification via context graph attention network[J].IEEE Transactions on Geoscience and Remote Sensing,2021.DOI:10.1109/TGRS.2021.3063255

    • [44] Fang C Y,Jia K B,Liu P Y,et al.Research on cloud recognition technology based on transfer learning[C]//2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).November 18-21,2019,Lanzhou,China.IEEE,2019:791-796

    • [45] Gan J R,Lu W T,Li Q Y,et al.Cloud type classification of total-sky images using duplex norm-bounded sparse coding[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10(7):3360-3372

    • [46] Zhang J L,Liu P,Zhang F,et al.Ensemble meteorological cloud classification meets Internet of dependable and controllable things[J].IEEE Internet of Things Journal,2021,8(5):3323-3330

    • [47] Rahman M M,Shakeri M,Tiong S K,et al.Prospective methodologies in hybrid renewable energy systems for energy prediction using artificial neural networks[J].Sustainability,2021,13(4):2393-2420

    • [48] Frniak M,Markovic M,Kamencay P,et al.Vehicle classification based on FBG sensor arrays using neural networks[J].Sensors,2020,20(16):4472

    • [49] Ahmad J,Farman H,Jan Z.Deep learning methods and applications[M]//Deep learning:convergence to big data analytics.Singapore:Springer Singapore,2018:31-42

    • [50] Liu Z H,Zhou S D,Wang M,et al.Ground-based visible-light cloud image classification based on a convolutional neural network[C]//2019 4th International Conference on Information Systems and Computer Networks(ISCON).November 21-22,2019,Mathura,India.IEEE,2019:108-112

    • [51] Shi C Z,Wang C H,Wang Y,et al.Deep convolutional activations-based features for ground-based cloud classification[J].IEEE Geoscience and Remote Sensing Letters,2017,14(6):816-820

    • [52] Ye L,Cao Z G,Xiao Y.DeepCloud:ground-based cloud image categorization using deep convolutional features[J].IEEE Transactions on Geoscience and Remote Sensing,2017,55(10):5729-5740

    • [53] Liu S,Li M,Zhang Z,et al.Ground-based cloud classification using task-based graph convolutional network[J].Geophysical Research Letters,2020,47(5):e2020GL087338

    • [54] Phung V H,Rhee E J.A high-accuracy model average ensemble of convolutional neural networks for classification of cloud image patches on small datasets[J].Applied Sciences,2019,9(21):4500

    • [55] Liu S,Li M.Multimodal GAN for energy efficiency and cloud classification in Internet of Things[J].IEEE Internet of Things Journal,2019,6(4):6034-6041

    • [56] Zhuang F Z,Qi Z Y,Duan K Y,et al.A comprehensive survey on transfer learning[J].Proceedings of the IEEE,2021,109(1):43-76

    • [57] Fang C Y,Jia K B,Liu P Y,et al.Research on cloud recognition technology based on transfer learning[C]//2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).November 18-21,2019,Lanzhou,China.IEEE,2019:791-796

    • [58] Manzo M,Pellino S.Voting in transfer learning system for ground-based cloud classification[J].Machine Learning and Knowledge Extraction,2021,3(3):542-553

    • [59] Li G S.A novel computer-aided cloud type classification method based on convolutional neural network with squeeze-and-excitation[J].Journal of Physics:Conference Series,2021,1802(3):032051

    • [60] Deng J,Berg A,Satheesh S,et al.ImageNet large scale visual recognition challenge 2012(ILSVRC2012)[EB/OL].[2021-06-06].https://image-net.org/challenges/LSVRC/2012/

    • [61] Shin H C,Roth H R,Gao M C,et al.Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging,2016,35(5):1285-1298

    • [62] Ker J,Wang L P,Rao J,et al.Deep learning applications in medical image analysis[J].IEEE Access,2018,6:9375-9389

    • [63] Mikołajczyk A,Grochowski M.Data augmentation for improving deep learning in image classification problem[C]//2018 International Interdisciplinary PhD Workshop(IIPhDW).IEEE,2018:117-122

    • [64] Budhiman A,Suyanto S,Arifianto A.Melanoma cancer classification using ResNet with data augmentation[C]//2019 International Seminar on Research of Information Technology and Intelligent Systems(ISRITI).December 5-6,2019,Yogyakarta,Indonesia.IEEE,2019:17-20

    • [65] Liu X,Chi M M,Zhang Y F,et al.Classifying high resolution remote sensing images by fine-tuned VGG deep networks[C]//2018 IEEE International Geoscience and Remote Sensing Symposium.July 22-27,2018,Valencia,Spain.IEEE,2018:7137-7140

    • [66] Szegedy C,Ioffe S,Vanhoucke V,et al.Inception-v4,inception-ResNet and the impact of residual connections on learning[J].arXiv e-print,2016,arXiv:1602.07261

    • [67] Chollet F.Xception:deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).July 21-26,2017,Honolulu,HI,USA.IEEE,2017:1800-1807

    • [68] Zhu Y,Newsam S.DenseNet for dense flow[C]//2017 IEEE International Conference on Image Processing(ICIP).September 17-20,2017,Beijing,China.IEEE,2017:790-794

    • [69] Sandler M,Howard A,Zhu M L,et al.MobileNetV2:inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:4510-4520

    • [70] Wang M,Zhou S D,Yang Z,et al.CloudA:a ground-based cloud classification method with a convolutional neural network[J].Journal of Atmospheric and Oceanic Technology,2020,37(9):1661-1668

    • [71] Xie W Y,Liu D,Yang M,et al.SegCloud:a novel cloud image segmentation model using a deep convolutional neural network for ground-based all-sky-view camera observation[J].Atmospheric Measurement Techniques,2020,13(4):1953-1961

    • [72] Li M,Liu S,Zhang Z.Dual guided loss for ground-based cloud classification in weather station networks[J].IEEE Access,2019,7:63081-63088

    • [73] Luo Q X,Meng Y,Liu L,et al.Cloud classification of ground-based infrared images combining manifold and texture features[J].Atmospheric Measurement Techniques,2018,11(9):5351-5361

  • 参考文献

    • [1] Tang Y Z,Yang P,Zhou Z M,et al.Improving cloud type classification of ground-based images using region covariance descriptors[J].Atmospheric Measurement Techniques,2021,14(1):737-747

    • [2] Zhang Y C.Calculation of radiative fluxes from the surface to top of atmosphere based on ISCCP and other global data sets:refinements of the radiative transfer model and the input data[J].Journal of Geophysical Research:Atmospheres,2004,109(D19):1121-1125

    • [3] Naud C M,Booth J F,del Genio A D.The relationship between boundary layer stability and cloud cover in the post-cold frontal region[J].Journal of Climate,2016,29(22):8129-8149

    • [4] Cui F,Ju R R,Ding Y Y,et al.Prediction of regional global horizontal irradiance combining ground-based cloud observation and numerical weather prediction[J].Advanced Materials Research,2014,1073/1074/1075/1076:388-394

    • [5] Yuan F,Lee Y H,Meng Y S.Comparison of radio-sounding profiles for cloud attenuation analysis in the tropical region[C]//2014 IEEE Antennas and Propagation Society International Symposium(APSURSI).July 6-11,2014,Memphis,TN,USA.IEEE,2014:259-260

    • [6] Phung V H,Rhee E J.A deep learning approach for classification of cloud image patches on small datasets[J].Journal of Information and Communication Convergence Engineering,2018,16(3):173-178

    • [7] Liu L,Sun X J,Chen F,et al.Cloud classification based on structure features of infrared images[J].Journal of Atmospheric and Oceanic Technology,2011,28(3):410-417

    • [8] Cheng H Y,Yu C C.Block-based cloud classification with statistical features and distribution of local texture features[J].Atmospheric Measurement Techniques,2015,8(3):1173-1182

    • [9] Hu J,Chen Z B,Yang M,et al.A multiscale fusion convolutional neural network for plant leaf recognition[J].IEEE Signal Processing Letters,2018,25(6):853-857

    • [10] Wu X P,Zhan C,Lai Y K,et al.IP102:a large-scale benchmark dataset for insect pest recognition[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).June 15-20,2019,Long Beach,CA,USA.IEEE,2019:8779-8788

    • [11] Singh M,Glennen M.Automated ground-based cloud recognition[J].Pattern Analysis and Applications,2005,8(3):258-271

    • [12] Calbo J,Sabburg J.Feature extraction from whole-sky ground-based images for cloud-type recognition[J].Journal of Atmospheric and Oceanic Technology,2008,25(1):3-14

    • [13] Heinle A,Macke A,Srivastav A.Automatic cloud classification of whole sky images[J].Atmospheric Measurement Techniques,2010,3(3):557-567

    • [14] Kazantzidis A,Tzoumanikas P,Bais A F,et al.Cloud detection and classification with the use of whole-sky ground-based images[J].Atmospheric Research,2012,113:80-88

    • [15] Lu J Y,Wang Y Z,Zhu Y Q,et al.P_Segnet and NP_Segnet:new neural network architectures for cloud recognition of remote sensing images[J].IEEE Access,2019,7:87323-87333

    • [16] Ye L,Cao Z G,Xiao Y,et al.Supervised fine-grained cloud detection and recognition in whole-sky images[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(10):7972-7985

    • [17] Li L Y,Li X Y,Jiang L Y,et al.A review on deep learning techniques for cloud detection methodologies and challenges[J].Signal,Image and Video Processing,2021,15(7):1527-1535

    • [18] Liu S,Zhang Z,Mei X.Ground-based cloud classification using weighted local binary patterns[J].Journal of Applied Remote Sensing,2015,9(1):095062

    • [19] Li Q Y,Zhang Z,Lu W T,et al.From pixels to patches:a cloud classification method based on a bag of micro-structures[J].Atmospheric Measurement Techniques,2016,9(2):753-764

    • [20] Ricciardelli E,Romano F,Cuomo V.Physical and statistical approaches for cloud identification using Meteosat Second Generation-Spinning Enhanced Visible and Infrared Imager Data[J].Remote Sensing of Environment,2008,112(6):2741-2760

    • [21] Taravat A,Del Frate F,Cornaro C,et al.Neural networks and support vector machine algorithms for automatic cloud classification of whole-sky ground-based images[J].IEEE Geoscience and Remote Sensing Letters,2015,12(3):666-670

    • [22] Pfister G,McKenzie R L,Liley J B,et al.Cloud coverage based on all-sky imaging and its impact on surface solar irradiance[J].Journal of Applied Meteorology,2003,42(10):1421-1434

    • [23] Urquhart B,Kurtz B,Dahlin E,et al.Development of a sky imaging system for short-term solar power forecasting[J].Atmospheric Measurement Techniques,2015,8(2):875-890

    • [24] Peng Z Z,Yu D T,Huang D,et al.3D cloud detection and tracking system for solar forecast using multiple sky imagers[J].Solar Energy,2015,118:496-519

    • [25] Chu Y H,Pedro H T C,Nonnenmacher L,et al.A smart image-based cloud detection system for intrahour solar irradiance forecasts[J].Journal of Atmospheric and Oceanic Technology,2014,31(9):1995-2007

    • [26] Mantelli Neto S L,von Wangenheim A,Pereira E B,et al.The use of euclidean geometric distance on RGB color space for the classification of sky and cloud patterns[J].Journal of Atmospheric and Oceanic Technology,2010,27(9):1504-1517

    • [27] Jayadevan V T,Rodriguez J J,Cronin A D.A new contrast-enhancing feature for cloud detection in ground-based sky images[J].Journal of Atmospheric and Oceanic Technology,2015,32(2):209-219

    • [28] Nouri B,Kuhn P,Wilbert S,et al.Cloud height and tracking accuracy of three all sky imager systems for individual clouds[J].Solar Energy,2019,177:213-228

    • [29] Yang J,Min Q,Lu W,et al.An automated cloud detection method based on the green channel of total-sky visible images[J].Atmospheric Measurement Techniques,2015,8(11):4671-4679

    • [30] Zhuo W,Cao Z G,Xiao Y.Cloud classification of ground-based images using texture-structure features[J].Journal of Atmospheric and Oceanic Technology,2014,31(1):79-92

    • [31] Xia M,Lu W T,Yang J,et al.A hybrid method based on extreme learning machine and k-nearest neighbor for cloud classification of ground-based visible cloud image[J].Neurocomputing,2015,160:238-249

    • [32] Kliangsuwan T,Heednacram A.FFT features and hierarchical classification algorithms for cloud images[J].Engineering Applications of Artificial Intelligence,2018,76:40-54

    • [33] Hughes M,Hayes D.Automated detection of cloud and cloud shadow in single-date landsat imagery using neural networks and spatial post-processing[J].Remote Sensing,2014,6(6):4907-4926

    • [34] Howard L.On the modifications of clouds[M].London:Cambridge University Press,2011

    • [35] Buch K A,Sun C H.Cloud classification using whole-sky imager data[J].Ninth Symposium on Meteorological Observations & Instrumentation,1995,16(3):353-358

    • [36] Zhen Z,Wang F,Sun Y J,et al.SVM based cloud classification model using total sky images for PV power forecasting[C]//2015 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference(ISGT).February 18-20,2015,Washington,DC,USA.IEEE,2015:1-5

    • [37] Zhou Z C,Zhang F,Xiao H X,et al.A novel ground-based cloud image segmentation method by using deep transfer learning[J].IEEE Geoscience and Remote Sensing Letters,2618,PP(99):1-5

    • [38] Kristollari V,Karathanassi V.Convolutional neural networks for detecting challenging cases in cloud masking using Sentinel-2 imagery[C]//Eighth International Conference on Remote Sensing and Geoinformation of the Environment(RSCy2020),2020,1152:188-201

    • [39] Song Q Q,Cui Z H,Liu P.An efficient solution for semantic segmentation of three ground-based cloud datasets[J].Earth and Space Science,2020,7(4):e2019EA001040

    • [40] Dev S,Lee Y H,Winkler S.Categorization of cloud image patches using an improved texton-based approach[C]//2015 IEEE International Conference on Image Processing(ICIP).September 27-30,2015,Quebec City,QC,Canada.IEEE,2015:422-426

    • [41] Zhang J L,Liu P,Zhang F,et al.CloudNet:ground-based cloud classification with deep convolutional neural network[J].Geophysical Research Letters,2018,45(16):8665-8672

    • [42] Liu S,Li M,Zhang Z,et al.Multi-evidence and multi-modal fusion network for ground-based cloud recognition[J].Remote Sensing,2020,12(3):464-483

    • [43] Liu S,Duan L L,Zhang Z,et al.Ground-based remote sensing cloud classification via context graph attention network[J].IEEE Transactions on Geoscience and Remote Sensing,2021.DOI:10.1109/TGRS.2021.3063255

    • [44] Fang C Y,Jia K B,Liu P Y,et al.Research on cloud recognition technology based on transfer learning[C]//2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).November 18-21,2019,Lanzhou,China.IEEE,2019:791-796

    • [45] Gan J R,Lu W T,Li Q Y,et al.Cloud type classification of total-sky images using duplex norm-bounded sparse coding[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10(7):3360-3372

    • [46] Zhang J L,Liu P,Zhang F,et al.Ensemble meteorological cloud classification meets Internet of dependable and controllable things[J].IEEE Internet of Things Journal,2021,8(5):3323-3330

    • [47] Rahman M M,Shakeri M,Tiong S K,et al.Prospective methodologies in hybrid renewable energy systems for energy prediction using artificial neural networks[J].Sustainability,2021,13(4):2393-2420

    • [48] Frniak M,Markovic M,Kamencay P,et al.Vehicle classification based on FBG sensor arrays using neural networks[J].Sensors,2020,20(16):4472

    • [49] Ahmad J,Farman H,Jan Z.Deep learning methods and applications[M]//Deep learning:convergence to big data analytics.Singapore:Springer Singapore,2018:31-42

    • [50] Liu Z H,Zhou S D,Wang M,et al.Ground-based visible-light cloud image classification based on a convolutional neural network[C]//2019 4th International Conference on Information Systems and Computer Networks(ISCON).November 21-22,2019,Mathura,India.IEEE,2019:108-112

    • [51] Shi C Z,Wang C H,Wang Y,et al.Deep convolutional activations-based features for ground-based cloud classification[J].IEEE Geoscience and Remote Sensing Letters,2017,14(6):816-820

    • [52] Ye L,Cao Z G,Xiao Y.DeepCloud:ground-based cloud image categorization using deep convolutional features[J].IEEE Transactions on Geoscience and Remote Sensing,2017,55(10):5729-5740

    • [53] Liu S,Li M,Zhang Z,et al.Ground-based cloud classification using task-based graph convolutional network[J].Geophysical Research Letters,2020,47(5):e2020GL087338

    • [54] Phung V H,Rhee E J.A high-accuracy model average ensemble of convolutional neural networks for classification of cloud image patches on small datasets[J].Applied Sciences,2019,9(21):4500

    • [55] Liu S,Li M.Multimodal GAN for energy efficiency and cloud classification in Internet of Things[J].IEEE Internet of Things Journal,2019,6(4):6034-6041

    • [56] Zhuang F Z,Qi Z Y,Duan K Y,et al.A comprehensive survey on transfer learning[J].Proceedings of the IEEE,2021,109(1):43-76

    • [57] Fang C Y,Jia K B,Liu P Y,et al.Research on cloud recognition technology based on transfer learning[C]//2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA ASC).November 18-21,2019,Lanzhou,China.IEEE,2019:791-796

    • [58] Manzo M,Pellino S.Voting in transfer learning system for ground-based cloud classification[J].Machine Learning and Knowledge Extraction,2021,3(3):542-553

    • [59] Li G S.A novel computer-aided cloud type classification method based on convolutional neural network with squeeze-and-excitation[J].Journal of Physics:Conference Series,2021,1802(3):032051

    • [60] Deng J,Berg A,Satheesh S,et al.ImageNet large scale visual recognition challenge 2012(ILSVRC2012)[EB/OL].[2021-06-06].https://image-net.org/challenges/LSVRC/2012/

    • [61] Shin H C,Roth H R,Gao M C,et al.Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging,2016,35(5):1285-1298

    • [62] Ker J,Wang L P,Rao J,et al.Deep learning applications in medical image analysis[J].IEEE Access,2018,6:9375-9389

    • [63] Mikołajczyk A,Grochowski M.Data augmentation for improving deep learning in image classification problem[C]//2018 International Interdisciplinary PhD Workshop(IIPhDW).IEEE,2018:117-122

    • [64] Budhiman A,Suyanto S,Arifianto A.Melanoma cancer classification using ResNet with data augmentation[C]//2019 International Seminar on Research of Information Technology and Intelligent Systems(ISRITI).December 5-6,2019,Yogyakarta,Indonesia.IEEE,2019:17-20

    • [65] Liu X,Chi M M,Zhang Y F,et al.Classifying high resolution remote sensing images by fine-tuned VGG deep networks[C]//2018 IEEE International Geoscience and Remote Sensing Symposium.July 22-27,2018,Valencia,Spain.IEEE,2018:7137-7140

    • [66] Szegedy C,Ioffe S,Vanhoucke V,et al.Inception-v4,inception-ResNet and the impact of residual connections on learning[J].arXiv e-print,2016,arXiv:1602.07261

    • [67] Chollet F.Xception:deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).July 21-26,2017,Honolulu,HI,USA.IEEE,2017:1800-1807

    • [68] Zhu Y,Newsam S.DenseNet for dense flow[C]//2017 IEEE International Conference on Image Processing(ICIP).September 17-20,2017,Beijing,China.IEEE,2017:790-794

    • [69] Sandler M,Howard A,Zhu M L,et al.MobileNetV2:inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:4510-4520

    • [70] Wang M,Zhou S D,Yang Z,et al.CloudA:a ground-based cloud classification method with a convolutional neural network[J].Journal of Atmospheric and Oceanic Technology,2020,37(9):1661-1668

    • [71] Xie W Y,Liu D,Yang M,et al.SegCloud:a novel cloud image segmentation model using a deep convolutional neural network for ground-based all-sky-view camera observation[J].Atmospheric Measurement Techniques,2020,13(4):1953-1961

    • [72] Li M,Liu S,Zhang Z.Dual guided loss for ground-based cloud classification in weather station networks[J].IEEE Access,2019,7:63081-63088

    • [73] Luo Q X,Meng Y,Liu L,et al.Cloud classification of ground-based infrared images combining manifold and texture features[J].Atmospheric Measurement Techniques,2018,11(9):5351-5361

  • Address:No. 219, Ningliu Road, Nanjing, Jiangsu Province

    Postcode:210044

    Phone:025-58731025