基于网络媒体的龙卷风灾情数据挖掘与智能识别方法探究

Research on data mining and intelligent identification methods for tornado disasters based on network media

摘要

为了充分挖掘龙卷风灾害的网络数据并研究其有效的智能识别方法,本文利用爬虫技术不间断搜索网络媒体中龙卷风相关信息,然后进行数据清洗,再运用词频-逆向文本频率(term frequency-inverse document frequency,TF-IDF)、文本排序算法(TextRank)、朴素贝叶斯分类方法(Naive Bayes)等算法对获取的文本信息进行搜索甄别。利用3次典型龙卷风个例的雷达探测等天气资料验证,龙卷风信息搜索甄别算法能够实现自动搜集、准确甄别与推送提醒,特别是江苏宿迁龙卷风个例,在其发生2h内即搜索到龙卷风发生的信息并推送提醒。搜索甄别算法针对强龙卷风个例识别的信息量大于弱龙卷风个例,强龙卷风个例信息识别的准确率达到94.4%,而弱龙卷个例为77.5%,源自于新浪微博(简称微博)的龙卷风信息量及时效性均高于百度网页(简称百度)。2022-2024年的对比检验表明,甄别龙卷风信息的准确率、召回率及F₁值分别达0.92、1.00、0.96,且自动搜索时间小于0.5h。该方法提取龙卷风信息时效性高,可以作为龙卷风监测预警的有效补充手段。

Abstract

To fully explore the network media data of tornado disasters and seek the effective intelligent identification methods,this paper utilizes crawling technology to continuously search the tornado disasters information on the network media sites. After data cleaning,algorithms such as term frequency-inverse document frequency(TF-IDF),TextRank,and Naive Bayes are applied to search and identify the obtained text information. Comparing with the weather data such as radar observations of three typical tornado cases,the tornado information search and identification algorithm has been verified to realize automatic collection,accurate identification and information push,especially for the tornadic case in Suqian,Jiangsu Province,information about this tornado was searched and pushed out within 2h of its occurrence. The search and identification algorithm can identify more information for strong tornado cases than for weak cases,achieving an accuracy rate of 94.4% for strong tornado cases and 77.5% for weak cases. The amount and timeliness of tornado information from Sina Weibo(Weibo) are higher than those from Baidu Web(Baidu). Comparative test from 2022 to 2024 show that the accuracy rate,recall rate and F ₁ value for identifying tornado information reached 0.92,1.00 and 0.96 respectively,and the automatic search time was less than half an hour. This method has high timeliness in extracting tornado information and can be used as an effective supplementary means for tornado monitoring and warning.

关键词

龙卷风信息 / 网络媒体 / 网络爬虫 / 自然语言处理 / 朴素贝叶斯

Key words

tornado information / network media / web trawling / natural language processing / Naive Bayes

参考文献

[1] 俞小鼎, 赵娟, 范雯杰. 中国龙卷的时空分布与关键环境参数特征[J]. 热带气象学报, 2021, 37(5/6): 681-692.

[2] YU Xiaoding, ZHAO Juan, FAN Wenjie. Tornadoes in China: Spatiotemporal distribution and environmental characteristics[J]. Journal of Tropical Meteorology, 2021, 37(5/6): 681-692. (in Chinese)

[3] XUE M, ZHAO K, WANG M, et al. Recent significant tornadoes in China[J]. Advances in Atmospheric Sciences, 2016, 33(11): 1209-1217.

[4] MENG Z Y, BAI L Q, ZHANG M R, et al. The deadliest tornado(EF4) in the past 40 years in China[J]. Weather and Forecasting, 2018, 33(3): 693-713.

[5] 刘宁微, 马雁军, 刘晓梅, 等 . 辽宁省“05-6”龙卷风过程的诊断与数值模拟[J]. 自然灾害学报, 2007, 16(5): 84-90.

[6] LIU Ningwei, MA Yanjun, LIU Xiaomei, et al. Diagnosis and numerical simulation of a tornado course in Liaoning Province in June 2005[J]. Journal of Natural Disasters, 2007, 16(5): 84-90. (in Chinese)

[7] 方丽娟, 姬菊枝, 陶国辉, 等 . 一次超级单体风暴中龙卷的天气过程分析及龙卷强度判定[J]. 自然灾害学报, 2009, 18(2): 167-172.

[8] FANG Lijuan, JI Juzhi, TAO Guohui, et al. Weather analysis and grade determination of a tornado occurring in a single super-storm[J]. Journal of Natural Disasters, 2009, 18(2): 167-172. (in Chinese)

[9] 俞小鼎, 郑永光. 中国当代强对流天气研究与业务进展[J]. 气象学报, 2020, 78(3): 391-418.

[10] YU Xiaoding, ZHENG Yongguang. Advances in severe convective weather research and operational service in China[J]. Acta Meteorologica Sinica, 2020, 78(3): 391-418. (in Chinese)

[11] 郑永光, 刘菲凡, 张恒进. 中国龙卷研究进展[J]. 气象, 2021, 47(11): 1319-1335.

[12] ZHENG Yongguang, LIU Feifan, ZHANG Hengjin. Advances in tornado research in China[J]. Meteorological Monthly, 2021, 47(11): 1319-1335. (in Chinese)

[13] 褚红星, 李兆明, 何珊, 等 . 11次龙卷个例的网络信息与灾情调查结果对比分析[J]. 气象科技进展, 2023, 13(3): 63-65.

[14] CHU Hongxing, LI Zhaoming, HE Shan, et al. Comparative analysis of Internet information and disaster survey results for 11 tornadoes[J]. Advances in Meteorological Science and Technology, 2023, 13(3): 63-65. (in Chinese)

[15] 李涛. 面向文本数据的分布式主题爬虫系统设计与实现[D]. 成都: 电子科技大学, 2023.

[16] LI Tao. Design and implementation of distributed focused crawler system for text data[D]. Chengdu: University of Electronic Science and Technology of China, 2023. (in Chinese)

[17] WANG L M, LI X L, CAO C H, et al. Combining decision tree and naive Bayes for classification[J]. Knowledge-Based Systems, 2006, 19(7): 511-515.

[18] LI Y M, WEI B G, LIU Y H, et al. Incorporating knowledge into neural network for text representation[J]. Expert Systems with Applications, 2018, 96: 103-114.

[19] 程盼, 徐弼军. 基于word2vec和logistic回归的中文专利文本分类研究[J]. 浙江科技学院学报, 2021, 33(6): 454-460.

[20] CHENG Pan, XU Bijun. Research on Chinese patent text classification based on word2vec and logistic regression model[J]. Journal of Zhejiang University of Science and Technology, 2021, 33(6): 454-460. (in Chinese)

[21] 辛梓铭, 王芳. 基于改进朴素贝叶斯算法的文本分类研究[J]. 燕山大学学报, 2023, 47(1): 82-88.

[22] XIN Ziming, WANG Fang. Research on text classification based on improved naive Bayes algorithm[J]. Journal of Yanshan University, 2023, 47(1): 82-88. (in Chinese)

[23] SUPRIYONO, WIBAWA A P,SUYONO, et al. Advancements in natural language processing: Implications,challenges,and future directions[J]. Telematics and Informatics Reports, 2024, 16, 100173.

[24] LIU W J, SUN Y Y, YU B, et al. Automatic text summarization method based on improved TextRank algorithm and K-means clustering[J]. Knowledge-Based Systems, 2024, 287: 111447.

[25] BONTHU S, MUDDAM S S G B, MUDUNURI K V, et al. TextRank-based keyword extraction for constructing a domain-specific dictionary[C]// Cognitive Computing and Cyber Physical Systems. Cham: Springer, 2024: 340-349.

[26] DOMINGOS P, PAZZANI M. On the optimality of the simple Bayesian classifier under zero-one loss[J]. Machine Learning, 1997, 29(2): 103-130.

[27] SALTON G, BUCKLEY C. Term-weighting approaches in automatic text retrieval[J]. Information Processing & Management, 1988, 24(5): 513-523.

[28] FIELD D A. Laplacian smoothing and Delaunay triangulations[J]. Communications in Applied Numerical Methods, 1988, 4(6): 709-712.

[29] 蔡康龙, 麦永耀, 张修远, 等 . 2023年6月1日辽宁省多龙卷风灾害调查及特征分析[J]. 气象与环境学报, 2024, 40(4): 27-36.

[30] CAI Kanglong, MAI Yongyao, ZHANG Xiuyuan, et al. Investigation and characteristic analysis of the multiple tornado disasters in Liaoning Province on June 1,2023[J]. Journal of Meteorology and Environment, 2024, 40(4): 27-36. (in Chinese)

基金资助

广东省气象局面上项目(GRMC2023M19)

中国气象局雷达气象重点开放实验室开放课题(2025LRM-B07)

广东省基础与应用基础研究基金项目(2025A1515510040)

广东省基础与应用基础研究基金项目(2024A1515510022)

广东省气象局科技创新团队项目(GRMCTD202509-QN02)