复杂养殖环境下基于骨架特征的牛体行为识别

Skeleton-based Cattle Behavior Recognition in Complex Farming Environments

摘要

牛的行为能够直观地反映其生理和健康状态，在复杂养殖环境中准确识别牛体行为对实现智能养殖至关重要。现有牛体行为识别方法受噪声敏感、牛体运动特性影响，导致在实际养殖环境中牛体行为识别的准确性不足。因此，本文提出一种适用于复杂养殖环境的基于骨架特征的牛体行为识别方法，该方法通过提升行为特征对复杂背景噪声及遮挡的抗干扰能力，提高其在实际养殖环境下的牛体行为识别效果。首先，本文基于HRNet对低分辨率特征利用不足，设计了多尺度特征融合（Multi-scale Feature Fusion，MSFF）模块，增强了HRNet 的多尺度表征能力，提升了复杂背景下牛体骨架特征提取的准确性；其次，考虑到牛体行为较为迟缓的特点，构建三维热图体（3D heatmap volume）作为输入，减少了噪声干扰，并保留了全局动态特征；最后，针对牛体行为类间相似性，在PoseConv3D行为识别网络的残差层引入帧间运动激励（Inter-frame Motion Excitation，IME）模块，增强不同行为的判别特征，提升类间相似牛体行为的区分能力。实验结果表明，所设计方法在骨架特征提取任务和行为识别任务中分别取得 92.3%（AP⁵⁰）和 93.9%（Top1_acc）的效果，相比原有方法分别提升了0.8%和4.7%，整体提升6.4%。本文所设计的方法能在遮挡等复杂养殖环境下更准确地提取牛体骨架特征，对行为的判别性特征提取能力更强，为智能养殖中的牛体行为识别提供了有效的解决方案。

Abstract

The behavior of cattle can directly reflect their physiological and health status,and accurately identifying cattle behavior in complex farming environments is crucial for achieving smart livestock farming. Existing cattle behavior recognition methods are susceptible to noise and affected by cattle motion characteristics, resulting in insufficient recognition accuracy in practical farming environments. To address these issues, this paper proposes a skeleton-based cattle behavior recognition approach suitable for complex farming environments, which enhances the robustness of behavioral features against complex background noise and occlusion, thereby improving recognition performance under practical farming conditions. First, to address the insufficient utilization of low-resolution features in HRNet, this paper designs a Multi-scale Feature Fusion (MSFF) module, which enhances the multi-scale representation capability of HRNet and improves the accuracy of cattle skeleton feature extraction under complex backgrounds. Second, considering the relatively slow movement characteristics of cattle, a three-dimensional heatmap volume (3D heatmap volume) is constructed as input to reduce noise interference while preserving global dynamic features. Finally, aiming at the inter-class similarity of cattle behaviors, an Inter-frame Motion Excitation (IME) module is introduced into the residual layers of the PoseConv3D behavior recognition network, enhancing the discriminative features of different behaviors and improving the distinction capability of inter-class similar cattle behaviors. The experimental results show that the designed method achieves a performance of 92.3% (AP⁵⁰) and 93.9% (Top1_acc) in skeleton feature extraction and behavior recognition tasks, respectively, representing improvements of 0.8% and 4.7% compared to the original method, with an overall enhancement of 6.4%. The method proposed in this paper can more accurately extract skeleton features of cattle in complex farming environments such as occlusion, and exhibits stronger discriminative capability for behavior feature extraction, providing an effective solution for cattle behavior recognition in intelligent livestock farming.

关键词

骨架特征 / HRNet / 行为识别 / PoseConv3D / 牛体

Key words

Skeleton feature / HRNet / action recognition / PoseConv3D / cattle

参考文献

[1] 许尚忠. 中国牛产业发展概况及展望[J]. 畜牧产业, 2021(07): 35-40.

[2] 汤文洁, 苏惟真, 姜 冰. 中国牛产业国际贸易回顾[J]. 乳品与人类, 2023(03): 3-10.

[3] Chen C, Zhu W, Norton T. Behaviour recognition of pigs and cattle: Journey from computer vision to deep learning[J]. Computers and Electronics in Agriculture, 2021, 187: 106255.

[4] Zhang L, Guo W, Lv C, et al. Advancements in artificial intelligence technology for improving animal welfare: Current applications and research progress[J]. Animal Research and One Health, 2024, 2(1): 93-109.

[5] 张宏鸣,孙 扬, 赵春平, 等. 反刍家畜典型行为监测与生理状况识别方法研究综述[J]. 农业机械学报, 2023, 54(03): 1-21.

[6] Shu H, Bindelle J, Guo L, et al. Determining the onset of heat stress in a dairy herd based on automated behaviour recognition[J]. Biosystems Engineering, 2023, 226: 238-251.

[7] Zhang Y, Zhang Y, Gao M, et al. Multimodal behavior recognition for dairy cow digital twin construction under incomplete modalities: A modality mapping completion network approach[J]. Artificial Intelligence in Agriculture, 2025, 15(3): 459-469.

[8] 程国栋, 吴建寨, 邢丽玮, 等. 基于IMU的细粒度奶牛行为判别[J]. 中国农业大学学报, 2022, 27(04): 179-186.

[9] Wang L, Arablouei R, Alvarenga F A P, et al. Classifying animal behavior from accelerometry data via recurrent neural networks[J]. Computers and Electronics in Agriculture, 2023, 206: 107647.

[10] Camargo V A, Pajor E A, Bayat S, et al. Predicting main behaviors of beef bulls from accelerometer data: A machine learning framework[J]. Smart Agricultural Technology, 2024, 9: 100683.

[11] Liang H T, Hsu S W, Hsu J T, et al. An IMU-based machine learning approach for daily bBehavior pattern recognition in dairy cows[J]. Smart Agricultural Technology, 2024, 9: 100539.

[12] Nasir M F, Fuentes A, Han S, et al. Multi-camera fusion and bird-eye view location mapping for deep learning-based cattle behavior monitoring[J]. Artificial Intelligence in Agriculture, 2025, 15: 724-743.

[13] Tassinari P, Bovo M, Benni S, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn[J]. Computers and Electronics in Agriculture, 2021, 182: 106030.

[14] 李恩泽, 王克俭, 司永胜, 等. 基于改进ConvNeXt 的奶牛行为识别方法[J]. 农业机械学报, 2024, 55(05): 282-289+ 404.

[15] 曹天一, 李宝山, 王月明, 等. 基于起卧频率异常的牛产犊自动预警[J]. 农业工程学报, 2022, 38(S1): 201-208.

[16] 付辰伏, 任力生, 王 芳. 基于改进YOLO v8的牛只行为识别与跟踪方法[J]. 农业机械学报, 2024, 55(05): 290-301.

[17] Jiang B, Yin X, Song H, et al. Single-stream long-term optical flow convolution nNetwork for action recognition of lameness dairy cow[J]. Computers and Electronics in Agriculture, 2020, 175: 105536.

[18] Chi H G, Ha M H, Chi S, et al. InfoGCN: Representation learning for human skeleton-based action recognition[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA: IEEE, 2022: 20154-20164.

[19] Cai J, Jiang N, Han X, et al. JOLO-GCN: Mining joint-centered light-weight information for skeleton-based action recognition[C]. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI, USA: IEEE, 2021: 2734-2743.

[20] Chen Y, Zhang Z, Yuan C, et al. Channel-wise topology refinement graph convolution for skeleton-based action recognition[C]. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada: IEEE, 2021: 13339-13348.

[21] Gupta P, Thatipelli A, Aggarwal A, et al. Quo vadis, skeleton action recognition?[J]. International Journal of Computer Vision, 2021, 129(7): 2097-2112.

[22] Yan S, Xiong Y, Lin D. Spatial temporal graph convolutional networks for skeleton-based action recognition[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1).

[23] Zhao Y, Feng L, Tang J, et al. Automatically recognizing four-legged animal behaviors to enhance welfare using spatial temporal graph convolutional networks[J]. Applied Animal Behaviour Science, 2022, 249: 105594.

[24] Zhou Y, Yan X, Cheng Z Q, et al. BlockGCN: Redefine topology awareness for skeleton-based action recognition[C]. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2024: 2049-2058.

[25] Li Y, Shu H, Bindelle J, et al. Classification and analysis of multiple cattle unitary behaviors and movements based on machine learning methods[J]. Animals, 2022, 12(9): 1060.

[26] Liu H, Reibman A R, Boerman J P. Video analytic system for detecting cow structure[J]. Computers and Electronics in Agriculture, 2020, 178: 105761.

[27] 张宏鸣, 李永恒, 周利香, 等. 基于改进YOLO v3的肉牛多目标骨架提取方法[J]. 农业机械学报, 2022, 53(03): 285-293.

[28] Hua Z, Wang Z, Xu X, et al. An effective PoseC3D model for typical action recognition of dairy cows based on skeleton features[J]. Computers and Electronics in Agriculture, 2023, 212: 108152.

[29] Yang Y, Yang J, Xu Y, et al. APT-36K: A large-scale benchmark for animal pose estimation and tracking[C].Koyejo S, Mohamed S, Agarwal A, et al (Eds.). Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates, Inc., 2022, 35: 17301-17313.

[30] Sun K, Xiao B, Liu D, et al. Deep high-resolution representation learning for human pose estimation[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA: IEEE, 2019: 5686-5696.

[31] Dai Y, Gieseke F, Oehmcke S, et al. Attentional feature fusion[C]. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI, USA: IEEE, 2021: 3559-3568.

[32] Duan H, Zhao Y, Chen K, et al. Revisiting skeleton-based action recognition[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA: IEEE, 2022: 2959-2968.

[33] Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation[C]. Leibe B, Matas J, Sebe N, Welling M (Eds.. Computer Vision-ECCV 2016. Lecture Notes in Computer Science, Vol. 9912. Cham: Springer International Publishing, 2016: 483-499.

[34] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016: 770-778.

[35] Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 4510-4520.

[36] Maji D, Nagori S, Mathew M, et al. YOLO-Pose: Enhancing YOLO for multi person pose estimation using object keypoint similarity loss[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans, LA, USA: IEEE, 2022: 2636-2645.

基金资助

国家自然科学基金项目(62202281)