面向车路云协同的时序数据湖仓一体架构与查询优化
Lakehouse Architecture and Query Optimization for Time-Series Data in Vehicle-Road-Cloud Collaboration
车路云协同数据同时包含车辆状态、路侧感知和交通事件,既要支持实时写入,也要保留轨迹回溯所需的历史记录.本 文在Apache Iceberg上组织这类时序数据:按时间和设备类型分区,在分区内按(vehicle_id,event_ts)聚类,再用布隆过滤 器、Min-Max统计、小文件合并和动态裁剪限制扫描范围.四类查询实验中,方案D的延迟较方案A下降75%以上,文件 数和I/O也明显减少.
The vehicle road cloud collaborative data includes vehicle status, roadside perception, and traffic events, which need to support real-time writing and retain historical records required for trajectory tracing. This article organizes this type of time-series data on Apache Iceberg: partitioning by time and device type, clustering within partitions by(vehicle_i, event_ts), and then using Bloom filters, Min Max statistics, small file merging, and dynamic cropping to limit the scanning range. In the four types of query experiments, the latency of scheme D decreased by more than 75% compared to scheme A, and the number of files and I/O were also significantly reduced.
| [1] |
王小丽, 陈华辉. ZFT索引: 基于分段线性回归的学习型多 维索引[J]. 计算机应用与软件, 2024, 41(10): 24-31. |
| [2] |
|
| [3] |
ARMBRUST M, DAS T, SUN L, |
| [4] |
|
| [5] |
谢星星, 张翠平, 田小燕, |
面向现代城市交通及城市治理的全生命周期 一体化智慧场景技术研发及示范应用(BG2025-0109-SZZY-FYH)
/
| 〈 |
|
〉 |