基于多头傅里叶KAN的大模型增强推荐技术研究

Large Language Model-enhanced Recommendation via Multi-head Fourier KAN

摘要

将大语言模型融入个性化推荐算法的各个阶段可以弥补传统推荐算法不足,但当前研究仍存在以下两个问题:1)将注意力集中于文本特征的表示,忽视了高维特征空间中的非线性关系,难以有效捕捉特征间的复杂交互,导致推荐精度受限;2)对于用户兴趣稀疏性问题,现有方法聚焦于用户本身属性或相似度匹配来解决用户历史数据不足,难以精准刻画用户的潜在兴趣并且无法适应用户的动态兴趣变化;3)盲目扩充用户特征而引入噪声,通过对用户特征扩充可以有效缓解用户稀疏性,但是对用户特征扩充超过指定限度会在用户表征中引入噪声,从而影响整体推荐效果.基于此,提出了MHFK-LLMRec模型.首先利用大语言模型对文本特征进行语义增强和深度编码,从而获得高维度的项目特征表示.其次,模型引入多头傅里叶特征子空间进行多层次非线性建模,精准捕捉特征间的复杂交互关系.此外,模型整合了大模型推理机制与置信因子主导的余弦相似度筛选策略,有效补充用户历史行为的建模,成功解决了用户兴趣稀疏性的问题.扩展性实验结果验证了所提模型有效性.

Abstract

The integration of large language models (LLMs) into personalized recommendation systems has shown great potential in addressing the limitations of traditional algorithms.However,existing studies still face three critical challenges:1) Most approaches overemphasize textual feature representation while overlooking nonlinear interactions within high-dimensional feature spaces,which hampers the model′s ability to capture complex cross-feature relationships and limits recommendation accuracy;2) To mitigate user interest sparsity,current methods typically rely on user profile attributes or similarity-based heuristics,which often fail to effectively infer latent user interests and adapt to dynamic preference shifts;3) Indiscriminate expansion of user features may introduce noise although feature augmentation can alleviate sparsity,excessive augmentation beyond a certain threshold degrades user representation quality and negatively impacts overall recommendation performance.To address these issues,we propose MHFK-LLMRec,a novel recommendation framework.The model first leverages LLMs for semantic enhancement and deep encoding of textual content,yielding high-dimensional,semantically rich item representations.Then,a multi-head Fourier feature subspace module is introduced to enable hierarchical nonlinear modeling,allowing the model to effectively capture intricate feature interactions.Furthermore,MHFK-LLMRec integrates a reasoning-inspired mechanism with a confidence-aware cosine similarity filtering strategy to enrich user behavior modeling and robustly mitigate interest sparsity.Extensive experimental results on benchmark datasets validate the effectiveness and generalizability of the proposed model.

关键词

个性化推荐 / 大语言模型 / 多头傅里叶特征子空间 / 非线性建模 / 语义增强

Key words

personalized recommendation / large language models / multi-head Fourier feature subspace / nonlinear modeling / semantic enhancement

[1] Lops P,De Gemmis M,Semeraro G.Content-based recommender systems:state of the art and trends[M].Berlin:Springer,2011.
[2] Liu Z M,Wang Y X,Vaidya S,et al.KAN:kolmogorov-arnold networks[C]//Proceedings of the 13th International Conference on Learning Representations,2025:66342-66388.
[3] Xu J F,Chen Z Y,Li J Z,et al.FourierKAN-GCF:fourier kolmogorov-arnold network-an effective and efficient feature transformation for graph collaborative filtering[J].arXiv preprint arXiv:2406.01034,2024.
[4] Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems,2017:6000-6010.
[5] Liu Q,Chen N,Sakai Tetsuya,et al.ONCE:boosting content-based recommendation with both open-and closed-source large language models[C]//Proceedings of the 17th ACM International Conference on Web Search and Data Mining,2024:452-461.
[6] Li J,Zhu J M,Bi Q W,et al.MINER:multi-interest matching network for news recommendation[C]//Findings of the Association for Computational Linguistics,2022:343-352.
[7] Wu C H,Wu F Z,Qi T,et al.Empowering news recommendation with pre-trained language models[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval,2021:1652-1656.
[8] Wu C H,Wu F Z,Qi T,et al.Fastformer:additive attention can be all you need[J].arXiv preprint arXiv:2108.09084,2021.
[9] Devlin J,Chang M W,Lee K,et al.Bert:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019:4171-4186.
[10] Lin J H,Dai X Y,Xi Y J,et al.How can recommender systems benefit from large language models:a survey[J].ACM Transactions on Information Systems,2025,43(2):1-47.
[11] Borisov V,Seβler K,Leemann T,et al.Language models are realistic tabular data generators[C]//Proceedings of the 11th International Conference on Learning Representations,2023:23614-23631.
[12] Zhang Q,Li J J,Jia Q L,et al.UNBERT:user-news matching BERT for news recommendation[C]//Proceedings of the 30th International Joint Conference on Artificial Intelligence,2021:3356-3362.
[13] Li J M,Zhang W T,Wang T,et al.GPT4Rec:a generative framework for personalized recommendation and user interests interpretation[J].arXiv preprint arXiv:2304.03879,2023.
[14] Liu G,Yang J,Wu L.Ptab:using the pre-trained language model for modeling tabular data[J].arXiv preprint arXiv:2209.08060,2022.
[15] Wei J,Tay Y,Bommasani R,et al.Emergent abilities of large language models[J].Transactions on Machine Learning Research,2022:1-37,doi:10.48550/arxiv.2206.07682.
[16] Yang Y H,Huang C,Xia L H,et al.Knowledge graph contrastive learning for recommendation[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval,2022:1434-1443.
[17] Guo H F,Tang R M,Ye Y M,et al.DeepFM:a factorization-machine based neural network for CTR prediction[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence(IJCAI),2017:1725-1731.
[18] Pennington J,Socher R,Manning C D.Glove:global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP),2014:1532-1543.
[19] Wu C,Wu F,An Mingxiao,et al.Neural news recommendation with attentive multi-view learning[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence,2019:3866-3872.
[20] Wu C,Wu F Z,Ge S Y,et al.Neural news recommendation with multi-head self-attention[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019:6389-6394.
[21] Zhang C Y,Sun Y F,Wu M H,et al.EmbSum:leveraging the summarization capabilities of large language models for content-based recommendations[C]//Proceedings of the 18th ACM Conference on Recommender Systems(RecSys),2024:1010-1015.
[22] Qiao S T,Gao C,Li Y,et al.LLM-assisted explicit and implicit multi-interest learning framework for sequential recommendation[J].arXiv preprint arXiv:2411.09410,2024.
[23] Sun Z X,Si Z H,Zang X X,et al.Large language models enhanced collaborative filtering[C]//Proceedings of the 33rd ACM International Conference on Information and Knowledge Management,2024:2178-2188.
[24] Gao S,Fang J B,Tu Q,et al.Generative news recommendation[C]//Proceedings of the ACM on Web Conference,2024:3444-3453.
[25] Lin X Y,Wang W J,Li Y Q et al.Data-efficient fine-tuning for LLM-based recommendation[C]//Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,2024:365-374.
[26] Yin B,Xie J J,Qin Y,et al.Heterogeneous knowledge fusion:a novel approach for personalized recommendation via llm[C]//Proceedings of the 17th ACM Conference on Recommender Systems,2023:599-601.
[27] Li L,Zhang Y F,Chen L.Prompt distillation for efficient llm-based recommendation[C]//Proceedings of the 32nd ACM International Conference on Information and Knowledge Management,2023:1348-1357.
[28] Wu F Z,Qiao Y,Chen J H,et al.Mind:a large-scale dataset for news recommendation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,2020:3597-3606.
[29] Wan M T,McAuley J.Item recommendation on monotonic behavior chains[C]//Proceedings of the 12th ACM Conference on Recommender Systems,2018:86-94.
[30] Touvron H,Lavril T,Izacard G,et al.LLaMA:open and efficient foundation language models.arXiv[J].arXiv preprint arXiv:2302.13971,2023.
[31] Fawcett T.An introduction to ROC analysis[J].Pattern Recognition Letters,2006,27(8):861-874.
[32] Voorhees E M.The trec-8 question answering track report[C]//Proceedings of the 8th Text Retrieval Conference(TREC),1999:77-82.
[33] Järvelin K,Kekäläinen J.Cumulated gain-based evaluation of IR techniques[J].ACM Transactions on Information Systems,2002,20(4):422-446.
[34] Liu Q,Fan L,Wu X M.Legommenders:a comprehensive content-based recommendation library with llm support[C]//Companion Proceedings of the ACM on Web Conference,2025:753-756.

基金资助

国家自然科学基金项目(62471122)资助.