• 《工程索引》(EI)刊源期刊
  • 中文核心期刊(综合性理工农医类)
  • 中国科技论文统计源期刊
  • 中国科学引文数据库来源期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度神经网络的点击率预测模型

刘弘历 武森 魏桂英 李新 高晓楠

刘弘历, 武森, 魏桂英, 李新, 高晓楠. 基于深度神经网络的点击率预测模型[J]. 工程科学学报. doi: 10.13374/j.issn2095-9389.2021.03.23.002
引用本文: 刘弘历, 武森, 魏桂英, 李新, 高晓楠. 基于深度神经网络的点击率预测模型[J]. 工程科学学报. doi: 10.13374/j.issn2095-9389.2021.03.23.002
LIU Hong-li, WU Sen, WEI Gui-ying, LI Xin, GAO Xiao-nan. Click-through rate prediction model based on a deep neural network[J]. Chinese Journal of Engineering. doi: 10.13374/j.issn2095-9389.2021.03.23.002
Citation: LIU Hong-li, WU Sen, WEI Gui-ying, LI Xin, GAO Xiao-nan. Click-through rate prediction model based on a deep neural network[J]. Chinese Journal of Engineering. doi: 10.13374/j.issn2095-9389.2021.03.23.002

基于深度神经网络的点击率预测模型

doi: 10.13374/j.issn2095-9389.2021.03.23.002
基金项目: 国家自然科学基金资助项目(71971025)
详细信息
    通讯作者:

    E-mail: weigy@manage.ustb.edu.cn

  • 中图分类号: TP183

Click-through rate prediction model based on a deep neural network

More Information
  • 摘要: 针对现有深度神经网络点击率预测模型在对用户偏好建模时,难以有效且高效地处理用户行为序列的问题,提出长短期兴趣网络(Long and short term interests network, LSTIN)模型,充分利用用户历史记录上下文信息和顺序信息,提升点击率预测精准性和训练效率。使用基于注意力机制的Transformer和激活单元结构完成用户长、短期兴趣建模,对用户短期兴趣进一步使用循环神经网络(Recurrent neural network, RNN)、卷积神经网络(Convolutional neural networks, CNN)进行处理,最后使用全连接神经网络进行预测。在亚马逊公开数据集上开展实验,将提出的模型与基于分解机的神经网络(DeepFM)、深度兴趣网络(Deep interest network, DIN)等点击率预测模型对比,结果表明提出的模型实现了考虑上下文信息和顺序信息的用户历史记录建模,接受者操作特征曲线下面积(Area under curve, AUC)指标为85.831%,相比于基础模型(BaseModel)提升1.154%,相比于DIN提升0.476%。且因区分用户长、短期兴趣,模型能够在提升预测精准性的同时保障训练效率。

     

  • 图  1  LSTIN模型结构

    Figure  1.  Structure of an LSTIN

    图  2  编码器结构

    Figure  2.  Structure of an encoder

    图  3  激活单元注意力机制原理示意图

    Figure  3.  Attention mechanism in an activation unit

    图  4  算法AUC对比

    Figure  4.  AUC of various algorithms

    图  5  部分模型训练时间对比

    Figure  5.  Training time comparisons

    表  1  数据集统计信息

    Table  1.   Statistical information of a dataset

    Data setNumber of usersNumber of categoriesNumber of commoditiesNumber of samples
    Amazon
    (Electronics)
    192403630018011689188
    下载: 导出CSV

    表  2  算法性能对比

    Table  2.   Algorithm performance

    CategoryNameAUC/%Best AUC/%RP(BaseModel)/%RP DIN)/%
    Existing modelsBaseModel84.85284.9460−0.670
    DeepFM85.01285.0950.189−0.482
    DIN85.42485.4680.6740
    Modesl of this paperLIN85.58185.6690.8590.184
    LINc84.80384.911−0.058−0.727
    LINr85.79685.8411.1130.435
    LSTINc85.78185.8341.0950.418
    LSTINr85.83185.9431.1540.476
    下载: 导出CSV
  • [1] Shen X L, Li Z J, He C H. Hybrid recommendation algorithm based on rating filling and trust information. J Comput Appl, 2020, 40(10): 2789

    沈学利, 李子健, 赫辰皓. 基于评分填充与信任信息的混合推荐算法. 计算机应用, 2020, 40(10):2789
    [2] Song H Q, Du S Y, Zhou Y C, et al. Big data intelligent platform and application analysis for oil and gas resource development. Chin J Eng, 2021, 43(2): 179

    宋洪庆, 都书一, 周园春, 等. 油气资源开发的大数据智能平台及应用分析. 工程科学学报, 2021, 43(2):179
    [3] Tao Z L, Wang X, He X N, et al. HoAFM: A high-order attentive factorization machine for CTR prediction. Inf Process Manag, 2019, 57(6): 102076
    [4] Zhou A Y, Zhou M Q, Gong X Q. Computational advertising: A data-centric comprehensive web application. Chin J Comput, 2011, 34(10): 1805 doi: 10.3724/SP.J.1016.2011.01805

    周傲英, 周敏奇, 宫学庆. 计算广告: 以数据为核心的Web综合应用. 计算机学报, 2011, 34(10):1805 doi: 10.3724/SP.J.1016.2011.01805
    [5] Liu M J, Zeng G C, Yue W, et al. Review on click-through rate prediction models for display advertising. Comput Sci, 2019, 46(7): 38 doi: 10.11896/j.issn.1002-137X.2019.07.006

    刘梦娟, 曾贵川, 岳威, 等. 面向展示广告的点击率预测模型综述. 计算机科学, 2019, 46(7):38 doi: 10.11896/j.issn.1002-137X.2019.07.006
    [6] Richardson M, Dominowska E, Ragno R. Predicting clicks: Estimating the click through rate for new ADs // Proceedings of the 16th International Conference on World Wide Web. Alberta, 2007: 521
    [7] Chen J X, Sun B G, Li H, et al. Deep CTR prediction in display advertising // Proceedings of the 24th ACM international conference on Multimedia. Amsterdam, 2016: 811
    [8] Rendle S. Factorization Machines // 2010 IEEE International Conference on Data Mining. Berlin, 2010: 995
    [9] Zhang W N, Du T M, Wang J. Deep learning over multi-field categorical data: A case study on user response prediction // Proceedings of European Conference on Information Retrieval. Padua, 2016: 45
    [10] Qu Y R, Cai H, Ren K, et al. Product-based neural networks for user response prediction // 2016 IEEE 16th International Conference on Data Mining. Barcelona, 2016: 1149
    [11] Cheng H T, Koc L, Harmsen J, et al. Wide & Deep learning for recommender systems // Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. Boston, 2016: 7
    [12] Guo H F, Tang R M, Ye Y M, et al. DeepFM: A factorization-machine based neural network for CTR prediction // Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne, 2017: 1725
    [13] Zhou G R, Song C R, Zhu X Q, et al. Deep interest network for click-through rate prediction // Proceedings of KDD International Conference on Knowledge Discovery & Data Mining. London, 2018: 1059
    [14] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computat, 1997, 9(8): 1735 doi: 10.1162/neco.1997.9.8.1735
    [15] Cheng Y, Yao L B, Zhang G H, et al. Text sentiment orientation analysis of multi-channels CNN and BiGRU based on attention mechanism. J Comput Res Dev, 2020, 57(12): 2583 doi: 10.7544/issn1000-1239.2020.20190854

    程艳, 尧磊波, 张光河, 等. 基于注意力机制的多通道CNN和BiGRU的文本情感倾向性分析. 计算机研究与发展, 2020, 57(12):2583 doi: 10.7544/issn1000-1239.2020.20190854
    [16] Cho K, Merriënboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation // Proceedings of the Conference on Empirical Methods on Natural Language Processing. Doha, 2014: 1724
    [17] Dai J H, Deng Y B. Extracting emotion-cause pairs based on emotional dilation gated CNN. Data Anal Knowl Discov, 2020, 4(8): 98

    代建华, 邓育彬. 基于情感膨胀门控CNN的情感-原因对提取. 数据分析与知识发现, 2020, 4(8):98
    [18] Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences // Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Baltimore, 2014: 655
    [19] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need // Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, 2017: 6000
    [20] Jiang D, Xu R B, Xu X, et al. Multi-view feature transfer for click-through rate prediction. Inf Sci, 2021, 546: 961 doi: 10.1016/j.ins.2020.09.005
    [21] Yu C M, Feng B L, An L. Sentiment analysis in cross-domain environment with deep representative learning. Data Anal Knowl Discov, 2017, 1(7): 73

    余传明, 冯博琳, 安璐. 基于深度表示学习的跨领域情感分析. 数据分析与知识发现, 2017, 1(7):73
    [22] Zhang Y F, Lu Z Q. Remaining useful life prediction based on an integrated neural network. Chin J Eng, 2020, 42(10): 1372

    张永峰, 陆志强. 基于集成神经网络的剩余寿命预测. 工程科学学报, 2020, 42(10):1372
    [23] Kim Y. Convolutional neural networks for sentence classification // Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Doha, 2014: 1746
    [24] Zhang Y, Wallace B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification. Comput Sci, https://arxiv.org/pdf/1510.03820.pdf
    [25] Kingma D P, Ba J. Adam: A method for stochastic optimization // International Conference on Learning Representations. San Diego, 2015
    [26] Li Y J, Guo H X, Li Y N, et al. A boosting based ensemble learning algorithm in imbalanced data classification. Syst Eng Theory Pract, 2016, 36(01): 189

    李诒靖, 郭海湘, 李亚楠, 等. 一种基于Boosting的集成学习算法在不均衡数据中的分类. 系统工程理论与实践, 2016, 36(01):189
  • 加载中
图(5) / 表(2)
计量
  • 文章访问数:  188
  • HTML全文浏览量:  159
  • PDF下载量:  43
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-03-23
  • 网络出版日期:  2021-08-12

目录

    /

    返回文章
    返回