大数据分析中的分布式抽样技术

发布者:文明办作者:发布时间:2020-12-15浏览次数:62


主讲人:艾明要  北京大学教授


时间:2020年12月18日14:00


地点:3号楼332


举办单位:数理学院


主讲人介绍:明要,北京大学数学科学学院统计学教研室主任、教授、博士生导师。兼任中国概率统计学会秘书长,中国数学会均匀设计分会副主任,中国现场统计研究会试验设计分会理事长,高维数据统计分会副理事长等。国际重要统计期刊《Statistica  Sinica》、《Journal of Statistical Planning and Inference》、《Statistics and  Probability Letters》、《Stat》副主编,国内核心期刊  《系统科学与数学》编委,科学出版社《统计与数据科学系列丛书》编委。主要从事试验设计与分析、大数据分析和应用概率统计的教学和研究工作,在Ann  Statist、JASA、Biometrika、《中国科学》等国内外顶尖期刊发表学术论文六十余篇,主持完成国家自然科学基金面上项目5项、重点项目子课题1项,参与完成国家科技部重点研发计划项目2项。  


内容介绍:Nonuniform subsampling methods are effective to reduce computational burden and  maintain estimation efficiency for massive data. Existing methods mostly focus  on subsampling with replacement due to its high computational efficiency. If the  data volume is so large that nonuniform subsampling probabilities cannot be  calculated all at once, then subsampling with replacement is infeasible to  implement. This paper solves this problem using Poisson subsampling. We first  derive optimal Poisson subsampling probabilities in the context of  quasi-likelihood estimation under the A- and L-optimality criteria. For a  practically implementable algorithm with approximated optimal subsampling  probabilities, we establish the consistency and asymptotic normality of the  resultant estimators. To deal with the situation that the full data are stored  in different blocks or at multiple locations, we develop a distributed  subsampling framework, in which statistics are computed simultaneously on  smaller partitions of the full data. Asymptotic properties of the resultant  aggregated estimator are investigated. We illustrate and evaluate the proposed  strategies through numerical experiments on simulated and real data sets.

k5电竞