基于Copula熵的互信息估计方法

Mutual information estimation based on Copula entropy

中文关键词  互信息  Copula  熵  概率密度函数  参数估计

英文关键词  mutual information  Copula  entropy  probability density function  parameter estimation

中文摘要

      互信息是一种常用的衡量变量相关性的方法, 但在互信息估计过程中, 联合概率密度的估计往往十分困难. 为了避免联合概率密度的估计, 同时有效提高互信息估计的准确度与效率, 本文提出一种基于Copula熵的互信息估计方法. 利用Copula熵与互信息之间的关系, 将互信息的估计转化为对Copula熵值的估计. 采用基于Kendall秩相关系数的参数估计方法对Copula函数的参数进行估计. 所提算法分别与直方图法、核方法、k近邻法和极大似然法进行比较. 二维高斯数据上的仿真结果表明, 所提方法能够快速准确地对互信息值进行估计.

英文摘要

      Mutual information is commonly used in the measure of dependency between variables. However, in the estimation of mutual information, the estimation of joint probability density function is always a hard problem. To avoid the estimation of joint probability density function and to improve both the effectiveness and efficiency of the estimation of mutual information, we propose a novel mutual information estimation method based on the entropy of the Copula density function. The estimation of mutual information is transformed into the estimation of Copula entropy by taking advantages of their relationships. Parameter estimation method based on Kendall’s rank correlation coefficient is used for the estimation of the parameters in Copula function. The proposed method is compared with the following four methods: histogram method, kernel method, k nearest neighbor method and maximum likelihood method. Simulation results on the two dimensional Gaussian distribution data substantiate the effectiveness and efficiency of the proposed method.