Optimal entropy estimation on large alphabet: fundamental limits and fast algorithms
Yang, Pengkun
Loading…
Permalink
https://hdl.handle.net/2142/90776
Description
Title
Optimal entropy estimation on large alphabet: fundamental limits and fast algorithms
Author(s)
Yang, Pengkun
Issue Date
2016-04-19
Director of Research (if dissertation) or Advisor (if thesis)
Wu, Yihong
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
entropy estimation
large alphabet
Abstract
Consider the problem of estimating the Shannon entropy of a distribution
over k elements from n independent samples. We obtain the minimax mean-
square error within universal multiplicative constant factors if n exceeds a
constant factor of k/log(k); otherwise there exists no consistent estimator.
This refines the recent result of Valiant and Valiant (2011) that the mini-
mal sample size for consistent entropy estimation scales. The apparatus of
best polynomial approximation plays a key role in both the construction of
optimal estimators and, via a duality argument, the minimax lower bound.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
