Entropy bounds on Bayesian learning

Olivier Gossner; Tristan Tomala

doi:10.1016/j.jmateco.2007.04.006

Entropy bounds on Bayesian learning

Olivier Gossner
, Tristan Tomala

France and IZA
Université Paris Dauphine

Research output: Contribution to journal › Article › peer-review

Abstract

An observer of a process (x_t) believes the process is governed by Q whereas the true law is P. We bound the expected average distance between P (x_t | x₁, ..., x_{t - 1}) and Q (x_t | x₁, ..., x_{t - 1}) for t = 1, ..., n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P.

Original language	English
Pages (from-to)	24-32
Number of pages	9
Journal	Journal of Mathematical Economics
Volume	44
Issue number	1
DOIs	https://doi.org/10.1016/j.jmateco.2007.04.006
Publication status	Published - 1 Jan 2008
Externally published	Yes

Keywords

Bayesian learning
Entropy
Repeated decision problem
Value of information

Access to Document

10.1016/j.jmateco.2007.04.006

Cite this

@article{61c43b96b42d4ac5b928bf8574f711b5,

title = "Entropy bounds on Bayesian learning",

abstract = "An observer of a process (xt) believes the process is governed by Q whereas the true law is P. We bound the expected average distance between P (xt | x1, ..., xt - 1) and Q (xt | x1, ..., xt - 1) for t = 1, ..., n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P.",

keywords = "Bayesian learning, Entropy, Repeated decision problem, Value of information",

author = "Olivier Gossner and Tristan Tomala",

year = "2008",

month = jan,

day = "1",

doi = "10.1016/j.jmateco.2007.04.006",

language = "English",

volume = "44",

pages = "24--32",

journal = "Journal of Mathematical Economics",

issn = "0304-4068",

number = "1",

}

TY - JOUR

T1 - Entropy bounds on Bayesian learning

AU - Gossner, Olivier

AU - Tomala, Tristan

PY - 2008/1/1

Y1 - 2008/1/1

N2 - An observer of a process (xt) believes the process is governed by Q whereas the true law is P. We bound the expected average distance between P (xt | x1, ..., xt - 1) and Q (xt | x1, ..., xt - 1) for t = 1, ..., n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P.

AB - An observer of a process (xt) believes the process is governed by Q whereas the true law is P. We bound the expected average distance between P (xt | x1, ..., xt - 1) and Q (xt | x1, ..., xt - 1) for t = 1, ..., n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P.

KW - Bayesian learning

KW - Entropy

KW - Repeated decision problem

KW - Value of information

U2 - 10.1016/j.jmateco.2007.04.006

DO - 10.1016/j.jmateco.2007.04.006

M3 - Article

AN - SCOPUS:36549043366

SN - 0304-4068

VL - 44

SP - 24

EP - 32

JO - Journal of Mathematical Economics

JF - Journal of Mathematical Economics

IS - 1

ER -

Entropy bounds on Bayesian learning

Abstract

Keywords

Access to Document

Fingerprint

Cite this