Learning the difference between partially observable dynamical systems

Sami Zhioua; Doina Precup; François Laviolette; Josée Desharnais

doi:10.1007/978-3-642-04174-7_43

Learning the difference between partially observable dynamical systems

Sami Zhioua
, Doina Precup
, François Laviolette
, Josée Desharnais

McGill University
Rennes et Université Laval (Canada)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (Scopus)

Abstract

We propose a new approach for estimating the difference between two partially observable dynamical systems. We assume that one can interact with the systems by performing actions and receiving observations. The key idea is to define a Markov Decision Process (MDP) based on the systems to be compared, in such a way that the optimal value of the MDP initial state can be interpreted as a divergence (or dissimilarity) between the systems. This dissimilarity can then be estimated by reinforcement learning methods. Moreover, the optimal policy will contain information about the actions which most distinguish the systems. Empirical results show that this approach is useful in detecting both big and small differences, as well as in comparing systems with different internal structure.

Original language	English
Title of host publication	Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings
Publisher	Springer Verlag
Pages	664-677
Number of pages	14
Edition	PART 2
ISBN (Print)	3642041736, 9783642041730
DOIs	https://doi.org/10.1007/978-3-642-04174-7_43
Publication status	Published - 1 Jan 2009
Externally published	Yes
Event	9th Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009 - Bled, Slovenia Duration: 7 Sept 2009 → 11 Sept 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 2
Volume	5782 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009
Country/Territory	Slovenia
City	Bled
Period	7/09/09 → 11/09/09

Access to Document

10.1007/978-3-642-04174-7_43

Cite this

Zhioua, S., Precup, D., Laviolette, F., & Desharnais, J. (2009). Learning the difference between partially observable dynamical systems. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings (PART 2 ed., pp. 664-677). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5782 LNAI, No. PART 2). Springer Verlag. https://doi.org/10.1007/978-3-642-04174-7_43

Zhioua, Sami ; Precup, Doina ; Laviolette, François et al. / Learning the difference between partially observable dynamical systems. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 2. ed. Springer Verlag, 2009. pp. 664-677 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2).

@inproceedings{071b4dbb3d064e12a61e836dbfb3ed35,

title = "Learning the difference between partially observable dynamical systems",

abstract = "We propose a new approach for estimating the difference between two partially observable dynamical systems. We assume that one can interact with the systems by performing actions and receiving observations. The key idea is to define a Markov Decision Process (MDP) based on the systems to be compared, in such a way that the optimal value of the MDP initial state can be interpreted as a divergence (or dissimilarity) between the systems. This dissimilarity can then be estimated by reinforcement learning methods. Moreover, the optimal policy will contain information about the actions which most distinguish the systems. Empirical results show that this approach is useful in detecting both big and small differences, as well as in comparing systems with different internal structure.",

author = "Sami Zhioua and Doina Precup and Fran{\c c}ois Laviolette and Jos{\'e}e Desharnais",

year = "2009",

month = jan,

day = "1",

doi = "10.1007/978-3-642-04174-7\_43",

language = "English",

isbn = "3642041736",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

number = "PART 2",

pages = "664--677",

booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings",

edition = "PART 2",

note = "9th Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009 ; Conference date: 07-09-2009 Through 11-09-2009",

}

Zhioua, S, Precup, D, Laviolette, F & Desharnais, J 2009, Learning the difference between partially observable dynamical systems. in Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 2 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 2, vol. 5782 LNAI, Springer Verlag, pp. 664-677, 9th Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009, Bled, Slovenia, 7/09/09. https://doi.org/10.1007/978-3-642-04174-7_43

Learning the difference between partially observable dynamical systems. / Zhioua, Sami; Precup, Doina; Laviolette, François et al.
Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 2. ed. Springer Verlag, 2009. p. 664-677 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5782 LNAI, No. PART 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Learning the difference between partially observable dynamical systems

AU - Zhioua, Sami

AU - Precup, Doina

AU - Laviolette, François

AU - Desharnais, Josée

PY - 2009/1/1

Y1 - 2009/1/1

N2 - We propose a new approach for estimating the difference between two partially observable dynamical systems. We assume that one can interact with the systems by performing actions and receiving observations. The key idea is to define a Markov Decision Process (MDP) based on the systems to be compared, in such a way that the optimal value of the MDP initial state can be interpreted as a divergence (or dissimilarity) between the systems. This dissimilarity can then be estimated by reinforcement learning methods. Moreover, the optimal policy will contain information about the actions which most distinguish the systems. Empirical results show that this approach is useful in detecting both big and small differences, as well as in comparing systems with different internal structure.

AB - We propose a new approach for estimating the difference between two partially observable dynamical systems. We assume that one can interact with the systems by performing actions and receiving observations. The key idea is to define a Markov Decision Process (MDP) based on the systems to be compared, in such a way that the optimal value of the MDP initial state can be interpreted as a divergence (or dissimilarity) between the systems. This dissimilarity can then be estimated by reinforcement learning methods. Moreover, the optimal policy will contain information about the actions which most distinguish the systems. Empirical results show that this approach is useful in detecting both big and small differences, as well as in comparing systems with different internal structure.

U2 - 10.1007/978-3-642-04174-7_43

DO - 10.1007/978-3-642-04174-7_43

M3 - Conference contribution

AN - SCOPUS:70349939641

SN - 3642041736

SN - 9783642041730

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 664

EP - 677

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings

PB - Springer Verlag

T2 - 9th Joint European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009

Y2 - 7 September 2009 through 11 September 2009

ER -

Zhioua S, Precup D, Laviolette F, Desharnais J. Learning the difference between partially observable dynamical systems. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 2 ed. Springer Verlag. 2009. p. 664-677. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2). doi: 10.1007/978-3-642-04174-7_43

Learning the difference between partially observable dynamical systems

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this