Mutually dependent Markov decision processes

Toshiharu Fujita, Akifumi Kira

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)


In this paper, we introduce a basic framework for mutually dependent Markov decision processes (MDMDP) showing recursive mutual dependence. Our model is structured upon two types of finite-stage Markov decision processes. At each stage, the reward in one process is given by the optimal value of the alternative process problem, whose initial state is determined by the current state and decision in the original process. We formulate the MDMDP model and derive mutually dependent recursive equations by dynamic programming. Furthermore, MDMDP is illustrated in a numerical example. The model enables easier treatment of some classes of complex multi-stage decision processes.

Original languageEnglish
Pages (from-to)992-998
Number of pages7
JournalJournal of Advanced Computational Intelligence and Intelligent Informatics
Issue number6
Publication statusPublished - 2014 Nov 1


  • Additive reward
  • Dynamic programming
  • Markov decision process
  • Nonserial system

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Mutually dependent Markov decision processes'. Together they form a unique fingerprint.

Cite this