On Controlled Finite State Markov Processes with Compact Control Sets
E. A. Fainberg · Theory of Probability and Its Applications · 1976
Previous article Next article On Controlled Finite State Markov Processes with Compact Control SetsE. A. FainbergE. A. Fainberghttps://doi.org/10.1137/1120093PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] O. V. Viskov and , A. N. Shiryaev, On controls which reduce to optimal stationary regimes, Trudy Mat. Inst. Steklov., 71 (1964), 35–45, (In Russian.) MR0173579 Google Scholar[2] Cyrus Derman, On sequential decisions and Markov chains, Management Sci., 9 (1962/1963), 16–24 MR0169685 CrossrefGoogle Scholar[3] Anders Martin-Löf, Existence of a stationary control for a Markov chain maximizing the average reward, Operations Res., 15 (1967), 866–871 MR0225565 0149.38103 CrossrefGoogle Scholar[4] John Bather, Optimal decision procedures for finite Markov chains. I. Examples, Advances in Appl. Probability, 5 (1973), 328–339 MR0368790 0262.90063 CrossrefGoogle Scholar[5] J. G. Kemeny and , J. L. Snell, Finite Markov Chains, Van Nostrand, Princeton, N.Y., 1965 Google Scholar[6] Ronald A. Howard, Dynamic programming and Markov processes, The Technology Press of M.I.T., Cambridge, Mass., 1960viii+136 MR0118514 0091.16001 Google Scholar[7] E. V. Denardo and , B. L. Fox, Multichain Markov renewal programs, SIAM J. Appl. Math., 16 (1968), 468–487 10.1137/0116038 MR0234721 0201.19303 LinkGoogle Scholar[8] A. A. Yushkevich, On a class of strategies in general controlled Markov models, Theory Prob. Applications, 18 (1973), 777–778 10.1137/1118099 0311.90081 LinkGoogle Scholar[9] L. G. Gubenko and , È. S. Štatland, On discrete time Markov decision processes, Teor. Verojatnost. i Mat. Statist., (1972), 51–64, 163, (In Russian.) MR0334957 0359.93048 Google Scholar[10] Samuel Karlin, Mathematical methods and theory in games, programming and economics. Vol. I: Matrix games, programming, and mathematical economics. Vol. II: The theory of infinite games, Addison-Wesley Publishing Co., Inc., Reading, Mass.-London, 1959Vol. I, x+433 pp. Vol. II, xi+386 MR0111634 Google Scholar[11] Cyrus Derman, Finite state Markovian decision processes, Mathematics in Science and Engineering, Vol. 67, Academic Press, New York, 1970xiii+159 MR0267686 0262.90001 Google Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Stochastic Optimization Methods for the Stochastic Storage Process Control24 February 2012 Cross Ref Optimal control of a queue under a quality-of-service constraint with bounded and unbounded ratesOperations Research Letters, Vol. 48, No. 6 Cross Ref Gittins Index for Simple Family of Markov Bandit Processes with Switching Cost and No DiscountingM. P. Savelov22 October 2019 | Theory of Probability & Its Applications, Vol. 64, No. 3AbstractPDF (199 KB)Индекс Гиттинса для простого семейства марковских игровых автоматов при штрафах за переход и отсутствии дисконтирования22 July 2019 | Теория вероятностей и ее применения, Vol. 64, No. 3 Cross Ref The average cost of Markov chains subject to total variation distance uncertaintySystems & Control Letters, Vol. 120 Cross Ref Equilibrium control policies for Markov chains Cross Ref Discounted Approximations for Risk-Sensitive Average Criteria in Markov Decision Chains with Finite State SpaceMathematics of Operations Research, Vol. 36, No. 1 Cross Ref Uniqueness and Stability of Optimal Policies of Finite State Markov Decision ProcessesMathematics of Operations Research, Vol. 32, No. 1 Cross Ref Policy iteration type algorithms for recurrent state Markov decision processesComputers & Operations Research, Vol. 31, No. 14 Cross Ref An Algorithm to Identify and Compute Average Optimal Policies in Multichain Markov Decision ProcessesMathematics of Operations Research, Vol. 28, No. 3 Cross Ref On Optimal Policies of Multichain Finite State Compact Action Markov Decision Processes Cross Ref A Note on the Existence of Optimal Policies in Total Reward Dynamic Programs with Compact Action SetsMathematics of Operations Research, Vol. 25, No. 4 Cross Ref Discrete-Time Controlled Markov Processes with Average Cost Criterion: A SurveyAristotle Arapostathis, Vivek S. Borkar, Emmanuel Fernández-Gaucherand, Mrinal K. Ghosh, and Steven I. Marcus14 July 2006 | SIAM Journal on Control and Optimization, Vol. 31, No. 2AbstractPDF (7400 KB)An Average Reward Criterion by Mandl Cross Ref An expected average reward criterionStochastic Processes and their Applications, Vol. 26 Cross Ref An $\varepsilon $-Optimal Control of a Finite Markov Chain with an Average Reward CriterionE. A. Fainberg28 July 2006 | Theory of Probability & Its Applications, Vol. 25, No. 1AbstractPDF (1074 KB)The Existence of a Stationary $\varepsilon $-Optimal Policy for a Finite Markov ChainE. A. Fainberg17 July 2006 | Theory of Probability & Its Applications, Vol. 23, No. 2AbstractPDF (1530 KB) Volume 20, Issue 4| 1976Theory of Probability & Its Applications History Submitted:26 June 1974Published online:17 July 2006 InformationCopyright © 1976 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1120093Article page range:pp. 856-862ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics