## Journal Papers

- N. Shimkin and A. Feuer, “Persistency of excitation in continuous-time systems,”
*Systems and Control Letters*9, 1987, pp. 225-233. - N. Shimkin and A. Feuer, “On the necessity of `block invariance’ for the convergence of adaptive pole-placement algorithms with persistently exciting input,“
*IEEE Trans. Automat. Contr.*33(8), 1988, pp. 775-780. - N. Shimkin and A. Shwartz, “Guaranteed performance regions in Markovian Systems with Competitive Decision Makers,”
*IEEE Trans. Automat. Contr.*38(1), pp. 84-95, 1993. - N. Shimkin, “Stochastic games with average cost constraints,”
*Annals of the International Society of Dynamic Games, Vol. 1: Advances in Dynamic Games and Applications*, Eds. T. Basar and A. Haurie, Birkhauser, 1994. - N. Shimkin, “Extremal large deviations in controlled i.i.d. processes with applications to hypothesis testing,”
*Adv. Appl. Prob.*25, pp. 875-894, Dec. 1993. - A. Orda, R. Rom and N. Shimkin, “Competitive routing in multi-user communication networks,”
*IEEE/ACM Trans. on Networking*1(5), pp. 510-521, Oct. 1993.. - N. Shimkin and A. Shwartz, “Asymptotically efficient adaptive strategies in repeated games. Part I: Certainty Equivalence strategies,”
*Mathematics of Operations Research*20, 1995, pp. 743-767. - N. Shimkin and A. Shwartz, “Asymptotically efficient adaptive strategies in repeated games. Part II: optimal strategies,“
*Mathematics of Operations Research*21, May 1996, pp. 487-512. - E. Altman and N. Shimkin, “Individual equilibrium and learning in processor sharing systems,”
*Operations Research*46, Dec. 1998, pp.776-784 (extended version). - A. Orda and N. Shimkin, “Incentive pricing in multi-class systems,”
*Telecommunication Systems*13, 2000, pp. 241-247. - I. Ben-Shachar, A. Orda and N. Shimkin, “Dynamic service sharing with heterogeneous preferences,”
*Queueing Systems*35, 2000, pp. 83-103. - E. Altman, A. Orda and N. Shimkin, “Bandwidth allocation for Guaranteed versus Best Effort service categories,”
*Queueing Systems*36, Nov. 2000, pp. 89-105. - A. Mandelbaum and N. Shimkin, “A model for rational abandonments from invisible queues,”
*Queueing Systems*36, 2000, pp. 141-173. - T. Jimenez, E. Altman, T. Basar and N. Shimkin, “Routing into two parallel links: game-theoretic distributed algorithms,”
*Journal of Parallel and Distributed Computing,*Special Issue on Routing in Computer and Communication Systems, 2001, pp. 1367-1381. - T. Jimenez, E. Altman, T. Basar and N. Shimkin, “Competitive routing in networks with polynomial costs,”
*IEEE Trans. on Automatic Control*47, Jan. 2002, pp. 92-96. - E. Zohar, A. Mandelbaum and N. Shimkin, “Adaptive behavior of impatient customers in tele-queues: theory and empirical support ,”
*Management Science*48(4), 2002, pp.566-583. (extended version). - R. Zaritsky, N. Peterfreund and N, Shimkin, “Velocity-guided tracking of deformable contours in three dimensional space,”
*International Journal of Computer Vision51*(3), Feb. 2003, pp. 219-238. - S. Mannor and N. Shimkin, “The empirical Bayes envelope and regret minimization in competitive Markov decision processes,”
*Mathematics of Operations Research*28(2), May 2003, pp. 327-345. - M. Jacobson, N. Shimkin and A. Shwartz, “Markov decision processes with slow-scale periodic decisions,”
*Mathematics of Operations Research*28(4), Nov. 2003, pp. 777-800. Extended version: “Piecewise stationary Markov Decision Processes, Part I: constant gain, Part II: state-dependent gain“. - N. Shimkin and A. Mandelbaum, “Rational abandonment from tele-queues: Nonlinear waiting costs with heterogeneous preferences,”
*Queueing Systems*47(1-2), June 2004, pp. 117-146. - S. Mannor and N. Shimkin, “A robust geometric approach to multi-criterion reinforcement learning“,
*Journal of Machine Learning Research*5, pp. 325-360, March 2004.. - I. Menache, S. Mannor and N. Shimkin, “Basis function adaptation in temporal difference reinforcement learning,”
*Annals of Operations Research*, special issue on The Cross Entropy Method, Vol. 134, Feb. 2005, pp. 215-238.. - E. Rippel, A. Bar-Gill and N. Shimkin, “Fast Graph-Search Algorithms for General Aviation Flight Trajectory Generation.”
*Journal of Guidance, Control and Dynamics*28(4), July-August 2005, pp. 801-811. - O. Richman and N. Shimkin, “Topological Uniqueness of the Nash Equilibrium for Atomic Selfish Routing.”
*Mathematics of Operations Research*32(1), Feb. 2007, pp.215-232. - I. Menache and N. Shimkin. “Capacity Management and Equilibrium for Proportional QoS“,
*IEEE/ACM Trans. Networking*16(5), Oct. 2008, pp. 1025-1037. (Extended version: EE/CCIT report No. 575, March 2007). - S. Mannor and N. Shimkin, “Regret Minimization in Repeated Matrix Games with Variable Stage Duration”,
*Games and Economic Behavior*63(1), May 2008, pp. 227-258. - I. Menache and N. Shimkin, “Efficient rate-constrained Nash equilibrium in collision channels with state information”,
*IEEE Journal on Selected Areas in Communications (JSAC)*26(7), special issue on Game Theory in Communication Systems, Sept. 2008, pp. 1070-1076. - M. Armony, N. Shimkin and W. Whitt, “The impact of delay announcements in many-server queues with abandonment”,
*Operations Research*57(1), Jan. 2009, pp. 66-81. With Supplementary Material to the main paper. - J. Y. Yu, S. Mannor and N. Shimkin, “Markov Decision Processes with Arbitrary Reward Processes“,
*Mathematics of Operations Research*34(3), August 2009, pp. 737-757. - S. Meerkov, N. Shimkin and L. Zhang, “Transient Behavior of Two-Machine Geometric Production Lines“,
*IEEE Trans. on Automatic Control ,*55(2), Feb. 2010, pp. 453-458. - D. Sigalov and N. Shimkin, “Cross entropy algorithms for Data Association in Multi Target Tracking“,
*IEEE Trans. on Aerospace and Electronic Systems*, 47(2), April 2011, pp. 1166-1185. - R. Atar, C. Giat and N. Shimkin, “The cm/q rule for many-server queues with abandonment“,
*Operations Research*58(5), Sept.-Oct. 2010, pp. 1427-1439. - A. Bernstein and N. Shimkin, “Adaptive-Resolution Reinforcement Learning with Efficient Exploration in Deterministic Domains“,
*Machine Learning Journal*81(3), Dec. 2010, pp. 359-397. - R. Atar, C. Giat and N. Shimkin, “On the asymptotic optimality of the cm/q rule under ergodic cost“,
*Queueing Systems*67(2), Feb. 2011, pp. 127-144. - R. Jain, S. Juneja and N. Shimkin, “The Concert Queueing Game: To wait or to be late“,
*Discrete Event Dynamic Systems*21(1), April 2011, pp. 103-138. - I. Menache and N. Shimkin, “Reservation-based distributed medium access in wireless collision channels“,
*Telecommunications Systems*47(1-2), June 2011, pp. 95-108. - S. Juneja and N. Shimkin, “The Concert Queueing Game: Strategic arrivals with waiting and tardiness costs“,
*Queueing Systems*74(4):369-402, August 2013. - R. Atar, H. Kaspi and N. Shimkin, “Fluid Limits for many-server systems with reneging under a priority policy,”
*Mathematics of Operations Research*39(3):672-696, August 2014. - A. Bernstein, S. Mannor and N. Shimkin, “Opportunistic approachability and generalized no-regret problems“, to appear in MOR (published online on April 2014).
- A. Bernstein and N. Shimkin, “Response-Based Approachability and its Application to Generalized No-Regret Algorithms,”
*Journal of Machine Learning Research*16, April 2015, pp. 747-773. (Full version: arXiv:1312.7658). - E. Altman and N. Shimkin, “The Ordered Timeline Game: Strategic Posting Times over a Temporally-Ordered Shared Medium,”
*Dynamic Games and Applications*, to appear. Published online on May 2015, DOI 10.1007/s13235-015-0158-y - N. Shimkin, “An Online Convex Optimization Approach to Blackwell’s Approachability.” Submitted to
*JMLR*, revised June 2016.

Conference Papers and Reports

- N. Shimkin and A. Shwartz, “Control of admission and routing in parallel queues operating in a random environment,” Proc. IEEE 28th Conference on Decision and Control, Tampa, FL, Dec. 1989, pp. 1064-1065.
- G. Davidov, A. Shavit, N. Shimkin and Y. Koren, “Identification of time varying systems by the internal model principle,” 16th Israel IEEE Conference, paper 3.4.3, March 1989.
- N. Shimkin and A. Shwartz, “Guaranteed performance regions for Markov models”, Proc. IEEE 30th Conference on Decision and Control, Brighton, England, Dec. 1991, pp. 459-464.
- N. Shimkin and A. Shwartz, “Guaranteed performance regions for Multi-User Markov models”, in” Stochastic Theory and Adaptive Control , edited by T.E. Duncan and B.Pasik-Duncan (proceedings of the workshop held in Lawrence, Kansas, 1991), Lect. Notes in Control and Information Sciences 184, Springer-Verlag, NY, 1992, pp. 436-450.
- A. Orda, R. Rom and N. Shimkin, “Competitive routing in multi-user communication networks,” Proc. IEEE INFOCOM, April 1993, pp. 964-971.
- A. Orda and N. Shimkin, “Incentive pricing in multi-class communications networks,” Proc. IEEE INFOCOM, April 1997, pp. 660-667.
- E. Altman A. Orda and N. Shimkin, “Bandwidth allocation for Guaranteed versus best-effort service categories,” Proc. IEEE INFOCOM, April 1998, NY, pp. 617-624.
- I. Ben-Shachar, A. Orda and N. Shimkin, “Best-effort resource sharing by users with QoS requirements,” Proc.IEEE INFOCOM, April 1999, NJ, pp. 883-890.
- E. Altman, T. Basar, T. Chimenez and N. Shimkin, “Competitive routing in networks with polynomial cost”, Proc. IEEE INFOCOM, April 2000, Tel-Aviv, pp. 1586-1593.
- A. Taliansky and N. Shimkin, “Behavior-based navigation system for an indoor mobile robot,” Proc. 21st IEEE Convention in Israel, April 2000, Tel-Aviv, pp. 281-284.
- R. Zaritski, N. Peterfreund and N. Shimkin, “Velocity-guided tracking of deformable contours in three dimensional space,” Proc. ECCV’2000 – 6th European Conference on Computer Vision, Dublin, June 2000. Lecture Notes in Computer Science, Vol. 1842, pp. 251-266.
- S. Mannor and N. Shimkin, “Adaptive strategies and regret minimization in arbitrarily varying Markov environments,” COLT 2001: 14th Annual Conference on Computational Learning Theory, Amsterdam, July 2001. Lecture Notes in Computer Science 2111,

Springer, pp. 128-142. - S. Mannor and N. Shimkin, “The steering approach for multi-criteria reinforcement learning,” Proc. NIPS 2001: 14th Conference on Neural Information Processing Systems, Vancouver, December 2001, pp. 1563-1570.
- I. Menache, S. Mannor and N. Shimkin, “Q-Cut – dynamic discovery of sub-goals in reinforcement learning,” ECML’02 – European Conference on Machine Learning,Helsinki, August 2002. In: T. Elomma et al. (eds.),
*Lect. Notes Artific. Intel.*2430, Springer, pp. 295-306. - S. Mannor and N. Shimkin, “Online learning with imperfect monitoring,” COLT’03 – 16th Annual Conference on Learning Theory, Washington, August 2003. In: B.Scholkopf and M. K. Warmuth (Eds.),
*Lect. Notes Artific. Intel.*2777, Springer, pp. 552-566. - O. Ziv and N. Shimkin, “Multigrid Methods for policy evaluation and reinforcement learning,” Proc. IEEE International Symposium on Intelligent Control (ISIC05),Limassol, Cyprus, June 2005 (expanded version).
- O. Zivand N. Shimkin, “Multigrid algorithms for Temporal Difference Reinforcement Learning“, Proc. ICML’05 Workshop on Rich Representations for Reinforcement Learning, Bonn, Aug. 2005 (expanded version). For full details see the technical report: O. Ziv,
*Algebraic Multigrid for Reinforcement Learning*. - E. Pinhasov, N. Shimkin and Y. Zeevi, “Optimal usage of color for disparity estimation is stereo vision”, Proc. EUSIPCO 2005 – 13th European Signal Processing Conference, Antalya, Turkey, Sept. 2005.
- O. Richman and N. Shimkin, “Uniqueness of the Nash equilibrium in convex routing games: Topological conditions”, Proc. CDC’05 – 44th IEEE Conference on Decision and Control, Seville, Spain, December 2005.
- I. Menache and N. Shimkin. “Proportional QoS in Differentiated Services Networks: Capacity Management, Equilibrium Analysis and Elastic Demands” (extended version, Sept. 2005). In:
*Proc. WINE05, First International Workshop on Internet and Network Economics*, Hong Kong, December 2005. LNCS, Vol. 3828, pp. 728 – 737. - S. Mannor and N. Shimkin, “Online learning with variable stage duration”, COLT’06, Pittsburgh, Pennsylvania, June 2006. In: Lugosi and H.U. Simon (Eds.), COLT 2006, LNAI 4005, pp. 408–422, 2006.
- I. Menache and N. Shimkin, “Capacity Assignment for Proportional QoS in Diffserv-Like Networks”, Proc. CDC’06 – 45th IEEE Conference on Decision and Control, San Diego, California, December 2006.
- D. Sigalov and N. Shimkin, “Cross Entropy data association for multiple target tracking”, 47th Israel Annual Conf. on Aerospace Sciences, Tel Aviv, Feb. 2007.
- I. Menache and N. Shimkin, “Fixed-Rate Equilibrium in Wireless Collision Channels“, Net-Coop 2007 – International Conference on Network Control and Optimization,Avignon, June 2007. In:
*Lect. Notes Comp. Sci.*4465, pp. 33–42. - N. Shimkin, “A Survey of Uniqueness Results for Selfish Routing“, Net-Coop 2007 – International Conference on Network Control and Optimization, Avignon, June 2007. In:
*Lect. Notes Comp. Sci.*4465, pp. 33-42. - I. Menache and N. Shimkin, “Efficient Rate-Constrained Nash Equilibrium in Collision Channels with State Information“, INFOCOM’08, Phoenix, Arizona, April 2008.
- I. Menache and N. Shimkin, “Decentralized Rate Regulation in Random Access Channels”, INFOCOM’08, Phoenix, Arizona, April 2008.
- I. Menache and N. Shimkin, “Noncooperative Power Control and Transmission Scheduling in Wireless Collision Channels“, SIGMETRICS’08, Annapolis, Maryland, June 2008,
- A. Bernstein and N. Shimkin, “Adaptive Aggregation for Reinforcement Learning with Efficient Exploration: Deterministic Domains“, COLT’08, Helsinki, July 2008.
- K. Dyagilev, S. Mannor and N. Shimkin, “Efficient Reinforcement Learning in Parameterized Models: Discrete Parameter Case”, 8
^{th}European Workshop on Reinforcement Learning, France, July 2008. In:*Lecture Notes in Computer Science*5323, Springer, pp. 41-54. (Full version). - K. Dyagilev, S. Mannor and N. Shimkin, “Reinforcement Learning with Polynomial Learning Rate in Parameterized Models“, Technical Report, 2008.
- D. Sigalov and N. Shimkin, “Cross-Entropy Based Data Association for Multiple Target Tracking“, ValueTools’08 – Third International Conference on Performance Evaluation Methodologies and Tools, Athens, Greece, October 2008.
- N. Shimkin, “Nonlinear Control Systems” (short overview),
*Encyclopedia of Neuroscience*, Eds. M. Binder et. al., Springer, 2008. - I. Menache and N. Shimkin, “Reservation-Based Distributed Medium Access in Wireless Collision Channels.”
*Proc. GameComm 2008: Second International Workshop on Game Theory in Communication Networks,*Athens, Greece, October 2008. - S. Meerkov, N. Shimkin and L. Zhang, “Transient Behavior of Two-Machine Geometric Production Lines“, Proc. INCOM’09 – 13th IFAC Symposium on Information and Control Problems in Manufacturing, Moscow, June 2009. (Best track-paper award.)
- J. Y. Yu, S. Mannor and N. Shimkin, “Markov Decision Processes with Arbitrarily Varying Rewards”, EWRL’08 – 8
^{th}European Workshop on Reinforcement Learning, France, July 2008. In:*Lecture Notes in Computer Science*5323, Springer, pp. 268-281. - A. Bernstein, S. Mannor and N. Shimkin, “Online Classification with Specificity Constraints“, Proc.
*NIPS 2010,*Vancouver, Dec. 2010. - M. Cohen and N. Shimkin, “Decentralized algorithms for sequential network time synchronization“, Proc.
*NETCOOP 2010 –**4*, Ghent, Belgium, Dec. 2010, pp. 97-104.^{th}Workshop on Network Control and Optimization - A. Ozdaglar, I. Menache and N. Shimkin, “Socially Optimal Pricing of Cloud Computing Resources“,
*ValueTools**2011: 5th ICST International Conference on Performance Evaluation Methodologies and Tools,*Paris, France, May 2011. - D. Pinkovich and N. Shimkin, “Clock Synchronization using Maximal Margin Estimation“,
*MED’11 – the 19th IEEE Mediterranean Conference on Control and Automation*, Corfu, Greece, June 2011. - K. Y. Levy and N. Shimkin, “Unified Inter and Intra Options Learning Using Policy Gradient Methods“,
*Proc. 9th European Workshop on Reinforcement Learning,*Athens, Sept. 2011. - A. Adler, A. Bar-Gill and n. Shimkin, “Optimal Flight Paths for Engine-Out Emergency Landing“,
*24*, Taiyuan, China, May 2012.^{th}CCDC – Chinese Control and Decision Conference - S. Juneja, T. Raheja and N. Shimkin, “The concert queueing game with a random volume of arrivals“,
*Proc. ValueTools 2012*, Cargèse, France, October 2013. (*Best paper award*.) - A. Bernstein, S. Mannor and N. Shimkin, “
*Opportunistic strategies for generalized no-regret problems”, Proc. COLT 2013**– 26th Annual Conference on Learning Theory,*Princeton, NJ, June 2013. - Y. David and N. Shimkin, “Infinitely many-armed Bandits with unknown value distribution,”
*Proc. ECML 2014*. Full version. - E. Altman and N. Shimkin, “Strategic posting times over a shared publication medium,”
*Proc. Intern. Conf. on Network Games, Control and Optimization (NetGCoop),*Trento, Italy, October 2014. - Y. David and N. Shimkin, “PAC Algorithms for the Infinitely-Many Armed Problem with Multiple Pools,” 12
^{th}European Workshop on Reinforcement Learning (EWRL), Lille, July 2015. - Y. David and N. Shimkin, “Refined Algorithms for Infinitely Many-Armed Bandits with Deterministic Rewards,” ECML 2015.
- Y. David and N. Shimkin, “PAC Lower Bounds and Efficient Algorithms for the Max-K arms Bandit Problem,” International Conference on Machine Learning (ICML), New York, June 2016.
- Y. David and N. Shimkin, “Pure Exploration for Max-Quantile Bandits,” European Conference on Machine Learning (ECML), Riva del Garda, Italy, September 2016.
- D Pure