Written by
Max
Projects
AI Poker
AI Poker Tutorial
Over the years, poker strategy has become more and more mathematical and many top players play by studying game theory optimal (GTO) strategy with “solver” software. This tutorial touches on how the math and algorithms behind the solvers work as well as important poker research papers and ideas.
- In progress
Jun 2019: Most Important Fundamental Rule of Poker
Using techniques from machine learning we have uncovered a new simple, fundamental rule of poker strategy that leads to a significant improvement in performance over the best prior rule and can also easily be applied by human players.
- With Sam Ganzfried
May 2017: Counterfactual Regret Minimization and Abstraction Analysis in Computer Poker
MSc thesis analyzing card abstraction in Kuhn (1-card) poker and betting abstraction in No Limit Royal Hold’em (same as No Limit Texas Hold’em, but with a 20-card deck with cards Ten and higher only).
- Technion Israel Institute of Technology with Professor Nahum Shimkin
Gambling Applications
Jan 2020: Policy and Value Iteration Gambler Problem
- Analyzing the gambler problem from the Sutton Barto reinforcement learning textbook
Jul 2020: Freethrow Bet Evaluation
- Reinforcement learning to optimize the strategy for Mike McDonald’s free throw bet where he had to make 90/100 free throws and was allowed to reset his attempts at any point
- Freethrow Bet Evaluation TLDR Version
- Freethrow Bet Evaluation on Medium
- With Mike Thompson
Sep 2020: Monte Carlo RL and Blackjack
- Finding optimal blackjack strategies using Monte Carlo reinforcement learning
AI Safety
Sep 2020: Assessing Generalization in Reward Learning
- Part 1: Intro and Background
- Part 2: Implementations and Experiments
- One of the primary goals in machine learning is to create algorithms and architectures that demonstrate good generalization ability to samples outside of the training set. In reinforcement learning, however, the same environments are often used for both training and testing, which may lead to significant overfitting. We build on previous work in reward learning and model generalization to evaluate reward learning on random, procedurally generated environments.
- Work as part of the 2020 AI Safety Camp with Anton Makiievskyi and Liang Zhou
Reinforcement Learning
Soon
Machine Learning/Deep Learning
Apr 2020: Prediction of Bayesian Intervals for Tropical Storms
- Building on recent research for prediction of hurricane trajectories using recurrent neural networks (RNNs), we have developed improved methods and generalized the approach to predict a confidence interval region of the trajectory utilizing Bayesian methods
- Accepted to ICLR 2020 Climate Change Workshop and FLAIRS 2020
- With Sam Ganzfried