@nathanlichtle profile picture
Nathan Lichtlé @nathanlichtle

PhD @UCBerkeley (@berkeley_ai)

Joined May 2019
  • Posts 30
  • Following 141
  • Followers 135
8 Photos and videos
Pinned Post
Nathan Lichtlé @nathanlichtle
14 Feb 2025
We ran 5,600 hyperparameter sweeps to compare RL algorithms on hidden-information games with billions of states. In our benchmark, we found that properly tuned policy gradient methods, such as PPO, performed the best. Paper: arxiv.org/abs/2502.08938

Reevaluating Policy Gradient Methods for Imperfect-Information Games

In the past decade, motivated by the putative failure of naive self-play deep reinforcement learning (DRL) in adversarial imperfect-information games, researchers have developed numerous DRL...

arxiv.org
Samuel Sokota @ssokota
14 Feb 2025
Model-free deep RL algorithms like NFSP, PSRO, ESCHER, & R-NaD are tailor-made for games with hidden information (e.g. poker). We performed the largest-ever comparison of these algorithms. We find that they do not outperform generic policy gradient methods, such as PPO. 1/N
4
10
60
13,668
Nathan Lichtlé reposted
Berkeley AI Research
 @berkeley_ai
Jul 1
We could not be prouder of this years BAIR PhD graduates! Follow their journey to see where they go next @baifeng_shi @sea_snell @databoydg @enfleisig @graceluo_ @zhuhl98 @HaozhiQ @jdzamfi @LianJiachen @joshminwookang @junhaobearxiong @kaylolittlejohn @KentKChang @kvablack @KunheYang @lisabdunlap @LongTonyLian @maulik__bhatt @michaelpsenka @nathanlichtle @neerjathakkar @nikita_mehandru @NiklasLauffer @qiyang_li @vbenara05 @vonekels @xiuyu_l @Yiheng_Li_Cal @ZheFu1 Sampada Deglurkar Wei-Jer Chang Yichen Xie Yigit Erginbas bair.berkeley.edu/blog/2026/…

2026 BAIR Graduate Showcase

The BAIR Blog

bair.berkeley.edu
2
12
123
25,623
Nathan Lichtlé reposted
Berkeley AI Research
 @berkeley_ai
25 Mar 2025
New Blog Post: Scaling Up Reinforcement Learning for Traffic Smoothing: A 100-AV Highway Deployment bair.berkeley.edu/blog/2025/…
3
12
51
18,957
Nathan Lichtlé reposted
Eugene Vinitsky 🦋 @EugeneVinitsky
14 Feb 2025
This paper has everything: large-scale empirical testing, new benchmarks, and subsequent empirical recommendations that we think might make solving imperfect information games a little easier
Nathan Lichtlé @nathanlichtle
14 Feb 2025
We ran 5,600 hyperparameter sweeps to compare RL algorithms on hidden-information games with billions of states. In our benchmark, we found that properly tuned policy gradient methods, such as PPO, performed the best. Paper: arxiv.org/abs/2502.08938
1
2
45
4,191
Tic-Tac-Toe... but the opponent's moves are hidden. Can you outsmart our top RL agents? Play here: nathanlichtle.com/research/2…
Nathan Lichtlé @nathanlichtle
14 Feb 2025
We ran 5,600 hyperparameter sweeps to compare RL algorithms on hidden-information games with billions of states. In our benchmark, we found that properly tuned policy gradient methods, such as PPO, performed the best. Paper: arxiv.org/abs/2502.08938
1
2
14
6,177
Nathan Lichtlé reposted
Joseph Suarez 🐡
 @jsuarez
9 Dec 2024
x.com/i/article/185380866896…
10
33
401
36,596
Coded in C with the amazing Raylib! Try it out at puffer.gg/assets/tactical/ga…
1
3
239
Fast turn-based tactical combat env I've been working on -- RL training incoming!
0:22
1
2
19
1,115
Nathan Lichtlé reposted
Amaury Hayat @Amaury_Hayat
17 Oct 2024
Can language models be trained to find solutions to as yet unsolved mathematical problems? The answer is yes! Check our new article 🙂 1/n
François Charton
 @f_charton
17 Oct 2024
Transformers can be trained to solve a 132-years old open problem: discovering global Lyapunov functions. New paper on Arxiv (accepted in NeurIPS 2024), with @albe_alfa and @Amaury_Hayat arxiv.org/abs/2410.08304 1/8
1
1
11
694
Another driving sim??
0:11
1
2
200
Nathan Lichtlé reposted
Eugene Vinitsky 🦋 @EugeneVinitsky
6 Aug 2024
We’re open-sourcing and arxiving GPUDrive, a GPU-accelerated 2.5D multi-agent driving simulator that runs at over a million FPS. Hundreds of scenes on one GPU means scalable multi-agent planning
19
142
915
70,975
Nathan Lichtlé reposted
Nathan Lichtlé @nathanlichtle
17 May 2024
Excited to share our work about the MVT, a large-scale highway field test for which we trained autonomous vehicles to smooth out traffic flow using deep reinforcement learning, then deployed our AI controllers onto 100 vehicles in busy morning traffic. Read more below! (1/n)
0:15
2
4
17
3,976
My amazing co-first authors: @JangKathy @EugeneVinitsky @aditshah00 Read more about our adventures in the RL team: 📄 Paper 1 (Controller design in simulation) - arxiv.org/abs/2401.09666 📄 Paper 2 (100 AV field test) - arxiv.org/abs/2402.17050
2
109
Extracting the data from dozens of overhead cameras from a week of experiments is a complex computer vision endeavor. Preliminary results suggest a trend of reduced fuel consumption behind AVs. Many more details and nuances available in the paper!
1
137
Going from simulation to real-world deployment is a massive hardware and logistical challenge! Thanks to our amazing collaborators (circles-consortium.github.io…), our RL controllers were safely and successfully deployed on 100 cars on a busy highway: we dubbed it the MegaVanderTest.
1
49
During training, we placed an agent behind a real-world trajectory and tasked it to optimize the energy consumption of everyone driving behind. The result: significant smoothing and fuel savings (>15%) on stop-and-go waves with only 4% of AVs! (Special thanks to our beloved PPO!)
1
1
153
Optimizing for both energy savings and movement is a careful balance! Cars shouldn’t stop (which is energy-optimal) or open excessive gaps. We introduced dynamic gap thresholds, giving AVs leeway to freely optimize for energy while being constrained to safe and reasonable gaps.
1
102
The trained agent only controls the speed of the current vehicle, like any adaptive cruise control system, but it attempts to do so in a smoother, altruistic way that benefits everyone on the road and not just the AV itself.
1
1
97
One priority was building controllers that can be deployed in the real-world on most recent vehicles. Our algorithms make decisions based only on basic sensor information of the vehicle right in front. No communication, no centralized server, only local observations.
1
1
93
Developing RL controllers for this required realistic, fast simulators. An easy way to get realism: use real-data. We collected trajectories from traffic jams and replayed them in sim, leading to generalization in more complex scenarios (eg. real life).
0:22
1
1
98
Load more