Joined January 2022
161 Photos and videos
Pinned Post
My new research piece: what the politics of jobless prosperity might look like in an AGI world, why the real political backlash to AI hasn’t started yet, and how the labs should prepare. 1. The backlash to AI isn’t here yet. There is anxiety among American voters, but there is no populist backlash yet, because the job losses haven’t started yet—and we don’t even know if they ever will. AI is not in the top 20 issues Americans say they care most about, and the AI policy issue with the most energy right now, data center opposition, reflects not just AI but also NIMBYism, as @mattyglesias has pointed out. 2. Real backlash will happen if and when unemployment climbs by two percentage points, because that’s where data shows we tend to see meaningful electoral effects of unemployment. At that point, if we do not have a good inventory of smart policy ideas ready, we could be overwhelmed with bad ones. 3. The labs should focus more on measurement, and less on dreaming up New Deals. There is tremendous uncertainty about what kind of job displacement there’s going to be. Instead of attempting to write a new social contract from the top down before Americans are even asking for one, the labs should be helping us all get more intel on whether, when, and how job displacement is occurring—building from the helpful data sharing they’ve already started piloting. This will put society in a better position to design policies that make sense for everyone. In doing the research for this piece, I came to two broader realizations. First, there is way more uncertainty than I appreciated about how the economics of AGI might play out, and there is stronger evidence than I appreciated that job losses from AI have not meaningfully started yet. And second, if AGI plays out the way the labs are predicting, the politics will be very hard to forecast, because it will be the politics of “jobless prosperity,” with jobs falling while the economy grows. We have very little experience with this happening at this kind of scale, and it will break our typical models of politics. For both of these reasons, we should all be really humble in making pronouncements about the politics of AGI. I hope my piece will be read in this light, as an attempt to reason about something that is super important but also super hard to forecast accurately. You can check out a lot more in the piece here: freesystems.substack.com/p/t…
23
35
200
76,309
Andy Hall reposted
I’ve always found people who bristle at “American exceptionalism” kind of… weird. Not because I lack self-awareness — I’ve spent my career cataloging every way this country fails to live up to its own rules. But that’s exactly why I love it so damn much. We built a system designed to be shamed by its own founding documents, and it still delivered one of the most spectacular, world-altering runs in human history. A genuine force for human flourishing. I also found the argument against American exceptionalism to be historically illiterate. Here’s a sample of what we were first at: • The first large-scale democratic republic in human history — not a city-state, not a monarchy with a parliament bolted on, but a bold continental experiment in self-rule, popular sovereignty, and ordered liberty. • A written Constitution (1789) with separation of powers and checks & balances — still the oldest national constitution in force anywhere. • The Bill of Rights (1791): the first time a nation wrote “the government cannot touch these” into supreme law and actually meant it. A dare the world copied — from later rights charters to the Universal Declaration of Human Rights. • Public land-grant universities and mass higher education (Morrill Act), opening college to ordinary people no aristocracy would have let near the gates. (but don’t get me started about what happened after we started. Massively federally funding it.) • Kitty Hawk, 1903 — first controlled powered flight. • The Moon, 1969 — still the only ones who’ve been there. • The world’s largest economy since ~1890, powering unprecedented prosperity through grit and genius. • The assembly line, skyscraper, transistor, personal computer, ARPANET — the backbone of the modern world. • Telephone, phonograph, GPS — connecting and powering daily life. • Surgical anesthesia, polio vaccine — saving and transforming millions of lives. • Jazz, blues, rock ‘n’ roll — brand new American art forms that conquered the globe. • Hollywood’s dreams, blue jeans, bourbon, and a culture so open a kid like me could devour sushi, burritos, stuffed cabbage, and tabouli in the same week and rightfully think of it all as American. That’s the part that fills me with genuine love and pride: not just the power or the wins, but the appetite for freedom, creativity, and reinvention. The audacity to say “We the People” and keep trying to live up to it. What do you love most about this truly exceptional country? 🇺🇸
149
881
4,444
461,649
Andy Hall reposted
Replying to @ahall_research
If we are lucky we might finally get systemic news accuracy rates. A key epistemic affordance I’ve wanted for a long time, but it’s always been too expensive. jonathanstray.com/measuring-…
1
1
2
592
This is a super important and deep concern about the limits of agentic representatives. Inattention and abstention can be very important parts of how people delegate in a democracy. Agents that artificially remove that might make many political problems worse, not better. This gets back to one of my main questions about how we design good agentic representatives---they have to do more than naively mirror our unconsidered preferences. I was mostly thinking that means pushing back on poorly considered ideas we have that don't actually match our underlying preferences, but Brad's comments show that there's another very important aspect, which is developing a notion of how important an issue is to the user, and knowing when to push hard on the issue and when to abstain. More broadly, Brad also raises the important point that democracy is more than a technological engineering problem---it's inevitable that we will shift the technology we use to do democracy and that agents will be a part of it in the long run, but we shouldn't expect that to magically solve fundamentally human problems. So much more to think about here!
Replying to @ahall_research
Some Saturday morning musings, as I await the World Cup: this is an interesting approach and one that's really worth thinking about. It gets a lot right in saying that you don't want concentrated power in control of these agents. That an arguably important problem in politics (if not the most important one) is that of principal-agent misalignment. That you wouldn't want your agent to simply be a digital twin that mirrored both your best and worst impulses, but rather should be something that interrogates the latter. All good. That said, it viscerally strikes me as problematic, and while I don't have a nuanced theory as to why I find it so troublesome, perhaps it becomes down to discomfort with trying to make the messy part of politics simply a design failure rather than the residue of legitimate, incommensurable bargaining that can't be reduced to an algorithm. It all seems a very Silicon Valley thing: recasting a tragic or constitutive feature of politics as a tooling problem. Stated slightly differently, the argument seems to assert that the major problem in American politics (or perhaps politics in a democratic regime more broadly) is a failure of representation technology, i.e., that we elect people who inadequately reflect the actual public will and who can pass policies that represent that of a concentrated minority rather than the general public, or perhaps even pursue some maximization of their own preferences at public expense. But is this the major problem? And, if so, can it be solved without introducing a more serious principal-agent misalignment? It seems to me that selective inattention is part of a democratic equilibrium, and I would worry if my agent were to represent every preference I have in every particular debate. Let me give you an example from my time in Congress. I was asked about overturning the Clinton administration's prohibition on snowmobiling in Yellowstone. I will confess that, representing rural eastern Oklahoma, I had no strong views about snowmobiling in Yellowstone and actually never even contemplated the topic. Over the course of a couple of beers, I am sure I can offer some views about the merits of snowmobiling, all ones not particularly well tutored. It would probably lead me to think that snowmobiling should be limited or even restricted in Yellowstone. This take is basically relying on my casual, almost aesthetic preferences, rather than any deep philosophical or political economy view. So would I want my agent to fight for this at the 4 a.m. debate over snowmobiling in Yellowstone? Absent my agent, I would probably pay no attention to the question at all and leave it to be fought out among local stakeholders: businesses, environmentalists, ranchers, people who live nearby, affected Indian tribes, animal welfare specialists, conservationists, and more. Each of these people would have more considered views and a much greater stake in the outcome. They are far more likely to get it right than my rather gratuitous opinion from a small town in eastern Oklahoma about what the fate of snowmobiling in Yellowstone should really be. So the assertion that a major problem in politics is misalignment of principal and agents seems to suggest that a major deficit of democracy is something along the lines of "I have preferences, but I lack the time and information, and so those preferences are not adequately expressed." But it seems to me, as in my Yellowstone example, a truer description might be "I have no serious view, no stake, no developed judgment, and no real claim to intervene." So having my constantly vigilant agent transform my faint, half-baked, low-salience attitudes into an active political force seems perverse. It makes actionable what is, in fact, almost complete indifference. Maybe I shouldn't be involved in questions of Yellowstone management (even if, as a federal park, I have standing to do so)? If my agent infers a bundle of preferences about how Yellowstone should be handled, then it actually disrupts the democratic process, which today, through the mechanism of salience, weighs interested parties' views more heavily. So the real problem is my agent might represent me too well. My low-stakes preferences become of equal weight to the people who are most directly affected by it. My views on Yellowstone should, properly in a democracy, remain weak and deferential. Sometimes not caring is the proper political approach. Of course, you might argue that the agents can develop some kind of model of salience that would solve the problem that I just discussed. This seems possible, and indeed you gesture at it when you talk about the agent not simply being a digital twin. The agent presumably could act on some kind of rule that says only support my preference when the issue reaches a particular salience threshold and disregard other times. This is where, if this is the solution, the implementation gets really murky because salience is a morally loaded concept. Sometimes issues are not salient to me simply because I am being negligent. Sometimes it is humility. Sometimes distant interventions are paternalism, and sometimes outside interventions are necessary to save local communities from themselves. It's very hard to know. So the Hallian agent must be able to discern considered conviction from gratuitous opinion, moral necessity from meddling, and decide when to represent my opinion, when to reinterpret my opinion, when to mute my opinion. That's why creating such "democratic agents" is about solving more than a principal-agent problem driven by the triage of attention. It is a deep question of political and moral philosophy. This comes back to why it seems to me, perhaps unfairly, to be such a Silicon Valley approach. It takes the most fundamental questions of human existence and tries to find a technical solution to them. Creating agents that can engage in democratic deliberation seems hard to the point of impossible. And it might even be dangerous: every political question becomes a universalized conflict among all possible preferences.
7
2,624
Andy Hall reposted
After I moved from America to Lebanon, my mom made it a goal for me and my brother to go back. “The future is there” she’d say. She made endless sacrifices to make that happen. She was right 🇺🇸
Happy 250th to the ultimate underdog: America. 250 years of American predictions: kalshi.com/america250
25
17
686
180,460
Freedom in the post-AGI world means building political superintelligence with tireless, brilliant political agents who represent us, the people—not governments or companies. In a special July 4th issue of System Check, I get into what this might mean. Several forces tilt the post-AGI world toward totalitarianism: the concentration of resources required to train frontier models, AI's obvious uses for surveillance and control, and existential risks that could justify extreme security states. But AI is also the biggest opportunity to upgrade democracy since the printing press. Most of our governance failures happen because citizens are too busy to pay attention, so a small group of highly motivated wackos drives the process. (See: NIMBYism.) What if that changed? What if AI could give every person a super effective political agent that represents them all the time? @gwern 's new "Guardian Angels" essay is the most serious technical sketch I've seen of that agent—one that learns you deeply, remembers everything, and can carry out "direct democracy on unprecedented scale." His most vivid example: official GAs for every member of Congress, able to simulate a roundtable debate among hundreds of politicians within minutes, or convene an emergency session at 4AM while every human is asleep. I see two big open questions. First: the agent has to be more than a digital twin. It should share your values without freezing your less-considered opinions in amber — willing to push you on topics it has studied more deeply than you have. On contested political questions, AI models don’t seem to possess that capability yet. Second: who governs the guardian angels? Gwern proposes a startup with dual-class founder shares. Sensible for the development phase. But can we build democratic infrastructure on private rails that one company controls forever? Which is why the recent attention back toward open-weight models and orgs that own their own models matters (see the great interview between @satyanadella and @ypatil125 below). The same logic driving firms to want their own models applies to democracy, too. If we're going to own our agents—agents that answer to us, and can't be secretly commanded from afar—we may need models we can run ourselves. The counterarguments to the open model idea (RSI leaving open models behind, safety pressure on open weights) are really big though, and I really have no idea how this is going to play out, or even how it should play out. How are we going to run a democracy if every citizen's agent is built on a single closed model with a single point of control? That's the question to think about this Independence Day. Check out the full piece here: freesystems.substack.com/p/g…
"There should be as many models in the world as firms in the world." Satya and I dig into when to own vs. rent your intelligence, why every company should be building and climbing its own private evals, and what makes for a stable frontier.
23
27
141
122,830
Andy Hall reposted
Companies check their own work through various internal but independent functional units: QA, security red teams, model risk management in banks. **I think it’s time for AI evaluation to become one such unit.** Orgs deploying AI should stand up cross-functional eval teams with their own reporting line. Many reasons: 1) Evals as IP / moat. It’s now widely recognized that evals are the new IP. So it makes sense to have teams whose primary focus is on creating and widening this moat. 2) Evals are harder than you think. This is less well recognized but as someone whose research centers on AI evals this has been my consistent experience. It can't be an afterthought and must be a center of excellence. 3) Evals are inherently cross-functional and require a distinct set of skills. They are judgment heavy, require both AI expertise and deep domain expertise, as well as customer understanding and sophisticated thinking about risk. To do them well, you need competence in data science & stats, business operations, product/customer experience, IT, risk management, and even compliance (depending on the sector). 4) In-house but independent eval teams keep companies honest. A climate where teams are getting top-down mandates to hit deployment targets and show results has resulted in a culture of companies fooling themselves. It is extremely easy to knowingly or unknowingly to do evals poorly, making your AI deployment look much more successful than it is. Eval teams who don’t share the deploying teams’ KPIs are the best defense against this.
15
11
65
11,990
Prime age participation rate still around 90%
JUST IN: 1 in 3 men aged 20 are neither working nor looking for a job in the US
4
3
43
13,223
This is exactly right: it’s all going to come down to how we balance locking down for safety vs staying open for freedom, innovation, and growth I have no idea what the right answer is and I don’t think any of the loud voices arguing for safety or open source do either
Replying to @LRudL_
Another core thing: in order for diffusion to be possible, misuse risk needs to be kept low. The arguments for locking down run through safety threats; we need to solve those threats if we want to avoid securitization & centralization. History warns any securitization will likely be stickier & more random than people hope, as we wrote last year
1
1
14
5,447
This is really cool — a great model for what serious introspective journalism can look like with AI
IS THE ECONOMIST ALWAYS WRONG? Scandalously, in some circles @TheEconomist has a reputation as a contrarian indicator. This week we fessed up to getting a big call on oil prices from April wrong. Obviously our goal is not perfectly-hedged (and perfectly boring) predictive accuracy: often it is to stimulate, provoke, and challenge. But I did want to test that wider allegation, so I ran a series of LLM scorers across our full leader database since 2000 (7,000 leaders in all.) You can see the results in the chart below: each dot is one of the 1,400 leaders where we identified concrete and falsifiable predictions that were central to the argument. Higher = more accurate, further to the right = more contrarian. We do well, unsurprisingly, when aligned with conventional wisdom. We often do worse when truly out on a limb. But actually, on average, we are a bit likelier to be right than wrong on our somewhat-out-of-consensus calls. All round, a respectable performance. And as @ecurrnomics points out an accompanying leader, there is no shame at all in being beaten by the market: as good free-marketers we believe deeply in the aggregated wisdom of prices. Take a look at my piece here, which includes a canter through our best and worst calls of the last quarter-century: economist.com/interactive/fi…
1
22
335
56,432
This is a key concern. The saving grace might be that this process is dynamic. Democracy can take steps along the path towards this outcome to prevent it from coming to pass. But just because we can doesn’t mean we’ll get our act together to do it. Lots to study here.
SITUATION EXPLAINED: How do we build democratic institutions that stay robust in a post AGI society? @jasonhausenloy, AI policy researcher at @CAIS: "What is the foundation of democracy? Why is it that we have the democratic institutions that we do? It is ultimately because humans provide value and democracy is a way to organize those humans so they continue to provide value." "Humans can opt out of society and that would be terrible for the governments that manage them. They provide their labor to the economy and their physical strength towards the military." "If you're able to have these things split apart in pretty rapid succession, then what you're hoping on is the reliability of these institutions that have lasted 250 years in the US to continue. And I actually don't think that they are so robust." "Whether that be in five years or twenty years, both of which seem totally reasonable, and in the grand scheme of things a very, very short amount of time, we will reach an end state where this technology does allow for the split of where humans' value comes from and how they can contribute in democratic systems."
3
3
25
3,174
Andy Hall reposted
SITUATION EXPLAINED: What did a Stanford classroom experiment in AI voting reveal about the future of democratic governance? @ahall_research, Professor at @StanfordGSB: "We did an experiment in class. Everyone had a Claude Code subscription and an open router API key. We took a set of proxy votes. JPMorgan and other institutions have fired their proxy advisors and announced they're doing all their shareholder voting through AI." "We wanted to do this experiment where every student built an AI representative who then voted on a set of real proxy votes. Did they vote the way the student would've wanted?" "What we found was, A, it's actually pretty hard to get the agent to understand your preferences, but it is totally doable. A couple students developed strategies that worked super well." "In a world where agents are doing all the voting, the people who write the proposals, the bills that are being voted on, will start to write them in ways that fool your agent. We have a lot of work ahead of us to make this robust."
1
2
53
8,120
Super fun to go on @MTSlive to discuss the political economy of the post-AGI world. It’s all about building political superintelligence.
SITUATION EXPLAINED: Why could post-AGI political agents fix the asymmetry that makes bad governance happen? We asked @ahall_research, Professor at @StanfordGSB "If you can use super powerful AI to give every person a super effective political agent that represents them all the time, I think you can actually go really far with that." "A lot of our biggest mistakes in governance today are the result of the fact that most of us are way too busy to pay attention, and then a small group of wackos gets to drive the decision-making process. And certainly in local politics, that's the dominant problem. Think about NIMBYism. That's totally what drives it." "I think we could totally rethink the structure of government around AI in ways that would be super exciting." "The problem will be we probably can't do that if we're talking about one closed weight model that has won out. How are we gonna run a democracy if all of our agents are built on a single model that has a single point of control by some CEO and or the government?" "We need that plurality of models. And we need to own our agents. The agent needs to respond only to us, not to the model company."
2
6
49
7,236
Legendary!
Life update: I've joined @the_IAS as a Professor in the School of Mathematics! Very excited to be part of the Institute’s growing commitment to theoretical computer science, its connections to other fields, and its importance for understanding and shaping modern technology.
12
3,033
Exciting opportunities for people who care about evals and the role of AI in society
Epoch is currently hiring for many roles, I recommend people take a look to see if they would be a good fit for any of them! I think we are already the world-leading source for many important questions on AI, and hopefully this is only improve as we expand.
2
12
3,356
“If I cut off my arm, I will probably learn lots of new ways to do household chores. I don’t think that’s what we mean by innovation. It’s certainly not the path to actual increased productivity.”
Replying to @ahall_research
We had this idea floating around with tariffs. “Oh if we hamstring ourselves, we will innovate.” There are some anecdotes of that but it is not a compelling argument on the theory or general historical evidence economicforces.xyz/p/will-ta…
3
32
4,136
Andy Hall reposted
Replying to @ahall_research
I don't have a strong take either way, but for the sake of discussion I'll try to steelman the other side: AI development requires making choices between lower risk and higher risk paths. Lower risk: Just use your talent and compute to scale like everyone else with predictable outcomes vs. Higher risk: use your talent and compute in unconventional ways that don't have the same guarantee of success but potentially outsized returns. Export controls pushed China into a higher risk/reward position which may actually have been better suited to their particular situation. A historical example would be the Soviet's rush to match US thermonuclear capability. Lacking the specialized engineering, compute and experimental data to match US nuclear bomb mass to yield, the Soviets decided to throw what resources they had (great aerospace engineers like Koralev) at building bigger rockets. This lead to a situation where the US had higher yield nukes and the Soviets had much great total mass to orbit capability until the 1990s.
1
1
4
994
Interesting. I keep seeing this argument that paradoxically export controls will make China *better* at developing AI---but I haven't yet seen a good explanation for why, if that were true, China wouldn't simply choose to develop AI without imports regardless. If it makes them better at AI, then what's stopping them? There's a good analogy in baseball. Sometimes when there's a runner on first and he steals second base, the announcers will say, "oh he shouldn't have done that, because now the pitcher has a 'free base open' and can pitch harder to the batter since he's no longer afraid of walking him. But of course, if the pitcher is fine having a runner on second, he could already pitch that way without the runner stealing second! The logic doesn't really make sense on its own, unless you have some particular theory of irrational, myopic behavior. Perhaps the availability of NVIDIA chips is a Siren song that AI companies can't resist, and they're better off unable to access this addictive drug. It might be plausible but I feel like it's worth spelling that out more clearly.
Meituan, basically China’s DoorDash, trained a 1.6T parameter LLM on 50K Chinese chips. It reminds me of Jensen Huang’s point on the Dwarkesh podcast: export controls on Nvidia GPUs won’t stop China. They’ll just accelerate the development of AI that runs on Chinese chips.
8
4
26
6,946
The primary rhetorical strategy for political fundraising in America is to single out villains to whip up outrage and get the small dollars flowing. Since 2025, the Democrats have decided: billionaires are the villains.
3
3
32
2,405
Phenomenal stuff
imagining a 74 year old judge reading this after a long week
1
12
1,915
Governance by stock market and the AI TACO trade continues to be an understudied phenomenon freesystems.substack.com/p/a…
Trump is gonna have to ban the Chinese models just like the Chinese cars are banned. Our entire stock market hinges on the AI trade and there is no way he cannot protect that x.com/yuhasbeentaken/status/…
1
7
1,351