Computer latency: 1977-2017
danluu.com/input-lag/
89
1,369
2,977
Active on mastodon.social/@danluu; also trying out bsky.app/profile/da. No longer read replies or notifications here now that tweetdeck is gated.
Joined December 2008
- Posts 5,581
- Following 43
- Followers 46,143
- Likes 1,585
910 Photos and videos
Exercises in benchmarking, evals, and experimental design, part 6:
patreon.com/posts/160473058
2
49
5,048
Nowadays, I often see people say they have access to unlimited tokens. Usually for work, although in response to x.com/danluu/status/20529245…, someone from Microsoft said they have unlimited tokens for personal projects.
I wonder what this means in practice. Surely there's a limit?
What do people like for token-intensive workflows for personal projects?
With a semi-autonomous workflow, I can exhaust a week's quota on the $200/mo codex plan in ~10 minutes of my time. If I'm in the loop, I get ~10x(?) the productivity per token, but it's time inefficient and
4
41
13,547
project that's using about 300M/hr. That one is running on a medium sized machine (64 cores, but old/slow ones) and is rate limited by CPU. It's less clear to me how productively that one scales up, but I think 10x for sure, and possibly 100x, with some more compute.
2
11
2,703
These are fairly autonomous, so I can run a bunch in parallel. If I worked for a company with "unlimited" tokens for personal projects, I could easily run multiple projects like above and spend > 1T/hr scratching personal itches.
Do you think they'd let me do it? Presumably not?
3
16
2,625
Why are so many people so sure that the big AI providers are losing money on inference? It reminds me of the comments about how Uber can never make money. Their unit economics were fine and they were only losing money because they chose to do so on customer acquisition.
57
14
278
21,928
It's not even obvious that inference should be money losing today. API cost for frontier models has gone way up and the people claiming a cost for inference are relying on made up assumptions that don't seem likely.
Widely cited folks, like Ed Zitron, aren't remotely credible.
11
3
62
5,736
The other widely cited reasoning is Garran's "AI has hit a wall" and similar. Every time people have said that, it's been wrong.
E.g., Garran wrote in 10/25. Since then, on $, 5.5 >> 5.4 >> 5.3. It's improved more than enough to allow massive price increases. There was no wall.
2
21
2,724
What do people like for token-intensive workflows for personal projects?
With a semi-autonomous workflow, I can exhaust a week's quota on the $200/mo codex plan in ~10 minutes of my time. If I'm in the loop, I get ~10x(?) the productivity per token, but it's time inefficient and
24
91
40,240
is to avoid burning a ton of money.
I've messed around a bit with having a more expensive model do planning with cheaper models doing implementation, but I'm not sure the savings is there with an autonomous workflow? Different story if I'm in the loop, but that takes time.
3
14
5,994
Since this is a common point of confusion, 10 minutes of my time != 10 min wall clock.
ChatGPT says multiple plans are allowed? chatgpt.com/c/69fe9b31-b6dc-…. There are stories of people getting banned, but at least it will be funny if I get banned after ChatGPT said it was allowed?
1
12
4,381
Can someone at OpenAI help me with an account issue? I'm doing non-security testing and constantly getting flagged for doing "cybersecurity" work.
I did the approval / ID scan on my personal account, but my work account is in an infinite loop and I can't use codex/GPT at work.
4
1
40
7,985
It's sort of amazing how quickly you can do things now. I wanted to try writing an alphazero-style AI for Azul. With no AI background, it took me maybe 2-3 hours to (2-3 days wall clock) to beat the best public AI I could find, training on my laptop CPU:
danluu.com/game/tile/
3
4
98
7,360
Looks like I spoke too soon about the AI not being superhuman.
The current Azul world champion played against it and thinks it's better than him at higher difficulties and a top 100 player played against it at default difficulty and thinks it's better than him at default.
1
15
2,255