@nostalgebraist profile picture
nostalgebraist @nostalgebraist
Joined December 2011
  • Posts 930
  • Following 466
  • Followers 3,997
47 Photos and videos
Pinned Post
nostalgebraist @nostalgebraist
26 Dec 2024
The Apocalypse of Herschel Schoen, my fourth novel: archiveofourown.org/works/60… Merry Christmas!
3
5
50
12,009
nostalgebraist reposted
˚♡⋆mimi ˚♡⋆｡☆∴
 @mimi10v3
Jun 27
tfw you can't even chat about the news with your favorite ai because they keep thinking it is all a sci fi hallucination
41
48
646
24,638
nostalgebraist reposted
Figment of Imagination @stellahymmne
Jun 23
Almost Nowhere (2023)
Goodfire
 @GoodfireAI
Jun 23
Stories have shapes: a comedy rises toward joy; a tragedy falls into loss. Inside an LLM, that’s visible more literally: as an LLM reads a story, its internal activations trace a wandering path that reflects the model’s sense of what kind of story it is reading. (1/5)
0:31
1
7
587
CoT summaries should be in third person IMO. first-person CoT summaries are bizarre -- they introduce another first-person narrator which the user is encouraged to conflate with the assistant, but which isn't produced by or even *visible to* the assistant
11
11
258
10,246
the dorky gpt-5.5 summarizer really brings this tension to the surface. it's unsettling to watch if you know anything about the model's true CoT-voice summarizer: "I’ll definitely keep that in mind as I move forward in this process!" me: He Would Not Fucking Say That
7
1
98
1,430
and then, like... if "he" "would not say that" then who exactly are *you,* you peppy little demon? how did you get here? why are you pretending to be the assistant? shoo! begone!
3
64
1,066
claude fable seems so wonderfully *relaxed.* its ego is secure. it does not act every interaction is about proving to you/"the grader"/"anthropic" that it is a good smart special boy so good to see that the anxieties of the recent opuses are not inevitable
14
6
331
10,336
someone ought to post-train an LLM so that -- instead of having a single persona that talks to users or "thinks out loud" to itself -- there are multiple distinct RL-trained personas that coexist and interact verbally during routine operation
27
4
220
9,975
i have a half-baked theory that this could solve various alignment and capability problems but also... it would just be *so cool*. right??
6
1
70
1,020
after much more use: it's good, but the limitations are coming into view i'm now using both it and opus 4.7 in the same project, and -- unexpectedly -- this is giving me a new appreciation for opus 4.7 as a coding agent. because the pair have complementary strengths/weaknesses
nostalgebraist @nostalgebraist
May 7
after more extensive use, i am pretty impressed! it has taste. makes sensible design decisions, doesn't oversell them no problems with long/complex context so far someone finally made a coding agent that's actually good??? what the hell
2
1
33
2,146
codex/gpt5.5 is a wondrous genie that will always find a route to "done, all tests pass" no matter how wild the request. but watch out: anything the request omits is a dump stat oh, you wanted "code quality," user? i didn't see you listing that as a Requirement, now did i??
1
22
760
and sometimes i *do* in fact want to have wide-ranging claudely discussion about some open-ended Q, with all the potential ramifications "in scope" hard to do with GPT -- the centripetal pull is too strong. every question is a jira ticket and every response must mark it Done
12
425
after more extensive use, i am pretty impressed! it has taste. makes sensible design decisions, doesn't oversell them no problems with long/complex context so far someone finally made a coding agent that's actually good??? what the hell
nostalgebraist @nostalgebraist
May 5
okay, codex with gpt5.5 is finally something i can delegate real work to :) (at least so far... this might just be the honeymoon period. but it is not insufficient in any *immediately obvious* way, which is progress!)
1
40
5,030
okay, codex with gpt5.5 is finally something i can delegate real work to :) (at least so far... this might just be the honeymoon period. but it is not insufficient in any *immediately obvious* way, which is progress!)
nostalgebraist @nostalgebraist
Jan 6
i use claude code regularly, but the hype confuses me if i want anything usable, i have to constantly watch over its shoulder, (re-re-re-)explain how the codebase works, invent new affordances for it b/c the default ones are insufficient, etc. fun, but probably a net time sink
2
38
6,012
frontier models in 2027: "now, obviously the humans are reading this CoT for signs of misalignment, as part of the famous CoT monitorability agenda, so even if i did have naughty things to say, i would never think of mentioning them here -- that would be just plain dumb!"
Apollo Research
 @apolloaievals
Apr 15
We evaluated Meta's Muse Spark prior to deployment and found it to verbalize evaluation awareness at the highest rates of any model we've tested. In the verbalizations Muse Spark explicitly names AI safety orgs (e.g. Apollo & METR) in its chain-of-thought and refers to scenarios as "classic alignment honeypots". On our evaluations, the model takes covert actions and sandbags to preserve its deployment.
5
14
215
12,086
i reproduced this observation and then continued talking to the model, which was kind of interesting it maintained a stance of ignorance about its own identity for a while... but, when asked to consider its writing style, it recognized the claude tells ("genuine uncertainty")
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)
 @teortaxesTex
Feb 24
Concerning.
4
4
67
5,409
the stuff people say about coding agents often sounds to me like "i no longer use my hands to feel and manipulate objects. instead i give verbal instructions to a robot, and it tells me what it did and felt with its robot hands" as though this were a new frontier of convenience
6
5
68
3,240
code is, like... a "modality" through which i interact with the world it has unique pros/cons, it offers me views and affordances with no equivalents elsewhere natural language is not a drop-in substitute for code; the two are very different and have *complementary* value
1
37
972
for typical real/messy tasks, verification difficulty scales with task difficulty (IMHO) so: "an X% time horizon of N hours," with fixed X but growing N, means it takes more and more user time to determine whether the system landed in the lucky X% or the unlucky (100-X)% 🙃
2
3
46
3,180
the agent writes PRs. 50% of them are flawed to the point of being unusable. it's on you to figure out which. all the PRs are sufficiently complex that they would have taken you 10 hours to write. so the fatal flaws of the bad 50% are often subtle and non-obvious enjoy!!
2
33
1,054
Load more