https://xkcd.com/808/ . I think the XKCD undersells the ability of capitalists to chase dumb ideas, especially in the short term. Like Theranos was only a step or two up from Homeopathy in terms of bad science?

VR did get a preemptive attempt to enshittify it with the Meta-verse, and the tech isn’t quite there, but it is coming along. 3d printers are nice, haven’t been fucked up yet as far as I known.

It was something stupider and sillier yeah. The link explains, but to tldr; exp(some complicated term) where the “some complicated term” has a division by a number that eventually goes to 0. I think it was… them trying to represent “horizon time” (as in that one stupid METR graph that gets repeated everywhere) increases happening faster and faster?

Actually it was literally an asymptote in the first version of their models they posted to github. They had a term that they were dividing by that ends up going to zero, so the entire equation breaks! https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-critique-of-ai-2027-s-bad-timeline-models

So, now we can answer the question from before: what time horizon does the red curve above predict for 2030?

Well, if you plug it into wolfram alpha, you get a time horizon of “-2542 - 11372i”

Yes, that is negative, and that is an imaginary number in there.

What’s actually happening is that when the term in the brackets above hit zero, we end up dividing by zero, and hit infinity. Beyond that, we get a negative number to the power of a non-integer, which gives nonsensical complex numbered answers.

The infinity point for this equation happens at a time of t = To/alpha, and will always occur at some point, no matter what initial parameters you use.

In fact, this infinity point is completely independent of both the time horizon and the SC threshold. If you keep the same alpha and doubling time, you could start with a time horizon of a nanosecond, and have a superhuman coding threshold of 1 trillion years, and the curve will still claim that superhuman coding will arrive before the end of 2029.

And indeed, reddit user mambo-12345 tried modifying the simulation parameters to drop the initial time horizon to 15 nanoseconds instead of 15 minutes, and the resulting curve still had a peak for the estimated superhuman coder arrival at 2026.5 years and a median at 2035. Credit to them for inspiring me to take a closer look here:

Judging by the way LLMs and generative AI has been used, even when we do get something vaguely like a realistic version of sci-fi fantasies, capitalism and people who can’t distinguish the limits of the real thing from the fantasy will manage to misuse it.

Is this the thing where the baked an asymptote to infinity into their “model”? It was both bad modeling and also made all of their other input “data” completely irrelevant. I remember someone pointing this out at the time AI 2027 dropped, but there was so many other problems with it in the sneer spaces we didn’t talk about it too much among all the other problems. And someone on lesswrong actually pointed it out, but somehow, “we accidentally baked an asymptote into our model that is supposed to be detailed and carefully thought out” wasn’t a breaking point for lesswrongers.

I don’t have to be thrilled about death to have a basic understanding of how basic our understanding of consciousness, biology, and neurology are.

I think its the mid-wit meme, lesswrongers know just enough to imagine some grand solution like brain uploading or nanobots might cure death and not enough to appreciate all the reasons each of those solutions won’t work.

Lesswrongers go full panpsychism: https://www.lesswrong.com/posts/wCY4oNHpH2HGYGSoP/the-user-in-the-llm-might-be-a-moral-patient-too-a-strange

To try to TLDR this multilayered bit of crazy… lesswrongers already theorize LLMs might be sapient and/or have morally meaningful internal experiences. Except… LLMs obviously “behave” wildly differently when prompted differently and especially when put into different “agent harnesses”… so the big brain follow-up thought is what if the different personas an LLM can exhibit are all different “people” (for the purposes of utilitarian accounting and/or what their internal experiences are like… the exact “theory” can vary by lesswronger). The even more insane full galaxy brain follow-up thought: what if the user’s input to the LLM is forming yet another “person”, like a copy/reflection/simulacrum of the user!

TLDR for the TLDR: What if not only the LLM was sentient/sapient, but also held a sentient/sapient simulacrum of the user inside!

Yep, so many of their favorite tropes and assumptions fail when an actual expert breaks them down. Like their belief in the possibility of Drexler-style nanotech has been taken down quite firmly several times (here is a good takedown that got posted to the EA forums), Drexler’s ideas pretty firmly ignore how things actually work at the molecular level…

What fullsquare said, they’ve got years of sci-fi filtered through lesswrong short stories and sequence posts saying it could happen, so they repeated that despite all the ways it is completely impossible given the limitations and hardware requirements of LLMs.

One of the classic sequence posts plays with the concept: https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien-message

But in a uno reverse twist, its the humans that are a metaphor for AI, with them manipulating out thinking the aliens from within the simulation.

Someone on Scott Aaronson’s blog suggested math will become an empirical field now, where they run “experiments” by asking AI for lean proofs.

Scott, is this a turning point where mathematics gets off its well trodden path of logic and becomes more like an empirical science? Heck, people were arguing if math is invented or discovered, this settles it. Math is discovered – by AI! From now on forth, Mathematicians will become scientists, wielding AI as a telescope, pointed towards new and disturbing alien worlds in the Platonic realm.

So yeah, it seems like this idiotic view is pretty popular among boosters.

The theory on the thread that

My guess is that this represents a shift in focus at OpenAI away from math: they had their big meaningless trophy prize with the Navier-Stokes solution, they appear to have distanced the competition (Anthropic) and the public bought the notion that “math is solved”

seems like a plausible explanation? I’ll guess we’ll if OpenAI moves on now that they’ve “solved” math or if they follow up with a few more drops of “proofs”.

True, but its a different form of disrespect, where they outwardly steal the style and aesthetics of the discipline to try to claim its legitimacy. Like the endless “whitepapers” pumped out by cryptobros.

I do not see the point of math for AI alone, since I do not think AI is or will ever be conscious.

Rationalists (and in general those with fear/hope or a technological singularity) have the idea that once the AI is smart enough, it will be able to recursively self-improve in an exponential process. Vibe-coding hasn’t made this come true, so they are trying to make it happen by shoving enough of the right sort of knowledge into the AI. Obviously being smart and being mathey go together, so if the AI just gets good enough at math, surely bootstrapping to godhood is just around the corner!

To them, the AI is not here for us, but we are here to serve the AI.

The techbros are giving mathematicians the humanities treatment. (I.e. they don’t respect the entire profession despite its immense importance to actually understanding reality.) Before, mathematics had sort of gotten half a pass for being rigorous and adjacent to stuff techbros do respect, like computer science. But, given just half a whiff of the idea that they may no longer need human mathematicians, they are ready to accept utter slop…

Place like /r/singularity and hackernews are eating up the hype and buying into it. Even with mathematicians explaining carefully the errors in the proofs and why they are badly written and useless to actually progress in mathematics the boosters are treating that like sour grapes or irrational fear. If mathematicians tried outright ignoring it, OpenAI’s narrative would go even more unchallenged.

You know how AI proofs are supposed to be credible because of formalization with Lean?

There is another problem… apparently Lean4 hasn’t actually been fully formally verified. And in fact could have more bugs lurking in it, maybe even some foundational problems… Lesswrong post explaining the problems.

OpenAI can more easily market that their AI is intended to fully replace mathematicians, and that this is only the herald of their general superintelligence (and not that math is the only thing working out for them).

Yeah, this has been driving me crazy… reading places like hackernews or r/singularity, I keep seeing people assuming mathematicians and just bitter or engaging in sour grapes and that is why they are complaining about the “proofs” OpenAI dumped out. Like it is just some small niggling detail that the proofs can’t be read by humans and that Lean may have bugs and that some of OpenAI’s proofs have already been shown to have critical mistakes.

Edit: I had the thought to collect some links in one place Here are some more discussions on lean and autoformalization for anyone looking for good sources:

• https://terrytao.wordpress.com/2026/10/09/what-mathematicians-should-know-about-the-lean-theorem-proverquestions-of-reliability-and-ai/ a general blog post breaking down some of the bugs found and potential for more
• https://arxiv.org/pdf/2610.08144 preprint discussing the problems with natural language to lean in detail and pointing out some blatant inconsistencies in the recent Navier-stokes “proof”
• /r/betteroffline discussion of above preprint: https://www.reddit.com/r/BetterOffline/comments/1x0b49i/navierstokes_lost_in_translation_why_lean/

Edit 2: Made a top level post: https://awful.systems/post/9997783

Or cite it, but in the most passive aggressive way you can get away with.

…You should check the literature really carefully to make sure that whatever proof the AI “generated” doesn’t actually already exist somewhere and it regurgitated it without proper accreditation.

Yeah lesswrong is weird about China. Its an ongoing thing. It alternates between fear (that China won’t appreciate the risk of making AI Satan or will beat the US to making AI God) and occasional hope (that they might manage to “safety-pill” China and China will find the magic formula to AI alignment). I pulled together some posts previously here: https://awful.systems/post/4103825

If it had been only a year they’ve been saying that, I might almost buy the logic. But it has been 4 years since Chat-GPT came out and they’ve been making the same promises the entire time.

Something of a tangent… thinking back to 2022-2023 my expectations for what the LLM companies would actually achieve by now were actually too high? Like I would have thought they would have figured out a better architecture for gluing functionality and symbolic logic to LLMs than just calling tools with the output text stream and feeding the tool outputs into the context window.