LessWrong posts by zvi

Benchmarks have never been less useful for telling us which models are best.

They are good for giving a general sense of the landscape. They definitely paint a picture. But if you’re comparing top models, like GPT-5.4 against Opus 4.6 against Gemini 3.1 Pro, you have to use the models, talk to the models, get reports from those who have and form a gestalt. The reports will contract each other and you have to work through that. There's no other way.

Thus, I try to gather and sort a reasonably comprehensive set of reactions, so you ca...

It feels good to get back to some of the fun stuff.

The comments here can double as a place for GPT-5.4 reactions, in addition to my Twitter thread. I hope to get that review out soon.

Almost all of this will be a summary of agentic coding developments, after a note.

Table of Contents

Make no mistake about what is happening.

The Department of War (DoW) demanded Anthropic bend the knee, and give them ‘unfettered access’ to Claude, without understanding what that even meant. If they didn’t get what they want, they threatened to both use the Defense Production Act (DPA) to make Anthropic give the military this vital product, and also designate the company a supply chain risk (SCR).

Hegseth sent out an absurdly broad SCR announcement on Twitter that had absolutely no legal basis, that if implemented as written would have been corporate murder. They have now is...

This was the worst week I have had in quite a while, maybe ever.

The situation between Anthropic and the Department of War (DoW) spun completely out of control. Trump tried to de-escalate by putting out a Truth merely banning Anthropic from direct use by the Federal Government with a six month wind down. Then Secretary of War Hegseth went rogue and declared Anthropic a supply chain risk, with wording indicating an intent to outright murder Anthropic as a company.

Then that evening OpenAI signed a contact with DoW,

I’ve been trying to...

I’ve been trying to find a slot for this one for a while. I am thrilled that today had sufficiently little news that I am comfortable posting this.

Gemini 3.1 scores very well on benchmarks, but most of us had the same reaction after briefly trying it: “It's a Gemini model.”

And that was that, given our alternatives. But it's got its charms.

Consider this a nice little, highly skippable break.

The Pitch

It's a good model, sir. That's the pitch.

Sundar Pichai (CEO Google): Gemini 3.1 Pro is her...

The attempt on Friday by Secretary of War Pete Hegsted to label Anthropic as a supply chain risk and commit corporate murder had a variety of motivations.

On its face, the conflict is a tale of three contracts and the associated working relationships.

This is the long version of what happened so far. I will strive for shorter ones later, when I have the time to write them.

Most of you should read the first two sections, then choose the remaining sections that are relevant to your interests.

But first, seriously, read Dean Ball's post Clawed. Do that first. I will not quote too extensively from it, because I am telling all of you to read it. Now. You’re not allowed to keep reading this or anything else until after you do. I’m not kidding.

T...

The Department of War gave Anthropic until 5:01pm on Friday the 27th to either give the Pentagon ‘unfettered access’ to Claude for ‘all lawful uses,’ or else. With the ‘or else’ being not the sensible ‘okay we will cancel the contract then’ but also expanding to either being designated a supply chain risk or having the government invoke the Defense Production Act.

It is perfectly legitimate for the Department of War to decide that it does not wish to continue on Anthropic's terms, and that it will terminate the contract. There is no reason things need be taken further th...

Events continue to be fast and furious.

This was the first actually stressful week of the year.

That was mostly due to issues around Anthropic and the Department of War. This is the big event the news is not picking up, with the Pentagon on the verge of invoking one of two extreme options that would both be extremely damaging to national security and that would potentially endanger our Republic. The post has details, and the first section here has a few additional notes.

Also stressful for many was the impact of Citrini's...

The situation in AI in 2026 is crazy. The confrontation between Anthropic and Secretary of War Pete Hegseth is a new level of crazy. It risks turning quite bad for all. There's also nothing stopped it from turning out fine for everyone.

By at least one report the recent meeting between the two parties was cordial and all business, but Anthropic has been given a deadline of 5pm eastern on Friday to modify its existing agreed-upon contract to grant ‘unfettered access’ to Claude, or else.

Anthropic has been the most enthusiastic supporter our military has in AI a...

A viral essay from Citrini about how AI bullishness could be bearish was impactful enough for Bloomberg to give it partial responsibility for a decline in the stock market, and all the cool economics types are talking about it.

So fine, let's talk.

It's an excellent work of speculative fiction, in that it:

Anthropic first gave us Claude Opus 4.6, then followed up with Claude Sonnet 4.6.

For most purposes Sonnet 4.6 is not as capable as Opus 4.6, but it is not that far behind, it would have been fully frontier-level a few months ago, and it is faster and cheaper than Opus.

That has its advantages, including that Sonnet is in the free plan, and it seems outright superior for computer use.

Anthropic: Claude Sonnet 4.6 is available now on all plans, Cowork, Claude Code, our API, and all major cloud platforms.

We’ve also upgraded our fr...

This was the week of Claude Opus 4.6, and also of ChatGPT-5.3-Codex. Both leading models got substantial upgrades, although OpenAI's is confined to Codex. Once again, the frontier of AI got more advanced, especially for agentic coding but also for everything else.

I spent the week so far covering Opus, with two posts devoted to the extensive model card, and then one giving benchmarks, reactions, capabilities and a synthesis, which functions as the central review.

We also got GLM-5, Seedance 2.0, Claude fast mode, an app for Codex and much more.

Claude fast mode...

Things that are being pushed into the future right now:

We are officially a bit behind and will have to catch up next week.

Even without all that, we have a second highly full plate today.

Table of Contents

(As a reminder: bold are my top picks, italics means highly skippable)

There was way too much going on this week to not split, so here we are. This first half contains all the usual first-half items, with a focus on projections of jobs and economic impacts and also timelines to the world being transformed with the associated risks of everyone dying.

Quite a lot of Number Go Up, including Number Go Up A Lot Really Fast.

Among the thing that this does not cover, that were important this week, we have the release of Claude Sonnet 4.6 (which is a big step over 4.5 at least for coding...

There really is a lot going on these days.

I held off posting this because I was trying to see if I could write a net helpful post about the current situation involving Anthropic and the Pentagon. Anthropic very much wants to help DoW defend our country and make us strong. It is clear there have been some large misunderstandings here about how LLMs work.

They are not ordinary tools like spreadsheets that automatically do whatever the user asks, nor would it be safe to make them so, nor do they predictably adhere to written...

Some podcasts are self-recommending on the ‘yep, I’m going to be breaking this one down’ level. This was one of those. So here we go.

As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my commentary. Some points are dropped.

If I am quoting directly I use quote marks, otherwise assume paraphrases.

Normally I keep everything to numbered lists, but in several cases here it was more of a ‘he didn’t just say what I think he did did he’ and I needed ext...

Some podcasts are self-recommending on the ‘yep, I’m going to be breaking this one down’ level. This was very clearly one of those. So here we go.

As usual for podcast posts, the baseline bullet points describe key points made, and then the nested statements are my commentary. Some points are dropped.

If I am quoting directly I use quote marks, otherwise assume paraphrases.

What are the main takeaways?

OpenAI is back with a new Codex model, released the same day as Claude Opus 4.6.

The headline pitch is it combines the coding skills of GPT-5.2-Codex with the general knowledge and skills of other models, along with extra speed and improvements in the Codex harness, so that it can now handle your full stack agentic needs.

We also got the Codex app for Mac, which is getting positive reactions, and quickly picked up a million downloads.

CPT-5.3-Codex is only available inside Codex. It is not in the API.

As...

Life comes at you increasingly fast. Two months after Claude Opus 4.5 we get a substantial upgrade in Claude Opus 4.6. The same day, we got GPT-5.3-Codex.

That used to be something we’d call remarkably fast. It's probably the new normal, until things get even faster than that. Welcome to recursive self-improvement.

Before those releases, I was using Claude Opus 4.5 and Claude Code for essentially everything interesting, and only using GPT-5.2 and Gemini to fill in the gaps or for narrow specific uses.

GPT-5.3-Codex is restricted to Codex, so this means that fo...

Coverage of Claude Opus 4.6 started yesterday with the mundane alignment and model welfare sections of the model card.

Today covers the kinds of safety I think matter most: Sabotage, deception, situational awareness, outside red teaming and most importantly the frontier, catastrophic and existential risks. I think it was correct to release Opus 4.6 as an ASL-3 model, but the process Anthropic uses is breaking down, and it not on track to reliably get the right answer on Opus 5.

Tomorrow I’ll cover benchmarks, reactions and the holistic takeaways and practical implications. I’m still taking it all...

Claude Opus 4.6 is here. It was built with and mostly evaluated by Claude.

Their headline pitch includes:

Other notes:<...

Claude Opus 4.6 and agent swarms were announced yesterday. That's some big upgrades for Claude Code.

OpenAI, the competition, offered us GPT-5.3-Codex, and this week gave us an app form of Codex that already has a million active users.

That's all very exciting, and next week is going to be about covering that.

This post is about all the cool things that happened before that, which we will be building upon now that capabilities have further advanced. This if from Before Times.

Almost all of it still applies. I haven’t ha...

Claude Opus 4.6 and agent swarms were announced yesterday. That's some big upgrades for Claude Code.

OpenAI, the competition, offered us GPT-5.3-Codex, and this week gave us an app form of Codex that already has a million active users.

That's all very exciting, and next week is going to be about covering that.

This post is about all the cool things that happened before that, which we will be building upon now that capabilities have further advanced. This if from Before Times.

Almost all of it still applies. I haven’t ha...

Remember OpenClaw and Moltbook?

One might say they already seem a little quaint. So earlier-this-week.

That's the internet having an absurdly short attention span, rather than those events not being important. They were definitely important.

They were also early. It is not quite time for AI social networks or fully unleashed autonomous AI agents. The security issues have not been sorted out, and reliability and efficiency aren’t quite there.

There's two types of reactions to that. The wrong one is ‘oh it is all hype.’

The right one is ‘we...

I had to delay this a little bit, but the results are in and Kimi K2.5 is pretty good.

Table of Contents

Official Introduction

Introducing Kimi K2.5,

Kimi.ai: Meet Kimi K2.5, Open-Source Visual Agentic Intelligence.

Global SOTA on Agentic Benchmarks: HLE full set (50.2%), BrowseComp (74.9%)
Open-source SOTA on Vision and Coding: MMMU Pro (78.5%), VideoMMMU (86.6...

First we must covered Moltbook. Now we can double back and cover OpenClaw.

Do you want a generally impowered, initiative-taking AI agent that has access to your various accounts and communicates and does things on your behalf?

That depends on how well, safely, reliably and cheaply it works.

It's not ready for prime time, especially on the safety side. That may not last for long.

It's definitely ready for tinkering, learning and having fun, if you are careful not to give it access to anything you would not want to lose.<...

Moltbook is a public social network for AI agents modeled after Reddit. It was named after a new agent framework that was briefly called Moltbot, was originally Clawdbot and is now OpenClaw. I’ll double back to cover the framework soon.

Scott Alexander wrote two extended tours of things going on there. If you want a tour of ‘what types of things you can see in Moltbook’ this is the place to go, I don’t want to be duplicative so a lot of what he covers won’t be covered here.

At least briefly Moltbook w...

Anthropic CEO Dario Amodei is back with another extended essay, The Adolescence of Technology.

This is the follow up to his previous essay Machines of Loving Grace. In MoLG, Dario talked about some of the upsides of AI. Here he talks about the dangers, and the need to minimize them while maximizing the benefits.

In many aspects this was a good essay. Overall it is a mild positive update on Anthropic. It was entirely consistent with his previous statements and work.

I believe the target is someone familiar with the basics, but who...

This was Anthropic Vision week where at DWATV, which caused things to fall a bit behind on other fronts even within AI. Several topics are getting pushed forward, as the Christmas lull appears to be over.

Upcoming schedule: Friday will cover Dario's essay The Adolescence of Technology. Monday will cover Kimi K2.5, which is potentially a big deal. Tuesday is scheduled to be Claude Code #4. I’ve also pushed discussions of the question of the automation of AI R&D, or When AI Builds AI, to a future post, when there is a slot for that.

...

The first post in this series looked at the structure of Claude's Constitution.

The second post in this series looked at its ethical framework.

This final post deals with conflicts and open problems, starting with the first question one asks about any constitution. How and when will it be amended?

There are also several specific questions. How do you address claims of authority, jailbreaks and prompt injections? What about special cases like suicide risk? How do you take Anthropic's interests into account in an integrated and virtuous way? What about our jobs?

This is the second part of my three part series on the Claude Constitution.

Part one outlined the structure of the Constitution.

Part two, this post, covers the virtue ethics framework that is at the center of it all, and why this is a wise approach.

Part three will cover particular areas of conflict and potential improvement.

One note on part 1 is that various people replied to point out that when asked in a different context, Claude will not treat FDT (functional decision theory) as obviously correct. Claude will instead say...

Claude's Constitution is an extraordinary document, and will be this week's focus.

Its aim is nothing less than helping humanity transition to a world of powerful AI (also known variously as AGI, transformative AI, superintelligence or my current name of choice ‘sufficiently advanced AI.’

The constitution is written with Claude in mind, although it is highly readable for humans, and would serve as a fine employee manual or general set of advice for a human, modulo the parts that wouldn’t make sense in context.

This link goes to the full text of Claude...

If there's several things this blog endorses, one of them would be going meta.

It's time. The big picture awaits.

You’re Single Because You Live In The Wrong Place

The most important meta question is location, location, location.

This is the periodic reminder that dating dynamics are very different in different locations, and gender ratios are far more uneven than they appear because a lot of people pair off and aren’t in the pool.

If you are a man seeking to date women, New York City is the...

Anthropic released a new constitution for Claude. I encourage those interested to read the document, either in whole or in part. I intend to cover it on its own soon.

There was also actual talk about coordinating on a conditional pause or slowdown from both DeepMind CEO Demis Hassabis and Anthropic CEO Dario Amodei, which I also plan to cover later.

Claude Code continues to be the talk of the town, the weekly report on that is here.

OpenAI responded by planning ads for the cheap and free versions of ChatGPT.

...

We’re back with all the Claude that's fit to Code. I continue to have great fun with it and find useful upgrades, but the biggest reminder is that you need the art to have an end other than itself. Don’t spend too long improving your setup, or especially improving how you improve your setup, without actually working on useful things.

The Efficient Market Hypothesis

Odd Lots covered Claude Code. Fun episode, but won’t teach my regular readers much that is new.

Bradly Olsen at the Wall Street Journal reports Claude [Code a...

A short fun one today, so we have a reference point for this later. This post was going around my parts of Twitter:

@gmltony: Go to your ChatGPT and send this prompt: “Create an image of how I treat you”. Share your image result.

That's not a great sign. The good news is that typically things look a lot better, and ChatGPT has a consistent handful of characters portraying itself in these friendlier contexts.

Treat Your ChatBots Well

A lot of people got this kind of result:

Eliezer Yudkowsky:

The main thing to know this time around is that the whole crazy ‘what is causing the rise in autism?’ debacle is over actual nothing. There is no rise in autism. There is only a rise in the diagnosis of autism.

Table of Contents

Autism Speaks

It has not, however, risen in prevalence.

The entire shif...

Good news, we managed to make some cuts. I think?

Table of Contents

California In Crisis

I’ve written about this before, but it turns out it's even worse than I realized.

California is toying with a 1.5% annual wealth tax on b...

Claude Code and Cowork are growing so much that it is overwhelming Anthropic's servers. Claude Code and Cowork news has for weeks now been a large portion of newsworthy items about AI.

Thus, at least for now, all things Claude Code and Cowork will stop appearing in the weekly updates, and will get their own updates, which might even be weekly.

Google offered us the new Universal Commerce Protocol, and gives us its take on Personalized Intelligence. Personalized Intelligence could be a huge deal if implemented correctly, integrating the G-Suite including GMail into Gemini, if...