People are using large language models more often and on more complex tasks than ever. Social media and marketers hammer us with the (real or dreamed-up) benefits of this uptake from all sides, but there are also voices pointing out the risks: we're told that our security is compromised, that intellectual property is undermined, and that—in case we didn't have enough already—more biases are being pushed onto us. These are all valid concerns that need to be addressed as soon as possible. But, as someone who calls himself a "thinking-tool artisan", I'm particularly concerned by two gnarly issues of generative AI:

1. There isn't strong evidence of major productivity gains from AI investments in organizations. Many individuals seem to be more productive on some tasks ¹ and consistently self-report saving a lot of time, but the empirical data for organizational improvements is underwhelming at best. ²³⁴⁵
2. Several studies have shown that using LLMs a lot can lead to over-reliance and "cognitive debt", i.e. it makes people dumber. ^67891011

I've seen both of these issues with my own eyes, but I also see a huge gap in our understanding of how they might come about. The scientific research on this seems to be in the Stone Age, still treating AI use as an on/off affair, asking what happens when people generically "use AI". Practitioners in software engineering, on the other hand, seem to have been teleported en masse to a distant and incomprehensible future. They've made huge strides in harnessing LLMs to write code for them—mostly via groping trial and error—but they're also caught in one hype super-tornado after the next, most people blindly copying whatever seems to work even a little for others. No one seems to be measuring how much all those techniques objectively help.

Much more research will be necessary, but I think we can already combine the little science available with the hacks that seem to be sticking, and begin to understand not just whether, but how and when to use generative AI to solve difficult tasks. If we want to export the productivity gains beyond coding and mitigate cognitive debt, we probably should figure out how this stuff works.

I think the research will eventually find that it's not mere "AI use" but proficiency in AI use, as well as the surrounding tooling, that make all the difference.

I've used LLM-enhanced tools and workflows for hours every day for a few years now, applying them to all sorts of complex problems: especially software, but also scientific research, math, learning advanced topics, product and process design, product management, and even leisure projects like dungeon-mastering and fantasy worldbuilding. These all sound like very different problem spaces, but I've found you need the same high-level skillset for all of them. The hard-earned AI lessons in software engineering can be distilled and reapplied in any other problem-solving domain. This is what this post is about.

We'll tackle the two problems above (cognitive debt and lacklustre productivity boosts) indirectly, by focusing on a third, more fundamental problem:

3. We still lack a clear, powerful language to talk and think about joint human-AI problem-solving.

This is the first step toward better discussions about the other two. Perhaps if we start speaking the same language, we'll find the answers already on the tips of our tongues.

How This Post Works

What I offer here is a framework to understand what happens when someone uses a large language model to tackle a complex, difficult question or problem of any kind. I'll call it the Anatomy of Augmented Thought framework.

While researching this, the biggest realization I arrived at is that even more so than critical thinking, it is metacognition that matters when you want to use AI without falling into its many pitfalls ¹². "Thinking about thinking" has always been a neat skill to practice, but in the past it has been easy to brush it off as something philosophers and a certain group of psychologists did. By the time you finish reading this post, I hope you'll be thoroughly convinced that the times have changed: we all have to become expert meta-thinkers if we are to actually reap the benefits of AI's advances.

To give some order to the fuzzy idea of "thinking", I'll borrow three simple frameworks from psychology and combine them into a larger, more detailed one composed of nine thinking categories. Then I'll consider the major factors that determine when and how we delegate things:

1. How tiresome is each category of thinking?
2. How much time does a thought take with and without AI?
3. How does expertise interact with AI use?

Mental fatigue matters a lot, because people tend to want to skip or delegate the tasks that would tire them out the most; time spent on a task is considered the key metric of productivity; and expertise matters because it drastically changes how you guide AI and evaluate its outputs.

Each of the nine thinking categories has different implications for these three factors. The structure that emerges adds nuance to this topic, and demolishes the naïve conception of thought as a single, monolithic activity. Instead, thought is highly complex, and the way it interacts with "external thinking machines" like LLMs must be equally sophisticated.

It's very tempting to treat a state-of-the-art LLM as a magic wand: you only have to want it, pronounce a couple dozen magic words, and it will somehow make it happen for you. I'll call this the Magic Wand Illusion, and I'll make the point that it couldn't be farther from the truth. The ideal way to divide work between a human and an LLM is a mixture of human-only work, AI-only work, and work requiring the close cooperation of both.

The Magic Wand Illusion is dispelled with the Anti-Magic Effect: unlike magic, using LLMs well for complex tasks is a difficult job in itself. I'll also try to sketch what an "optimal" engagement with AI would look like for different kinds of people and different tasks.

Disclaimers

You can skip directly to the next section, or read the answers to any of these legitimate fine-print questions. You can find even more caveats and assumptions in Appendix II.

(Click on "read" next to the questions you have to show the answers.)

Table of Contents

• How This Post Works
• Disclaimers
• 📍 You are here
• Part 1: Drawing the Boundaries
  • Cognition in 3D
  • Pre-Augmentation Anatomy of Thought
  • Thoughts, Mental Fatigue, and Time
• Part 2: Augment
  • Careful What You Hand Over
  • Augmented Thinking: Faster and Slower
  • When Fatigue Steers in Your Stead
  • Expertise Matters Doubly
• Conclusions
• Appendix I: Pitfall Recap
• Appendix II: Caveats and Assumptions

Part 1: Drawing the Boundaries

Cognition in 3D

Our starting point is the rejection of the lazy all-or-nothing framing of AI-augmented thought: it is probably not true that using AI is always bad for productivity, or always bad for your brain's sharpness. So what we need to figure out is when is it bad, and when okay.

In other words, we need to open up the black box we call "thought" and distinguish some subtypes by drawing new boundaries inside it in a way that is meaningful for our purpose.

There are various ways to cut up "thought", and more than one may do the job. I've settled for a three-dimensional division based on a few scientific theories, which works nicely. It is formed by three axes: the Mode axis distinguishes thoughts by their function, the Level axis is about the target of cognition, and the Phase axis separates thoughts by when they happen.

Mode Axis: Stanovich's Tripartite Mind

The dual-process theory of cognition—popularized by Nobel laureate Daniel Kahneman in Thinking, Fast and Slow—says the mind has two modes of operation, System 1 and System 2, that work quite differently: the former is quick and instinctive, while the latter is slow and rational. Keith Stanovich built on that to add a further division within System 2, effectively bringing the number of "minds" to three ¹³:

1. The autonomous mind is largely equivalent to Kahneman's System 1, i.e. it is about pattern recognition, instinctive associations, and heuristics.
2. The algorithmic mind is raw computation: following logical steps, applying predefined rules (for example mathematical rules) in sequence to arrive at a desired answer.
3. The reflective mind thinks about collecting information, chooses strategies, sets and updates goals, and so on. It's the control layer, the how rather than the what.

Is this the best way to distinguish types of thinking? Maybe not, and the topic is still debated among experts. But it is a very defensible theory as of now, and (most importantly) it works extremely well for the specific case of human-LLM thinking I want to tackle. Stanovich split System 2 into algorithmic and reflective because he wanted to explain why smart people can still do dumb things—he called this dysrationalia: you can be very good at algorithmic thinking but bad at reflecting on whether what you're doing makes sense. I think this boundary is especially illuminating when it comes to language models.

I'll begin by drawing this first axis on our map.

Level Axis: Object and Meta

The next dimension of thought concerns the realm the thought is interested in, of which there are two possibilities: object and meta ¹⁴.

Object-level (or first-order) thinking is concerned with the actual problem at hand—it's the most obvious interpretation of the phrase "thinking about something", so no surprise there. Meta-level (or second-order) thinking, also called metacognition, on the other hand, is thinking about the thinking process itself. A couple of examples might help here.

Suppose you're trying to solve a crossword puzzle:

• object-level cognition means thinking about the hints and the word candidates;
• meta-level cognition is thinking things like "am I focusing too much on this word?" or "things aren't adding up, I might have written the wrong word somewhere."

If, instead, the problem you're tackling is climate change:

• interpreting scientific measurements and considering possible mitigation actions is mostly object-level thinking;
• monitoring your success in solving climate change and wondering about how to best communicate the problem to others are examples of meta-level thinking.

Like the others, this distinction will prove critical to understanding how best to delegate tasks to an LLM. On the map, I'll show it as a splitting of each mode of thought in two: each mode can be focused at the object level or at the meta level.

Phase Axis: Divergent and Convergent Thinking

This third way to divide cognition is concerned with the natural lifecycle of thought: you first need to diverge (search and explore many possible directions and approaches) before you can converge (prune the list of possibilities, discard the bad ones, whittle them all down until you reach a single output).

This theory was first proposed by J. P. Guilford in 1967 ¹⁵¹⁶, but even today you'll find it in most business courses on management and design (the design thinking framework has it as one of its tenets ¹⁷).

Divergent thinking is largely about creativity, because it implies coming up with novel ideas and yet-unverified paths to your goal. It tends to be messy and noisy, because the goal of this phase is to expand your range, not to bring clarity. It relies on the brain's default mode network (DMN)¹⁸.

Convergent thinking, on the other hand, is all about bringing structure, removing the errors and the unnecessary fluff, and selecting the best option—a job for the regulation skills of executive-control neural networks.

Normally, adding this third axis would mean splitting every one of the 6 existing categories (3 for Mode times 2 for Level) into two, leading to 6 ✕ 2 = 12 categories. But I've found that metacognition doesn't really need a distinction between convergent and divergent thought. That seems to be mostly relevant to object-level thoughts, which usually involve an exploration of what's “out there” before drawing conclusions. At the meta level, there is no “out there”, and separating divergent and convergent processes, while arguably possible, doesn't offer much insight. So I'll avoid making that distinction for metacognition, leaving us with 9 different categories of thought.

Putting Them Together: the AAT Framework

With these three axes, we get a grid of nine thought categories. To keep the terms short, in this post I'll omit mentioning the word "object" in the names of object-level thinking categories. Unless I write "meta-", you should assume I'm talking about the good ol' object level. I will also shorten "autonomous" to "auto" and "algorithmic" to "algo".

This is the Anatomy of Augmented Thought (AAT) framework.

All nine thinking categories are useful for something, and—with the exception of diverge → converge—there is no predefined order in which you use (or should use) them. All can happen more or less simultaneously and in loops, and you shift your attention from one to another very quickly whenever you're actively thinking.

But there is one pattern that seems absolutely unavoidable in complex problem solving: no matter what paths you weave on the AAT grid, you must periodically make decisions that bring you closer to the solution, be they object-level decisions on direct actions (writing a word in the crossword puzzle) or meta-level indirect actions (resolving to ask your friend for help with the puzzle).

Every thinking process, then, is a tangle of many types of thinking dotted with decision points.

Pre-Augmentation Anatomy of Thought

Before we can stick the "Augmented" on and say anything useful about LLMs, we need to understand some things about the plain Anatomy of Thought, unaided by technology. Below you can read (AI-sourced) examples and (my own) clarifications about each of the nine types of thought in the AAT framework.

1. Auto-Divergent

Object-level autonomous divergent thought

What it is

Representative examples of auto-divergent thoughts are associative leaps, pattern recognition, and remote connections.

It's the hunches you have—often suddenly and unexpectedly—about new possibilities and ideas to explore about a problem, but without the certainty of which is right.

Examples

• While showering, you suddenly see a connection between a historical migration pattern and your thesis argument. Intriguing, but you have no sense of how you got there.
• A designer glances at a cracked sidewalk and instantly gets a hunch that the branching motif could work for a poster concept.
• Halfway through a conversation about music theory, you get the nagging feeling the same concept might explain a bug in your code.

2. Auto-Convergent

Object-level autonomous convergent thought

What it is

Expert intuition, fast heuristic matching, recognizing the right answer without deliberation.

It's the a-ha moments when the solution reveals itself without you really trying, or when the implications of what you just learned suddenly click.

Examples

• A chess player looks at a position and immediately knows it's lost: no calculation, just pattern recognition from thousands of prior games.
• A seasoned editor reads one sentence and instinctively spots the passive construction that's draining the paragraph's energy.
• A nurse glances at a patient's posture and skin tone and knows, before looking at any chart, that something is wrong.

3. Meta-autonomous

Meta-level autonomous thought

What it is

Gut-level monitoring: surprise, dissonance, the nagging sense something's been missed (alerting), as well as felt coherence, pieces clicking together, fluency-based confidence (confirming).

Examples

• Reading a technically correct AI-generated paragraph and feeling something is off. You feel a vague unease before you can name what's wrong.
• The satisfying click when the last piece of a complex proof slots in and the whole thing suddenly feels obvious.
• Unexpected boredom while reading a source, before consciously realizing you've already extracted everything useful from it.

4. Algo-Divergent

Object-level algorithmic divergent thought

What it is

Systematic exploration, enumerating options, combinatorial search, structured analogy, running thought experiments.

It's the mechanical work of finding information, setting up boilerplate, or collecting raw data. When Sherlock Holmes is hunting for clues in the middle of a case, he is mostly doing algo-divergent work.

Examples

• Systematically searching the literature for all studies on a topic, building a spreadsheet of findings and keywords, before drawing any conclusions.
• Listing every possible cause of a software bug by category—environment, input, logic, dependencies—before testing any of them.
• Running through all combinations of available fridge ingredients before deciding what to cook, to avoid missing a better option.

5. Algo-Convergent

Object-level algorithmic convergent thought

What it is

Step-by-step derivation, calculation, following logical chains.

It's the mathematical, logical, rational work of reaching an answer. What Sherlock does once enough pieces of the puzzle have surfaced, culminating in the grand reveal of the murderer.

Examples

• Evaluating a definite integral step by step, checking each algebraic manipulation before proceeding to the next.
• Tracing a program's execution line by line to find exactly where the output diverges from the expected result.
• Eliminating crossword candidates one by one using letter constraints until only one option remains.

6. Meta-algorithmic

Meta-level algorithmic thought

What it is

Systematic process-checking: tracking coverage, noticing repetition, checking breadth, but also auditing whether the problem decomposition itself is right. Verifying steps against constraints and checking consistency.

Examples

• Realizing mid-literature-review that you've only searched English sources, and the most relevant work might be in another language.
• Checking your task list and noticing you've been ticking off easy items while consistently deferring the one that actually matters.
• Pausing to ask whether the categories organizing your research are exhaustive, or whether you've silently omitted an entire dimension.

7. Reflective-Divergent

Object-level reflective divergent thought

What it is

Questioning your framing, generating alternative perspectives, asking "what if I'm looking at this from the wrong angle?"

When you take a step back and reassess the situation, you're doing reflective-divergent work.

Examples

• Midway through analyzing a business failure, asking: "Am I assuming this was a management problem? What if it was a market-timing problem?"
• Noticing you've never made your definition of "success" explicit, and that might have misled other people and caused confusion.
• Recognizing that the problem you've been solving is actually a symptom, and trying to name the upstream cause instead.

8. Reflective-Convergent

Object-level reflective convergent thought

What it is

Applying values, weighing incommensurables, deciding: committing under uncertainty and accepting trade-offs.

This is when you act on what you've thought so far. It's no longer just about the problem; it's about your own stance, relation, and effect on it (real or potential).

Examples

• After weeks of weighing job offers, accepting one: it's not perfect, but you feel like it's high time to commit to something.
• Cutting a beloved essay section that undermines the core argument, even though it took a week to write.
• Deciding to report a colleague's misconduct despite the social cost, because your values leave no option you could respect.

9. Meta-reflective

Meta-level reflective thought

What it is

Epistemic self-examination: generating hypotheses about your own biases and limitations ("Am I anchored? What would someone different see?") and judging whether your reasoning process has met your own standards of intellectual honesty.

Here is the difference from "mere" reflective thought: in the (object-level) reflective-divergent category, I gave the example of asking "what if I'm looking at this from the wrong angle?"; the meta-reflective equivalent is asking "what factors lead me to look at things from the wrong angle?"

Examples

• Asking yourself: "I keep reaching the same conclusion on this topic—is that because I'm right, or because I built my whole framework to produce it?"
• Noticing that using an AI to research this subject always leaves you feeling reassured, and wondering if that's a sign of confirmation bias¹⁹.
• After a decision: "My argument felt airtight, but I only tested it with people who already agreed with me. What would a real stress-test look like?"

Thoughts, Mental Fatigue, and Time

Some categories of thought can be safely delegated to an AI and, unfortunately, these aren't always the categories that you would like to delegate. So what would people gladly outsource, if possible? The most consistent answers you'll get are: the mentally tiring tasks, which are perceived as "difficult", and the time-consuming tasks.

In this section I'll offer very rough estimates of these two salient implications of thought. Needless to say, these things can't be generalized without gross oversimplification, because thought can be literally about anything imaginable. The estimates I give are general trends based on the few scientific experiments about these things and on my own experience in various fields and roles.

The Panting Mind

The human body is excellent at optimizing its energy consumption, and the brain is no exception. As you're probably all too aware, the brain naturally falls into the least-tiring approach that it considers enough²⁰ to solve a problem—even if it's not the best approach ²¹. You can usually tell which thought processes would be tiring because you intuitively feel they are hard.

What are the most tiring categories of thought along the three axes of AAT? Knowing the typical mental cost (a.k.a. cognitive load) of each category will tell you how the average person will tend to think, and what they'll try to offload to AI.

A major "cost center" of the brain, acting as a biological bottleneck for intense thought, is the lateral prefrontal cortex (LPFC), on the front-facing surface of the brain. This area handles working memory, reasoning, planning, and conscious imagination—all ingredients for the most advanced forms of rational thought. Several studies ²²²³ have shown that the metabolism of the LPFC (and related regions, like the anterior cingulate cortex and the parietal cortex) gradually changes when it is activated for longer periods. Glutamate and other toxic substances accumulate, making activations in those areas increasingly costly for the brain. This may explain why the brain seems to fall back to lower-effort options when fatigued ²⁴.

(By the way, this is different from the now-infamous "ego depletion" effect by Baumeister et al., which has now been largely debunked²⁵.)

Other regions of the brain, like the default mode network (DMN), don't experience the fatiguing effect seen in the LPFC. Although people (understandably, given the name) tend to think of the DMN as "the thing the brain does when it's idle", this brain-spanning web of neurons also plays a role in low-effort thinking ²⁶, including memory retrieval, future planning, and hypothesis generation ²⁷. On the Phase Axis of the AAT framework, these are particularly important during the divergent thinking phases, which correspond with creative output.

Unfortunately, things are not as simple as they sound, because divergent thinking is a collaboration between various networks, and the brain has to switch back and forth between the DMN, the PFC, and other regions ²⁸, which likely adds to the cognitive overhead ²⁹. The details depend on the specific task, but as a rule of thumb the divergent phase starts with a heavier focus on memory retrieval, which is not very tiring, but draws on the executive networks (like the PFC) more and more as the brain runs out of low-hanging fruit from memory—it has to get busy actively hunting for the less-obvious ideas ³⁰. This means that, in a single session of divergent thought—especially in unfamiliar algorithmic search tasks—the mental fatigue profile can rise from moderate to high.

On the Mode Axis, unfamiliar algorithmic work is generally rather tiring for the PFC. It requires you to hold multiple interdependent elements in your working memory at the same time ³¹³². For repetitive, pattern-like algorithmic tasks, however, you usually learn to break them down into small, near-effortless, standardized, but tedious steps, turning this work into something almost autonomous. For example, with the right technique (probably involving pen and paper), making a list of all possible combinations of three letters becomes a mechanical, even easy task—but still time-consuming. The same happens with algebra, once you've internalized its rules: it lets you simplify any algebraic expression with the same ease, regardless of how huge or complex it is.

This point bears repeating, as it will come up again: whether a logical chain of thought is pattern-following or irregular makes all the difference for how tiring it is for a human, how fast they can execute it, and how much reflection is needed to guide it.

Using the reflective mind also tends to be rather tiring, especially in the convergent phase. Overriding your first instincts, inhibiting bad habits, marshalling the necessary information and, crucially, deciding what to do next are some of the added demands on your prefrontal cortex.

This gets even tougher with the meta-level thought processes on the Level Axis. fMRI studies tell us that "thinking about thinking" is an extra layer of thought on top of the object-level activations, associated with its own specific areas (the rostrolateral PFC) ³³, and that it adds its own contribution to the metabolic shift and fatiguing effect of thought ³⁴. If you've ever tried genuine rational self-examination, this much shouldn't surprise you.

People tend to skimp most on the meta-reflective ⓘ work, especially on routine tasks or under time pressure ³⁵. In practice, I think most people avoid spending any significant time doing meta-reflective work, partly because of its high cognitive load.

The problem is counterintuitive, because the main job of metacognition is optimizing object-level thought. You know what I'm talking about if you've ever caught yourself thinking "I'm getting fixated on this again, but I know it's never worth my time" or "I already know how this discussion ends, let me change approach." Used well, the meta level can actually decrease the amount of effort (and time) needed to solve the problem, perhaps by correcting a misleading bias of yours, or prompting a reevaluation of your goals. But it is itself a tiring task ¹².

Guesstimating Time Spent

The second factor that matters when delegating (or not delegating) thought is the time you spend in each cognitive category. Some problems need to be tackled regardless of how mentally tiring they are, and how long each step takes can matter a lot. If a computer can do it at 10x or 100x the speed, you'll probably want to have the computer do it for you.

Unfortunately, on this front we have to rely on breadcrumbs of research, because empirical data about the time costs of each cognitive category is scarce. The ninefold subdivision of this framework is my invention, so it's natural that no research has explicitly measured process durations in each of those nine cells. We'll have to work with what we have and fill in the gaps with educated guesses, one AAT axis at a time.

First, the Tripartite Mind axis. We do know that the autonomous mind is fast (it's Kahneman's System 1, i.e. the "fast" in Thinking, Fast and Slow), and that the other kinds are slow. But, as far as I can tell, there hasn't been a systematic study of the timing differences between Stanovich's algorithmic and reflective minds. They may be too intertwined to measure meaningfully, or it may just be too early.

What we can say is that algorithmic work can be indefinitely long, while reflective work tends to be more bounded: any specific calculation approach can have any number of steps, but there are usually only a handful of alternative approaches you can consider. As an extreme example, consider the problem "find the smallest prime number that has exactly 10 digits". Claude Opus proposes five strategies to solve it, but tells me that the most promising (i.e. fastest) of these strategies would take about 400 mathematical operations, and the least promising about 75,000! The mechanical work, if you were to actually carry it out, would involve orders of magnitude more steps than the reflective work that guides it.

In other words, both algorithmic and reflective thoughts are slow, but algorithmic work can (though it doesn't always) take much, much longer than the reflective work that guides it.

The picture isn't much clearer for the object vs. meta axis: metacognition takes time ³⁷, but we don't know how much. The deeper problem is that different people seem to operate at the meta level far more than others, and some seem to avoid it entirely (this is my own observation; frequency of metacognitive thoughts is a recognized research gap ³⁸).

So the question of how long metacognition actually takes is probably moot. Instead, let's consider how long it should take ideally: too little and you won't (even) learn from your deepest mistakes, too much and you get paralyzed in self-analysis. The counterintuitive part is that, done long enough, metacognition can significantly reduce the time you need to think at the object level: you avoid many pitfalls and false starts, stop wasting time on irrelevant thoughts, and generally keep your thought process healthy and efficient.

For this reason, I think a good target for metacognition in complex problem-solving is somewhere around 20% to 30% of total thinking time (with massive uncertainty bars!). This is probably more than most people actually spend at the meta level, or consider appropriate. Maybe I'm overestimating it, but lowering the estimate only makes my discussion below about augmented cognition stronger, so I'll be conservative here.

Now for the third axis: what are typical divergent and convergent thinking times? There seems to be even less research comparing these two phases than the other two subdivisions. Psychologists do seem to agree that they are not two sequential steps but a cycle, with thought going back and forth between diverging and converging at irregular intervals. But how long do (or should) we dwell in those intervals? No one seems to have even asked the question. In a case like this, I'll just shrug and say that, to the best of our (poor) knowledge, the two have a 50-50 split of time. Precise estimates are not important here.

All in One Table

Based on all the above, we get a map like this:

Part 2: Augment

Careful What You Hand Over

At this point you might be wondering whether defining all those categories and making crude estimates of mental fatigue and time costs was really worth the trouble. I think it was, because—imprecisions and all—now we can use it to structure our answer to the big questions of if, when, and how to delegate thought to a (general purpose) LLM. We get nine answers instead of one, and even if each may not be very precise, I think the ensemble lends some much-needed nuance to the discussion on augmented thought.

Here is a brief review of how to think about delegation in each of the nine thinking categories, one by one. After that, we'll step back and look at the full picture.

The AI's Territory: Things You Can Happily Delegate

General-purpose language models like GPT, Claude, and Gemini come with vast computational power but zero accountability, lived intuition, and consistent intellectual honesty, so we can start with the obvious insight: algorithmic work is the paradise of augmented thought.

I'm talking about the systematic, quantitative, or combinatorial heavy lifting. These are the areas that usually take up the most time in "knowledge work", and are often (though not always) cognitively tiring. An LLM can do them at a mind-numbing speed while you stare out the window sipping your tea. It's a match made in paradise, and the part that gets closest to being an actual magic wand.

Algo-Divergent: Delegate 🤖

You can usually delegate most Algo-Divergent ⓘ work to an LLM, which consists mainly in the systematic exploration of options, sources, and ideas. Especially in the early low-hanging-fruit phases, simply letting the model produce options from its training data is valuable and cheap, with little downside—filtering out errors and hallucinations is the job of other categories.

This is brainstorming territory, so there are (almost) no wrong answers. Still, letting the AI run wild almost guarantees that it will miss certain possibilities and directions. Models tend to bias their searches toward whatever their initial attention lands on.

This is where the human comes in: you need to set precise goals and boundaries, steer the model when it gets stuck or clearly biased, and ask it to ground its search in reliable external sources or even use code for more systematic surveys (all reflective and/or meta inputs).

Do this well, and the LLM can enhance your creativity ³⁹. The point is not to get high-quality options on average (you won't) but to unearth the occasional big leap in insight.

Algo-Convergent: Delegate 🤖

You can fruitfully delegate this category to an LLM too, but with more care. Artificial neural networks like today's transformers are probabilistic algorithms: they tend to give the most accurate next answer, but offer no guarantees. This makes them unreliable at logic and math in general, and you can't trust them to one-shot the right answer ⁴⁰⁴¹.

AI models are cajoled into parroting logical thought like a trained monkey dances to a tune. Trained or not, you don't want a monkey piloting your aircraft.

Still, the benefits of what it can get right are so enormous that people have put a lot of work into mitigating this unreliability. This is the realm of most software engineering use cases (at its core, coding is algorithmic convergence). Engineers have devised approaches like iterative refinement⁴², adaptations of test-driven development, multi-model reviews, and many more to improve the trustworthiness of AI algo-convergent work. We're still far from 100% reliability, but it's an exciting and fast-moving field.

The role of the human, here—and not only in software—is to vet the process and the results critically, do reality and plausibility checks, and interrogate the LLM whenever it seems to have made leaps in reasoning. In other words, you trade object-level grunt work for meta-level control work. More on this below.

A key tool you should use whenever possible to drastically improve the results is asking the LLM to solve the problem by writing a program, rather than relying on worded reasoning alone. Modern language models are very good at putting together quick scripts, and usually come with the infrastructure to run them on the spot. You can think of code as the routinization of an LLM's algorithm, similar to how internalizing a routine algorithmic task (e.g. mastering algebra) makes your own algorithmic work less mentally tiring and more reliable. The difference is that code is mind-numbingly faster.

The Middle Ground: Tight AI-Human Collaboration

Being "algorithmic" isn't all that an LLM can do. The large state-of-the-art models of today have immense latent spaces—maps of knowledge about almost any topic known to humankind. AI knows about everything.

The problem is that these knowledge spaces are unstructured (compared to any reliable database) and too vast to fully wrap one's head around. This is why some AAT thinking categories need to be joint efforts, where the AI provides the data and patterns but the human is the one who actively navigates.

Auto-Divergent: Joint Work 🧑‍🦰🤖

This is the kind of divergence work that comes automatically. Unlike systematic explorations and algorithmic brainstorming, auto-divergent ⓘ thinking just happens, unplanned. For a human, this means hunches and leaps of insight. For an LLM, it's the surprising connections it pulls out of its training data ⁴³. If you ask it for ideas to name your baby, it can produce any number of ideas seemingly from thin air. Auto-Divergent is where most of creativity happens.

This is a delicate task to delegate to an AI. On the one hand, it can genuinely expand the range of ideas you consider; on the other, it can skew you toward the same pool of ideas as every other AI-assisted thinker ⁴⁴⁴⁵. On top of that, reading the novel associations offered by an LLM can reduce the number of original ideas your own brain produces and make you feel less invested in the work ⁴⁶.

Often this doesn't matter: you just need a good enough idea, and being maximally original is not a priority. In these cases, giving the LLM free auto-divergent reins is probably fine. But when originality matters, you can get the best of both worlds by first brainstorming without the AI, then asking it for its ideas (without biasing it with yours). The resulting pool will be a mix of your own creativity and that of the model's training set.

Reflective-Divergent: Joint Work 🧑‍🦰🤖

It's very common to struggle with a problem because you're thinking about it the wrong way. You make the wrong assumptions, draw the wrong boundaries, and things don't make sense to you. Or you might be able to reason your way through the question but find it very confusing and tiring. You need to reframe, to change your point of view and angle of attack. That's easier said than done, because framings are very sticky—the longer you've held one, the more it will seem like the way reality works.

This is where an LLM can really get you out of your thinking rut. It will offer as many new framings as you want and, although most will be clumsy or moot, some might be just what you need to see the limitations of your original way of thinking. Under the right conditions, asking an AI for reflective-divergent assistance can speed up the work of reframing.

I worded that last sentence carefully. There are two strong limitations in using LLMs here. First, you apparently need to be already quite competent in the problem's domain to benefit from the speed-up in reflective-divergent ⓘ thought ⁴⁷. If you're a beginner and don't have the problem-solving muscles for that kind of problem, an AI may not help you reframe much at all. Second, even if you're a domain expert, whatever benefit there is to augmenting this thinking category seems to be mainly in speed—the number of reframes you can explore in a given time—and not in their quality ⁴⁷. (I'll return to the role of expertise in the Expertise Matters Doubly section below.)

In other words, an LLM might—maybe—make your reflective-divergent thinking somewhat smoother, but only if you're both already good at it and you put in your own effort.

Meta-Algorithmic: Joint Work 🧑‍🦰🤖

Meta-Algorithmic ⓘ thinking is about structuring the process in a logical and appropriate way. The good news is that part of this work can be standardized and shared. There is, in fact, a whole cottage industry of pre-packaged "mental models" and "n steps to do x" recipes—it's a big portion of the self-help industry and business manuals: Five Whys, OODA loops, six thinking hats, project management waterfalls, product lifecycles, lean processes, six sigma, code boilerplate, and many more.

The even-better news is that your LLM knows all of them. Throw a common problem at it and it will readily produce a neat, well-organized thinking framework to solve it. This is excellent and there is no reason not to delegate this otherwise tedious work to the machine.

Except... your deep involvement is still necessary. The fact that an AI can produce an advanced framework made by seven PhDs and battle-tested by three generations of Druckers doesn't mean it's the right framework for your specific case.

Only you know the full details of the problem you're trying to solve, your short- and long-term goals, and the risks and opportunities involved. No matter how much context you give the LLM, a fundamental decision like which thinking process to use can only be vetted and chosen appropriately by your carbon-based brain.

So yes, let the AI handle the skeleton of the process, but audit its decisions critically at every step.

The Human's Territory: DIY

Things may change in the future, but there are some forms of thought where current LLMs do more harm than good ⁴⁸. These are the mental tasks that keep the delegated categories of the previous two sections on track. They rely on lived experience, accountability, and full metacognition, and they should remain fully human-generated. Delegating them can lead to cognitive atrophy ^9101149, and will make it almost impossible to reach consistently good, reliable results.

People who delegate these categories to LLMs are, in my assessment, the main cause of the lack of measurable performance boosts from AI: they have the machine generate lots of nice-sounding outputs, but those outputs are answers to the wrong questions, or contain bugs that pass unnoticed until it's too late.

Auto-Convergent: Do It Yourself 🧑‍🦰

This is the instinct that lets you recognize the "right" answer on the spot, without reasoning steps. It's shaky in humans and utterly unreliable in LLMs. Since it takes very little time and effort regardless, there is no reason to delegate it.

Reflective-Convergent: Do It Yourself 🧑‍🦰

This cognitive category is one of the most important and consequential, because it's where you decide what to do in the next step. To decide well you need to apply your values, weigh fuzzy trade-offs, and commit under uncertainty. An LLM can lay out the options and costs (an algo- or reflective-divergent job), but only you can make the call. Reflective-Convergent thinking is where accountability and consequences reside.

Trying to think with an LLM in this cell of the AAT matrix is a risky business.

One problem is that tasks that seem algorithmic on the surface sometimes hide reflective-convergent ⓘ decision points inside them, where uncertainty or contrasting options exist. The model might simply choose the wrong path at some point along its long chain of algorithmic steps and silently proceed to crunch the wrong numbers for a while without you noticing. It's very important to prune as many of these branching points as possible before the LLM begins its algorithmic work, and to ask it to stop and request clarification whenever it's unsure about something.

Then you have to deal with the even more insidious hazard of being hypnotized by the AI's own narrative ⁵⁰. An LLM will always try to quietly colonize the reflective-convergent mode—it will offer fluent explanations and rationalizations for why the answer it gave you is absolutely the right one and, if you're like most people, it will probably persuade you. Research has shown that people not only tend to go along with the AI's stories even when they're wrong, but they usually feel like they made the call ⁵⁰¹⁹!

If you use your LLM-augmented output for anything serious, "but... AI wrote it!" won't be accepted as an excuse when that output turns out to be bad.⁵¹ The only sane choice is to never delegate value judgements and key design decisions to AI, and to actively distrust its proposals—all the more so when they seem convenient to you.

It's not a coincidence that most of the impressive advances in LLM use for coding are ways to prevent the models from doing reflective-convergent work. ⁵²⁵³⁵⁴ The software world nowadays has an endless stream of new techniques to effectively structure the AI's process before starting, to provide all the context that might become necessary along the way, and to ensure the human gets to make all the important decisions. One of my projects is to free these methods of their engineering baggage and make them accessible to non-technical people, too.

Meta-Autonomous: Do It Yourself 🧑‍🦰

Gut-level monitoring is the intuition of whether something is missing or off (cognitive dissonance) or whether everything checks out and clicks (fluency). The LLM can't do that for you—it will always craft things in a way that appears to click. Meta-autonomous thinking is your first line of defense against those treacherously smooth stories the AI tells you, and you should pay attention to these instincts. Fortunately, this is not a mentally tiring operation, so you can keep it "always on" in the background as you work through the other thinking categories.

But not all gut feelings are equally reliable. If you get an alert signal ("something feels wrong here"), you can bet it's right. Pursue that lead like a hound, dig as deep as you need to, until you clear it completely. When a confirming signal fires in your mind ("this makes sense"), on the other hand, you might very well be wrong, tricked by the artificial fluency of the transformer's output. Treat such confirming gut reactions as noise, and perform the algorithmic and reflective due diligence to ensure the output is really correct.

Meta-Reflective: Do It Yourself 🧑‍🦰

Lastly, metacognitive reflection is the most meta of thought processes, and the single most important human contribution to the quality of thought—augmented or not.

Meta-reflective ⓘ thinking is, in general, the realm of epistemic self-examination; when working with an LLM, it's how you keep the model on track and near peak efficacy. Understanding and applying this Anatomy of Augmented Thought framework is itself a meta-reflective ⓘ function.

You should never skimp on this, even though it's one of the more mentally demanding jobs ⁵⁵⁵⁶. In fact, I don't think it's possible to produce anything more than highly generic and mediocre results without deep meta-reflective effort. Keep this cell of the AAT matrix fully yours: steer it, manage its context, craft careful prompts, and use "doubt engineering" techniques (these are the topic of a future post).

Fortunately, LLMs are steadily improving at meta-reflective support thanks to techniques like structured chains of thought, prompted self-critique⁴², and multi-model reviews. These improvements can take some of the cognitive load off your brain, but not much: LLM mistakes are also becoming subtler and harder to spot, so the harder part of this work remains squarely a human task.

In Summary: The Delegation Grid

Augmented Thinking: Faster and Slower

Using an LLM for work is alluring, first and foremost, for its promise of incredible time savings. It feels like magic! Things that might have taken hours can now be produced in a minute. People who value their time can't resist the temptation, and they generally report feeling much faster and more productive with AI. But the empirical data tells a different story: companies are still not seeing the wondrous explosion of productivity promised by the advocates ^{57 58}.

We have a paradox. If everyone says they're producing more, why doesn't macro-level productivity soar? It could be that individuals are absorbing the time savings to finish early and take longer breaks from work ⁵⁹, but that's a rather extreme hypothesis at the aggregate level, and there are studies saying that people are actually busier than ever because of LLMs ⁶⁰.

I find somewhat more convincing the idea (partly backed by research ⁶¹) that AI-assisted people end up trying to do more in the same amount of time—including nice-to-have things and gold-plating that add complexity and don't contribute much value in the grand scheme of things. This increase in number of tasks leads to more multi-tasking, which is slow, error-prone, and mentally tiring (more on this later). I know this firsthand: I've gotten into the habit of working on three or four completely unrelated projects at the same time, all the time: while an AI agent is working on one of my prompts I switch to the next one awaiting my input, review its work, prompt it again, and move on. It feels productive, but I wonder how good an idea it really is in the long term.

But at least part of the explanation lies in how people use LLMs, and on what tasks. That, I think, makes an enormous difference in how they can affect productivity. We need to break free of the monolithic "is AI useful or not" framing and look into which kinds of tasks can benefit more or less from the technology, and how to get the most out of it while doing them. This is where the AAT framework can help.

Mapping Time: The Anti-Magic Effect

What time savings can an LLM offer you, assuming you follow my Delegation Matrix to the letter? Something along these lines.

First, augmented autonomous thinking categories take roughly the same amount of time: they are fast without AI, and they stay fast with its help. Not much to gain directly here. Reflective categories, according to the Delegation Matrix, remain human tasks, so the time savings in that area don't change much—if anything, you have to reflect more while working with an AI, because you need to monitor and control its approach in addition to your own. The same goes for metacognition, which is largely the human's territory and becomes especially important—non-negotiable, even—when working with the flakiness of language models.

Divergent thinking phases tend to be low-risk while surfacing the occasional gem of insight, so an LLM's results can add value even when they don't save you much time directly (as in autonomous categories). This added value, for example in the form of better framings or additional points of view, can lead to indirect time savings later on. Conversely, convergent (alliteration not intended) phases executed by LLMs are risky and potentially deceptive, often leading to indirect time costs when you later find the mistakes and have to go back and fix things.

Algorithmic tasks are the clear winners in terms of time savings. You can tell because coding remains by far the biggest AI spending category, and other major ones like IT operations, document processing, and customer support automation are mostly algorithmic in nature ⁶²⁶³. These are the systematic, tedious, sometimes mind-numbing tasks that would take you a lot of time but that an AI can work through tirelessly and quickly. It might even be able to write code that does it in milliseconds.

In summary, all things considered, my rough assessment of the time effects of each AAT category is as follows:

Big savings in algorithmic (divergent more than convergent) work, more modest and indirect savings for auto-divergent and meta-algorithmic, and slowdowns in all other categories.

The strangest result, though, is in the meta-reflective ⓘ layer. As the delegation grid showed, this is a human task, yet you should expect to spend significantly more time on it when working with an LLM, not less. AI introduces a host of failure modes that typically don't exist in solo work, like confabulation and plausible-sounding flawed narratives, and these require you to constantly ask: "am I trusting this output because I've verified it, or just because it made a long and coherent argument?" ⁵⁰

Call this the Anti-Magic Effect: using the "magic wand" can be harder work than not using it! Without heavy meta-reflective effort, it doesn't matter how much time you save on the algorithmic work: the outputs will be useless at best, harmful at worst, in the wrong format, and their flaws hard to fix ¹²⁴¹.

In practice, this means we all need to invest in honing our metacognitive and reflective skills. The fact that we naturally tend to shy away from precisely these kinds of tiring mental chores ⁶⁴ (see When Fatigue Steers in Your Stead below) makes this habit all the more urgent.

To Augment or Not to Augment

Problem-solving in the real world is always some mix of all nine cognitive categories. If not all augmented thinking is equally time-saving, it follows that not all projects will benefit equally from an LLM's participation. Let's take two somewhat extreme examples.

Some office jobs are almost entirely algorithmic in nature. The work isn't necessarily difficult or confusing in itself, just long and mechanical. This includes tasks like inventories, formatting long documents, cross-checking sources, and writing variant after variant of the same kind of code. In such cases, the before-after picture of how you spend your time might look like this:

In other words, although the Anti-Magic Effect means you have to spend more time on reflection than you would otherwise, the LLM's time savings in the algorithmic part offset that by a wide margin, and you get a huge boost in productivity. This is the ideal use case for AI, and honestly a no-brainer: go for it!

Other tasks look more like this:

Here, from the outset, the algorithmic part accounts for a much smaller fraction of the task's necessary work, and the reflective part is already predominant without AI. In cases like this, the algorithmic boost may not be enough to outweigh the added reflection time. There are many tasks at this end of the spectrum: opinionated creative work, tackling very novel and exotic problems, working through emotions are some that come to mind.

The point is that some jobs can actually take longer with an LLM than without. This doesn't mean you should never delegate tasks like that to a machine—the inspiration and hints it gives you might still be valuable, and help you increase the quality of your work—but you should at least be aware of its negative effects on productivity.

What kinds of tasks are most reflection-demanding? As a rule of thumb, you need to put the most effort into carefully monitoring and steering the LLM during convergent phases, because this is where every wrong step has the biggest consequences. And a convergent thought process needs more attention the less pattern-like it is. Some algo-convergent ⓘ tasks, in particular, can be long sequences of easy, standardized steps, which an LLM can complete accurately by referring to analogous patterns in its training data. The risks spring up mostly when the process is new, randomness-driven, or very specialized: precisely the mental tasks we found to be most tiring for humans.

What this framing gives you is a view on the landscape of augmented productivity:

• Tasks high in algorithmic volume and based on pattern-like convergent work are LLM heaven: incredibly fast and quite accurate.
• In processes with many algorithmic steps but requiring custom and non-standard reasoning you might be more productive delegating the divergent part only, keeping the careful convergent work for yourself.
• When there isn't much algorithm involved but the convergence follows standard patterns, you can safely delegate the work to a model—just don't expect extreme time savings.
• And when the work is both non-algorithmic and highly custom, you need to seriously consider doing it all yourself to save time.

When Fatigue Steers in Your Stead

The Anti-Magic Effect means that properly augmented thought is more cognitively demanding than solo thought, and many users skimp on the meta and reflective work precisely to avoid that strain. My hope is that simply being aware that using an LLM is supposed to be hard can be enough to spur more nuanced approaches ⁶⁵. The real danger, however, is that this extra strain is largely invisible.

Mental fatigue is a metabolic byproduct of using the lateral prefrontal cortex ²². The more intense, deliberate thinking you do, the more toxic substances like glutamate accumulate in your LPFC, and the less control you have over your thought processes and decision-making ²⁴. Crucially, the same experiment that measured these objective effects also asked participants to rate their subjective perception of their cognitive abilities, and found that people were not aware of their own degradation. When your LPFC is flooded with glutamate after hours of careful prompting, you'll be more likely to accept the first reasonable-looking output the LLM throws at you, to skip the necessary "step back and critique" work, and to be generally more careless with your procedural decisions. This may be how you get experienced software developers who think they're 20% faster with AI while actually being 20% less productive ⁵⁷⁷⁵, and how people end up burning out more and feeling less engaged with their work ^61666768 when using AI.

What to do about this problem? I don't have a very good answer yet. At the very least, being aware of this inevitable cognitive degradation during long AI sessions might help you plan your workday more carefully. In my own experience, learning to detect your own prefrontal fatigue is not entirely impossible. It took me years of daily, intensive thinking sessions and a lot of scheduling trial-and-error, but I'm now usually quite aware of when my brain has become too dumb for proper reasoning (typically after about 3-4 hours of reflective/meta thought). I'm not sure I'm able to fully compensate for all the negative effects yet, but I did see big productivity gains when I started using that knowledge.

Expertise Matters Doubly

So the optimal application of LLMs for difficult thought depends on the category of thinking you delegate, the time savings depend on the type of task, and the pitfalls are amplified by your brain's fatigue. That's a good refinement of the default black-or-white view so common online. But there's one last space we need to explore before this framework can be of use to most people: are the benefits the same for novices and experts?

Science offers some hints of an answer in this case. The short version is: of course the benefits are not the same. A longer one: not the same, and it depends in interesting ways on what you mean by "expert". In very rough terms, you could be an expert in the problem's domain, an expert in using LLMs for problem-solving, both at the same time, or neither. These are four distinct scenarios.

What does it mean to be a "domain expert" in this framework? Stanovich's tripartite mind is helpful again: when you're an expert at something, you have learned to "chunk" domain information into a few meaningful patterns ⁶⁹ that you have internalized and automatized ⁷⁰, i.e. you can unconsciously simplify the problem space to the minimum necessary. Your autonomous mind is your most powerful asset—essentially a sophisticated set of alarm bells to spot every awkward detail and mistake. This, in turn, supports your algorithmic thinking, because you need to hold in your working memory only those few tried-and-true learned patterns, rather than all potentially meaningful information.

A demonstration of this was given by a famous experiment by Chase & Simon (1973), in which they showed that chess masters are vastly better than novices at remembering chessboard states from actual games, but equally bad at remembering randomly-generated board states. This proved that expert chess players must be doing powerful simplification work without even thinking about it, picking up and retaining the meaningful bits of information while ignoring the unimportant details ⁷¹. This is the expert autonomous mind at work.

But specialists fare less well with the reflective mind and metacognition, because they become overconfident in their own abilities ⁷²⁷³⁷⁴, their thinking becomes entrenched ⁷⁵⁷⁶, and (like novices) they remain susceptible to bias from external inputs.⁷⁷

As for the definition of "LLM expert", I define it as someone who

1. excels at reflection and meta-level considerations,
2. knows when to delegate thought to AI and when not to,
3. knows which tasks are going to benefit the most from augmentation, and
4. can account for their own cognitive fatigue to some extent.

This skillset is almost exactly complementary to that of the domain expert. In fact, it is largely domain agnostic.

Here is what everything we've seen up to now suggests for various types of users.

Domain Novice × LLM Novice: Tentative and Shallow

Someone who is a beginner in both LLM use and the complex problem they're tackling isn't going to be very effective with AI, because they'll be slow (lots of uncertainty, trial and error) and mostly wrong (all the pitfalls we've already covered). They don't have the autonomous monitoring skills to spot errors, so the only way they can hope to catch AI mistakes is by slowly working through the results themselves, defeating the point to some extent.

Add a sprinkling of Dunning Kruger to the mix (but not too much), and you can see how fraught an endeavor this is.

Still, AI can speed up at least the easy and standardized parts, so it's not always a bad idea for this group. The double-novice should treat the LLM as a tutor rather than an engine, using it to learn and explore more than to produce polished outputs.

Domain Expert × LLM Novice: Powerful but Inefficient

People with deep experience in a topic but unfamiliar with LLM-augmentation find themselves in an interesting predicament. They have an excellent autonomous mind that lets them catch mistakes early, but they also tend to be overconfident. If the AI gets into Yes-Man Mode (as it's wont to do), it won't point out the human's errors, simply playing along with whatever the human asserts with any conviction.

The work is also tiring for the expert, because they're not used to all the steering and prompting tricks required to get good results. They tend to under-specify or over-specify their demands, get poor results on the first try, and either start fighting the machine or give up.

I think this complex relationship with the LLM's role explains, at least in part, why experts seem to gain less from AI than domain beginners in controlled experiments (see the "Isn't AI a boon for novices?" box below). But one has to begin somewhere, and this initial struggle to get even a small productivity boost from an LLM is a necessary step toward proper thought augmentation.

Domain Novice × LLM Expert: Double or Nothing

This quadrant is a honeypot for veteran prompters. An LLM's latent space is vast beyond comprehension. No human knows so much. The fact that it is there to be extracted at any moment is a huge temptation. Why limit myself to solving problems in my own domain of expertise—asks the expert AI user—when an LLM can fill in everything I don't know?

When this succeeds, it feels enormously empowering. The world is at your cognitive fingertips—you are a super-generalist!

But this quadrant is also the riskiest. People here can't really apply their object-level reflective mind effectively, and are forced to delegate that part to the LLM. Without domain knowledge, you also have to rely on the AI to do the convergent work for you. Both of these go against the insights of the AAT framework. The usual result is highly fluent, coherent, and deeply flawed LLM outputs that the human is unable to debug.

Even assuming you do manage to get correct outputs on unfamiliar topics, you face the danger of deskilling or perpetual inexperience, never learning anything new again ⁷⁸. We don't know yet how this will play out. Maybe the models and our techniques will eventually become good enough that this is the normal state for everyone; maybe it's the beginning of a global intellectual disaster.⁷⁹

We'll see. Some people are pessimistic about it. For now, consider this way of using AI a high-risk, high-return gamble.

Domain Expert × LLM Expert: Full Augmentation

Needless to say, the Goldilocks zone is when someone is both experienced in the topic of study and skilled at augmented cognition. This combination allows you to delegate exactly what can be delegated and nothing more, while catching mistakes early and limiting the most tiring mental work to the categories of thought with the most leverage.

In my very anecdotal experience, almost all of those working in this quadrant today are software developers. I haven't heard of many people expertly doing highly complex, sustained, useful work with LLMs outside of coding. This doesn't mean there aren't people doing that (there must be some), but I do believe there is an enormous gap here. This is the main reason I'm writing this post: I feel we can and should democratize LLM expertise.

Conclusions

You can't conquer a land you don't know the lay of. The relevant science is still in its embryonic stages, and there is much left to research about human-AI problem-solving. Outside of the labs, we hear lots of fragmentary reports of how much AI helps, whether it hurts more than it helps, and all the risks and horror stories, but we don't hear enough about how and when and why we can use AI to greater or lesser benefit (with notable exceptions, of course: see for example Ethan Mollick, Maggie Appleton, Andy Clark).

Nor is it a theoretical topic we can leave to specialists the way we generally can with particle physics or medicine. We are all practitioners. This post is my attempt at mapping the landscape as granularly as possible, hoping it helps others orient themselves toward a nuanced relationship with this strange emergent skillset.

I found that simply having a rough map like this—shaky as it is—offers useful insights almost for free. In closing, I'll summarize my three major takeaways from this exploration.

First, contrary to popular belief, serious augmented thought isn't supposed to decrease the mental strain on people's brains. In fact, if you're not mentally strained after an LLM session, you might be doing it wrong, and already bankrupt on cognitive debt. How sustainable this high-intensity work is in the long run, and how it affects the quality of your results, is something to ponder ⁶¹.

Second, it's tempting to believe that AI, if it is any good at all, is good for every part of a project and useful for everyone and for all projects equally. This fallacy will push you into monolithic delegation: you ask the machine to cover all categories of thought, especially the tiring work of carefully steering, deciding, and monitoring the models ²¹. As we've seen, AI can be very helpful, but only on certain processes, and only for people with certain know-how. For some projects it won't even be worth the mental overhead.

The last takeaway is that being very good at the meta-work of using an LLM is critical for any serious endeavor with it⁸². Contrary to popular belief, this new technology is not creating a generation of sorcerers who solve every problem simply by purchasing a subscription. When it comes to LLMs, cognition becomes the main subject of human thought. Understanding it as well as we can is non-negotiable, like mastering human anatomy is non-negotiable for any artist who wants to draw people. Metacognition is the skillset that makes the difference.

At the very beginning I mentioned a worrisome lack of large-scale productivity gains in organizations that have invested heavily in AI. Individuals often claim (and sometimes prove) that LLMs help a lot, but companies beg to differ. The AAT framework offers an explanation for that—perhaps it's a problem of wrong expectations.

Investing millions to equip your workforce with state-of-the-art LLMs won't make a difference unless they all suddenly become meta-reflectively ⓘ skilled and willing to burn through their precious cognitive resources every day. They need to sweat their prompts, orchestrate their context, and fine-tune uncertainty at every step (techniques I'll cover in a more practical post in the near future). Adjust the expectations, redirect some of that spending to training people, and things might just turn around for companies.

Most people agree that we'll soon be using AI to do much more than we do now. For simple routine tasks, I don't think we need to worry too much, and the warnings of this post probably won't apply that strictly. But if we start using LLMs to handle all of our difficult tasks, then meta-reflective ⓘ self-examination will rise to the top spot as the single most important human contribution in everything we do, and the one role that won't be replaced by machines for a good while yet.

It's time to begin teaching metacognition in all school curricula. This is the meta-era. ●

References