Anthropic published a fascinating new piece of research on July 6, describing what the company calls a “global workspace” within modern language models. This “J-space” reflects what the model is “thinking” about, in some sense, as distinct from other information processing. Anthropic shows that modifying these values directly changes the system’s outputs significantly. Setting aside the anthropomorphism, there seems plenty of potential utility in this research, including improving explainability and safety. Consider, for example, that a governor (similar to what I’m trying to build through https://axrlab.net/) could look at the J-space in a model and identify and change outputs before they are produced; it’s only one extra earlier step of visibility, but I think even that could have profound benefits. Anthropic’s paper gives some examples of the possibilities: in one experiment, the “J-lens” shows a model fabricating benchmark data mid-task, and in another, it surfaces a deliberately misaligned model organism’s hidden intent on an ordinary coding prompt where the output itself looked unremarkable.
I’m going to write somewhat skeptically in the rest of this piece, but I wanted to be clear up front that I do think this research is potentially very useful. What I’m bothered by is the consciousness framing wrapper around it. It doesn’t feel to me like a space per se, but something more like probabilistic vibes (which, after all, is the core of how LLMs work, so that doesn’t undermine its utility!). I wrote in my book that LLM-based AI systems are inherently above 0 on the abstraction scale, but I don’t see the J-space concept as reflecting a measurable further climb up the consciousness curve.
There’s no consensus definition of consciousness, but global workspace theory is one of the more widely subscribed notions. The idea, in a nutshell, is that we maintain a privileged space within our thinking, a relatively small mental workspace that is available to the rest of our brain’s processes (thus the “global” piece). Anthropic solicited external feedback on this work from a few outside experts, notably including cognitive neuroscientist Stanislas Dehaene, whom I referenced extensively in my own writing and who has written some terrific books on consciousness and how to think about machine intelligence. Dehaene and Naccache identify two key properties of global workspace theory to analyze Anthropic’s research: 1) global availability, ability to select and highlight and prioritize specific info; and 2) self-monitoring, including information about itself in reasoning.
What is happening in this J-space, and does it exhibit these two properties?
The “J-space” is a particularly useful area of token activation for models to report on, according to Anthropic. It is named for a mathematical operation, a Jacobian transformation, that gauges how strongly a given activation predicts which word the model will eventually produce — a kind of sensitivity score. To summarize, hopefully not excessively so, Anthropic’s research shows that a consistently small number of vectors are usefully activated within a middle set of layers within the neural network in a way that allows for greater utility in the core prediction function of its LLM tools.
The theory is that these layers represent a functional equivalent of a global workspace, and the consistently small (“a few dozen”) tokens activated enough within those layers are the tokens being “thought about” in a more “conscious” way in that moment. This improves reportability on what the model is “thinking”: rather than having to articulate every activated token at every layer (which depending on thresholds could be vastly unworkable), the model can make its “J-space” visible. In theory, this provides a tractable, yet more meaningfully internal look into the model’s thinking than you would get by using prompts to ask the AI about its “thoughts”. The researchers also built a demo tool using open weight models, showing tokens that reach the threshold of activation in the relevant layers to be part of the J-space.
Anthropic shows that manipulating the elements in the J-space changes the output significantly. It also shows that removing this layer from the process, effectively closing off the J-space, doesn’t prevent the system from functioning, but does significantly limit its reasoning capabilities: “Without its J-space, Claude speaks fluently, classifies sentiment, answers multiple-choice questions, and pulls facts out of passages roughly as well as before. What it loses, though, are the tasks that require some higher-order thinking.” Looking at the two factors articulated by Dehaene and Naccache, it seems as if every token within the system’s vocabulary has the potential to be highlighted in the J-space, and that the J-space activations are used in reasoning.
Is this then a workspace, in the sense of global workspace theory? Erik Hoel doesn’t think so. He offers at least three separate criticisms in his newsletter, The Intrinsic Perspective:
As a theory of consciousness, J-space is, in Hoel’s words, unfalsifiable and trivial. In particular, it waters down the definition of global workspace theory to make it fit: “In order to make the definition work, they drop most requirements for a ‘global workspace’ outside of things that naturally go along with reportability.”
The Jacobian process is not inherently meaningful. Hoel quotes from the underlying technical report to say that a logit lens approach to isolating the most activated tokens (which is not novel) captures “much of the workspace-like structure” already. At minimum, it feels like the use of the Jacobian transformation doesn’t merit elevation to naming rights for the concept.
Anthropic’s delineation of a special “space” within the neural layers, as distinct from motor and sensory, is by their own account “post-hoc” and not always accurate. Hoel notes that on open-weight models using Anthropic’s own released code, there is no clear tripartite sensory/workspace/motor structure at all, which he frames as a falsification-by-substitution argument rather than mere fuzziness. (To be fair, note that Anthropic separately invited Neel Nanda of DeepMind to replicate their findings on an open-weight model as part of the official commentary, with a more positive outcome.)
Anthropic says that their “space” has limited capability: “It turns out that the J-space holds only a few dozen concepts at a time, and accounts for less than a tenth of the overall activity in Claude's internal processing.” How did they calculate this limit of “a few dozen”? Their tests determined a sparsity level, the sort of thing mathematicians always refer to as ‘k’, a subset of the total set which is referred to as ‘n’. Basically, beyond the ‘k’ most activated tokens, the rest of the tokens are not at meaningful levels of activation within the layers identified and described as the ‘workspace’, and thus the minimal work that those non-active tokens are doing within the very complex neural network system is not particularly meaningful at shaping the final output of the system.
So, to summarize: Anthropic identified some middle layers within the neural network (through a demarcation Hoel argues is blurry at best), runs a Jacobian transformation, picks out a few dozen most activated tokens, and describes that as the “conscious” thinking that the LLM is doing, in that the system can identify those elements and report them (and even allow them to be directly modified) in a useful way.
In human thinking, access consciousness describes a clear delineation between conscious and subconscious activations in our brains. We don’t need to be consciously thinking about a concept in order for it to influence our actions. This isn’t just true at the motor layer, where we catch a thrown ball or pull our hand away from a hot surface without thinking consciously about our actions; it’s true with higher-order cognition tasks as well. There is a finite number of activated concepts that have more, or at least qualitatively different, kinds of influence over our thinking than other, non-activated concepts do, and we have some (limited) amount of agency over them. (Though it’s difficult or impossible to suppress specific concepts, as commonly understood by the phrase “don’t think of an elephant”.)
Anthropic’s piece distinguishes access consciousness from phenomenal consciousness, the more mysterious notion of having subjective experiences that we typically associate with an entity that merits moral rights considerations. Here’s a quote:
Our experiments don't show Claude can have experiences, or feel things in the way humans do—in fact, it’s unclear whether any scientific experiment could prove this to be true or false. But philosophers often distinguish this capacity to have experiences, often referred to as phenomenal consciousness, from another idea, so-called access consciousness, which is defined in purely functional and computational terms. A thought is “access-conscious” (or “consciously accessible”) if you can report it, reason with it, and use it to guide what you do. It remains a contested philosophical question whether or not access consciousness implies phenomenal consciousness, or if the ability to have experiences requires some other property.
The many references to consciousness in this research emphasize one specific feature of access consciousness: it’s easy for us to articulate, or report on, the things we are actively thinking about, those things in our access consciousness zones. And in contrast, it’s extremely difficult to articulate what’s going around in our subconscious. We can make some educated guesses based on our emotions, but it’s a very different exercise. The takeaway from this research, from that perspective, is that using this “J-lens” process helps identify a bounded, small subset of influential and reportable elements within the AI’s thinking.
This is still, ultimately, an activation measurement and a threshold of utility. It’s shoehorned into a binary outcome -- a token is or is not in the J-space -- but that doesn’t feel very similar to me to how our brains form a distinctly binary access consciousness space. We are, or we are not, thinking about a thing; it’s not a probabilistic question. Neurons fire or they don’t.
In my book, I spent multiple chapters arguing that consciousness as a moral concept is not a binary, even though colloquially we treat is as one, and look for a magic lever to flip after which machines will be conscious (full stop, yes/no, not a spectrum). But, in contrast, assessments of access consciousness should perhaps be a binary matter. Dehaene and Naccache’s critique is notable here: “Ignition remains to be fully demonstrated.” In other words, there’s no equivalent binary activation function as there is in the brain, because the J-space is created and measured and modeled as a probability function.
Language is inherently an abstraction. The exercise of comparing this property of LLM operation to global workspace theory is inherently partially correct in that there are similarities, and partially incorrect in that there are differences. I don’t want to argue over how similar or how different, as that feels academic and unhelpful. Instead, the question should be whether a particular abstraction is useful, and for what. The technique Anthropic describes can improve reportability and potentially governance. But the label, calling it a global workspace, doesn’t seem useful to me for any particular purpose, except for vibes, and perhaps marketing.