What exactly is time? Is it an unshakeable dimension in the universe, or a subjective experience created by our brains?
In physics, time is often described as a static dimension, where past, present, and future coexist in a “block universe,” and the flow of time is merely an illusion of human consciousness. However, from the perspective of neuroscience, time is dynamic and indivisible, a core force that shapes memory, perception, and behavior.
“There’s a persistent tension between physics and neuroscience; they like to go at each other. Neuroscientists will often say, if physics tells you that time doesn’t flow, then physics must be wrong,” says Dean Buonomano, a neuroscientist at UCLA, vividly describing the divergence between the two views of time.
But where does this divergence come from? And are they truly irreconcilable? Buonomano has been working with physicist Carlo Rovelli to find common ground between these two perspectives. In a recent dialogue, Buonomano and Paul Middlebrooks from Carnegie Mellon University delved into these questions.
From neuroscience’s research on time, to how artificial intelligence understands time and causality, and the impact of the concept of time on theories of consciousness, they attempted to provide answers to these complex questions from different disciplinary perspectives. This discussion offers us new ways to rethink the nature of time and its implications for understanding intelligence and the universe.
Paul Middlebrooks
Assistant Researcher at Carnegie Mellon University, host of the podcast “Brain Inspired”
He received his PhD in Cognitive Neuroscience from Mark Sommer’s lab at the University of Pittsburgh. He then pursued postdoctoral research in the labs of Jeffrey Schall, Geoff Woodman, and Gordon Logan at Vanderbilt University, studying how neuronal population activity in the motor cortex and basal ganglia affects natural behavior in freely moving mice.
Dean Buonomano
Neuroscientist, Author
Professor at UCLA, specializing in computational neuroscience and the brain's sense of time. His major works include “Brain Bugs: How the Brain’s Flaws Shape Our Lives” and “Your Brain Is a Time Machine.”
Table of Contents:
01 Has AI Neglected Brain Science?
02 Physics “Time” vs. Neuroscience “Time”
03 Why is Integrated Information Theory Considered “Unscientific”?
04 How Do Organotypic Slices Differ from Traditional Brain Slices?
05 Can the Brain Perceive Time?
06 Does AI Need to Perceive Time?
07 Translator’s Afterword
Has AI Neglected Brain Science?
Paul Middlebrooks: My first question today, let’s start with AI. Earlier this year, you published a tweet, “The brain holds no exclusive rights on how to create intelligence,” which read a bit pessimistically. You mentioned that AI has neglected the brain’s dynamic nature and advances in brain science, and this trend is likely to intensify in the future. Does this mean that AI development has deviated from the path of neuroscience?
Dean Buonomano: Yes, I think I can add some context.
First, AI and neuroscience have been sister fields since their inception. I never implied in any way that AI has neglected neuroscience. In fact, the foundations of AI are some of the basic principles of neuroscience. For example, neuroscience holds that information and memory storage depend on the strength of synaptic connections and their maintenance, and learning depends on changes in synaptic connection strength. This is also central to all AI based on artificial neural networks.
From convolutional neural networks to regularization (which reflects some form of homeostatic plasticity), to neural network dropout (similar to synaptic failure), and the design of convolutional neural networks based on the V1 architecture, these are clearly inspired by neuroscience. The relationship between AI and neuroscience is very close. When von Neumann wrote the first code for the first architectures we now call “digital computers” or “von Neumann computers,” he was also inspired by the brain. However, computer science and neuroscience diverged very quickly.
I think we are now at a watershed – the connection between AI and neuroscience is gradually weakening.
Around 2010, when AI began to seriously address problems like speech recognition, action recognition, and interaction with the world, recurrent neural networks (RNNs) were considered the future, mimicking the brain’s way of perceiving time and having their own internal dynamics.
However, the emergence of the Transformer architecture in 2017 changed everything. That paper, “Attention is All you Need,” proposed a new approach that quickly shifted the field from RNN methods to Transformers. For me, the most astonishing thing about Transformers is that they don’t need to perceive time yet still perform well on many tasks.
Transformers are time-agnostic, but they can distinguish order and sequence. For example, they can distinguish between “I am” and “Am I.” How do they do this? The answer is through “positional encoding.” Instead of processing “I” then “am,” Transformers process “I am” or “Am I” in parallel, marking the position of tokens with positional encoding. For instance, “I” is marked as the first token, and “am” as the second. This way, even without perceiving time, they can still understand order.
For example, if you type “wait 10 seconds then tell me the capital of France” into ChatGPT. It won’t faithfully execute that, it will immediately reply “OK,” and give you the answer, without waiting. Of course, if you use the appropriate prompt, it might call a Python compiler to wait 10 seconds before answering.
Paul Middlebrooks: Positional encoding itself doesn’t utilize time, but rather the position in a sequence. Are you pessimistic or optimistic about this, or are you just pointing out that this situation might continue—what conclusion do you draw from this?
Dean Buonomano: I think this is a response to the idea that “AI will only take its next step under the guidance of neuroscience.” I am neutral on this, agnostic. In some ways, the NeuroAI perspective has some validity: to reach higher levels of intelligence, AI might indeed need neuroscience support. However, as I just said, the progress in AI over the past decade has been astonishing, and it hasn't strictly followed the fundamental principles of neuroscience, such as the continuity of time or the brain's dynamic processing of sequences.
By the way, I might not like using the word “sequence” because it often implies time. I prefer the word “ordinality,” which only denotes the order of first, second, and third… The terminology here is very vague and indeed causes some confusion.
Joey Guidone
Paul Middlebrooks: You keep using the word “progress” to describe AI achievements. How do we define “progress”? And how do we define “Artificial General Intelligence” (AGI), and what's the difference between AGI and the singularity?
Dean Buonomano: What some people call progress might be regression for others, depending on whether you are ultimately pursuing Artificial General Intelligence (AGI) or the singularity.
Ironically, AGI simply refers to Artificial General Intelligence. Never mind what it is, just define it as “intelligence.” In fact, the cause of intelligence is currently undecided. And it's not wise to talk about intelligence without defining it, although many fields do—whether in the field of consciousness or free will, or even the meaning of genes, we sometimes have to deal with undefined abstract concepts. AGI is ill-defined, but intelligence itself is equally ill-defined. Are humans smart? Is this standard high or low? The standard for human intelligence is very low.
“Time” in Physics and Neuroscience
Paul Middlebrooks: We agree on that. Now let’s talk about the different understandings of time in physics and neuroscience. Perhaps we can start with your collaboration with physicists.
Dean Buonomano: We just discussed Transformers, and that might be a good starting point. In Transformers, past and present coexist because you feed the entire content “in parallel” to the Transformers at once. This bears some resemblance to the two perspectives on the nature of time that I discuss with my physicist colleague, Carlo Rovelli.
In the article “Bridging the neuroscience and physics of time” that Carlo and I collaborated on, we attempted to bridge two perspectives on the nature of time in philosophy, physics, and neuroscience:
One is “presentism,” which is our intuitive view as neuroscientists: the present is real; the past was real, but it is no longer real at this moment, and our memories of the past are merely residual impressions in the brain; while the future has not yet become real.
The other is “eternalism,” or the block universe view, or even more extreme “static eternalism.” This view posits that the universe is a four-dimensional block universe, and in some sense, time has transpired. All time—past, present, and future—are equally real. A convenient analogy is that “now” is to time as “here” is to space; just as we accept that we are in a certain “here,” we also acknowledge that other locations can become “here.”
In presentism, we do not accept the possibility of other “nows.” In eternalism, all nows are equally valid.
Transformers largely embody the “eternalism” or “block universe” view—past and present coexist. RNNs, on the other hand, are closer to “presentism,” processing information step-by-step.
Block universe. Image source: Scientific Diagram
Paul Middlebrooks: I always imagine the block universe as a block of wax that can be sliced into arbitrarily thin layers, each representing a point in time, and then different slices can be accessed. I’d love to hear how you collaborated with Carlo Rovelli. I don’t know if you knew each other before, how did your collaboration happen? Was it philosophy that connected you?
Dean Buonomano: Yes, absolutely. Carlo and I attended several conferences on time, and a few philosophical workshops on time. There’s always been a tension between physics and neuroscience, and we like to go at each other. Neuroscientists like to say: “Listen, we live in a presentist universe, time flows, time changes, and if physics tells you otherwise, then physics must be wrong.” I’m exaggerating a bit here.
And some physicists believe that the flow of time is an illusion of the mind, and they would retort: “Listen, neuroscientists, you should figure out why we have this illusion of time flowing, because it doesn’t exist at all.” Some physicists (not all) take eternalism as a given doctrine, and therefore find it hard to imagine how time flows, why neuroscientists have this subjective feeling of time flowing.
Here I see the issue as two-fold. “Time is static” is very counter-intuitive for most neuroscientists. So why do physicists hold this view? There are several reasons. One is that many equations in physics are time-reversible; you can use them to predict the future forwards or to trace the past backwards. This time symmetry is unrelated to relativity, although relativistic equations are certainly time-reversible.
In relativity, time and space are a trade-off, and there’s a concept called “Minkowski space,” which suggests that time and space can be unified into a four-dimensional whole. The best way to visualize this space is: if there’s no absolute “now,” then if I move at one speed and you move at another, our clocks will run at different speeds. Returning to the “wax block” analogy, imagine we are all in a wax block, and if we cut it at different angles (orthogonal or diagonal), we might find ourselves on different “planes of simultaneity”—this demonstrates the spatialization of time.
Paul Middlebrooks: So on this issue, neither neuroscience nor physics has strong evidence to support. These are fundamental open questions. The key to this problem lies in our subjective experience of the passage of time.
Spanish surrealist painter Salvador Dalí's interpretation of time. Image source: imuseum
At the mesoscopic time scale where we humans exist, the presentist view held by neuroscientists is acceptable, as this is the full extent of what we can study; our sense of time is based on limited experience of the mesoscopic world. As for the philosophical understanding of time by physicists, it goes beyond our epistemological scope, because we cannot directly measure or observe phenomena at that scale.
Dean Buonomano: Of course, we live at the mesoscopic scale; relativistic speeds or quantum phenomena do not exist in our daily lives. This explains why our intuition fails when trying to understand these phenomena. Our understanding of “presentism” is a product of the mesoscopic world, and I don’t think it’s an illusion, because the block universe operates at that time scale. The block universe is independent of time scale.
I don’t want to draw too many conclusions from this; I think that’s part of the problem. Unlike gravity near black holes or time at relativistic speeds, the flow of time should occur at all scales. My view is that, since the brain evolved to understand the universe we live in, our mesoscopic universe and the world, and they are all governed by physical laws, it is part of reality that we see time flowing.
Paul Middlebrooks: It’s governed by approximately Newtonian physics, which is the world we live in. So our sense of time is like that too, right?
Dean Buonomano: Yes, that's our sense of physics, and our sense of time. But even in the block universe view, the sense of time applies at the mesoscopic scale. I would say that our perception of time is not limited to this world we live in that operates at the mesoscopic scale.
We propose that we perceive time, but on this point, Carlo and I disagree. I believe we perceive time because it does flow. To be clear, this is absolutely not contradictory to the laws of physics. The laws of physics allow for presentism, and also for eternalism, provided it’s local presentism, not Newtonian absolute time.
We abnormally smart apes are indeed incapable of truly figuring out these laws, whether at the quantum or cosmic scale. Assuming we are not living near black holes or colliders, these laws behave stably. I don’t know why you don't believe in the laws.
Paul Middlebrooks: Right. I think my understanding of laws is different now than it used to be. Maybe that’s the right way to put it. I remember Yael Niv saying that the beauty of mathematics is that “you can design experiments to test it, you can do mathematical operations, perform all sorts of mathematical operations, and you can apply these mathematical operations to different things, and it all works. That’s the beauty of mathematics.” I like that statement, and I think that’s what embodies the “laws.”
Dean Buonomano: The brain is full of cognitive biases; we make wrong decisions.
The brain evolved, not to satisfy our intellectual desires, and its purpose is clearly not to understand the laws of physics or the nature of consciousness, let alone the mind.
To me, mathematics is by far the best intellectual tool invented. Perhaps one day, someone will propose a set of equations describing abstract things, whether general motion or Schrödinger’s equation in quantum physics, and these equations, though complex, if used correctly, will allow us to predict the world around us. Yes, I think these equations are our tools for understanding these laws.
I need to clarify that I am not a Platonist at all. I do not believe they reflect entities that exist in the universe.
Mathematics is not perfect. Mathematics can be perfect in certain configurations. Mathematics can capture truth, but there’s also a lot of bad mathematics. Some things you can record with mathematics. That’s just applied mathematics. Similarly, mathematics may not be particularly accurate in reflecting reality. Sometimes mathematics can capture the equations of how the universe works, and sometimes it cannot.
Why is Integrated Information Theory Considered “Unscientific”?
Paul Middlebrooks: Can we talk about Integrated Information Theory (IIT) now? A large group of scholars believes IIT is pseudoscience. You recently wrote an article detailing why you think it is unscientific. Could you please explain what IIT is, why it’s unscientific, yet still makes predictions?
Dean Buonomano: I think it’s more meaningful to understand why one hundred people signed an open letter expressing concern that IIT is unscientific than to debate the wording of “is it pseudoscience.” First, let me state a few points. Rather than forming an IIT concern alliance, this should perhaps be seen as just a group of people coming together. And the fact that they managed to gather a hundred people to sign a document, I think that’s what people should reflect on, rather than why it appeared. Also, I cannot speak for this group; I can only share my opinion, and nothing more—I do not represent any group in any way, shape, or form.
Let’s go back to your question. The COGITATE project is collecting data, and will open-source high-quality data in the future, which is always a good thing, and I think everyone would agree with that. All my concerns about IIT are unrelated to the COGITATE project, and only related to IIT and the laws of physics.
I think people need to understand this (many people misunderstand this): IIT is not a neuroscience theory; it is a fundamental physics theory. It proposes a new ontology for the structure and properties of the universe, arguing that certain material structures possess consciousness. This is why some people consider it a form of panpsychism. Under IIT, many forms of matter, whether neural or not, can possess consciousness. This is the first reason why I focus on the laws of physics here.
The most concise explanation for why I consider IIT unscientific is this: new physical laws must be integrated into existing laws; otherwise, what you get is a free-floating, disconnected, unintegrated rule or law that cannot be reliably tested and whose consistency with existing laws is unknown. Physics is like a beautiful, unfinished puzzle with many pieces. You cannot arbitrarily add new pieces without integrating them with existing ones, because then you wouldn’t know if you’re violating the laws of other existing pieces.
Therefore, I believe that formulating entirely new laws without integrating them with existing ones does not conform to normal scientific methodology. One needs to combine them with old laws to know if there are violations. In my opinion (and it’s just my personal opinion), “yes, I do think IIT might violate certain laws of physics.”
The real key is that it’s like an island—IIT, despite its name “Integrated Information,” ironically isn’t integrated with other scientific fields. IIT is often described as a system with intrinsic causality, and the intuitive meaning of these three words isn’t always immediately obvious.
I’ll try to explain with a real example. Since IIT is not a neuroscience theory and doesn’t require neurons, we can explain it using a system of logic gates.
Suppose there are two logic gates, A and B, which are simple copy gates or threshold gates. Their state is either on (1) or off (0); they receive 0 and output 0, receive 1 and output 1, and that’s all. IIT requires that the current state of the system contains information about past and future states. In reality, this involves many complex concepts like partitioning, sub-networks, channels, mechanisms, etc., but in this simple system composed of A and B, one only needs to consider one question: if B is currently 1, does this constrain the system’s past? The answer is yes, because if B is now 1, then A must have been 1 in the previous time step; this is the only reason B became 1.
This means that this system might have consciousness; it might have a positive φ value (IIT’s measure of consciousness). But we also must consider the future: Does B impose any constraints on the future? The answer is no, because B’s state does not feed back to itself or A.
In other words, this system has no constraining power over the future. Its behavior contains no extra information compared to random states, nor can it predict the future, as you just mentioned.
Here, a neuroscientist might ask: “What exactly do you mean by past and future? Are you referring to specific points in the past and future?” This is IIT’s difficulty, because it sometimes requires us to look at all time points within a certain range in the past and future. It is discrete. For a computer, this discrete time step is clear, because computer time is clearly partitioned. But for the brain, this is not the case, which can raise some issues.
Now, if we reconnect B to A, forming a recurrent neural network. This connection does make B’s state begin to influence the future and constrain what might happen in the future. In this way, the system exhibits a certain degree of consciousness. By adding this connection, the system’s consciousness phi value becomes 1.
Next, we can try a thought experiment to further understand this. One characteristic of IIT is that it is independent of space and time. If we separate the two units in the system and increase the distance between them, this separation actually does not change the system’s state of consciousness. As long as the connection remains good, the system remains conscious. Even if both units are in the off state (both values are 0), the system is still conscious.
Suppose we further change the state of the system: we take out the two units, cut the wires between them, connect them with light beams, and then place them somewhere at opposite ends of the solar system, say, one on Earth and the other on Neptune. According to IIT theory, as long as the connection is good, its phi value remains 1. This is truly important because it raises a question: where exactly is consciousness—is it on Earth, on Neptune, or somewhere in between?
We can conduct another thought experiment—now we are going to block the light path. Remember, the light path is not transmitting anything because they are in a 0 state, meaning it’s blocked without activation. According to our traditional physicist's way of thinking, nothing should happen; there is no precedent for this in physics. At this point, the only thing that happens is that the phi value tends to zero.
So, how long does it take to reset to zero? Most people think it will instantly reset—even if the two gates are far apart, it will instantly reset. However, you should be wary of instantaneous changes in physics, because “faster-than-light transmission” is a big taboo in physics. Thus, the Phi value would instantly drop, but in reality, no information is transmitted.
Now, let me further expand this thought experiment. We can name this experiment after you—“Middlebrooks’ Split-Brain Experiment.” With your permission, we will take out half of your brain, leave one half on Earth and place the other half on Neptune, connected by light beams. According to most people, you would still be conscious. And a characteristic of conscious entities is the ability to communicate changes in consciousness. Now, if we interfere with some of these connections, and we know what we’re doing, what happens?
I’m actually asking a question: if these connections between the brain hemispheres are blocked, you can now convey these changes through the phi value (your level of consciousness). How does your conscious state change, and can it still communicate? Do we now have a way to transmit information instantly?
Here, I think IIT supporters would make a few points:
- Impact of Delay: Increasing delay changes signal distribution, your consciousness is on Neptune or Earth, but communication might be impossible. I think this can be resolved because it's a thought experiment, and we can adjust based on the delay of inter-atmospheric connections versus intra-atmospheric connections.
- Result of Interference: IIT supporters would say: “Well, because I interfered with the lines on Earth, this result occurred.” Now it’s a bit complex, and there might be an argument that the phi value tends to zero, or the system changes in advance because it “knows” at some future moment I will interfere with these connections, somewhat like clairvoyance.
In short, things get a bit complicated. I apologize for that, but this is precisely the challenge with IIT and its associated thought experiments: it’s difficult to pinpoint whether it violates physical laws. My point is simple: because IIT is “free-floating,” disconnected from other branches of physics, it cannot answer these questions I raised. I don't know if it violates physical laws, because its definition is not precise enough, and its integration with other branches of physics is not tight enough to know if these laws are violated.
I think the problem isn't whether we agree with the new ontology, but that if the new ontology is supported, then we will agree. Before we agree with the new ontology, I want to clarify whether it violates the existing ontology. To me, this is the first step in the scientific process. As I said before, physical laws are not immutable; there is room for change. But I need to know whether the proposed new ontology is consistent with the existing one, without any empirical data.
Paul Middlebrooks: It’s worth noting that, from Karl Popper’s perspective, the essence of science is that it can actually be continuously falsified, depending on our existing entire body of scientific knowledge.
Karl Popper (1902-1994) was a prominent philosopher of the 20th century. He proposed the principle of “falsifiability,” emphasizing that scientific theories must be capable of being disproven by empirical facts, meaning a theory needs the possibility of being proven wrong. Popper opposed induction, advocating that the development of scientific knowledge relies on the formulation of hypotheses and critical testing. Popper also explored the concept of an open society, criticizing historicism and utopian socialism in “The Open Society and Its Enemies,” and emphasizing the importance of democracy and individual freedom. His philosophical ideas had a profound impact on philosophy of science, social sciences, and education, promoting the development of scientific methodology and the understanding of how knowledge grows.
Interestingly, listening to your “long discourse,” I found that we’re not discussing the COGITATE project, but Neptune and Pluto, visual pathways, and instantaneousness. Ultimately, the prediction is that IIT is located in the posterior parietal cortex.
Dean Buonomano: I reiterate, I said the COGITATE project is great. I think the data collection is good, but I’m not saying its interpretation or how it’s used is ideal. I think his problem is that it’s a test of two theories, and both theories lack constraints. If you read Stan Dehaene’s conclusion, he explicitly points out that IIT actually hasn’t been tested; what was tested was a loose interpretation of IIT, which predicted why IIT is located in the posterior parietal cortex.
Honestly, I don't fully understand. IIT is math-centric (which has not been tested yet). IIT is built upon mathematical pillars, it’s too complex to compute. The φ value of a C. elegans with 321 neurons, if you started computing today, the sun would have completely disappeared by the time you finished. This mathematical structure has limited use.
There's another point I want to clarify. If IIT supporters believed that the φ value is merely correlated with consciousness, or a measure of consciousness, I don't think IIT would receive as much attention. IIT's mathematical framework is commendable, but they don't actually examine its mathematical framework. Because by definition, φ cannot be computed; at best, it's only estimated.
Paul Middlebrooks: In the case of the COGITATE project, they eventually conducted a comparative test of IIT against the Global Neuronal Workspace Theory (GNWT). Why is the scientific community able to accept GNWT, but not IIT?
From your epistemological perspective, GNWT seems difficult to define. Bernard Baars’ philosophical definition is more abstract, and Stanislas Dehaene subsequently made it more neuroscientific.
Global Neuronal Workspace Theory (GNWT) attempts to explain how consciousness emerges from brain activity and how conscious content is represented and processed in the brain. GNWT posits that consciousness is the result of information propagating across widespread brain regions, especially through a “global information broadcasting” mechanism involving the prefrontal cortex. GNWT emphasizes the global accessibility of conscious content, meaning once information enters consciousness, it can be accessed by all information processing systems in the brain. This theory has had a significant impact in cognitive science and consciousness research and has been experimentally verified using various neuroimaging techniques.
Bernard Baars is one of the founders of GNWT. In 1988, he proposed an early version of this theory, emphasizing that consciousness is a dynamic, integrated process in which information is transmitted and shared among multiple brain regions. Baars believed that the generation of consciousness requires a global workspace that can integrate information from different sensory and cognitive processes and transform this information into content we can be aware of.
Stanislas Dehaene is another important contributor to GNWT. He further developed the theory building on Baars' work. Dehaene’s research mainly focuses on the neural mechanisms of consciousness, and he uses various neuroimaging techniques (such as fMRI, MEG, and iEEG) to study the neural basis of consciousness. Dehaene proposed the concept of a “global neuronal workspace,” suggesting that the generation of consciousness requires specific brain regions (such as the prefrontal cortex) to broadcast information to other regions, thereby making the information globally accessible.
Dean Buonomano: I think what you're trying to express is that the GNWT theory is very vague, amorphous, and severely lacks constraints. This is precisely what causes confusion, and I'm glad you brought it up.
In fact, many things are like this, causing people confusion. I cannot find a simpler answer or a simpler dichotomy. As for