The Neuroscience of Decision Making

IN AN ATTEMPT TO PUT MATTER OVER MIND, researchers are beginning to decipher what exactly is happening in our brains when we are making decisions.

Our thoughts, though abstract and vaporous in form, are determined by the actions of specific neuronal circuits in our brains. The new field known as “decision neuroscience” is uncovering those circuits, thereby mapping thinking on a cellular level. Although still a young field, research in this area has exploded in the last decade, with findings suggesting it is possible to parse out the complexity of thinking into its individual components and decipher how they are integrated when we ponder. Eventually, such findings will lead to a better understanding of a wide range of mental disorders, from depression to schizophrenia, as well as explain how exactly we make the multitude of decisions that ultimately shape our destiny.

Recently, three experts in decision neuroscience discussed their work, describing the genesis of this cutting-edge field and why it incorporates several disciplines. They also identified the driving questions in the field and reflected on the potential practical applications of this research. The investigators who participated are:

• DAEYEOL LEE, PhD, Department of Neurobiology and Kavli Institute for Neuroscience, Yale University School of Medicine
• C. DANIEL SALZMAN, MD, PhD., Department of Psychiatry and Neuroscience and Kavli Institute for Brain Science, Columbia University School of Medicine
• XIAO-JING WANG, PhD., Department of Neurobiology, Physics and Psychology; Director, Swartz Program in Theoretical Neurobiology; Kavli Institute of Neuroscience, Yale University School of Medicine

The following is an edited transcript of the teleconference. Participants were also provided the opportunity to amend and add to their remarks.

THE KAVLI FOUNDATION (TKF): Dr, Salzman, to begin, can you discuss how you became interested in decision neuroscience?

If you think about our own decision making, we might fool ourselves into thinking we’re perfectly rational beings, but of course that is far from the case. Clearly emotional factors affect how we make decisions all the time. And for many psychiatric disorders, patients that are symptomatic are frequently making poor decisions about numerous things throughout the day, such as how they handle their anxiety and other emotional states. If you’ve ever had a friend or family member with depression, you can see they are not making decisions the way they normally do. So there clearly has to be dysfunction in the neurocircuits of psychiatric patients affecting their decisions, and we need to understand this better in order to come up with better treatments for mental disorders. So you can see that I come at this field a little bit as a psychiatrist and a little bit as a neuroscientist.

TKF: Dr. Lee, you were trained in economics before you ventured into neuroscience as a graduate student. How did you end up getting involved in this area of research?

DAEYEOL LEE: I was also interested in the mind/brain problem from a philosophical point of view. My ultimate goal or desire to study neuroscience was to understand the physical substrates of mental phenomena—how our thoughts are generated by brain processes; what happens in your brain when you are thinking. For me, the essence of the thinking during decision making is mental simulation—you are trying to predict before you take an action what outcome may occur by using analogies of your previous experiences, or by observing and remembering the outcomes of other people’s behaviors. These give you some clues about possible and likely outcomes of your actions, and you can do this mental simulation for many different actions before you actually make a choice. Having studied economics, I knew the mental simulations involved in decision making could be studied using economic models and a rigorous mathematical approach. Such a quantitative approach is really important for studying biologically-based thoughts, because thoughts have a lot of subjective components that could get in the way of studying something scientifically. Thoughts mean very different things to different people, which makes them hard to quantify. But economic theories show that you can assign numerical values called utilities to outcomes. You can assign the same numerical values to both real outcomes and mentally simulated outcomes, whether they are juice or money so you have something concrete to work with that you can measure and connect to neuronal activity.

TKF: Dr. Wang, why have you focused on this field?

XIAO-JING WANG: I am interested in understanding basic neural processes and computations that underlie cognitive functions. Decision making opens a window to cognition. It bridges neurobiology and cognitive science. What makes this field exciting is that it really is cross disciplinary. I trained in theoretical physics and about 10 years ago got into the field of neurobiology of decision making accidentally. Before that, using biologically based neural circuit modeling, we had been studying the activity of neurons in the parts of the brain that play a role in working memory—the ability to hold and manipulate information in your mind in the absence of direct sensory stimulation. When people started to uncover the neuronal correlates of decision computations, I realized that the neurocircuit computer model we developed for working memory could also explain behavioral and neurophysiological observations seen with decision-making. This realization there may be a shared neural circuit mechanism for working memory and decision making led us to study all kinds of decision processes.

TKF: This is a young field. How did it take root and what is new?

SALZMAN: In many ways, this area picked up steam a little over 20 years ago with research on perceptual decision making. Monkeys were trained to report the direction of motion they were seeing on a visual display— whether the dots were going up or down. During these experiments, we were aware the monkeys were doing this for one reason: to get a reward for a correct discrimination. [This reflected how] often, in a seamless way, we assign values to different options related to what we are seeing. We are using our perceptual system while at the same time kicking in parts of the brain that are more specialized for placing a value on what we are seeing. And that leads into the whole economic decision making that Daeyeol was talking about.

WANG: What is new is that, using experimental decision tasks, people like Daniel and Daeyeol and others are now recording neuronal activity from brain areas that are downstream from sensory areas, and these downstream brain areas are probably where decisions or choices are made. We don’t quite know yet, but at least we now have some evidence of how a deliberative choice is made and how its outcome is represented in terms of neural computations and their underlying neural processes. That is new research—it started about 10 years ago.

LEE: There definitely has been a change in the culture and the minds of the neuroscientists that study decision making. This occurred somewhere between 10 to15 years ago. People started using more common economic terms like “rewards” and “values” in decision making and asking questions like how the uncertainty about the reward or risk is represented in the brain and influences decision making, and how the brain handles the tradeoff between the overall magnitude of the reward and how immediately you get the reward. Daniel is looking at how desirable and aversive information is integrated into decision making. People weren’t asking these kinds of questions 10 or 20 years ago because we didn’t yet have the basic understanding of how perception and motor control worked—how simple movements and sensory information were coded in the brain. This knowledge was a prerequisite to understanding how the brain makes a decision because now when we test animals in our experiments we can distinguish changes in neuronal activity due to perception and motor control from those related to mental simulations and decisions. About 10 years ago there was a paradigm shift when people realized they could ask more complicated questions related to thinking and decision making.

SALZMAN: Often, the main parameter being manipulated in an experiment nowadays is rewards. Your decisions are typically really about some aspect of rewards, whether immediate or delayed into the distant future, and researchers are in a position where they can actually study how neurons are representing rewards, and how information on rewards may be integrated over time in order to reach a decision. Researchers have also come up with formal neurocircuit models of neurons to show how signals that are representing information about the sensory world are integrated over time and accumulated. Some of these models suggest you have to reach some kind of threshold of neuronal activity to make a decision like “yeah, I saw something move up.” The real beauty with these models is that you can use them to make quantitative predictions about [how sensory input is coded] and you then can see if the [predicted coding] matches the neural signals in the single neurons [that you record in your experiments].

WANG: It’s quite fascinating that what we are seeing now in single-neuron recordings is not coding for what we see or do—sensory and motor coding—but for the processes involved in how we value and make choices. That’s an important advance in neuroscience. I also think it’s fascinating that, when it comes to decision making, behavior is very adaptive. You can really watch and see how your choice behavior adapts and changes from trial to trial, according to environment and task design, and such changes are reflected in the recorded activity of single neurons.

TKF: And are we starting to understand how that adaptability comes about at the neural level?

WANG: Yes. One of the important ingredients is reinforcement learning and its neural implementation. Reinforcement learning occurs when you are not explicitly taught what you are supposed to learn, but rather learn it by trial and error—by getting feedback about how well you predicted the outcomes of your behavioral choices. There is a substantial body of work showing that the neurotransmitter dopamine plays a central role in reward signaling, and that it can greatly affect changes in the synaptic connections between neurons in a way consistent with reinforcement learning.

LEE: There is a lot of movement in the field to figure out what theoretical framework is useful and should be used by investigators in the field to study decision making, and one of them is reinforcement learning theory. This theoretical framework, which has roots in many different disciplines, including psychology, artificial intelligence and machine learning, computer science, and economics, is actually playing a central role in neurobiological studies of decision making. Such an economic framework draws a lot of people doing neuroscience studies at multiple levels, including people doing neuroimaging studies in humans and those doing single neuron recordings in animals.

TKF: To do research in decision making neuroscience, you can do multiple single-neuron recordings to detect which neurons are active during a decision making task, or you can do PET scanning or functional MRI to show heightened activity in larger swathes of the brain during a decision making task. How do the two types of techniques complement each other?

LEE: A minority of people look at the decision-related signals at the level of single neurons. The majority of people studying neurobiology of decision making are using noninvasive neuroimaging techniques like PET and fMRI in humans. Looking at the signals you get from the human brain and trying to relate those to the signals you observe at the single-cell level in animals has been very fruitful and taught us a lot of things about the neural basis of decision making.

TKF: Has the research using noninvasive neuroimaging techniques for the human studies confirmed what you are finding on the single-cell level with the monkey studies?

SALZMAN: Yes and no. The type of questions you can ask in humans versus animals studies are different due to the different level of resolution in an imaging study compared to recording from single neurons in brain.

Functional imaging is not looking at single neurons, but thousands of neurons. You are also not looking at the activity of those neurons with millisecond resolution, but looking at the activity of those neurons averaged over the course of about a second. So they are complementary—functional imaging looks at the whole brain, while Daeyeol can look at only a handful of neurons at a time. In some cases they confirm each other and in other cases they don’t, but when they don’t it probably also has to do with the nature of the information that each type of study can obtain. For example, you can have two groups of neurons anatomically close to each other with opposite response properties. If you were imaging the area of the brain that houses those two neuronal groups, you would only be able to record the average activity, so you will not see the fact that one group of neurons signals one thing, and the other group of neurons signals something completely opposite. You might conclude that the brain area does not encode the parameter you manipulated, because the activity across the entire group of neurons is the same in both conditions as far as an imaging experiment can tell. But if you could record from these neurons individually, you would see that you have two groups of neurons doing opposite things right next to each other in the brain and they do encode the parameter of interest.

LEE: One of the puzzling things is that in humans, if you have a lesion in one part of the brain that causes a certain deficit, and then you compare that to deficits that you get from another lesion in a different part of the brain, often you get very clear behavioral differences. But when you try to then go and measure the neural activities in the corresponding two different areas in monkey brains, often times you get neuronal signals that are very similar. The reason for this discrepancy is people have not figured out what kind of tasks you need to use to reveal differences across different brain areas. So trying to characterize the unique functions of different brain areas is one of the most important challenging questions we face. To do that we have to improve the quality and resolution of the measurements we are making, as well as think a lot harder about the task that you need to use to elicit certain kinds of cognitive processes during decision making.

TKF: What are some of the other current driving questions?

SALZMAN: The field is still in its infancy, but one of the driving forces behind the field now is to try to understand more precisely what are the computations performed in different brain areas, and how they are similar or different. Also how do they communicate with each other, and how is information transformed as it moves around in the brain. How do these different representations about important variables for decision making come together and allow you to form a decision?

TKF: When someone makes a decision, this involves pulling up a memory and a value system and an emotional response. Isn't one growing area of interest figuring out how the brain coordinates all these things?

WANG: That’s one of the biggest challenges today. Many of the questions so far have been addressed by focusing on local circuits. For example, there is a lot of outstanding work on understanding how neurons in the primary visual cortex—the part of the brain where visual signals are processed—become selectively active in response to specific visual stimuli, such as the orientation of lines seen. You can study the local circuit mechanisms when you try to answer these kinds of questions. But a decision involves many processes--you need to accumulate evidence for or against different choice options, evaluate their possible outcomes and risks, suppress certain learned responses and biases, etc. So when you are interested in decision making, you cannot avoid looking at how the different parts of the brain do different computations in a coordinated way. There are no good theories about the operation of a larger-scale brain system with many interactive modules. That’s going to be one of the big questions people are going to try to address in the next 10 years or more.

TKF: How can we figure out such a complex scenario?

In these two different view of the monkey brain, three areas thought to play a key role in making decisions are highlighted (dorsal anterior cingulate cortex, lateral intraprietal cortex and dorsolateral prefrontal cortex.) Researchers are trying to decipher how such far-flung regions of the brain work together in decisionmaking. (Credit: Daeyeol Lee)

SALZMAN: We study very specific tasks so we can study the different encoding process of neurons—how information is represented in different brain centers. But ultimately, if you want to study functional interactions between brain structures, you have to do experiments that target specific neuronal subtypes in specific locations and manipulate their activity while you are studying the effects on other parts of the neural systems that are mediating the behavior. It’s the integration of techniques that will let you test very specific hypotheses about the interaction between neurons in different parts of the brain on each other and also on behavior. Hopefully all of us in 10 or 20 years will have gotten to the point where we can start to manipulate the neurons that we have studied, and predict their effects on processing in other neurons in other brain structures, as well as their effects on behavior.

LEE: We’re finding that very different functions, like memory and perception and motor control, are not handled separately in the brain. There’s a lot of overlap. I don’t think they are truly identical, but the degree of separation across these different systems is much less compared to what a lot of people thought in the past. That makes it more important for us to try to figure out how these anatomical structures in the brain actually coordinate their activity and work together.

TKF: In physics, there is a pursuit to identify simple, universal laws of nature. Do you think this is possible in neuroscience--to find some underlying simplicity to all the complexity?

WANG: It comes back to the notion of building blocks. The hope is that if we can understand with our magnifying lens the mechanisms for some very important basic computations, such as how neuronal circuits accumulate and value information about different choices, then this understanding will become the basic building blocks that we can put together to explain all kinds of possible complex thinking and behaviors. In my lab we are currently putting those things together in a larger-scale brain circuitry model to explain how your brain switches from task to task, which is rather complex and involves a number of different processes. We and other researchers in the field are testing the idea that when you put those building blocks together in a circuit, you may be able to explain more complex behavior. That’s the hope.

LEE: I’m skeptical that something like a universal law that describes the decision making function of the brain is going to happen in the foreseeable future. My favorite analogy for the brain is that it’s like a toolbox where you have this random collection of tools, and you have to grab something that is most appropriate to solve the particular problem. Since the design of the toolbox may not be completely rational, it may not be easy to come up with one set of equations for how the tools are collected in the first place.

SALZMAN: Another one of the real challenges in terms of understanding decision making is figuring out how the brain represents new situations in order to understand how the brain makes decisions about such new situations. But our brains are not big enough, and we don’t have enough neurons, to represent every situation that we might possibly encounter. It’s a very fundamental question we are going to have to answer in order to put the whole puzzle together.

TKF: So what do you envision as the practical applications of this research?

LEE: The most immediate one is to understand the biological basis of mental disorders.

SALZMAN: We really don’t understand at a systemic level how psychiatric disorders result from dysfunction of neural circuits that produce different cognitive and emotional symptoms, and that thereby affect decision making. We also don’t understand how psychiatric treatments work -- how they change neural activity in the brain. The reason we know almost nothing about this is because we really don’t know in detail enough about how those neural circuits work normally. It’s like bringing your car to a car mechanic and the mechanic doesn’t know how the car works normally, so how in the world can he fix your car? Some people interested in psychiatric disorders will do more clinically oriented research and that’s because you can make contributions to understanding what is the better treatment for patients right now. But if you are asking what will be the better treatment in 20, 40 or 100 years, those treatments are only going to arise if we come up with a better understanding of about how cognition and emotion work and interact in the brain to produce things like decisions, because then we can begin to understand how they become dysfunctional. That kind of long-term investment in the basic science of the brain is going to be key to figuring out a wide range of psychiatric disorders.

WANG: Yes. In fact, as an example, our new knowledge about the cellular and circuit mechanisms of working memory and decision processes in the brain has already had a significant impact on clinical studies of mental illness. For instance, addiction is fundamentally a problem of making bad choices, resulting from impaired reward signaling and decision-making circuits in the brain. Understanding these circuits has become key to linking genes and molecules with behavior in clinical studies.

LEE: We also need to understand the neurobiological basis for individual variability in decision making. When people face the same decision, they tend to make different choices. Some of that is due to their different experiences and learning environment. There are also fundamental genetic differences that give rise to different decision making styles. Getting a better understanding of the neurobiological basis for those individual differences in decision making will have enormous implications. It can explain a lot of problems in our society, including differences in the tendency to develop psychiatric illnesses.

SALZMAN: I agree. I talk about how the brain works, but obviously everyone has a different brain. So hopefully we’ll uncover principles that apply to most brains, but at the same time if you really want to start extending this to clinical populations, you’re absolutely going to have to understand not just how the prototypical brain works, but you have to understand the variability in the brain and how that’s related to variability in individual processing of information, personality differences, and emotional processing differences. That will be absolutely essential and that’s going to be even a more challenging problem than the one we’re currently facing. It’s challenging for a number of reasons, including the difficulties in finding appropriate animal models for studying this. The good news is I don’t think any of us is running out of things to do so we’ll have a job for our entire careers as long as there’s funding and a long-term commitment to solving these problems. There’s no danger of us figuring it all out anytime soon.