Grants and Contributions:

Biological Grounding of Continuous Reinforcement Learning via the Basal Ganglia

1008399

$202,070.00

Sep 1, 2023 - Mar 31, 2026

The aim of this project is to take an existing high-level abstract model of how reinforcement learning (RL) occurs, and ground it in detailed biology and molecular-level dynamics of the basal ganglia. That is, the project team will show how these low-level molecular details combine with biological structures (neurons and synapses in the basal ganglia) to produce overt behavior. Animals and humans can select actions that are continuous, e.g. how hard to turn a knob, or that are extended through time, e.g. reach and grasp trajectories. The basal ganglia in the mammalian brain is thought to be associated with this kind of action selection, acting as a control mechanism for initiating actions. Models of the basal ganglia, however, are generally based on reinforcement learning theory that treat both action spaces and time as inherently discrete. However, in recent work (funded )under the NRC’s AI for Logistics program), a completely continuous theory has been developed. This theory, while compatible with the biology of the basal ganglia, is specified at a high level with abstract components and variables. This project proposes to connect this theory to the actual biological details of the basal ganglia at the level of biological connectivity, neurotransmitters, and other molecules. The primary focus will be on two areas: the biological realism of the inputs to the basal ganglia, including reward representations; and the biological realism of the continuous outputs from the basal ganglia, enabling selection in continuous action spaces. Given the resulting model, the team will then explore the effects of varying low-level molecular details (such as the concentrations of GABA, glutamate, cholesterol, etc.) on high-level behaviour such as navigating T-mazes and the Morris Water Maze.

National Research Council Canada

In the short term, anticipated outcomes will be strengthened collaborations across industry, academia, and government to support research excellence. In the medium term, anticipated outcomes will be the development of new and potentially disruptive technologies with collaborators. In the long term, find collaborative solutions to public policy challenges and create stronger innovation systems.

Waterloo, Ontario, CA N2L 3G1

172-2023-2024-Q2-1008399

Grant

Grants and Contributions

119260685

Academia

This agreement has been amended 1 time(s). The end date of this agreement has been modified by 211 days.

Aug 7, 2025

University of Waterloo

Waterloo

35114

Collaborative Science, Technology and Innovation Program - Collaborative R&D Initiatives

Collaborate on multiparty research and development programs to catalyze transformative, high-risk, high-reward research with the potential for game-changing scientific discoveries and technological breakthroughs in priority areas.

541710