1287_ch11.pdf

Learning Dynamics

of Reaching

Reza Shadmehr, Opher Donchin, Eun-Jung

Hwang, Sarah E. Hemminger, and Ashwini K. Rao

CONTENTS

Abstract

11.1 Introduction

11.2 Neural Correlates of Learning Internal Models of Dynamics

11.3 Generalization As a Function of Arm Position

11.4 Computing an Internal Model with a Population Code

11.5 Inferring Coding of Limb Position and Velocity from Patterns of

Generalization

11.6 Generalization from One Arm to the Other

11.7 Activity Fields with Respect to Color of the Target

11.8 Inferring Shape of the Tuning Curves from Patterns of Generalization

11.9 Relating the Inferred Activity Fields to the Neurophysiology of the

Motor System

11.10 Consolidation

11.11 Major Shortcomings of the Theory

11.12 Summary

References

ABSTRACT

When one moves one’s hand from one point to another, the brain guides the arm by

relying on neural structures that estimate the physical dynamics of the task. For

example, if one is about to lift a bottle of milk that appears full rather than empty,

the brain takes into account the subtle changes in the dynamics of the task and this

is reﬂected in the altered motor commands. The neural structures that compute the

task’s dynamics are “internal models” that transform the desired motion into motor

commands. Internal models are learned with practice and are a fundamental part of

voluntary motor control. What do internal models compute, and which neural struc-

tures perform that computation? We approach these problems by considering a task

where physical dynamics of reaching movements are altered by force ﬁelds that act

on the hand. Experiments by a number of laboratories on this paradigm suggest that

internal models are sensorimotor transformations that map a desired sensory state

0-8493-1287-6/05/$0.00+$1.50

of the arm into an estimate of forces, i.e., a model of the inverse dynamics of the

task. If this computation is represented as a population code via a ﬂexible combi-

nation of basis functions, then one can infer activity ﬁelds of the bases from the

patterns of generalization. We provide a mathematical technique that facilitates this

inference by analyzing trial-to-trial changes in performance. Results suggest that

internal models are computed with bases that are directionally tuned to limb motion

in intrinsic coordinates of joints and muscles, and this tuning is modulated multi-

plicatively as a function of the static position of the limb. That is, limb position acts

as a gain ﬁeld on directional tuning. Some of these properties are consistent with

activity ﬁelds of neurons in the motor cortex and the cerebellum. We suggest that activity

ﬁelds of these cells are reﬂected in human behavior in the way that we learn and

generalize patterns of dynamics in reaching movements.

11.1 INTRODUCTION

The arm has inertial dynamics that dictate a complex relationship between motion

of the joints and torques. In order to reliably produce even the most simple move-

ments — for example, ﬂexion of the elbow — the brain must activate not only elbow

ﬂexors, but also shoulder ﬂexors that counter the shoulder extension torque that is

produced by the acceleration of the elbow. The importance of these interaction forces

was quite apparent when engineers were trying to control motion of robots.

This has led to the idea

passive properties of muscles are not enough

to compensate for the complex physics of our limbs. Rather, the brain must

predict

the speciﬁc force requirements of the task in generating the motor commands that

eventually reach the muscles.

To illustrate this idea, consider picking up an opaque carton of milk that appears

full but has been drained empty. The brain overestimates the mass of the carton by

only a couple of pounds (the weight of the missing milk) yet the error is sufﬁcient

so that the resulting motor commands produce a jerky motion of the hand. The visual

appearance of the bottle apparently retrieves a motor memory in a neural system

that predicts the forces that are necessary to move the bottle. Motor commands are

constructed based on this prediction and the predicted forces must be accurate if we

are to produce smooth movements.

The accuracy of force prediction is particularly important for control of our arm

because our hands evolved in large part to support manipulation. For example, a trip

to your local natural history museum will conﬁrm that the hand of a chimpanzee

has a much longer palm length as compared to a human hand. This means that while

we can easily touch our index ﬁnger to our thumb and hold an object, say a string

that is attached to a yo-yo, a chimpanzee’s hand is poorly suited for this. Holding

different objects can dramatically change the mechanical dynamics of our arm. The

neural system that predicts force properties would have to be able to accommodate

this variability and adapt. To study the properties of the neural system with which

the brain learns to predict forces, we have used a paradigm ( Figure 11.1 ) where arm

Yet the

principle is the same for control of biological limbs, as has been conﬁrmed in

electromyographic (EMG) recordings from the human arm.

that, contrary to earlier hypotheses,

Experimental setup and typical data. (A) Subjects hold the handle of the robot

and reach to a target. The plot shows hand trajectory (dots are 10 msec apart) for typical

movements to eight targets. (B) Examples of two force ﬁelds produced by the robot.

(D) Simulation results for movements in the saddle ﬁeld. (E) Hand trajectories during catch

trials. (F) Simulation results during catch trials. The controller in this simulation had fully

adapted to the ﬁeld and was expecting the ﬁeld to be present in these movements. (Adapted

from Reference 4, with permission.)

FIGURE 11.1

The subject is

provided with a target and asked to reach while holding the handle of a robot. When

the robot’s motors are off (null ﬁeld condition), movements are straight

( Figure 11.1A ) . The forces in the ﬁeld typically depend on the velocity of the hand

(Figure 11.1B). When the ﬁeld is applied, movements are perturbed (Figure 11.1C).

With practice, hand trajectories once again become smooth and nearly straight. The

brain’s ability to modify motor commands and predict the novel forces is revealed

as a sudden removal of force in

. Very early in training, the hand’s

trajectory in the catch trials is a straight path to the target. With further training,

trajectories in ﬁeld trials become straight. More importantly, the trajectories in catch

trials (Figure 11.1E) become an approximate mirror image of the early ﬁeld trials

(Figure 11.1C). The trajectories in these catch trials are called after-effects.

Improvement in performance occurs because training results in a change in the

motor commands. One possibility is that movements improve because subjects co-

contract antagonist muscle groups. This motor strategy can be sufﬁcient to resist

perturbations imposed by the robot. However, in a catch trial, this kind of adaptation

would not produce any after-effects.

An alternate hypothesis is that the composition of motor commands by the brain

relies on a neural system that, for any given movement direction, predicts the forces

that will be imposed on the hand by the robot. One way to do this is to imagine a

tape that is played out as a function of time for each movement direction. This tape

may be an average record of forces that were sensed in the previous movements in

that direction. Mathematically, the inputs to this system are direction and time and

the output is force. To test this idea, Conditt et al.

catch trials

trained subjects to reach to a

small number of targets in a force ﬁeld and then suddenly asked them to draw a circle

in the same ﬁeld. They reasoned that if what was learned was like a tape recording

of the forces encountered in reaching to each target, then the neural system that had

been trained to predict forces in short, brief reaching movements should contribute

little to longer, circular movements. However, they found that performance was quite

good in circular movements when the ﬁeld was on and, importantly, the subjects

showed after-effects when the ﬁeld was off.

This suggested that the neural system did not predict forces explicitly as a

function of time. Rather, in performing the reaching movements, the neural system

had learned to associate the sensory states of the limb — especially limb position

and velocity — to forces. The particular order in which those states were visited and

the trajectory at the time they were visited (e.g., in a straight line trajectory or in a

curved movement) was immaterial. What was important was the region of the state

space — the limb’s velocity at a given position — that the reaching movements had

visited. If the temporal order of the states were changed from the “training set” in

which the system had experienced the forces, the neural system could still predict

forces because the states themselves were part of the initial training set.

However, one could argue that the reason why the subjects learned to associate

states to forces, rather than some other input that explicitly included time, was

because the force ﬁeld that was imposed on the hand was itself not explicitly time-

dependent. Rather, it was dependent on hand velocity. Conditt and Mussa-Ivaldi

tested this by asking whether subjects could adapt to force ﬁelds that explicitly

dynamics are systematically changed through forces on the hand.

depended on time. Remarkably, the experimental results indicated that they could

not. When a predictable, time-dependent pattern of force was imposed on reaching

movements, generalization trials (circular movements) suggested that subjects still

learned to associate states of the arm to forces. Therefore, the brain’s ability to

predict force did not explicitly depend on movement time. Rather, that prediction

depended on an input that described the desired state of the arm.

These experiments suggested that with practice, participants learned a sensory

to motor transformation where a velocity-like input signal was transformed into a

force-like output signal. This is an

internal model

of the force ﬁeld.

11.2 NEURAL CORRELATES OF LEARNING INTERNAL

MODELS OF DYNAMICS

We have not speciﬁed how information is represented in this internal model, or how

this information is acquired through experience. All we can say at this point is that

at the start of training the internal model is “empty” (i.e., it predicts zero force for

all input states) and, after a long period of training, it has adapted in the sense that

it correctly predicts forces that are produced for typical states visited in reaching

movements. However, there is sufﬁcient information in this statement to allow us

to test whether our formulation thus far is consistent with measurements.

If a simulation of the dynamics of the arm acquires an internal model of a force

ﬁeld, what will its trajectories of motion look like? The dynamics of the arm (in

this case, a two-joint planar system) are derived from Newton’s laws and are written

as equations that describe how the limb’s acceleration depends on forces. They

describe how the mass of the limb responds to force input from the muscles. To

represent the error feedback system of the muscles and the spinal reﬂexes, we add

to the equations a simple low-gain spring-damper element that stabilizes the limb

about the desired trajectory (the straight line). To produce a movement, we assume

that the joint torques are commanded based on knowledge of the inverse dynamics

of the limb, i.e., a map that transforms the desired sensory state of the limb into

torques so that it compensates for the arm’s inertial dynamics. This is an internal

model of the arm’s physical dynamics. These equations have been detailed in Shad-

mehr and Mussa-Ivaldi.

Now we

change the internal model so that it completely takes into account the added dynamics

of the force ﬁeld. That is, we assume that the internal model is fully trained. If we

now simulate a catch trial, the resulting movement (Figure 11.1F) is an approximate

mirror image of the ﬁeld trials early in training. Therefore, the trajectories that

we had recorded in the reaching movements of our subjects are consistent with

learning an internal model that transformed desired sensory states into forces.

It is an easy next step to extend the mathematical formulation and predict not

just the limb’s trajectory before and after the internal model adapts to imposed forces,

4,7

Initially in training, the simulated internal model has no knowledge of the robot-

imposed forces. Because of this, the simulated arm does not move straight to the

target (Figure 11.1D). Rather, it moves along a trajectory that is similar to what we

have recorded in our participants, that is, a peculiar hooking pattern.

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika: