Active sensing Ehud Ahissar 1
Active sensing Passive vs active sensing (touch) Comparison across senses Basic coding principles -------- Perceptual loops Sensation-targeted motor control Proprioception Controlled variables Active vibrissal touch: encoding and recoding 2
Eye movements during fixation 3
sensory encoding: What receptors tell the brain Sensory organs consist of receptor arrays: somatosensation audition vision ~200 µm Finger pad 10 µm cochlea 10 µm retina Spatial organization => Spatial coding ( which receptors are activated ) Movements => Temporal coding ( when are receptors activated ) 4
Temporal coding in action 5
Coding space by time 1. Spatial frequency 2. Spatial phase 6
Touch: Temporal encoding of spatial features Darian-Smith & Oke, J Physiol, 1980 anesth. monkey, MR fibers 7
RA fiber Vel - constant f = SF * V dt = dx / V 8
SF Vel SA fiber 9
SF Vel RA fiber V1 V2 V3 G1 G2 G3 10
SF Vel PC fiber 11
Coding ranges 12
Temporal filtering (by intrinsic factors) eye K P M W X Y whisker 0.5 2 8 32 Frequency (Hz) Frequency (Hz) finger SA RA PC 1 10 100 1000 Frequency (Hz)
Coding space by time 1. Spatial frequency 2. Spatial phase 14
Vision: Temporal encoding due to eye movement space Veye RF(1) RF(2) space retinal outputs 1 2 time 15
Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 16
Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 17
Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 18
Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 19
Spatial vs temporal coding Spatial Temporal faster better resolution scanning allows sensing in between receptors 20
Passive vs Active sensing of stationary objects Passive Active threshold low high accuracy low high Systems involved sensory Sensory + motor coding spatial Spatial + temporal Processing speed fast slow Used in detection Exploration Localization Identification 21
Central processing of touch where touch begins? Text book: at the receptors 22
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 23
Sensory-motor loops of the vibrissal system Cortex E D C Identification ( what ) Localization ( where ) B Whisking A m a The old view Thalamus WT T W Brainstem WT T W 24
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 25
Cortex E D C Identification ( what ) Localization ( where ) B Whisking A m a Thalamus WT T W Brainstem WT T W 26
Sensory-motor loops of the vibrissal system E D C B A m a WT T W WT T W 27
Central processing of touch where touch begins? Text book: at the receptors Active touch does not begin at the receptors Sensor motion determines the interaction between the receptors and external objects 28
Break?
Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 30
Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 31
Excitation Contraction Coupling Phase 1: Firing of Motor Neuron Phase 2: Release of Neurotransmitter 32
Excitation Contraction Coupling Phase 1: Firing of Motor Neuron Phase 2: Release of Neurotransmitter Phase 3: Muscle contraction 33
Open-loop system Information flows in one direction (from neurons to muscles 34
Open-loop system Information flows in one direction (from neurons to muscles Closed-loop system Information flows in a closed loop: from neurons to muscles and from muscles to neurons What kind of information? 35
Closed-loop system The direct feedback from muscles and joints is mediated by proprioceptive signals Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension 36
Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Location: Fleshy part of the muscle Between muscle and tendon Joint capsule Parallel to muscle fibers Serial to muscle fibers 37 Between bones
Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 38
What proprioceptors encode? 39
Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension From Arthur Prochazka, University of Alberta 40
Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: force f = k 1 F 41
Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: Length + velocity f = k 1 L + k 2 V 0.6 force f = k 1 F angle f = k 1 θ 42
Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: Length + velocity f = k 1 L + k 2 V 0.6 force f = k 1 F angle f = k 1 θ θ θ θ 43
PID control Proportional (to the controlled variable) Integral (of the controlled variable) Derivative (of the controlled variable) Present Past Future θ θ θ 44
Negative feedback loop Characteristic: The effect of a perturbation is in the opposite direction Requirement: The cumulative sign along the loop is negative Function: Can keep stable fixed points 45
Positive feedback loop Characteristic: The effect of a perturbation is in the same direction Requirement: The cumulative sign along the loop is positive Function: amplifies perturbations + 46
Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 47
The stretch reflex probes the control function of muscle spindles 48
Is the loop positive or negative? The stroke stretches the muscle As a result the muscle contracts The result opposes the perturbation => negative FB loop 49
the anatomical loop Muscle spindle excites the motor neuron Motor neuron excites muscle fibers Muscle contraction suppresses spindle response 50
Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: force f = k 1 F Why proprioceptors fire at rest? And why aren t we aware of it? 51
What about the flexor muscles? Positive or negative loop? What is the underlying circuit? Take it as homework may appear in the exam 52
Pain reflex Positive or negative? What is the underlying 53 circuit? Same
Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 54
Break?
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 56
Basic principles of closed-loop control 57
Set point - + f Vd Vs Vm=f(-Vs) Vs=g(Vm) Vs Vm V s0 V m0 Vm V 58
Set point - + f Vd Vs Vm=f(Vd-Vs) Vs=g(Vm) Vs Vm V s0 Vsd V md V m0 Vm V 59
Direct control without direct connection - + f Vd Vs Vm=f(Vd-Vs) Vs=g(Vm) Vs Vm V s0 Vsd V md V m0 Vm V 60
Nested loops + - f 2 - + f Vd Vs Vm=f(Vd-Vs) Vs=g(Vm) Vs Vm V s0 Vsd V md V m0 Vm V 61
Parallel loops + - f 2 Vs Vm2=f(Vd-Vs)Vs=g(Vm2) - + f Vm2 V s0 Vsd V md V m0 Vm2 Vs Vm1 Vs V s0 Vsd Vm1=f(Vd-Vs)Vs=g(Vm1) V V md V m0 Vm1 62
Parallel loops + - f 2 Xs Xm=f(Xd-Xs) Xs=g(Xm) - + f Xm X s0 Xsd X md X m0 Xm Xs Vs Vm Vs Vm=f(Vd-Vs) Vs=g(Vm) V s0 Vsd V V md V m0 Vm 63
Closed loops in active sensing The controlled variables can be - - + f 2 + f Motor (Xm) (velocity, amplitude, duration, direction, ) Sensory (Xs) (Intensity, phase, ) Object (via Xm Xs relationships) (location, SF, identity, ) Xs Vs Vm Xm V 64
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 66
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 67
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 68
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 69
Active sensing in the vibrissal system 70
Sensory signal conduction The vibrissal system 71
whisker Sensory signal conduction The vibrissal system Meisner Merkel Ruffini Lanceolate free endings 72
73
Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 74
Motor control of whiskers Intrinsic muscles 75 Dorfl J, 1982, J Anat 135:147-154
Follicle as a motor-sensory junction Motor signals move the follicle and whisker Follicle receptors report back details of self motion = proprioception Plus perturbations of this motion caused by the external world Dorfl J, 1985, J Anat 142:173-184 76
Motor control of whiskers Intrinsic muscles 78 Dorfl J, 1982, J Anat 135:147-154
Vibrissal proprioception Each follicle contains ~2000 receptors About 20% of them convey pure proprioceptive information 79
Vibrissal system Skeletal system Proprioceptive loop Proprioceptive loop 80
Whiskers come with different muscle sizes Intrinsic muscles 0.5 mm 81 Dorfl J, 1982, J Anat 135:147-154
Whisking behavior reflections of control loops 82
Perception of external objects Object localization What signals must the brain process in order to infer a location of an external object in space? Reafferent + exafferent signals 86
What the whiskers tell the rat brain Reafference: Their own movement ( Whisking ) Exafference: Touch 88
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 89
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 90
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 91
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 92
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 93
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 94
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 95
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 96
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 97
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 98
What the whiskers tell the rat brain Whisking space Whisker position vs. time time 99
What the whiskers tell the rat brain Reafference: Their own movement ( Whisking ) Exafference: Touch 100
What the whiskers tell the rat brain Touch space Whisker position vs. time time 101
What the whiskers tell the rat brain Touch space Whisker position vs. time time 102
What the whiskers tell the rat brain Touch space Whisker position vs. time time 103
What the whiskers tell the rat brain Touch space Whisker position vs. time time 104
What the whiskers tell the rat brain Touch space Whisker position vs. time time 105
What the whiskers tell the rat brain Touch space Whisker position vs. time time 106
What the whiskers tell the rat brain Touch space Whisker position vs. time time 107
What the whiskers tell the rat brain Touch space Whisker position vs. time time 108
What the whiskers tell the rat brain Touch space Whisker position vs. time time 109
What the whiskers tell the rat brain Touch space Whisker position vs. time time 110
What the whiskers tell the rat brain Touch space Whisker position vs. time time 111
Whisking: What the whiskers tell the rat brain How can the brain use this information? space Whisker position vs. time time Touch: contact with object space Whisker position vs. time time 112
Whisking: What the whiskers tell the rat brain How can the brain use this information? space? Whisker position vs. time time Touch: contact with object space? Whisker position vs. time time 113
How can the brain extract the location of the object Whisking: space Whisker position vs. time time Touch: contact with object 114
How can the brain extract the location of the object Whisking: space Whisker position vs. time time Touch: contact with object 115
sensory encoding: What receptors tell the brain Sensory organs consist of receptor arrays: somatosensation audition vision ~200 µm Finger pad 10 µm cochlea 10 µm retina Spatial organization => Spatial coding ( which receptors are activated ) Movements => Temporal coding ( when are receptors activated ) 116
Orthogonal coding of object location Vertical object position is encoded by space Horizontal object position is encoded by time Radial object position is encoded by rate 117
Active sensing The End 118