Active sensing. Ehud Ahissar

Transcription

1 Active sensing Ehud Ahissar 1

2 Active sensing Passive vs active sensing (touch) Comparison across senses Basic coding principles Perceptual loops Sensation-targeted motor control Proprioception Controlled variables Active vibrissal touch: encoding and recoding 2

3 Eye movements during fixation 3

4 sensory encoding: What receptors tell the brain Sensory organs consist of receptor arrays: somatosensation audition vision ~200 µm Finger pad 10 µm cochlea 10 µm retina Spatial organization => Spatial coding ( which receptors are activated ) Movements => Temporal coding ( when are receptors activated ) 4

5 Temporal coding in action 5

6 Coding space by time 1. Spatial frequency 2. Spatial phase 6

7 Touch: Temporal encoding of spatial features Darian-Smith & Oke, J Physiol, 1980 anesth. monkey, MR fibers 7

8 RA fiber Vel - constant f = SF * V dt = dx / V 8

9 SF Vel SA fiber 9

10 SF Vel RA fiber V1 V2 V3 G1 G2 G3 10

11 SF Vel PC fiber 11

12 Coding ranges 12

13 Temporal filtering (by intrinsic factors) eye K P M W X Y whisker Frequency (Hz) Frequency (Hz) finger SA RA PC Frequency (Hz)

14 Coding space by time 1. Spatial frequency 2. Spatial phase 14

15 Vision: Temporal encoding due to eye movement space Veye RF(1) RF(2) space retinal outputs 1 2 time 15

16 Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 16

17 Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 17

18 Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 18

19 Vision: Temporal encoding due to eye movement x space Veye RF(1) RF(2) space retinal outputs 1 2 t time 19

20 Spatial vs temporal coding Spatial Temporal faster better resolution scanning allows sensing in between receptors 20

21 Passive vs Active sensing of stationary objects Passive Active threshold low high accuracy low high Systems involved sensory Sensory + motor coding spatial Spatial + temporal Processing speed fast slow Used in detection Exploration Localization Identification 21

22 Central processing of touch where touch begins? Text book: at the receptors 22

23 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 23

24 Sensory-motor loops of the vibrissal system Cortex E D C Identification ( what ) Localization ( where ) B Whisking A m a The old view Thalamus WT T W Brainstem WT T W 24

25 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 25

26 Cortex E D C Identification ( what ) Localization ( where ) B Whisking A m a Thalamus WT T W Brainstem WT T W 26

27 Sensory-motor loops of the vibrissal system E D C B A m a WT T W WT T W 27

28 Central processing of touch where touch begins? Text book: at the receptors Active touch does not begin at the receptors Sensor motion determines the interaction between the receptors and external objects 28

29 Break?

30 Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 30

31 Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 31

32 Excitation Contraction Coupling Phase 1: Firing of Motor Neuron Phase 2: Release of Neurotransmitter 32

33 Excitation Contraction Coupling Phase 1: Firing of Motor Neuron Phase 2: Release of Neurotransmitter Phase 3: Muscle contraction 33

34 Open-loop system Information flows in one direction (from neurons to muscles 34

35 Open-loop system Information flows in one direction (from neurons to muscles Closed-loop system Information flows in a closed loop: from neurons to muscles and from muscles to neurons What kind of information? 35

36 Closed-loop system The direct feedback from muscles and joints is mediated by proprioceptive signals Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension 36

37 Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Location: Fleshy part of the muscle Between muscle and tendon Joint capsule Parallel to muscle fibers Serial to muscle fibers 37 Between bones

38 Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 38

39 What proprioceptors encode? 39

40 Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension From Arthur Prochazka, University of Alberta 40

41 Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: force f = k 1 F 41

42 Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: Length + velocity f = k 1 L + k 2 V 0.6 force f = k 1 F angle f = k 1 θ 42

43 Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: Length + velocity f = k 1 L + k 2 V 0.6 force f = k 1 F angle f = k 1 θ θ θ θ 43

44 PID control Proportional (to the controlled variable) Integral (of the controlled variable) Derivative (of the controlled variable) Present Past Future θ θ θ 44

45 Negative feedback loop Characteristic: The effect of a perturbation is in the opposite direction Requirement: The cumulative sign along the loop is negative Function: Can keep stable fixed points 45

46 Positive feedback loop Characteristic: The effect of a perturbation is in the same direction Requirement: The cumulative sign along the loop is positive Function: amplifies perturbations + 46

47 Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 47

48 The stretch reflex probes the control function of muscle spindles 48

49 Is the loop positive or negative? The stroke stretches the muscle As a result the muscle contracts The result opposes the perturbation => negative FB loop 49

50 the anatomical loop Muscle spindle excites the motor neuron Motor neuron excites muscle fibers Muscle contraction suppresses spindle response 50

51 Proprioceptive receptor types Name: Muscle spindle receptors Golgi tendon organs Joint receptors Sensitive to: muscle length muscle tension Flexion, extension Encode: force f = k 1 F Why proprioceptors fire at rest? And why aren t we aware of it? 51

52 What about the flexor muscles? Positive or negative loop? What is the underlying circuit? Take it as homework may appear in the exam 52

53 Pain reflex Positive or negative? What is the underlying 53 circuit? Same

54 Motor control Closed loops Proprioceptive feedback Reflexes tool for probing loop function Controlled variables motor vs sensory 54

55 Break?

56 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 56

57 Basic principles of closed-loop control 57

58 Set point - + f Vd Vs Vm=f(-Vs) Vs=g(Vm) Vs Vm V s0 V m0 Vm V 58

59 Set point - + f Vd Vs Vm=f(Vd-Vs) Vs=g(Vm) Vs Vm V s0 Vsd V md V m0 Vm V 59

60 Direct control without direct connection - + f Vd Vs Vm=f(Vd-Vs) Vs=g(Vm) Vs Vm V s0 Vsd V md V m0 Vm V 60

61 Nested loops + - f f Vd Vs Vm=f(Vd-Vs) Vs=g(Vm) Vs Vm V s0 Vsd V md V m0 Vm V 61

62 Parallel loops + - f 2 Vs Vm2=f(Vd-Vs)Vs=g(Vm2) - + f Vm2 V s0 Vsd V md V m0 Vm2 Vs Vm1 Vs V s0 Vsd Vm1=f(Vd-Vs)Vs=g(Vm1) V V md V m0 Vm1 62

63 Parallel loops + - f 2 Xs Xm=f(Xd-Xs) Xs=g(Xm) - + f Xm X s0 Xsd X md X m0 Xm Xs Vs Vm Vs Vm=f(Vd-Vs) Vs=g(Vm) V s0 Vsd V V md V m0 Vm 63

64 Closed loops in active sensing The controlled variables can be f 2 + f Motor (Xm) (velocity, amplitude, duration, direction, ) Sensory (Xs) (Intensity, phase, ) Object (via Xm Xs relationships) (location, SF, identity, ) Xs Vs Vm Xm V 64

65 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 66

66 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 67

67 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 68

68 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 69

69 Active sensing in the vibrissal system 70

70 Sensory signal conduction The vibrissal system 71

71 whisker Sensory signal conduction The vibrissal system Meisner Merkel Ruffini Lanceolate free endings 72

72 73

73 Sensory-motor loops of the vibrissal system Primary Sensory Cortex Secondary Cortex Primary Motor Cortex Zona Incerta VPM-dm VPM-vl Thalamus POm - VL Thalamic Nuclei Sensory extralemniscal Cerebellar/Olivary Red Nucleus Pontine Reticular Nucleus Superior Colliculus Brainstem Reticular Formation Motor Trigeminal Nuclei Brainstem Loop + Facial Nucleus Trigeminal Ganglion Vibrissae 74

74 Motor control of whiskers Intrinsic muscles 75 Dorfl J, 1982, J Anat 135:

75 Follicle as a motor-sensory junction Motor signals move the follicle and whisker Follicle receptors report back details of self motion = proprioception Plus perturbations of this motion caused by the external world Dorfl J, 1985, J Anat 142:

76 Motor control of whiskers Intrinsic muscles 78 Dorfl J, 1982, J Anat 135:

77 Vibrissal proprioception Each follicle contains ~2000 receptors About 20% of them convey pure proprioceptive information 79

78 Vibrissal system Skeletal system Proprioceptive loop Proprioceptive loop 80

79 Whiskers come with different muscle sizes Intrinsic muscles 0.5 mm 81 Dorfl J, 1982, J Anat 135:

80 Whisking behavior reflections of control loops 82

81 Perception of external objects Object localization What signals must the brain process in order to infer a location of an external object in space? Reafferent + exafferent signals 86

82 What the whiskers tell the rat brain Reafference: Their own movement ( Whisking ) Exafference: Touch 88

83 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 89

84 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 90

85 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 91

86 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 92

87 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 93

88 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 94

89 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 95

90 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 96

91 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 97

92 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 98

93 What the whiskers tell the rat brain Whisking space Whisker position vs. time time 99

94 What the whiskers tell the rat brain Reafference: Their own movement ( Whisking ) Exafference: Touch 100

95 What the whiskers tell the rat brain Touch space Whisker position vs. time time 101

96 What the whiskers tell the rat brain Touch space Whisker position vs. time time 102

97 What the whiskers tell the rat brain Touch space Whisker position vs. time time 103

98 What the whiskers tell the rat brain Touch space Whisker position vs. time time 104

99 What the whiskers tell the rat brain Touch space Whisker position vs. time time 105

100 What the whiskers tell the rat brain Touch space Whisker position vs. time time 106

101 What the whiskers tell the rat brain Touch space Whisker position vs. time time 107

102 What the whiskers tell the rat brain Touch space Whisker position vs. time time 108

103 What the whiskers tell the rat brain Touch space Whisker position vs. time time 109

104 What the whiskers tell the rat brain Touch space Whisker position vs. time time 110

105 What the whiskers tell the rat brain Touch space Whisker position vs. time time 111

106 Whisking: What the whiskers tell the rat brain How can the brain use this information? space Whisker position vs. time time Touch: contact with object space Whisker position vs. time time 112

107 Whisking: What the whiskers tell the rat brain How can the brain use this information? space? Whisker position vs. time time Touch: contact with object space? Whisker position vs. time time 113

108 How can the brain extract the location of the object Whisking: space Whisker position vs. time time Touch: contact with object 114

109 How can the brain extract the location of the object Whisking: space Whisker position vs. time time Touch: contact with object 115

110 sensory encoding: What receptors tell the brain Sensory organs consist of receptor arrays: somatosensation audition vision ~200 µm Finger pad 10 µm cochlea 10 µm retina Spatial organization => Spatial coding ( which receptors are activated ) Movements => Temporal coding ( when are receptors activated ) 116

111 Orthogonal coding of object location Vertical object position is encoded by space Horizontal object position is encoded by time Radial object position is encoded by rate 117

112 Active sensing The End 118