1. What changes when we look at a red cup?1. 看一只红色杯子时,究竟发生了什么?

Place a red cup on a pale table. In daylight its color, rim and handle are easy to distinguish. Dim the light: its outline may remain visible after adaptation while its color becomes unreliable. Look beside it: fine details become harder to resolve. Finally, slide it across the table. The object is unchanged, but the light reaching each retinal location changes.把一只红色杯子放在浅色桌面上。白天,杯子的颜色、杯沿和把手都容易分辨;调暗灯光,适应一段时间后可能仍能看到轮廓,却难以可靠辨色;把目光移到杯子旁边,细小纹理又变得难以看清。最后推动杯子:物体没有变,各个视网膜位置接收到的光却随时间变化。

01
One cup, several kinds of information一只杯子,几种不同的信息

Color: the light reflected by the glaze has a spectrum.颜色:杯面反射光包含一段光谱。

Outline: the cup and its background produce different local intensities.轮廓:杯身与背景在局部产生不同的光强。

Motion: moving the cup shifts that boundary across the retinal image.运动:移动杯子,边界便在视网膜上的像中移位。

The small strip isolates an achromatic boundary for later calculations. Its two intensities are chosen examples, not brightness values inferred from this red illustration.右下角抽出一条明暗边界,供后文计算。强度数值是独立设定的例子,不是从这幅红色插图推算出的亮度。
01 · Detecting dim light01 · 暗处为什么逐渐看得见Photon capture sets the available signal; adaptation changes the visual system’s sensitivity.光子捕获决定可用信号,适应过程改变视觉系统的灵敏度。
02 · Seeing red02 · 为什么暗处难辨红色Color depends on comparing cone signals, not on a wavelength label attached to each photon.辨色依赖不同视锥信号的比较,不是给每个光子贴上颜色标签。
03 · Resolving detail03 · 为什么注视处更清楚Optical blur and retinal sampling jointly limit the detail available at each location.光学模糊和视网膜采样共同限制不同位置可保留的细节。

The stages carry different quantities: light intensity and spectrum, membrane voltage, transmitter release, then neural spike trains. We will follow these quantities through the same scene before building filters and event encoders. A pixel value, a membrane potential and a spike count are not interchangeable measurements.这条通路中传递的量依次发生变化:光强与光谱、膜电位、递质释放,随后是神经脉冲序列。下面沿着同一场景追踪这些量,再落实到滤波器和事件编码。像素值、膜电位和脉冲计数是不同的测量,不能直接混用。

2. From an image on the retina to a cellular circuit2. 从视网膜上的像到细胞回路

Light reflected from each point on the cup enters the pupil and is refracted onto the retina. Light from the top of the cup reaches the lower retinal image; the optical image is inverted. “Seeing upright” does not require a second screen inside the brain: spatial relations are represented by neural connections and activity.杯子每一点反射的光经过瞳孔,由眼的光学系统折射到视网膜上。杯子上方的点成像在视网膜较下方,形成倒置的光学图像。我们看见正立的世界,并不意味着脑中还需要一块把图翻转回去的屏幕;空间关系由神经连接和活动来表示。

The cornea provides most of the eye’s refractive power; changing the shape of the lens adjusts focus for different distances. The iris changes the pupil diameter and therefore the light admitted. An optical image forms on the retina, but the optic nerve does not transmit a rectangular array of pixel values: its axons belong to retinal ganglion cells, whose responses have already been transformed by local circuits.角膜提供眼球大部分屈光力,晶状体通过形状变化调节不同距离的焦点。虹膜改变瞳孔直径,从而调节入光量。光学图像落在视网膜上,但视神经传输的并不是一张矩形像素表:其中的轴突来自视网膜神经节细胞,其响应已经过局部回路的处理。

National Eye Institute: cornea, lens, retina and optic nerve美国国家眼科研究所:角膜、晶状体、视网膜与视神经

Where the image meets the retina像落在视网膜的哪里
Oblique eye cutaway showing the lens behind the iris, zonular fibers, layered eye wall and optic nerve / 眼球斜剖面:虹膜后方的晶状体、悬韧带、分层眼球壁与视神经
05Retina / eye wall视网膜/眼球壁
The salmon-colored inner lining is the retina. The cup’s optical image falls here; photoreceptors then convert absorbed light into electrical responses. The clear vitreous is omitted to expose the wall.内侧的鲑红色衬层是视网膜。杯子的光学图像落在这里,感光细胞随后把吸收的光转换为电响应。图中省去透明玻璃体,以显示眼球壁。
Locate the inner lining, then follow the retinal section and cell connections below. The cornea and lens focus light onto this curved surface.先定位眼球内侧的视网膜,再看下方的视网膜切面与细胞连接。角膜和晶状体将光会聚到这片弯曲表面。
02
From an optical image to a neural circuit从光学图像到神经回路
A · The eyeA · 眼球
①②
① Cup in the scene. ② Inverted retinal image. The outlined patch leads to the cell layers.① 场景中的杯子。② 视网膜倒像。框出的局部对应相邻的细胞分层图。
B · A peripheral retinal sectionB · 周边视网膜切面
hνRPEONLOPLINLIPLGCL12345
  1. 1Photoreceptors感光细胞
  2. 2Bipolar cells双极细胞
  3. 3Ganglion cells神经节细胞
  4. 4Horizontal cell水平细胞
  5. 5Amacrine cell无长突细胞
Positions, layer thickness and cell counts are schematic. Rays show image inversion through an equivalent optical element, not a quantitative trace through every refracting surface. Light and neural signals travel in opposite directions through the layer stack. RPE supports the outer segments; ONL/INL/GCL contain nuclei or somata, OPL/IPL contain synaptic connections.位置、层厚与细胞数量为示意。光线以等效光学元件表示成像倒置,不是对每个折射面的定量追迹。光与神经信号在层间的主要传播方向相反。RPE 支持外节;ONL/INL/GCL 容纳细胞核或胞体,OPL/IPL 容纳突触连接。

In most of the retina, incoming light crosses inner neural layers before reaching the photoreceptor outer segments next to the retinal pigment epithelium (RPE). The principal signal route then runs in the opposite direction: photoreceptor → bipolar cell → ganglion cell. Horizontal cells connect across the outer retina; amacrine cells modify signaling in the inner retina. Their lateral interactions contribute to spatial contrast, temporal filtering and other computations.在视网膜的大部分区域,入射光先穿过内层神经组织,才到达色素上皮(RPE)附近的感光细胞外节。主要信号通路随后朝相反方向运行:感光细胞 → 双极细胞 → 神经节细胞。水平细胞在外层提供横向联系,无长突细胞在内层调节信号;这些横向作用参与空间对比、时间滤波等计算。

The central fovea is specialized: overlying layers are displaced sideways and cone sampling becomes dense. The optic disc is a separate location where ganglion-cell axons exit; its lack of photoreceptors creates the anatomical blind spot. RPE cells absorb stray light, recycle visual pigment and remove shed outer-segment material. Müller glia help maintain the ionic and metabolic environment. A functional retina requires both the signaling circuit and this support tissue.中央凹具有特殊结构:覆盖其上的内层向两侧移开,视锥采样更加密集。视盘则是另一处位置,神经节细胞轴突由此离开;由于缺少感光细胞,它对应生理盲点。RPE 吸收杂散光、参与视觉色素循环并清除脱落外节。Müller 胶质细胞协助维持离子与代谢环境。视网膜的正常工作同时依赖信号回路和支持组织。

Webvision: retinal layers, cell types and foveal specializationWebvision:视网膜分层、细胞类型与中央凹结构

Webvision: retinal pigment epithelium and photoreceptor maintenanceWebvision:色素上皮与感光细胞的维护

3. The physical budget: photons, blur and sampling3. 物理约束:光子、模糊与采样

Dimming the lamp reduces the light reflected by the same cup. Before considering any neuron, ask how much evidence can be collected in a short interval. At this stage, the relevant quantity is the number of detected photons, not the amplitude of a neural spike.调暗台灯,同一只杯子反射的光就会减少。在讨论神经元之前,先问一个物理问题:短时间内究竟能收集多少证据?此时关注的是探测到的光子数,不是神经脉冲的幅度。

A dim scene supplies fewer photons during a given integration interval. Under a Poisson counting model, the variance equals the mean: a mean count of 25 has a standard deviation of 5, while a mean of 100 has a standard deviation of 10. The absolute fluctuation grows, but its fraction of the signal falls. Quadrupling collected photons therefore doubles the shot-noise-limited signal-to-noise ratio.暗场景在固定积分时间内提供的光子更少。在泊松计数模型下,方差等于均值:平均计数 25 对应标准差 5,平均计数 100 对应标准差 10。绝对涨落增加了,但它在信号中所占的比例下降。因此,收集光子数增加到四倍,散粒噪声限制下的信噪比才增加到两倍。

For a monochromatic camera-sensor example, mean photoelectron count is set by quantum efficiency η, irradiance E at the pixel, sensitive area A, integration time T and photon energy hc/λ. The equation below makes the exposure trade-off explicit. Photoreceptors instead initiate biochemical responses to absorbed photons; their amplification and neural noise are additional stages, not camera electrons with different names.以单色照明下的相机像素为例,平均光电子数由量子效率 η、像素处辐照度 E、感光面积 A、积分时间 T 与光子能量 hc/λ 决定。下式直接给出曝光取舍。生物感光细胞则由被吸收的光子启动生化响应;其放大与神经噪声属于后续过程,不能直接等同于相机中的电子计数。

Increasing electronic gain scales the signal and the noise already present; it does not increase the number of captured photons. Extending T does collect more photons, but a moving image travels farther during the exposure. For an image velocity v measured in pixels per second, the approximate motion-blur length is vT pixels. Pooling adjacent samples is another way to gain sensitivity, at the cost of spatial detail.提高电子增益会同时放大已有的信号与噪声,并不增加实际捕获的光子。延长 T 可以收集更多光子,却使运动图像在曝光期间移动得更远。若像面速度 v 的单位是像素/秒,则运动模糊长度约为 vT 个像素。合并邻近样本也能提高灵敏度,代价是空间细节减少。

EMVA 1288: quantitative characterization of image sensorsEMVA 1288:图像传感器的定量表征

03
Two limits before neural computation神经计算之前的两道限制
A · Counting noiseA · 计数噪声
010020030040001020SNRμ

4× detected photons → 2× SNR探测光子数 ×4 → 信噪比 ×2

B · DiffractionB · 衍射
D = 2 mm1.15′D = 4 mm0.58′λ = 550 nm

2× aperture → ½ diffraction angle孔径 ×2 → 衍射角 ÷2

Calculated ideal limits, not measurements of a person or a camera. μ is the mean detected count; real systems also have aberrations, sampling limits and other noise sources.计算得到的理想极限,并非对人眼或相机的测量。μ 是平均探测数量;真实系统还受像差、采样与其他噪声限制。

Light is also a wave. Even a perfectly focused point forms a diffraction pattern through a finite circular aperture. At wavelength λ and aperture diameter D, the Rayleigh angle is approximately 1.22λ/D. For λ = 550 nm and D = 2 mm this is 3.36 × 10⁻⁴ rad, or about 1.15 arcminutes. Doubling D halves this ideal angle, but real eyes also have aberrations, scattering and nonuniform receptor sampling.光也具有波动性。即使完全合焦,点光源经过有限圆孔后仍形成衍射斑。波长为 λ、孔径直径为 D 时,瑞利角约为 1.22λ/D。取 λ = 550 nm、D = 2 mm,得到 3.36 × 10⁻⁴ rad,约 1.15 角分。将 D 加倍可使这一理想角度减半,但真实眼睛还存在像差、散射以及不均匀的感光细胞采样。

An engineering image model separates these stages: first convolve scene irradiance with a point-spread function, then integrate over each pixel area, then sample in space and time, and finally add the relevant sensor noise. More pixels cannot recover spatial frequencies already lost in the optical blur. Sampling too coarsely can alias surviving detail into false patterns.工程成像模型应分开描述这些步骤:先让场景辐照度与点扩散函数卷积,再对每个像素面积积分,随后在空间与时间上采样,最后加入相应的传感器噪声。增加像素不能恢复已被光学模糊滤掉的空间频率;采样过疏还会把残留的高频细节混叠成错误纹理。

OpenStax University Physics: circular apertures and the Rayleigh criterionOpenStax 大学物理:圆孔衍射与瑞利判据

4. A brighter patch changes four signals4. 一处增亮,四种信号怎样变化

Follow one cone viewing a small patch near the cup’s rim. Suppose the light on this cone increases while its surroundings remain unchanged. Membrane potential is the voltage inside the cell relative to outside: becoming more negative is hyperpolarization; becoming less negative is depolarization. These words describe voltage, not the number of spikes.跟踪一枚接收杯沿附近局部光线的视锥。假设它接收到的光增强,周围照明保持不变。膜电位是细胞内部相对外部的电压:变得更负叫超极化,变得不那么负叫去极化。这两个词描述电压,不是脉冲数量。

A cone releases glutamate at its synapse even before the increment. Brightening makes its voltage more negative and reduces that release. Its two downstream bipolar pathways interpret the same decrease differently: the ON cone bipolar cell depolarizes, while the OFF cone bipolar cell hyperpolarizes. “ON” and “OFF” name response preference, not two switches that simply turn the whole cell on and off.视锥在增亮之前就通过突触释放谷氨酸。光增强后,它的膜电位更负,谷氨酸释放减少。下游两类双极细胞对同一次减少作出不同响应:ON 型视锥双极细胞去极化,OFF 型视锥双极细胞超极化。ON 和 OFF 表示响应偏好,不是把整个细胞简单打开或关闭的两个开关。

04
One cone input, two bipolar responses同一个视锥输入,两类双极细胞响应
Light at the cone视锥处的光强Same bright interval同一段增亮刺激
t₀t₁
Cone voltage视锥膜电位More negative during light光照时变得更负
t₀t₁
Glutamate release谷氨酸释放Less transmitter released递质释放减少
t₀t₁
Bipolar voltage双极细胞膜电位ON solid · OFF dashedON 实线 · OFF 虚线
t₀t₁
Shading marks the same light increment, t₀–t₁. Curves show response directions on separate vertical scales; their amplitudes and delays are schematic. Horizontal dashed lines mark the pre-stimulus levels.阴影标出同一次增亮的 t₀–t₁ 区间。曲线表示响应方向,各行纵轴尺度独立,幅度与延迟均为示意;水平虚线标出刺激前的水平。

A sign-inverting synapse means that a change at its input causes the opposite direction of voltage change downstream. ON cone bipolar cells use the mGluR6 pathway; OFF cells use ionotropic glutamate receptors. Under dim conditions, rods predominantly reach these output pathways through rod bipolar and AII amacrine circuits, so the cone diagram should not be read as a direct rod wiring diagram.反号突触的意思是:输入发生变化时,下游电位的变化方向与上游相反。ON 型视锥双极细胞采用 mGluR6 通路,OFF 型采用离子型谷氨酸受体。暗光下视杆主要通过视杆双极细胞、AII 无长突细胞等回路进入输出通路,因此不能把上图直接当成视杆的连接图。

Molecular mechanism: why the inward current falls分子机制:内向电流为什么减少

A vertebrate photoreceptor is electrically active in darkness. cGMP keeps cyclic-nucleotide-gated (CNG) channels in its outer-segment membrane open, allowing an inward cation current. Light absorption changes the visual pigment’s conformation, activates transducin and then PDE6, and accelerates cGMP hydrolysis. Fewer CNG channels remain open. The reduced inward current makes the membrane potential more negative: hyperpolarization.脊椎动物的感光细胞在黑暗中并非没有电活动。cGMP 使外节膜上的环核苷酸门控(CNG)通道保持开放,产生内向阳离子电流。光吸收引起视觉色素构象变化,激活转导蛋白及 PDE6,加快 cGMP 水解。保持开放的 CNG 通道减少,内向电流随之下降,膜电位变得更负,这就是超极化。

Purves et al.: the phototransduction cascadePurves 等:光转导的分子级联

This voltage change reduces glutamate release from the photoreceptor terminal. ON cone bipolar cells use a sign-inverting mGluR6 pathway: less glutamate permits depolarization through its downstream mechanism. OFF cone bipolar cells use ionotropic glutamate receptors and preserve the sign: less glutamate reduces their excitation. One light increment can therefore drive ON and OFF channels in opposite directions. “Light means more activity everywhere” is the wrong circuit model.膜电位变化使感光细胞末端释放的谷氨酸减少。ON 型视锥双极细胞采用反号的 mGluR6 通路:谷氨酸减少后,其下游机制允许细胞去极化。OFF 型视锥双极细胞采用离子型谷氨酸受体,保留信号符号:谷氨酸减少使兴奋性输入降低。因此,同一次增亮会使 ON 与 OFF 通道朝不同方向变化;不能把回路理解为“光更强,所有细胞都更活跃”。

Webvision: ON/OFF bipolar signaling and rod pathwaysWebvision:ON/OFF 双极细胞信号与视杆通路

At the retinal output, ganglion cells communicate with action potentials. Stronger input can change their spike timing and firing rate; it does not encode brightness by making every spike proportionally taller. In engineering, this separates a continuously varying state from the discrete events used to transmit it.到达视网膜输出端,神经节细胞通过动作电位传递信息。输入变化可以改变脉冲时刻和放电率,并不是靠让每个脉冲按比例变高来表示更亮。工程上,这对应连续变化的内部状态与用于传输的离散事件这两个层次。

From membrane voltage to spikes: the LIF approximation从膜电位到脉冲:LIF 近似

Photoreceptor and many bipolar responses are graded. Ganglion-cell output uses action potentials: information is carried by spike times, firing rates and the population pattern, not by proportionally scaling spike height. A leaky integrate-and-fire approximation treats the membrane as a capacitor with a leak conductance, driven by synaptic current. Crossing a threshold emits an event and resets the model voltage.感光细胞以及许多双极细胞的响应是渐变的。神经节细胞输出采用动作电位:信息由脉冲时刻、放电率与群体模式携带,而不是通过等比例增高脉冲幅度表达。泄漏积分发放近似将膜视为带漏电导的电容,由突触电流驱动;膜电位越过阈值后产生事件,并复位模型电位。

The membrane time constant τ = Cₘ/gL determines how quickly the model integrates and forgets its input. This is a useful circuit-level approximation; it omits the voltage-gated conductances that generate the biological spike waveform.膜时间常数 τ = Cₘ/gL 决定模型积累和遗忘输入的速度。这是电路层面的有效近似,但没有描述生成真实动作电位波形的电压门控电导。

Gerstner et al.: membrane circuits and integrate-and-fire modelsGerstner 等:膜等效电路与积分发放模型

Webvision: recovery and adaptation in rod/cone phototransductionWebvision:视杆与视锥光转导中的恢复与适应

The LIF dynamics and threshold-crossing derivation develops the membrane model in more detail.LIF 动力学与越阈时刻推导进一步展开了这一膜电位模型。

UTHealth Neuroscience Online: retinal imaging, photoreceptor voltage and ON/OFF signalingUTHealth Neuroscience Online:视网膜成像、感光细胞电位与 ON/OFF 信号

5. Color is a spectral measurement, not a wavelength label5. 颜色是光谱测量,不是波长标签

The red cup reflects a range of wavelengths; it does not send only one “red wavelength.” Change the lamp and the spectrum reaching the eye changes too. The visual system has to compare cone responses under that illumination, while the camera has to account for its own filters and color calibration.红杯子反射的是一段波长范围,不是只发出某一个“红色波长”。换一盏灯,到达眼睛的光谱也会改变。视觉系统需要在这种照明下比较视锥响应,相机则需要考虑自己的滤光片与颜色标定。

The light reaching the eye combines the illuminant spectrum with the surface’s spectral reflectance and viewing geometry. Human S, M and L cones have overlapping sensitivities. A single cone response is ambiguous: fewer photons near its sensitivity peak and more photons away from the peak can produce the same response. Comparing different cone classes provides spectral information unavailable from any one class.到达眼睛的光由照明光谱、物体的光谱反射率以及观察几何共同决定。人的 S、M、L 三类视锥具有重叠的敏感性。单个视锥的响应存在歧义:在敏感峰附近接收到较少光子,与远离峰值处接收到较多光子,可能产生相同响应。比较不同类型的视锥,才能获得单一类型无法提供的光谱信息。

Here Φ(λ) is the incident photon spectrum integrated over the collection area and time; sᵢ includes the wavelength-dependent probability of contributing to that cone’s response. Mapping an entire spectrum to three responses discards information. Different spectra can therefore match in cone response under specified viewing conditions: metamerism. Rod-dominated dim-light vision lacks the same three-cone comparison, which helps explain the loss of reliable color discrimination at low light.这里 Φ(λ) 是在收集面积和时间上积分后的入射光子光谱,sᵢ 包含不同波长对该类视锥响应的贡献概率。把整条光谱映射为三个响应必然丢失信息,因此不同光谱在特定观察条件下可能产生相同的视锥响应,即同色异谱。以视杆为主的暗光视觉缺少同样的三类视锥比较,这有助于解释低照度下可靠辨色能力的下降。

05
Color is inferred from a population response颜色来自多类细胞的联合响应
A · Overlapping sensitivitiesA · 重叠的光谱敏感性
400500600700SMLnm
B · A many-to-one measurementB · 多对一的测量
∫
SML
A spectrum → three responses一条光谱 → 三个响应
Different spectra can give the same triplet.不同光谱可以产生相同的三元响应。
The curves illustrate overlap only; their peak positions and widths are not a calibration dataset. A cone counts absorbed photons without attaching a wavelength label to each photon.曲线仅说明光谱重叠,峰值与带宽不能作为标定数据。视锥对吸收的光子产生响应,不为每个光子附加波长标签。

OpenStax Behavioral Neuroscience: photoreceptors and color codingOpenStax 行为神经科学:感光细胞与颜色编码

A camera’s RGB filters are not the human S/M/L sensitivities. Color reproduction requires calibrated transformations and an assumed or estimated illuminant. White balance corrects channel gains for illumination; it does not reconstruct an arbitrary lost spectrum. For physically meaningful averaging, blur and exposure operations, work in a linear-light space before applying a display transfer function. Averaging gamma-encoded values generally gives a different result from averaging the underlying light.相机的 RGB 滤光片并不等于人的 S/M/L 敏感性。颜色再现需要标定变换,并依赖假定或估计的照明。白平衡校正照明引起的通道增益差异,但不能重建任意丢失的光谱。要对光进行有物理意义的平均、模糊和曝光运算,应先在线性光空间计算,再施加显示传递函数。直接平均经过伽马编码的数值,通常不等于平均它们所代表的光。

W3C CSS Color 4: linear-light color mixingW3C CSS Color 4:线性光空间中的颜色混合

6. From center–surround circuitry to an edge filter6. 从中心—周围回路到边缘滤波

For many ON-center ganglion cells, a bright spot in the center increases firing, while light in the surround suppresses that increase. A large uniformly lit area can excite both components, partly canceling their effects. The spatial arrangement makes a boundary between light and dark regions especially informative. Horizontal-cell and inner-retinal mechanisms both contribute; one circular filter cannot represent every ganglion-cell type or all brightness perception.对许多 ON 中心神经节细胞,中心亮斑使放电增加,周围受光则会抑制这种增加。大范围均匀照明同时激活两个分量,它们的效应部分抵消。这种空间组织使明暗区域之间的边界成为重要信息。水平细胞与内层视网膜机制均有贡献;一个圆形滤波器不能代表所有神经节细胞类型,也不能解释全部亮度知觉。

Webvision: receptive-field centers, surrounds and ganglion-cell responsesWebvision:感受野中心、周围与神经节细胞响应

A cell’s receptive field is the region of visual space whose illumination affects its response. Imagine moving a small measuring window across the cup’s edge. A center–surround comparison asks whether the middle differs from its neighbors, so it responds to a boundary even when the two broad regions on either side are uniform.一个细胞的感受野,是照明变化会影响它响应的那片视觉空间。把一个小测量窗口沿杯沿平移:中心—周围比较问的是中间与邻域是否不同。因此,即使边界两侧各自均匀,经过边界时仍会产生响应。

First use only three samples: C is the middle intensity, S is the mean of the two neighbors, and r = C − S. The window and weights stay fixed in all three positions. The outlined square marks the center.先只取三个样本:C 是中间强度,S 是两侧邻居的平均值,r = C − S。以下三个位置使用同一个窗口和同一组权重,带框的格子表示中心。

06
The same window, three positions同一个窗口,三个位置

Uniform region均匀区域

202020
C = 20S = (20 + 20) / 2 = 20r = 20 − 20 = 0

Dark side of the edge边界暗侧

202080
C = 20S = (20 + 80) / 2 = 50r = 20 − 50 = -30

Bright side of the edge边界亮侧

208080
C = 80S = (20 + 80) / 2 = 50r = 80 − 50 = 30
Intensities are arbitrary linear units. This three-tap zero-sum filter explains the subtraction: its weights are −½, 1, −½. It is not a measured receptive field or the Gaussian/log-domain implementation below. Negative and positive responses identify the two sides of the boundary.强度使用任意线性单位。三点零和滤波器的权重为 −½、1、−½,用来算清楚相减过程;它不是实测感受野,也不是下方的高斯/对数域实现。负、正响应分别标记边界的两侧。

For a smoother engineering approximation, replace the single center and two neighbors by narrow and broad Gaussian averages. Below, those averages operate on log intensity. The resulting q is still “center minus surround,” but its numerical scale differs from the three-point example. A flat region cancels, and the edge leaves a negative and a positive lobe.更平滑的工程近似用窄、宽两个高斯平均代替单个中心和两侧邻居。下图在对数强度上计算这些平均,q 仍是“中心减周围”,但数值尺度与三点例子不同。均匀区域相互抵消,边界留下负、正两个响应波瓣。

07
An edge becomes two signed responses一条边界被分解为正负响应
A · Center minus surroundA · 中心减周围
-9090.40k
G₁G₃W = G₁ − G₃
B · Filtered log intensityB · 对数强度的滤波结果
2080I(x)0OFFONq(x)x
Computed from the discrete kernels and replicated boundary samples used in the C++ program below. Each Gaussian is normalized after truncation to ±9 pixels. The dashed curve is W. OFF = max(−q, 0); ON = max(q, 0).按下方 C++ 程序的离散核与边界复制方式计算。两个高斯核截断到 ±9 像素后分别归一化,虚线表示 W。OFF = max(−q, 0),ON = max(q, 0)。
Gaussian model, exposure invariance and runnable C++高斯模型、曝光不变性与可运行的 C++

A useful engineering approximation subtracts two normalized Gaussian averages. The narrow kernel samples a local center; the wider kernel estimates its neighborhood. For a one-dimensional signal, let W[k] = G₁[k] − G₃[k], where each sampled kernel is normalized after truncation. This guarantees a zero-sum filter even with a finite implementation.一种实用的工程近似是将两个归一化高斯平均相减。窄核采样局部中心,宽核估计邻域。对于一维信号,令 W[k] = G₁[k] − G₃[k],每个离散核都在截断后重新归一化。这样,有限长度实现仍能保证滤波核总和为零。

Apply W to z[x] = ln(I[x]/Iref), with strictly positive intensities and a fixed positive reference. A uniform exposure multiplier a then adds the constant ln(a), which the zero-sum filter removes. This exact property belongs to this log-domain model; it is not a claim that every retinal neuron is perfectly exposure invariant. Clipping, zeros, sensor noise and nonlinear adaptation break the idealization.令 z[x] = ln(I[x]/Iref),其中强度严格为正、参考值固定且为正,再施加 W。全局曝光乘数 a 会在对数域增加常数 ln(a),而零和滤波器会去掉这一常数。这是该对数域模型的严格性质,并不意味着所有视网膜神经元都完全不受曝光影响。饱和截断、零值、传感器噪声与非线性适应都会破坏这一理想化条件。

The program uses a 20-to-80 intensity step and replicated boundary samples. It checks two consequences of the model: a constant field gives zero response, and multiplying every intensity by three preserves q. ON and OFF below are nonnegative filter outputs, not measured firing rates.程序使用从 20 到 80 的强度阶跃,并以复制边界样本处理越界。它检查模型的两个结论:恒定场响应为零;全部强度乘以三后 q 不变。下方 ON 与 OFF 是非负滤波输出,不是实测放电率。

#include <algorithm>
#include <cmath>
#include <iomanip>
#include <iostream>
#include <vector>

std::vector<double> gaussian(double sigma, int radius) {
    std::vector<double> g(2 * radius + 1);
    double sum = 0;
    for (int k = -radius; k <= radius; ++k) {
        g[k + radius] = std::exp(-0.5 * k * k / (sigma * sigma));
        sum += g[k + radius];
    }
    for (double& value : g) value /= sum;
    return g;
}

std::vector<double> response(const std::vector<double>& image) {
    constexpr int radius = 9;
    const auto center = gaussian(1, radius);
    const auto surround = gaussian(3, radius);
    std::vector<double> out(image.size(), 0);
    for (int x = 0; x < static_cast<int>(image.size()); ++x) {
        for (int k = -radius; k <= radius; ++k) {
            const int j = std::clamp(x - k, 0,
                                    static_cast<int>(image.size()) - 1);
            out[x] += (center[k + radius] - surround[k + radius])
                      * std::log(image[j]); // Iref = 1; image[j] > 0
        }
    }
    return out;
}

int main() {
    std::vector<double> image(61, 20);
    std::fill(image.begin() + 30, image.end(), 80);
    const auto q = response(image);
    for (double& value : image) value *= 3;
    const auto brighter = response(image);
    const auto flat = response(std::vector<double>(61, 50));
    double exposure_error = 0, flat_error = 0;
    for (std::size_t i = 0; i < q.size(); ++i) {
        exposure_error = std::max(exposure_error,
                                 std::abs(q[i] - brighter[i]));
        flat_error = std::max(flat_error, std::abs(flat[i]));
    }
    std::cout << std::fixed << std::setprecision(6)
              << "dark-side OFF = " << std::max(-q[29], 0.0) << '\n'
              << "bright-side ON = " << std::max(q[30], 0.0) << '\n'
              << std::boolalpha
              << "flat field ~ 0: " << (flat_error < 1e-12) << '\n'
              << "exposure invariance: " << (exposure_error < 1e-12) << '\n';
    return flat_error < 1e-12 && exposure_error < 1e-12
           && q[29] < 0 && q[30] > 0 ? 0 : 1;
}

This direct implementation makes the operator explicit. For two-dimensional images, Gaussian separability reduces a K × K convolution to horizontal and vertical K-tap passes; subtract the two blurred images afterward. Boundary treatment and kernel normalization must remain consistent when comparing implementations. Benchmark throughput only after checking the numerical result.这一直接实现把运算写得清楚。对于二维图像,高斯核可分离性将 K × K 卷积降为水平、垂直两次 K 点滤波,随后再将两张模糊图相减。比较实现时,边界处理与核归一化必须一致。先验证数值结果,再做吞吐量基准测试。

7. From temporal contrast to event cameras7. 从时间对比到事件相机

Now slide the cup across the table and hold one image location fixed. As the boundary passes it, that location changes from background to cup, or from cup to background. The spatial edge in the previous section has become a time-varying input. This connects edge filtering to motion-sensitive measurements.现在沿桌面推动杯子,固定观察像上的一个位置。边界经过时,这个位置会从背景变成杯身,或从杯身变回背景。上一节空间上的边界,变成了这个位置随时间变化的输入;边缘滤波与运动测量由此联系起来。

Retinal output contains multiple temporal response types, including sustained and transient channels. Adaptation changes sensitivity according to recent input; eye movements also change the retinal image of an otherwise static scene. Engineering can borrow the principle of reporting change without reproducing the entire biological circuit. A dynamic vision sensor implements a particularly concrete version at each pixel.视网膜输出包含持续型与瞬态型等多种时间响应。适应使敏感性随近期输入调整;即使场景静止,眼动也会改变视网膜上的像。工程上可以借鉴“报告变化”的原则,而不必复制整套生物回路。动态视觉传感器在每个像素上实现了一个明确的版本。

An ideal event pixel compares log intensity L(t) with a stored reference. Once their difference reaches +C or −C, it emits an event with position, timestamp and polarity, then advances its reference by that signed threshold. Equal log thresholds correspond to equal ratios in opposite directions, not equal percentage increases and decreases. With C = ln(1.2), the positive step is +20%, while the negative step is about −16.7%.理想事件像素将对数强度 L(t) 与保存的参考值比较。差值达到 +C 或 −C 时,它发出带位置、时间戳和极性的事件,并将参考值沿相应方向更新一个阈值。对称的对数阈值对应相反方向的等比例变化,而不是相同百分比的增减。C = ln(1.2) 时,正变化为 +20%,负变化约为 −16.7%。

08
Transmit changes at the time they occur在变化发生时传输变化
A · One ideal event pixelA · 一个理想事件像素
0C2C0204060ln I − ln I₀ms
Log intensity对数强度Reference参考值
B · A threshold, not a frame clockB · 阈值触发,不是帧时钟
C = ln 1.2+20% from the reference → ON相对参考值 +20% → ON
exp(−C) = 1 / 1.2About −16.7% → OFF约 −16.7% → OFF
(x, y, t, p)Address · time · polarity位置 · 时刻 · 极性
Analytical example: I rises exponentially from 100 to 144 between 10–20 ms, remains constant until 40 ms, then falls to 100 by 50 ms. The ideal events occur at 15, 20, 45 and 50 ms; the reference moves by ±C after each event.解析示例:I 在 10–20 ms 从 100 指数上升至 144,保持到 40 ms,再于 50 ms 降至 100。理想事件发生在 15、20、45、50 ms,每次事件后参考值移动 ±C。

The following samples include every threshold-crossing time in the plotted trajectory. The output is 15 ms ON, 20 ms ON, 45 ms OFF and 50 ms OFF. With more coarsely sampled data, this loop can recover event counts per interval but not the original crossing times; a real event sensor detects changes asynchronously.下列样本包含图中轨迹的每个越阈时刻,输出依次为 15 ms ON、20 ms ON、45 ms OFF、50 ms OFF。若输入采样更稀疏,这个循环只能恢复每个采样间隔内的事件数量,无法恢复原始越阈时刻;真实事件传感器是异步检测变化的。

C++: reproduce the four threshold crossingsC++:复现四次越阈事件
#include <array>
#include <cmath>
#include <iostream>

int main() {
    const double C = std::log(1.2);
    const std::array<int, 8> time_ms{0, 10, 15, 20, 40, 45, 50, 60};
    const std::array<double, 8> log_ratio{0, 0, C, 2*C, 2*C, C, 0, 0};
    double reference = log_ratio.front();
    int count = 0;
    for (std::size_t i = 1; i < time_ms.size(); ++i) {
        // The tolerance only handles floating-point roundoff.
        while (std::abs(log_ratio[i] - reference) >= C - 1e-12) {
            const int polarity = log_ratio[i] > reference ? 1 : -1;
            reference += polarity * C;
            std::cout << time_ms[i] << " ms "
                      << (polarity > 0 ? "ON" : "OFF") << '\n';
            ++count;
        }
    }
    return count == 4 ? 0 : 1;
}

A constant scene ideally produces no events only when its image is also stationary and sensor noise is absent. Motion or illumination flicker can generate a dense stream. Threshold mismatch, background activity and bandwidth limits matter in hardware. Event polarity resembles an ON/OFF distinction, but an event camera’s threshold circuit is not a retinal ganglion cell or the LIF membrane model above.只有像面也保持静止、且没有传感器噪声时,恒定场景才会在理想情况下不产生事件。运动或照明闪烁可以带来密集事件流。硬件中的阈值不一致、背景活动与带宽限制都不可忽略。事件极性具有 ON/OFF 的区分,但事件相机的阈值电路并不等于视网膜神经节细胞,也不等于前面的 LIF 膜模型。

Gallego et al., Event-based Vision: A Survey — sensor model and limitationsGallego 等,《Event-based Vision: A Survey》——传感器模型与限制

8. From neural signals to perception and engineering8. 从神经信号到视觉知觉与工程系统

We now have local changes and spike trains, but have not yet explained seeing a cup. The optic nerve carries the outputs of many retinal circuits. Their spatial arrangement matters: neighboring image locations remain systematically related in subsequent maps, while multiple pathways process different aspects of the scene.到这里已有局部变化和脉冲序列,却还没有解释怎样看见“一只杯子”。视神经携带许多视网膜回路的输出。这些信号的空间关系很重要:相邻成像位置在后续映射中仍有系统性的对应,多条通路并行处理场景的不同方面。

09
From local responses to seeing the cup从局部响应到看见杯子

01 · Retina01 · 视网膜

Parallel signals for local contrast, color and temporal changes leave as ganglion-cell spike trains.局部对比、颜色与时间变化等并行信号,通过神经节细胞脉冲序列输出。

02 · LGN02 · 外侧膝状体

Relays and regulates input in a spatially organized map, with substantial feedback from cortex.保持空间组织,调节并传递输入,同时接受来自皮层的大量反馈。

03 · V103 · 初级视觉皮层

Many neurons respond selectively to orientation, spatial structure or binocular disparity.许多神经元对方向、空间结构或双眼视差具有选择性。

04 · Further processing04 · 后续视觉处理

Distributed circuits combine features and context for object perception, motion and visually guided action.分布式回路结合特征与上下文,参与物体知觉、运动处理和视觉引导的动作。

Right visual field: input from both eyes is represented chiefly in the left hemisphere. Nasal retinal fibers cross at the optic chiasm; this is not one eye per hemisphere.右半视野:来自双眼的相应输入主要由左半球处理。鼻侧视网膜纤维在视交叉交叉,并不是一只眼睛对应一个半球。
The main geniculostriate pathway is summarized here. Processing is parallel and recurrent, with feedback between regions; the four columns are not four serial software layers. Detecting an edge, perceiving an object and choosing a grasp are distinct operations.此处概括主要的膝状体—纹状皮层通路。各区域之间存在并行处理、循环与反馈,四列不是四个串行的软件层。检测边界、形成物体知觉和选择抓取动作属于不同操作。

For the cup rim, retinal channels report local contrast at particular locations. Many V1 neurons additionally respond selectively to edge orientation and spatial scale. Similar local edges can belong to a cup, a book or the table; identifying the object requires integrating features across locations and using context.对于杯沿,视网膜通道报告某些位置上的局部对比,V1 中许多神经元进一步对边缘的朝向、空间尺度产生选择性响应。相似的局部边缘也可能属于书本或桌面;识别物体需要结合不同位置的特征与上下文。

Cortical processing combines orientation, binocular disparity, motion, context and feedback across multiple areas. A center–surround filter detects a local contrast pattern; it does not identify the object responsible. Likewise, an event stream is a measurement representation, not a recognition result. A robotic system still needs geometry, tracking or learned inference and a decision rule tied to its task.皮层处理在多个区域中结合方向、双眼视差、运动、上下文与反馈。中心—周围滤波器检测的是局部对比模式,并不会识别产生该模式的物体。同样,事件流是一种测量表示,而不是识别结果。机器人系统仍需几何估计、跟踪或学习推断,并需要与任务相匹配的决策规则。

UTHealth Neuroscience Online: visual fields, LGN and cortical processingUTHealth Neuroscience Online:视野、外侧膝状体与皮层处理

For a camera tracking this same cup, keep three outputs distinct: the acquired image, a representation such as filtered edges or timestamped events, and the estimated cup position or identity. Test the first with noise and blur, the second with response correctness, and the third with localization error and missed detections. These engineering stages borrow principles; they are not anatomical equivalents of retina, LGN and cortex.让相机跟踪同一只杯子时,应区分三个输出:采集到的图像、滤波边缘或带时间戳事件等表示,以及估计出的杯子位置或类别。第一层测噪声与模糊,第二层检查响应是否正确,第三层测定位误差与漏检。这些工程阶段借鉴了生物视觉的原则,并不是视网膜、外侧膝状体和皮层的一一对应物。

Constraint约束Implementation实现What to measure测量什么
Photon-limited detection光子受限的探测Longer integration or spatial pooling before thresholding延长积分,或先空间汇聚再设阈值SNR and detection error, together with blur and localization error同时测量信噪比、探测错误、模糊与定位误差
Local contrast局部对比Normalized center–surround filtering at relevant scales在相关尺度上做归一化中心—周围滤波Flat-field response, exposure sensitivity, noise amplification and edge localization恒定场响应、曝光敏感性、噪声放大与边缘定位
Color reproduction颜色再现Linear-light processing, white balance and calibrated color transforms线性光处理、白平衡与标定颜色变换Color error across illuminants; clipping and metameric failures不同照明下的色差、饱和截断与同色异谱失配
Fast motion快速运动Timestamped event streams with motion-aware aggregation带时间戳的事件流,配合考虑运动的聚合End-to-end latency, tracking error and peak event throughput端到端时延、跟踪误差与峰值事件吞吐量
Limited processing budget有限的处理预算Fine sampling near the task target, coarser context elsewhere任务目标附近精细采样,其余区域保留较粗上下文Task error versus compute, plus missed peripheral targets任务误差与计算量,以及周边目标漏检