Skip to content跳到主要内容
f.FENG/Knowledge Lab知识实验室
THE NOTEBOOK / 16 NOTES

Notebook知识库.

Begin with a question. Explore equations, diagrams, and experiments.从一个问题开始，在公式、图形与实验之间找到答案。

SYSTEMS / NOTESInteractive essay交互长文
Systems & Performance系统与性能

From torch.compile to generated kernels: capture, guards and fusion从 torch.compile 到生成内核：图捕获、守卫与融合

TorchDynamo captures tensor operations into FX graphs and guards the assumptions used to specialize them. AOTAutograd prepares differentiation graphs; TorchInductor lowers and schedules computation, fusing compatible operations to avoid materialized intermediates. Cached code is reused only when its guards hold.TorchDynamo 将张量操作捕获为 FX 图，并用守卫检查特化所依赖的条件。AOTAutograd 准备求导图，TorchInductor 将图转换为低层中间表示并调度计算，将兼容操作融合以减少中间张量的物化。只有守卫满足时，缓存代码才能复用。

PyTorchtorch.compileTorchInductor
Published发布 15 min read15 分钟阅读
SYSTEMS / NOTESInteractive essay交互长文
Systems & Performance系统与性能

Autograd nodes and the backward engine: how gradients meetAutograd 节点与反向引擎：梯度如何汇合

Backward nodes apply vector–Jacobian products. The autograd engine accumulates incoming contributions in input buffers and schedules a node when its dependencies are satisfied. AccumulateGrad then writes or adds the resulting gradient to a leaf tensor’s .grad.反向节点计算向量–雅可比积。Autograd 引擎用输入缓冲区汇总梯度贡献，在依赖满足后调度节点；AccumulateGrad 随后将结果写入或累加到叶子张量的 .grad。

PyTorchAutogradComputation graph
Published发布 13 min read13 分钟阅读
SYSTEMS / NOTESInteractive essay交互长文
Systems & Performance系统与性能

CUDA streams, kernels and timing: when is the GPU actually done?CUDA 流、内核与计时：GPU 到底什么时候算完？

CUDA streams order submitted work; events connect dependencies across streams. Host return, event completion and tensor storage lifetime are different boundaries. Timing a matrix multiplication requires choosing which boundary to measure and accounting for warm-up, precision and synchronization.CUDA 流为提交的操作建立顺序，事件连接跨流依赖。主机调用返回、事件完成和张量存储生命周期是不同的边界。测量矩阵乘法前，需要明确计时边界，并控制预热、计算精度与同步方式。

PyTorchCUDAStreams
Published发布 16 min read16 分钟阅读
SYSTEMS / NOTES
Systems & Performance系统与性能

Operator registration and dispatch: how PyTorch selects a kernel算子注册与调度：PyTorch 如何选择内核

PyTorch separates operator schemas from implementations. The dispatcher combines tensor dispatch keys with thread-local state, selects an operator-specific table entry, and lets wrappers redispatch to lower layers. Autograd dispatch and gradient recording are separate decisions.PyTorch 将算子接口与实现分开注册。调度器合并张量的调度键与线程局部状态，从算子专属的调度表中选择入口；包装层再通过 redispatch 进入下层实现。进入 Autograd 调度层与记录反向图是两个不同的判断。

PyTorchDispatcherATen
Published发布 14 min read14 分钟阅读
SYSTEMS / NOTES
Systems & Performance系统与性能

Tensor, strides, and storage: why a transpose can be freeTensor、步幅与存储：转置为什么可以不搬数据

For a dense strided tensor, strides and storage_offset map logical indices to storage locations. Transpose shares the original storage by swapping sizes and strides; making a noncontiguous result contiguous requires copying its data.对稠密步幅张量，strides 与 storage_offset 决定逻辑索引对应的存储位置。转置通过交换形状和步幅共享原存储；将非连续结果转为连续布局则需要复制数据。

PyTorchTensorMemory layout
Published发布 12 min read12 分钟阅读
INTERACTIVE / 05 LABSInteractive essay交互长文
Mathematics数学与算法

How do spikes learn to compute?脉冲如何学会计算？

LIF describes membrane integration, threshold firing and reset. BPTT propagates gradients through time, STDP updates synapses from spike timing, and EventProp differentiates continuous event trajectories. UltraLIF modifies the forward model to support differentiable training.LIF 描述膜电位积分、阈值发放与重置。BPTT 沿时间展开图传播梯度，STDP 根据脉冲时序更新突触，EventProp 对连续时间的事件轨迹求导。UltraLIF 则调整前向模型以支持可微训练。

SNNDynamical systems动力系统Interactive lab交互实验
Published发布 30 min read30 分钟阅读
LANGUAGE / COMPARISON
Systems & Performance系统与性能

Rust vs C++: two paths to performanceRust vs C++：高性能的两条路

Safe Rust uses ownership and borrowing rules to restrict invalid references and unsynchronized shared mutation. C++ uses RAII to manage resource lifetimes, while reference validity depends on API contracts and program design. Performance in either language depends on data layout, algorithms and generated code.Safe Rust 通过所有权和借用规则约束无效引用与未同步的共享修改。C++ 用 RAII 管理资源生命周期，引用有效性仍依赖接口约定和程序设计。两者的性能都取决于数据布局、算法与生成的机器码。

RustC++Memory safety内存安全
Published发布 22 min read22 分钟阅读
COMPUTE / GPU KERNEL
Systems & Performance系统与性能

CUDA Rust: from threads to tilesCUDA Rust：从线程到 Tile，理解两条 Kernel 路线

SIMT kernels assign scalar work to individual threads; tile kernels express operations over data blocks and leave thread mapping to the compiler. Correct output partitioning and boundary handling prevent duplicate writes and omissions, while memory access and data reuse shape performance.SIMT 内核为单个线程分配标量计算，Tile 内核则描述数据块上的运算，由编译器组织线程映射。正确的输出分区与边界处理避免重复写入和遗漏，访存方式与数据复用影响性能表现。

CUDARustSIMT
Published发布 13 min read13 分钟阅读
SYSTEMS / NOTES
Systems & Performance系统与性能

What is PyTorch? Inside torch.matmulPyTorch 到底是什么？跟踪一次 torch.matmul

For dense 2-D inputs in eager mode, torch.matmul calls mm and dispatches to a CPU or CUDA implementation. With gradient recording enabled and inputs requiring gradients, autograd records dependencies for backward to apply the chain rule and accumulate leaf gradients.在即时执行模式下，稠密二维张量的 torch.matmul 调用 mm，并分派到 CPU 或 CUDA 实现。开启梯度记录且输入需要梯度时，autograd 记录依赖关系，供反向传播应用链式法则并累积叶子张量的梯度。

PyTorchATenAutograd
Published发布 21 min read21 分钟阅读
EXPERIMENTS / NOTES
Benchmark基准测试

How much faster? Benchmark with evidence快了多少？从一次测量到可信的基准测试

Warm-up reduces initial JIT compilation effects, batched timing amortizes timer overhead, and repeated samples reveal runtime variability. Performance comparisons require consistent inputs and equivalent results; medians and quartiles describe the measured runtime distribution.预热减轻首次 JIT 编译的影响，批量计时摊薄计时开销，重复采样揭示耗时波动。性能比较以输入一致、结果等价为前提，再用中位数和四分位数描述实测耗时的分布。

Benchmark基准测试Performance性能优化JavaScript
Published发布 3 min read3 分钟阅读
PHYSICS / NOTES
Physics & Models物理与模型

Waves, phase & superposition波、相位与叠加：让公式动起来

In a linear model, waves superpose by adding their displacements. For two sinusoidal waves with equal amplitude, frequency and wave number, phase difference sets the resultant amplitude: zero phase difference doubles it, while π produces complete cancellation.在线性模型中，波的叠加等于位移相加。两列振幅、频率和波数相同的正弦波，其合成振幅由相位差决定：同相时振幅加倍，相差 π 时完全相消。

Waves波动Phase相位Interactive lab交互实验
Published发布 3 min read3 分钟阅读
SYSTEMS / NOTES
Systems & Performance系统与性能

Band Storage GAXPY带状矩阵的紧凑存储

Band storage keeps only diagonals within the lower and upper bandwidths, allowing GAXPY to skip entries outside the band. For an n×n narrow-band matrix with stored band width w, storage and arithmetic work scale as O(nw) rather than O(n²).带状存储只保留上下带宽以内的对角线，使 GAXPY 跳过带外元素。对于 n×n 窄带矩阵，若存储带宽为 w，存储量与运算量均由 O(n²) 降为 O(nw)。

Memory layout存储布局Matrix computation矩阵计算
Published发布 5 min read5 分钟阅读
SYSTEMS / NOTES
Systems & Performance系统与性能

Symmetric Storage GAXPY对称矩阵的压缩存储

A real symmetric matrix can store its lower triangle in n(n+1)/2 entries. In GAXPY, each off-diagonal entry contributes to two output components, so storage is nearly halved while arithmetic work remains O(n²).实对称矩阵只需保存下三角的 n(n+1)/2 个元素。GAXPY 中，每个非对角元素都要贡献到两个输出分量，因此存储量接近减半，运算量仍为 O(n²)。

Memory layout存储布局Linear algebra线性代数
Published发布 5 min read5 分钟阅读
MATHEMATICS / NOTES
Mathematics数学与算法

Fast Matrix–Vector Products快速矩阵向量乘法

Radix-2 FFT recursively splits a DFT into transforms over even- and odd-indexed samples. Butterfly operations reuse these results, reducing arithmetic from O(n²) to O(n log n) while computing the same discrete Fourier transform.基 2 FFT 将 DFT 递归分解为偶数位置与奇数位置样本的变换，再通过蝶形运算复用子问题的结果。它计算的仍是同一个离散傅里叶变换，运算量却从 O(n²) 降为 O(n log n)。

FFTLinear algebra线性代数
Published发布 6 min read6 分钟阅读
SYSTEMS / NOTES
Systems & Performance系统与性能

Matrix Multiplication矩阵乘法：结构与效率

Loop order determines memory strides and opportunities for data reuse. Contiguous accesses use cache lines efficiently, while blocking reuses submatrices in cache to reduce data movement without changing the arithmetic work.矩阵乘法的循环顺序决定访存步长与数据复用机会。连续访问有利于利用缓存行，分块则反复使用缓存中的子矩阵，在算术量不变的情况下减少数据搬运。

BLASMatrix computation矩阵计算Performance性能优化
Published发布 9 min read9 分钟阅读
MATHEMATICS / NOTES
Mathematics数学与算法

Frank–Wolfe AlgorithmFrank–Wolfe 条件梯度法

Frank–Wolfe minimizes a linear approximation over a convex feasible set, then updates the iterate by a convex combination, preserving feasibility without projection. For a differentiable convex objective, the Frank–Wolfe gap bounds the remaining objective error.Frank–Wolfe 在凸可行域上求解线性化子问题，再以凸组合更新迭代点，从而无需投影即可保持可行性。对可微凸目标，Frank–Wolfe 间隙给出剩余目标误差的上界。

Convex optimization凸优化Frank–Wolfe
Published发布 6 min read6 分钟阅读
16 notes in the collection共 16 篇笔记Complete archive →完整归档 →

Search the notebook搜索知识库

Try “matrix”, “Rust”, or “spikes”试试「矩阵」「Rust」或「脉冲」

Full-text search ·全文检索 · ESC to closeESC 关闭