ZCA白化是一种有效的图像预处理技术，通过消除特征之间的线性相关性，减少数据冗余，标准化数据分布，从而加速模型训练并提升性能。其独特之处在于在白化的同时尽可能保留原始数据的结构和视觉信息，使得白化后的图像在视觉上与原始图像相似。

简要回顾 PyTorch 中关于 Tensor 形状变换的 API（如 view, reshape, unsqueeze, squeeze）以及 Einops 库中 rearrange, reduce, repeat 的用法。

This blog revisits the underlying principles of the Vision Transformer (ViT) and proposes explorations on extending the Transformer architecture to other modalities. For instance, in tasks such as "Weight2Weight," prior approaches have often simply flattened weights into one-dimensional tensors, without leveraging positional encodings.

1. 梯度范数 (Gradient Norm)在深度学习中,梯度是一个向量,表示损失函数相对于模型参数的偏导数。对于具有 n 个参数的模型,梯度可以表示为: $$\nabla L = (\frac{\partial L}{\par