本期 code：https://github.com/chunhuizhang/pytorch_distribute_tutorials/blob/main/tutorials/3D-parallel/ring-allreduce.ipynb

DDP 早期视频：BV1Uo4y1K7wM
FSDP原理：BV1Kx4y187Te