Tags collectives Inside NCCL's all-reduce: ring, double binary tree, or neither? Aug 2026 distributed-training Inside NCCL's all-reduce: ring, double binary tree, or neither? Aug 2026 FSDP collectives 101: why reduce-scatter, and why not broadcast? Jul 2026 fsdp FSDP collectives 101: why reduce-scatter, and why not broadcast? Jul 2026 gpu Inside NCCL's all-reduce: ring, double binary tree, or neither? Aug 2026 nccl Inside NCCL's all-reduce: ring, double binary tree, or neither? Aug 2026 pytorch FSDP collectives 101: why reduce-scatter, and why not broadcast? Jul 2026