PyTorch - torch.distributed

NOTE
Please refer to PyTorch Distributed Overview for a brief introduction to all features related to distributed training.