Issues

• [WIP] A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networks

  gTop-k S-SGD
  CompML/survey-deep-gradient-compression#20 · Filter by author mitsuhiko-nozawa opened on Jun 4, 2021
  

• Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees

  pipliningTop-k S-SGD
  CompML/survey-deep-gradient-compression#19 · Filter by author mitsuhiko-nozawa opened on Jun 4, 2021
  

• Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization

  gradient quantizationQSGD
  CompML/survey-deep-gradient-compression#18 · Filter by author ryoherisson opened on Jun 4, 2021
  

• Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks

  gradient quantizationsignSGD
  CompML/survey-deep-gradient-compression#17 · Filter by author mitsuhiko-nozawa opened on Jun 4, 2021
  

• GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training

  gradient quantization
  CompML/survey-deep-gradient-compression#16 · Filter by author ryoherisson opened on Jun 4, 2021
  

• Quantized compressive sampling of stochastic gradients for efficient communication in distributed deep learning

  gradient quantizationQSGDsignSGD
  CompML/survey-deep-gradient-compression#15 · Filter by author mitsuhiko-nozawa opened on Jun 3, 2021
  

• [WIP] Election coding for distributed learning: Protecting SignSGD against byzantine attacks

  signSGD
  CompML/survey-deep-gradient-compression#14 · Filter by author nocotan opened on Jun 2, 2021
  

• Communication-Efficient Distributed Deep Learning with Merged Gradient Sparsification on GPUs

  pipliningTop-k S-SGD
  CompML/survey-deep-gradient-compression#13 · Filter by author mitsuhiko-nozawa opened on Jun 1, 2021
  

• Error Feedback Fixes SignSGD and other Gradient Compression Schemes

  gradient quantizationQSGDsignSGDtheoretical analysisTop-k S-SGD
  CompML/survey-deep-gradient-compression#12 · Filter by author nocotan opened on Jun 1, 2021
  

• signSGD with majority vote is communication efficient and fault tolerant

  gradient quantizationsignSGD
  CompML/survey-deep-gradient-compression#11 · Filter by author nocotan opened on May 31, 2021
  

• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning

  gradient quantizationTernGrad
  CompML/survey-deep-gradient-compression#10 · Filter by author nocotan opened on May 28, 2021
  

• One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis

  federated learningQSGD
  CompML/survey-deep-gradient-compression#9 · Filter by author mitsuhiko-nozawa opened on May 28, 2021
  

• On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning

  gradient quantizationQSGDtheoretical analysisTop-k S-SGD
  CompML/survey-deep-gradient-compression#8 · Filter by author nocotan opened on May 26, 2021
  

• Sparsified SGD with Memory

  MEM-SGDtheoretical analysisTop-k S-SGD
  CompML/survey-deep-gradient-compression#7 · Filter by author nocotan opened on May 25, 2021
  

• The Convergence of Sparsified Gradient Methods

  theoretical analysisTop-k S-SGD
  CompML/survey-deep-gradient-compression#6 · Filter by author nocotan opened on May 19, 2021
  

• Asynchronous Distributed Learning with Sparse Communications and Identification

  asynchronous distributed learningtheoretical analysis
  CompML/survey-deep-gradient-compression#5 · Filter by author mitsuhiko-nozawa opened on May 14, 2021
  

• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

  gradient quantizationQSGD
  CompML/survey-deep-gradient-compression#4 · Filter by author ryoherisson opened on May 12, 2021
  

• 99% of Worker-Master Communication in Distributed Optimization Is Not Needed

  theoretical analysis
  CompML/survey-deep-gradient-compression#3 · Filter by author nocotan opened on May 11, 2021
  

• A Convergence Analysis of Distributed SGD with Communication-Efficient Gradient Sparsification

  gTop-k S-SGDtheoretical analysisTop-k S-SGD
  CompML/survey-deep-gradient-compression#2 · Filter by author nocotan opened on May 10, 2021
  

• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

  Top-k S-SGD
  CompML/survey-deep-gradient-compression#1 · Filter by author mitsuhiko-nozawa opened on May 7, 2021