Distributed Learning Systems Group

Overview

The Distributed Learning Systems Group occupies a specific niche in machine learning research: we study what happens to learning algorithms when the data they need is spread across many autonomous agents, when communication is expensive or unreliable, and when the cost of computation must be shared. These conditions are the norm in industrial deployments, healthcare consortia, and IoT sensor networks — yet most published methods assume unrestricted access to centralized, homogeneous data.

Our work sits at the intersection of optimization theory, statistical learning theory, and systems research. We develop methods with provable convergence guarantees under realistic distributional assumptions: non-iid data across nodes, Byzantine-faulty participants, heterogeneous hardware, and intermittent connectivity. We pay close attention to the privacy properties of our algorithms, particularly in settings where raw gradients or activations could leak sensitive information about local datasets.

Beyond theory, we maintain active collaborations with the Meridian HPC facility for large-scale empirical validation, and with the Probabilistic Inference Lab on uncertainty-aware federated learning. Several group members contribute to open-source frameworks for federated learning used by external institutions. The group participates in the graduate program through the CDS doctoral track.

Research themes

• Federated optimization under data heterogeneity and partial participation
• Communication-efficient distributed training (compression, quantization, sparsification)
• Byzantine-robust aggregation and adversarial robustness in distributed settings
• Differential privacy in gradient-based learning
• Asynchronous and decentralized learning over peer-to-peer topologies
• Continual and lifelong learning across distributed agents
• Incentive design and fairness in multi-stakeholder learning systems