BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20210402T160548Z
LOCATION:Track 3
DTSTART;TZID=America/New_York:20201112T112500
DTEND;TZID=America/New_York:20201112T115500
UID:submissions.supercomputing.org_SC20_sess209_ws_mlhpce102@linklings.com
SUMMARY:EventGraD: Event-Triggered Communication in Parallel Stochastic Gr
 adient Descent
DESCRIPTION:Workshop\n\nEventGraD: Event-Triggered Communication in Parall
 el Stochastic Gradient Descent\n\nGhosh, Gupta\n\nCommunication in paralle
 l systems consumes significant amount of time and energy which often turns
  out to be a bottleneck in distributed machine learning. In this paper, we
  present EventGraD - an algorithm with event-triggered communication in pa
 rallel stochastic gradient descent. The main idea of this algorithm is to 
 modify the requirement of communication at every epoch to communicating on
 ly in certain epochs when necessary. In particular, the parameters are com
 municated only in the event when the change in their values exceed a thres
 hold. The threshold for a parameter is chosen adaptively based on the rate
  of change of the parameter. The adaptive threshold ensures that the schem
 e can be applied to different models on different datasets without any cha
 nge. We focus on data-parallel training of a popular convolutional neural 
 network used for training the MNIST dataset and show that EventGraD can re
 duce the communication load by up to 70% while retaining the same level of
  accuracy.\n\nRegistration Category: Workshop Reg Pass
END:VEVENT
END:VCALENDAR