GitHub Hit by 1.35Tbps Memcached DDoS

Written by

Code repository GitHub was hit by a distributed denial of service (DDoS) attack which peaked at 1.35Tbps via 126.9 million packets per second.

According to a statement the incident occurred on February 28 and persisted for around nine minutes and originated from over a thousand different autonomous systems (ASNs) across tens of thousands of unique endpoints. 

“The first portion of the attack peaked at 1.35Tbps [between 17:21 and 17:30 UTC] and there was a second 400Gbps spike a little after 18:00 UTC,” said Sam Kottler, manager of Site Reliability Engineering.

This attack registered even larger than the peak of the attack on Dyn in 2016, according to Wired

Tod Beardsley research director at Rapid7, said that the attack used a memcached amplification attack technique, which produces many thousands of bytes of UDP response to a very short UDP request.

“This request can be easily spoofed and leveraged by attackers with low skill and few resources, and does not require any authentication,” he said. “After all, the design purpose of memcached is to deliver popular content quickly and without much warning – but the design of memcached over UDP is patently inappropriate for today's internet.

A blog by Akamai claimed that memcached is a protocol allowing a server to be queried for information about key value stores and is only intended to be used on systems that are not exposed to the internet, as no authentication is required.

“When this is added to the ability to spoof IP addresses of UDP traffic, the protocol can be easily abused as a reflector when it is exposed to the internet; Akamai has seen multiple attacks, some in excess of 190 Gbps, with the potential for much larger attacks.”

Sammy Migues, principal scientist at Synopsys, said: “This massive DDoS attack was possible because organizations operating memcached servers failed to implement some very basic security practices. The impact was minimal because GitHub was commendably prepared to survive an attack much larger than this.

“Unless the unwitting operators of these memcached servers take corrective action, it is inevitable that other ill-equipped targets will fall victim to similar DDoS attacks and suffer a much longer outage.”

Ashley Stephenson, CEO of Corero Network Security, said: “Of additional note is the GitHub report of the time delay in the response to this attack. Time to mitigation was around 10 minutes meaning the attack succeeded in impacting Github service, mission accomplished for the attackers who were flexing their DDoS muscles.”

Kottler said that Github has more than doubled its transit capacity recently, which has allowed it to withstand certain volumetric attacks without impact to users, and it is focused on making its edge infrastructure more resilient to current and future conditions of the internet, and less dependent upon human involvement, requires better automated intervention.

“We’re investigating the use of our monitoring infrastructure to automate enabling DDoS mitigation providers and will continue to measure our response times to incidents like this with a goal of reducing mean time to recovery.”

What’s hot on Infosecurity Magazine?