Wave G Denial-of-Service

I started to have intermittent outages around the middle of November 2019. Initially I thought it was a misconfiguration on my internal network, but I've since isolated it to a problem with Wave G. Even worse, it appears that I can cause denial-of-service to myself with as little as 50 megabits/s of incoming traffic. That amounts to only 5% of my total bandwidth.

How widespread this issue is I do not know, as I feel it would be unethical to attempt to cause a network outage for someone else without their consent.

As of this time, I have been unable to get Wave G to diagnose this issue despite many hours on the phone and talking to a tech in-person, despite the ability to reliably cause service disruption with such a small amount of traffic.

The Problem

A network outage can be triggered by simply sending 50 megabits/s or more of UDP traffic to my public IP. After exactly 30 minutes it will cause a partial outage for about 4-5 minutes before recovering, only to fail in another 30 minutes if the incoming traffic is maintained. Sending more than 50 megabits/s will not change the timing of the failure; it will still be 30 minutes.

Being a partial outage makes it harder to diagnose. As far as I can tell, the following appears to happen:

  • All incoming UDP traffic is dropped
  • All outgoing UDP traffic is allowed
  • All ICMP traffic is allowed
  • TCP traffic is unstable. Currently established connections will still usually work to some extent. They will sometimes fail if used much during the outage window. Web browsing is virtually impossible.

Since ICMP traffic is allowed, basic connectivity testing using tools such as ping will work successfully, assuming the IP address for a domain has been cached. Domain resolution will fail since that requires inbound UDP traffic.

It does not seem to stop the outage if traffic is also simultaneously being sent to the remote machine, nor does the outage trigger with only outgoing traffic

Reproducing the Problem

This can be reproduced simply by sending UDP traffic to my public IP. There are a variety of ways to do this, but I use iperf as it's able to send both UDP and TCP traffic, as well as running on almost every operating system.

Remote Machine

Send 50 megabits/s UDP to public ip for one hour:

iperf -c $public_ip -u -b 50M -t 3600

Local Machine

There is no need to actually receive the traffic to trigger an outage but you can do so if you wish to monitor:

iperf -s -i 5 -u

Now just wait thirty minutes, and you will see a partial outage.

Tested Configurations

I have tested with the following remote and local configurations, bypassing any firewalls or packet forwarding on my local network. All local machines plugged directly into Wave G wall outlet:

Local

  • OS X Darwin 18.7.0 - MacBook Pro Thunderbolt Ethernet Adapter
  • Linux 5.3.11 - Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller
  • Linux 4.19.84 - Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller

Remote

  • Linux 4.9.0 - Google Cloud (Oregon)
  • Linux 5.3.0 - Digital Ocean (San Francisco)

Example From Local Machine

bmon and htop status showing incoming traffic dropping and system load low.
Traffic drops after thirty minutes.
bmon and htop status showing incoming traffic returning and system load low.
Traffic returns after almost five minutes.

Additional Failures

Before replicating this with iperf, I was able to reproduce it with Wireguard connected to a variety of endpoints in the US and Europe, so the outage appears to happen regardless of the source of the incoming traffic.

Conclusion: Possible Network Hardware Misconfiguration

The exact timing of the outage, every 30 minutes, with only select traffic being blocked, regardless of remote or local hardware or software configuration, or remote network location, leads me to believe this problem is somewhere in Wave G's network. There are other indications of misconfigured network hardware in Wave G's network. For example, IPv6 packets (even ICMPv6) do not appear to traverse despite being assigned a prefix by Wave G via DHCPv6-PD, indicating a partially configured network environment.