Site Index
• Wiki
• Blog
Verizon DSL Router Caught Red-Handed !
My in-home network was working fine with a Netgear WGR614 v7
wireless 4-port Ethernet router using a 192.168.1.0/24 subnet
LAN.
I signed up for the Verizon DSL
service. Verizon's ads show up many on this page in the ad bar on this page. You can compare DSL
vs. Cable vs. MegaPath
Ethernet
options. T.J. Nelson used to have a good DSL vs.
Cable modem comparison web page, but the page seems dead summer
2012; any good replacement suggestions?
I installed Verizon's Westell 6100G DSL modem/router combo upstream on the WAN
side
of my
router following the instruction that came with the modem. Within
a day after the DSL service went live, every
computer on the LAN side of my Netgear router switched from
192.168.1.0/24 to 10.0.0.0/24. Additionally, all the DHCP ranges
and reserved addresses inside the Netgear router were reset to
10.0.0.0/24 addresses. The Westell modem took the
192.168.1.1/24 network, and by taking this address, any 192.168.1.1
gateway requests from my
computers
(intended for my Netgear to handle) were now routed through to the
Westell
modem. How in the world did the
new modem, on the WAN side of my Netgear router take over the LAN side
of my
router?
I reset
all the computers and routers the
way I wanted and removed the DSL modem (I disconnected the Ethernet
cable between the Westell and the Netgear). When everything was
stable logging, I plugged the Netgear modem into the DSL modem and went
to Thanksgiving dinner. I left Wireshark, on my computer, logging
all the packets it could hear from its connection to the Netgear router.
When I got home, I perused the logs. Sure enough! My
10.0.0.101 computer had been switched to a 192.168.1.18 number
again.
It was easy to scroll down the Source column and find where the IP
address switched about 21632 seconds into the log. It appears
Verizon performed an ARP attack to steal my computer's IP, swapped
addresses inside my LAN
network, and executed a perfect
Man-in-the-middle (MITM) attack on my
network. It appears they culled admin passwords to the router,
and modified my router
settings.
Unbelievably, their DSL
User Agreement
paragraph 10(4) actually says they will "access and adjust your
computer" settings! Do you know what your internet provider is
doing?!
Here is one security expert write-up on why MITM
attacks are threatening.
I used to do a lot of work with PLC and SCADA devices (computers used
to control hardware in the real physical world). Here is a
description of how the Stuxnet
MITM attack is used to modify behavior of a Siemens PLC. The
best Windows
OS analysis of Stuxnet that I know of is from Mark Russinovich,
author of the SysInternals suite of tools. If you're
interested, you can read about the murder of an Iranian
computer expert trying to purge this worm out of their nuclear
power plants.
Ethernet Log Documents Verizon Hack into My Network
Below is a screen-shot of the Wireshark
log of my network traffic while I was out of the house on a holiday
trip to friends house. Wireshark is the same program as Ethereal, which apparently quit
development in about November
2006, and became Wireshark. Wireshark runs on Windows, OSX, and
Linux. You can even get wireshark on the USB-drive bootable U3
operating system. Comparitech provides a comparison of network sniffers.
In this case, Wireshark was running on my Mandriva OS computer on the
LAN-side of the Netgear in promiscuous mode. None of the activity
logged below was initiated by human activity. Topology
was the DSL modem
hooked into the "uplink" port of a Netgear router. The only
computer powered up was a Mandriva Linux OS plugged into the LAN side
of the Netgear router. The DSL modem owned the 192.168.1.1/24
network, and the Netgear router owned the internal 10.0.0.1/24
network. Missing packet numbers in the sequence are because many
127.0.0.1 housekeeping
transmissions are not shown. More specfically, the packet filter
on the Wireshark
display was "! (ip.addr == 127.0.0.1)". This log picks up 6:00
hours (21611 seconds) after logging was started...
What you see in the above Wireshark log, on the third blue line, is
my computer (10.0.0.101 Elitegro network card) announcing its CUPS
printer server available on port 631. The logs show this occuring
every 30 seconds or so for a few hours. Verizon is using the
Westell modem to sequentially ARP-scan 192.168.1.1/24 addresses
to find anybody on the network. This is done at a rate (every few
seconds) below the threshold that most firewall programs will
report a scan attack. Perhaps this is a way to find
rogue DHCP servers that someone may have put on the net to conflict
with the Verizon's services. I don't know why Verizon does
this, whether from remote servers, or from Jungo DSL middleware
installed on the modem (as suggested by one reader).
When Verizon found my Netgear router
at 192.168.1.18, the
Linux box responded with the five purple
packets. Then, about 0.01 seconds
later, the modem sends a bogus
attack packet
followed by an immediate DHCP offer of a new IP address, and a DHCP
ACK. Notice there are no missing packets in this sequence.
All three are shotgun out in quick sequence. You can duplicate
this type of behavior using the Ettercap
program if you'd like to experiment yourself. In fact, acquire a
copy of the entire Helix Linux
distribution, and you can have all sorts of fun. I have
successfully
captured other computers on a network, and forced them to send
all their traffic through me. Caution:
you do this where you're not welcome and it is criminal activity.
This fast triple sequence (NAK OFFER ACK) is intended to snatch the
attention of the target computer before any ~other~ DHCP server can
figure out what happened. It worked. Notice the packet was
sent to 255.255.255.255 - this means all other networks, not just the
subnetwork that the Westell is on. There is no reason to do this except an
intent to reach into other subnets and hack into them. Remember,
the Westell was on
the WAN side of the Netgear; this data is being logged on the LAN side
of the Netgear. I need to look into whether or not the Netgear router
can be configured to block "broadcast all" packets from WAN side to LAN
side.
As an aside, maintaining TOTAL silence on the network (replying not
at all) would have caused the Westell modem to pass me by and go
searching at other addresses. This is the theme strongly
advocated by the grc.com
ShieldsUP! web pages in order to make your computer less attacked
while on the Internet. However, your computer can not remain silent
about DHCP ARP packets, because that's how your computer gets an IP
address to do all the rest of the required network communication.
You'll see that the attack worked because 0.08 seconds later, my
computer (Elitegro MAC address) announced its new IP address with the
bright tan-colored packet. It has been hacked!
Simultaneously,
it's netmask and gateway has been changed to point it toward the
Westell modem (not visible in this log). The Westell has inserted
itself in the middle of any communication between my computer and my
LAN router. This is a
Man-In-The-Middle attack.
In other words, if my
computer now tries to send to the Netgear router at 10.0.0.1, the
packet
will be sent to the Westell modem first because this address is off its
newly assigned subnet. I'm not sure how the Westell modem
succeeded in reprogramming my internal LAN network away from
192.168.1.1/24 to 10.0.0.1/24.
Verizon Hacks and Collects your LAN Information
Below is the main page of the web server inside the Westell
modem. Only the top panel exists when you first install the
modem, before it has ever communicated on a DSL line; the bottom
3 panels of data (with the red banners) do not show up. Whenever the DSL cable
was disconnected, the data for the panels was
unavailable. The
point here is that displaying these pages apparently require
communication with
the "mother home ship" back inside the Verizon network somewhere.
The Westell modem does not do these things itself (disconnected from
the DSL).
It appears that any information about my internal network is
traveling
over the DSL lines back to Verizon. Of note, the modem home page
displayed
below, at
192.1681.1, was recorded after I fixed the IP-snatching problem.
The one listed "computer" on My Network below is one device Verizon
should see: the Netgear router. When I had other computers inside
my LAN, they also appeared on this
page
until I tuned them to refuse DHCP ARP attacks. Realize that all
this internal LAN information is being collected and used by Verizon,
available to them because they hack into the network on the LAN side of
your router! See Verizon's DSL
User Agreement, paragraphs 10(2).
Solution
I turned off DHCP acquisition of IP
addresses from every hard-wired omputer inside the LAN. It seems
to be okay to let the Netgear issue IP addresses to wireless computers.
In other words,
manually set your computer addresses to 10.0.0.x, netmask to
255.255.255.0, and your gateway to 10.0.0.1 (the router). This
requires you to also go into the Netgear router and specify each of the
10.0.0.x addresses
used by one of your computers as a
reserve assignment, so the router doesn't attempt to give it to any
other computers that happen to log onto your LAN network and request an
IP
from the normal DHCP pool. Also, go into the Netgear router and
set its WAN side IP address to a fixed value (I use
192.168.1.19). It seems there should be some way to stop passing
"broadcast all" messages from one side of the router to the other, but
I haven't found it.
Update December 2010 - I added
another hardwired Windows computer downstream of the Netgar router that issued a DHCP request. Boy, did that gum up the works!
Somehow the DSL modem issued a 192.168.1.x address to it before the
Netgear could issue a 10.0.0.x address. This situation got my
Netgear modem into some sort of routing loop or reprogramming state it
would not come out of
(check light lit up on the front). Even pressing the hardware
button on the Netgear router would not reset it! It cost me 5
hours of dancing configuration to get this fixed.
The only way I found to fix this was to let the Netgear router get a
DHCP address from the DSL modem (192.168.1.0/24) and let the Netgear
router issue DHCP addresses to the LAN (10.0.0.0/24). Then,
within
seconds of all the addresses being issued, I had to separate the DSL
modem and the Netgear modem so that they would not go back into the gum it up
mode. Once they were separated, I admined into the Netgear with a
computer manually set to a 10.0.0.0/24 number, and
locked it down to a fixed IP address on the WAN side. I admined
into the DSL modem using a computer manually set to a 192.168.1.0/24
number and turned off it's network scanning, so it wouldn't
try to look inside my network.
I also wanted to use Tor, so
I
admined into the DSL modem and set up a port forward from 9100-9100 to
a new base of 9100 (doing nothing, but giving me a port forward rule to
use). However, the port forwarding rule could not be completed by
manually specifying the 192.168.1.19 Netgear IP. Instead, I had
to turn network scanning back on, let the DSL modem find my Netgear,
and then it would allow me to select the Netgear IP as part of the port
forwarding rule. Lastly, I entered in a port forward inside the
Netgear to pass packets to my Tor computer.
Update April 2011 - This page
was referenced on DSLReports.com and Reddit. I received a number
of thoughtful emails suggesting further tests, or questioning
things, or abusive comments. Below are some thoughts and
additional information.
I should have explained up front that I didn't go hunting for these
problems. Instead, I had a stable local area net running on the LAN
side of the Netgear router, using 192.168.1.1/24 addresses issued from
the Netgear router's internal DHCP server. I purchased Internet
service, and hooked up the DSL modem to the WAN port on the Netgear
router to provide Internet access. Problems began.
Within hours, all my computers on the LAN-side of the Netgear were
reprogrammed to 10.0.0.1/24 IP addresses. There was no reason the
computers should have asked for or obtained new IP addresses, and
nothing *I* had configured would even make 10.0.0.1/24 addresses
available. Everything was reset to the
way I wanted it, and it happened again. I looked deeper and found
that
the Netgear DHCP server was now programmed to hand out 10.0.0.1/24
numbers. The third time, I logged packets to see what happened.
Seeing the issues, I locked everything down with static IP addresses
and things have gone well for several months.
Whether or not this fits your definition of an attack, I cannot
decide. This much I know: without the DSL modem, things were
stable with my choice of IP addresses. With
the DSL modem, everything got re-programmed three separate times.
I wrote about these results to learn all I can (and part of what I'm
learning
is there are a lot of smart
people reading DSLReports that are willing to help!), but in the end,
the presence of the DSL modem was undeniably a causal factor.
I don't want to engage with syntactical debates about what exactly
constitutes an "attack". If an un-welcomed WAN-side identity
changes my router and my LAN-side computers, and sets themselves up
as the DHCP and DNS server/gateway, that is an attack in my
dictionary. Sorry, disbelieving guys and gals, that's what
happened.
If you don't believe me, please suggest what experiments you'd like to
do and document. I assure you I didn't have have time to sit down and
fake a Wireshark log.
Specific clarifications:
- Someone asked, "What makes you think material posted on your page
is credible?" As with
any research, I would say documentation, data, and community discussion.
- How was Wireshark being run? The Mandriva
Linux OS was the only computer running. I added an additional
sentence
to clarify that Wireshark was running on it. Anything on the
ethernet cable plugged into the Linux
computer would be available to the Wireshark log. This includes
traffice sourced from anywhere on the LAN or WAN side of the Netgear
(including the world-wide Internet) provided the Netgear put it on the
LAN-side ethernet cable to the Linux box.
- Wireshark was set to filter packets as explained in the main
text, using the rule
"! (ip.addr == 127.0.0.1)". All other packets are shown. I
believe
this filter changes none of the conclusions.
- At least one readers bristled at my claim of "bogus attack
packet", quoting sources describing how a NAK packet is suppose
to be used, and pointing out that my claim doesn't fit the
protocol. I
apologize
for not being clear: I know this is not normal NAK usage, and using a
NAK packet in an incorrect way is why I called it bogus!
- Network topology came up in some dicussions. A
clarification is in
order for those not familiar with the terminology: a Man-In-The-Middle
attack doesn't mean the attacker is physically in the middle. I
was
critiqued for not recognizing that my Netgar was in the middle.
Of
course. With a MITM attack, the "middle man" can be on either
side of
any network or firewall. I've done MITM attacks using a wireless
laptop link into a WAP, and was able to reach any other computer on the
network. By "reach", I mean cause that other computer to route
all
traffic through me, and accept traffic through me as if it came from
elsewhere. The concept of being "in the middle" is an operational concept, not an attempt
to describe physical
connection topology.
- I was critiqued for not recognizing a modem can detect if there
is a
DSL signal, and asked to provide "proof" that the data displayed on the
Westell's configuration pages was being accessed from elswhere. I
clarified the relevant passage in the main text.
- To
the folks pointing out the problems are impossible with static IP
addresses, I'm sorry for being unclear. Static IP addresses were
used as a solution to fix the
problems, they weren't part of the problem.
- One interesting discussion highlighted that if my claims were
really happening, 99% of the people would never know, but the remaining
1% would be vocal about it. I agree! Well, I'm being vocal, and I'm all
ears for whatever data or documentation polite readers would like to
cooperatively collect. I propose: setup your own 192.168.1.1/24
network,
plug in your own DSL modem, network log what happens, and let's see if
your local network computers are reprogrammed to 10.0.0.1/24.
- I received rather violent disagreement and personal slams that no
outside network can
"hack into my wireless Netgear router". I can only refer that
reader
back to the text above starting with "This much I know: ..." I do
have
ideas of how this can be done. I'll document that it can be done
with
the assistance of some people behind more friendly emails, and the
maybe we'll spill the beans of how.
Update May 2011 - I've received
a number of suggestions that "double NAT" arrangments are unstable or
unpredictable or "may cause problems". Some people ridiculed me
for using a double NAT setup or that I tried to port forward through
it. Okay, if you don't want to fine. But there's nothing
illigitimate about doing so because I want to. By double-NAT, I mean
that the Westell had different nets on the WAN and LAN side, the
LAN side of the Westell was on a different subnet connected to the WAN
side of the Netgear, and the Netgear had a third sub-net on it's own
LAN
side. For me, once I locked things down with static IPs, the double-NAT
setup has
been stable for 6 months or more, so double-NAT with port forwarding
can work fine.
Nonetheless, I like the simplicity
of simply a bridge and my router, so I decided to give it a try. I
changed the Westell to a Bridge mode instead of Routed Bridge and
reset
it. Looking at the modem, you can tell it's in Bridge mode because the
Internet light stays off. In this mode, the Westell obtains no IP
address, and issues no IP addresses. It behaves as a simple
modem, converting DSL signals and voltages to ethernet packets. The Verizon
documentation doesn't say you have to then change
your local router MAC (Netgear in my case) to adopt what the Westell
MAC used to be.
In my case, I found Verizon would not issue an IP address set to the
router without doing this, so I did the MAC cloning according to other Westell 6100
instructions. After
cloning the Westell MAC onto the Negear WAN side, the Netgear router
was given a WAN IP of
71.x.y.z, mask 255.255.255.0, and gateway 71.x.y.1. DNS servers
were assigned
68.238.64.12 and 68.238.128.12.
Only one problem: the assigned gateway at 71.x.y.1 is
unresponsive. It gives no
response to any protocol I can send it, including simple pings.
The gateway is necessary because it has to handle all traffic to/from
anywhere other than my local network. In simple terms, I could
not access the Internet. Looks like the Bridge mode has problems, or
I'm still missing something.
Back to the double-NAT or Routed Bridge mode...
Unplug the Netgear from power. Poke the reset button on the
Westell for 12 or so seconds. About 20 seconds later, the Westell
Internet light comes back on, meaning it's back on the net with an
IP. Power back up the Netgear. I manually put the Netgear
DNS servers back to Google's
numbers, WAN side of the Netgear set to static 192.168.1.19, and
gateway set to 192.168.1.1, and MAC number back to default.
WWW through the Netgear to the Westell and change the default password
(admin/password) to something a bit more obscure. Main page
reports "Routed Bridge" mode. Main tab no changes. Advanced tab select
Private LAN and un-select DHCP services. Strangely the Westell
responds with "Warning: Enabling Private LAN will disable Public
LAN. Your Modem will reboot automatically due to IP address
modifications. After the reboot you may need to release and renew your
IP address to communicate with the modem." I note that I am not
enabling Private LAN, but rather disabling the DHCP server. I
don't know what this has to do with the Public LAN (aka WAN) side of
the Westell. Click through and the Westell provided a reset banner page
to my web browser for about 40 seconds. Reset date and time to my
timezone. Put back my custom port forwards.
Browsing the Westell logs under the Advanced System Log tab shows a
normal reset and bootup, except a continuing error gets logged over and
over: "daemon.err statsd[140]: supplyDnsLocalDev: failed to open
/var/etc/dnsmasq.leases: No such file or directory", and "user.err
syslog: addDHCPDevicesToARP: failed to open
/var/etc/dnsmasq.leases." Appears the on-board firmware is
reporting this error several times per minute. Anybody else
seeing these? Google reports 1 or 2 hits of others with the same
errors, but no replies.
Update October 2014 - After
being gone for a year, I came back and Verizon changed their pricing
model. They now charge $35/mo for a 1Mbit/760Mbit DSL line.
Charter Cable charges $40 for an "up to" 60 Mbit/sec line, with no
contract, and prorated charges for any partial month. Intall has
been within 1 day each of 3 times I signed up with them in different
states. The extra $5/mo or $60/yr was worth it to me for the
speed and customer service and configuration convenience. I
actually get about 30 Mbit/sec when direct connected to their modem and
slower inside my network, so now my hardware is the slowest thing in
the system.
Bibliography / Additional Resources
Wikipedia on ARP
Spoofing
- describes the technique used to steal an IP address from your
computer. Once your IP address taken, the other computer can be
you on the network.
Wikipedia on
DHCP and ARP Snooping.
Dartmouth
College Enumeration of DHCP Flaws-
describes how the DHCP protocol lets another computer (or DSL modem)
steal the IP address of your router gateway and penetrate your network
to intercept whatever it wants! This includes the password to
your router/gateway, which it can then use to log-in and do secret
administration of your router settings!
http://www.tekbar.net/hackers-and-security/seven-easy-way-to-help-you-defend-against-arp.html
- How to defend against ARP attacks.
Thes
skeleton of this document was originally created using AbiWord
under a Gnome desktop. It was subsequently edited by KompoZer to
become the web page you are reading. Created November 2010. Last
updated January 2016.