Shoretel, LLDP and DHCP headache

During the process of migrating to new DHCP servers my colleague noticed a lot of inactive leases in the DHCP scope for our data subnets. After cross referencing MAC addresses it became apparent that the leases belonged to our Shoretel IP phones. All of the phones also have active leases in the VoIP DHCP Scope.

First of all we took a look at the DHCP scope options. Option 156 was enabled on both scopes, containing the following.

ftpservers=192.168.***.***,country=7,language=4,layer2tagging=1,vlanid=200

We decided to break out trusty Wireshark to try and figure out what was going on. From here we could see what was going on. When the phones were booting up they were doing the following:

  1. Trying LLDP
  2. Requesting DHCP lease from the untagged vlan on the switchport they were plugged into (data subnet)
  3. Retrieving option 156 from the data scope and reconfiguring themselves to tag voice traffic with vlanid.
  4. Requesting DHCP lease from the tagged vlan on the switchport they were plugged into (voice vlan)
  5. Continuing normal boot.

Step 2 is where the phones were getting the lease in the data subnet.

We decided to try lldp-med on the Cisco 2960S switches that we use for access.

This is probably a good time to mention that all of this was tested thoroughly in a controlled environment before it was rolled out to end users. We are not responsible if you break your phone system after reading about our investigations here.

conf terminal
!
lldp-run
!
interface range Gi1/0/1-48
switchport mode access
switchport access vlan 100
switchport voice vlan 200

I won’t go into detail about the QoS config we are running. It is there though, just not shown here.

Next we removed option 156 from the data scope and changed the string in option 156 on the voice scope to the following.

ftpservers=192.168.***.***

After rebooting the phones they did the following.

  1. Negotiated LLDP with the switch.
  2. Requested DHCP lease from the tagged vlan on the switchport they were plugged into (voice vlan)
  3. Retrieved option 156 from the voice scope and reconfigured themselves NOT to tag voice traffic.
  4. Requested DHCP lease from the untagged vlan on the switchport they were plugged into (data subnet)
  5. Used a cached config file and continued booting.

Strange. We did some more tinkering and found that if we added the layer2tagging=1,vlanid=200 options back to the option 156 string and rebooted the phone they stayed in the correct vlan. From this we took an educated guess that the phones were assuming the defaults of layer2tagging=0,vlanid=0 if the option were not specified in the option 156 string.

Next we removed option 156 from the voice scope and added option 66, which it a Boot Server Address, and set it to the ftpservers address from the option 156 string, 192.168.***.***.

We rebooted the phones and they did the following.

  1. Negotiated LLDP with the switch.
  2. Requested DHCP lease from the tagged vlan on the switchport they were plugged into (voice vlan)
  3. Retrieved option 66 from the voice scope.
  4. Contacted the FTP server.
  5. Continued booting normally.

Success! Or so we thought. The country and language of the phones had both reverted to 1 and 1, meaning the dial tone was different, although the language was still English.

To get around this we changed the config files for the phones on the FTP server. The files were stored in c:inetpubftproot on the Shoreware Director server. The file names are shore_model.txt, where model is referred to on the white label on the back of every model. The original config file looked like this.

There is an updated method to accomplish this, whcih you can find in my newwer post ShoreTel LLDP Followup.

shore_s6g_text

We changed it to the following.

shore_s6g_text_modified

After a quick reboot the phones were back to their normal selves.

 

Cisco 887VA VDSL – Ethernet bridge on Sky Fibre Unlimited Pro

After successfully configuring the Cisco 887VA on my Sky Fibre Unlimited Pro connection I started to configure NAT and ACLs to allow all of my devices to work properly. For the most part this wasn’t an issue, until I came to all of the games consoles in the house.

We pretty much have a console in every room in the house, totalling 3 x Xbox 360s, a PS3 and a PS4. I blogged a while ago about setting them all up to use UPnP on pfsense to map inbound ports on demand in order to get an open NAT type in game. Now that worked well, but unfortunately the Cisco box doesn’t support UPnP or NAT-PNP by design. The reason for the deliberate lack of these features is simple; Cisco IOS devices are enterprise devices, and no enterprise want users to be able to dynamically NAT ports to internal resources.

While I agree that UPnP or NAT-PNP is a security risk in the enterprise, many other vendors support the features but provide means to restrict which devices may use them, similar to how pfsense does.

The console all tend to use the same ports to connect to the internet. However, when they use UPnP they can use alternative ports if the UPnP router refuses to open the requested ports because another device is using them. This is all good on consumer routers which tend to have UPnP enable as standard. The biggest problem I have with the 887 is that the ports would have to be manually NATed to the console that as currently in use, and the other console would struggle to work properly.

This issue pretty much rules out the feasibility of using the 887 in our house as a conventional router. I did however wonder is I could simply replace the Openreach Modem with the 887 and continue to use my trusty Firebox x750 running pfsense as my firewall. I started to play with my config. After a quick config erase and reload I had a blank canvas to play with.

I decided to try a bridge group first. I shutdown the ATM interface and created the required sub interface on the eth0 interface as below. I also put the sub interface in a bridge group.

interface Ethernet0
no ip address
no ip route-cache
!
interface Ethernet0.101
encapsulation dot1Q 101
no ip route-cache
bridge-group 1
!
interface ATM0
no ip address
no ip route-cache
shutdown
no atm ilmi-keepalive

I then tried to pt a FastEthernet interface into the bridge group, which failed as layer 2 interfaces are not allowed in bridge groups. To get around this I created a vlan and placed that into the bridge group. I then stick Fa0 into the clan. Notice the config line “ip virtual-reassembly in”. It is required.

interface Vlan100
no ip address
ip virtual-reassembly in
no ip route-cache
bridge-group 1
!
interface FastEthernet0
description ~ Uplink to Firewall ~
switchport access vlan 100

Then I set the protocol on the bridge group.

bridge 1 protocol ieee

Finally I disable the routers routing engine.

no ip routing

It worked. I was impressed!.

I know many people this the 887VA is an expensive router to just use as a bridge. I disagree. I have had issues with my line for a few months, caused by broken insulation on the drop wire. The drop wire has been replaced now but OpenReach didn’t re-enable DLM, meaning the line never really built any speed up since the drop wire was replaced. Considering it was sitting at 52Mbs sync our of 80, and the DSLAM is approximately 80 meters from my master socket, this wasn’t acceptable. After 8, yes eight, engineer to my house, none of which were interested in the history of the fault and none of which were willing to do the OGEA reset I requested to set the line speed back to 80Mbs to train down to a stable speed, I have pretty much given up on OpenReach.

Enter 887VA. When I started using this router as a bridge two days ago, the sync speed was already 5Mbs up from the OpenReach Modem. I decided to hammer the connection using Iperf and monitor it for errors. I set iperf away all night at the maximum speed of the line and checked it in the morning. There were a total of 7 CRCs and no drop outs. Result. I shutdown “controller vdsl 0” and brought it back up to find another 1.3Mbs sync speed. I repeated this procedure again the next night and yet again gained another 0.9 Mbs sync speed, bringing me to just under 60Mbs.

Another benefit of using the 887VA is the fact I can see my full line stats. Bonus.

I’m going to continue to try and increase my line speed over the next week and see how high I can get it. If only I could use the “del noise-margin” command.

 

Sky Fibre Unlimited Pro on a Cisco 887VA

I recently decided to look for a replacement for the crappy white OpenReach modem that was installed as part of my Sky Fibre Unlimited Pro FTTC connection. The problem was that I didn’t want to fork out for an expensive VDSL2 modem to find I couldn’t get it working with the silly MER authentication used by Sky to try and prevent you from using your own router.

Luckily, a Cisco 887v a became available to test with before I took the plunge and bought one. I started googling and couldn’t find one success case of using this router with Sky’s service. Undeterred, I started to tinker and eventually got it working….

Before you begin you will need your mac address,  user-id and password. I won’t cover how to obtain these in this post as I provided steps (steps 1 to 7) to obtain them in an earlier post.

Once you have your mac, username and password, you will need to use them to create three bits of information.

MAC:             <0000.0000.0000> (remove the :’s and place a . after every four characters)
Hostname:    <username>|<password>
Client-ID:      <hexadecimal string of Hostname> (A converter is available here.)

I won’t go into any other configuration in this post, just the interface configuration.

First of all you want to disable the ATM interface as it shared a physical interface with the VDSL controller.

interface ATM0
no ip address
shutdown
no atm ilmi-keepalive

The VDSL modem should automatically connect to the DSLAM. You can check it’s progress by using “show controller vdsl 0”.

When the VDSL modem connects it brings interface Ethernet0 up. Eth0 is a virtual port but is used as your outside interface. OpenReach encapsulate traffic for different ISPs in Vlans. In the case of Sky it is Vlan 101 so you need to use a sub interface of Eth0.

interface Ethernet0
mac-address <mac>
no ip address
!
interface Ethernet0.101
encapsulation dot1Q 101
ip dhcp client request classless-static-route
ip dhcp client client-id hex <client-id in hex>
ip dhcp client hostname <username>|<password>
ip address dhcp
no ip redirects
no ip proxy-arp
ip flow ingress
ip flow egress
ip nat outside
no ip virtual-reassembly in

Thats it. I’ll post my full config below which includes some basic NAT. It doesn’t include any security though. And no, you don’t need a dialer interface!

version 15.1
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
memory-size iomem 10
crypto pki token default removal timeout 0
!
!
ip source-route
!
!
!
!
!
ip cef
no ipv6 cef
!
!
multilink bundle-name authenticated
license udi pid CISCO887VA-K9 sn FCZ1633C05Z
license boot module c880-data level advipservices
!
!
!
!
!
!
controller VDSL 0
!
!
!
!
!
!
!
!
interface Ethernet0
mac-address <mac>
no ip address
!
interface Ethernet0.101
encapsulation dot1Q 101
ip dhcp client request classless-static-route
ip dhcp client client-id hex <client-id>
ip dhcp client hostname <username>|<password>
ip address dhcp
no ip redirects
no ip proxy-arp
ip flow ingress
ip flow egress
ip nat outside
no ip virtual-reassembly in
!
interface ATM0
no ip address
shutdown
no atm ilmi-keepalive
!
interface FastEthernet0
switchport access vlan 1
!
interface FastEthernet1
!
interface FastEthernet2
!
interface FastEthernet3
!
interface Vlan1
ip address 192.168.1.1 255.255.255.0
ip flow ingress
ip flow egress
ip nat inside
ip virtual-reassembly in
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
!
ip nat inside source list NATACL interface Ethernet0.101 overload
!
ip access-list standard NATACL
permit 192.168.1.0 0.0.0.255
!
logging esm config
access-list 1 permit 192.168.1.0 0.0.0.255
dialer-list 1 protocol ip permit
!
!
!
!
!
control-plane
!
!
line con 0
no modem enable
line aux 0
line vty 0 4
login
transport input all
!
end

 

Sky Fibre Unlimited – pfsense

I took the day off work today to wait in the house for an OpenReach engine to switch me to FTTC from Sky. The engineer turned up at the door at 8:10am… Perfect! Up and running by 8:30am… Sky router in the cupboard and pfsense doing the hard work by 8:45am.

The only complaint I have about the whole order process was that I couldn’t upgrade my order. By that I mean I ordered the 40-10Mb/s package initially and then called back to change it to the 80-20Mb/s package. The lovely lady on the phone said “no problem!” As it transpires, however, I cannot actually upgrade until I have had the lower package for a month. Gutted!

I know Sky don’t like people using their own routers / firewalls with their internet service but frankly, I don’t give a shit! Their router is utter pants. A quick iPerf to a known high speed network and I found the throughput on the Sky router was approximately 34.2Mb/s download and 7.6Mb/s upload. After switching to my pfsense box I was getting a consistent 39.4Mb/s download and 9.2Mb/s upload. Case closed!

Now. How did I get it working with pfsense? I’ll show you. Just follow the steps below.

1. Connect to your Sky router either via WiFi or Ethernet. Make sure its plugged in and switched on as well. Obviously.
2.Open your web browser and type in the routers IP address. The default is http://192.168.0.1.
3.Click on the Maintenance link at the top of the page. It will ask you to login. The default username is “admin” and password is “sky” without quotes.
4. Scroll down the page until you find the “LAN Port” section. You will see the following.


5. Copy the Mac Address into notepad for use later. Make sure it is the LAN Mac Address that you use otherwise you will fail.
6. Head to http://www.cm9.net/skypass/ and click the button for F@ST2504 once you have read and accept the T&C’s.
7. Input the Mac Address from notepad to the LAN MAC Address field and your Default WPA Key in the other field. The WPA key is the “Your Password” section on the little slip of paper inside the router box. It is also printed on the back of the router.
8. Copy and paste bother the username and password to notepad for later use.
9. Connect to your pfsense box and login.
10. Go to Interfaces.
11. Fill in the information as follows. Type: Set to DHCP. Mac Address: Copy and paste the LAN Mac from notepad. Hostname: <username>|<password> as copied from the cm9 site.

12. Click “Save”
13. Click Apply Changes.
14. Plug your OpenReach Modem (Lan 1 port) into your pfsense box (WAN port).

That’ it! Simple eh?

I believe the hostname field is DHCP option 61. Providing your router supports this option i don’t see why this wouldn’t work with any other “cable” router or firewall.

1: The Hardware and Topology

I know it’s been a while since I initially announced this project, but unfortunately this is a “side project” and my day job needs to take priority.

First of all I thought i would show you the hardware that we will be using for this project, along with the topology that we intend to deploy in order to provide our tenants with a second to none service.

First of all lets talk about the topology. I have created a diagram to depict this, available here, but I will attempt to explain the theory behind each section.

First of all we get our Internet connection from our ISP’s data centre via two disparate gigabit private fibre circuits that are fed from two separate BT exchanges. These circuits terminate into two separate “presentation” switches on our site to create a resilient Internet connection.

It is from these switches that our corporate firewalls are fed, and also where we will be connecting our pfSense box. We currently have 32 public IP’s that can be bound to by any device plugged into these switches, providing it passes the security checks when it is plugged in of course.

On the tenant side of the pfSense box we will link to a switch (or two). To prevent each tenant from seeing each other , and to prevent to them from binding to each others external IP’s we intend to use the PPPoE Server feature of pfSense for authentication. This will also allow us to track how much bandwidth each tenant uses.

Here are a few pics of the server we will be running pfSense from.

IBM x3550

To the rear of the server you can see six gigabit LAN interfaces (the seventh is an on board management port and cannot be used for networking). Just to the right there are two power supplies.

For testing we are using a Cisco C2960PD-8TT-L switch which is an eight port 10/100 switch with one Gigabit uplink port. The switch is powered via POE on the uplink port and is completely silent.

We are still unsure of the configuration we will be using on our pfSense box at the moment but over the next few weeks we will be testing various setup, each of which will be documented by me on this blog.

If you have any suggestions on this project then please share them in the comments!