Cisco 887VA VDSL – Ethernet bridge on Sky Fibre Unlimited Pro

After successfully configuring the Cisco 887VA on my Sky Fibre Unlimited Pro connection I started to configure NAT and ACLs to allow all of my devices to work properly. For the most part this wasn’t an issue, until I came to all of the games consoles in the house.

We pretty much have a console in every room in the house, totalling 3 x Xbox 360s, a PS3 and a PS4. I blogged a while ago about setting them all up to use UPnP on pfsense to map inbound ports on demand in order to get an open NAT type in game. Now that worked well, but unfortunately the Cisco box doesn’t support UPnP or NAT-PNP by design. The reason for the deliberate lack of these features is simple; Cisco IOS devices are enterprise devices, and no enterprise want users to be able to dynamically NAT ports to internal resources.

While I agree that UPnP or NAT-PNP is a security risk in the enterprise, many other vendors support the features but provide means to restrict which devices may use them, similar to how pfsense does.

The console all tend to use the same ports to connect to the internet. However, when they use UPnP they can use alternative ports if the UPnP router refuses to open the requested ports because another device is using them. This is all good on consumer routers which tend to have UPnP enable as standard. The biggest problem I have with the 887 is that the ports would have to be manually NATed to the console that as currently in use, and the other console would struggle to work properly.

This issue pretty much rules out the feasibility of using the 887 in our house as a conventional router. I did however wonder is I could simply replace the Openreach Modem with the 887 and continue to use my trusty Firebox x750 running pfsense as my firewall. I started to play with my config. After a quick config erase and reload I had a blank canvas to play with.

I decided to try a bridge group first. I shutdown the ATM interface and created the required sub interface on the eth0 interface as below. I also put the sub interface in a bridge group.

interface Ethernet0
no ip address
no ip route-cache
!
interface Ethernet0.101
encapsulation dot1Q 101
no ip route-cache
bridge-group 1
!
interface ATM0
no ip address
no ip route-cache
shutdown
no atm ilmi-keepalive

I then tried to pt a FastEthernet interface into the bridge group, which failed as layer 2 interfaces are not allowed in bridge groups. To get around this I created a vlan and placed that into the bridge group. I then stick Fa0 into the clan. Notice the config line “ip virtual-reassembly in”. It is required.

interface Vlan100
no ip address
ip virtual-reassembly in
no ip route-cache
bridge-group 1
!
interface FastEthernet0
description ~ Uplink to Firewall ~
switchport access vlan 100

Then I set the protocol on the bridge group.

bridge 1 protocol ieee

Finally I disable the routers routing engine.

no ip routing

It worked. I was impressed!.

I know many people this the 887VA is an expensive router to just use as a bridge. I disagree. I have had issues with my line for a few months, caused by broken insulation on the drop wire. The drop wire has been replaced now but OpenReach didn’t re-enable DLM, meaning the line never really built any speed up since the drop wire was replaced. Considering it was sitting at 52Mbs sync our of 80, and the DSLAM is approximately 80 meters from my master socket, this wasn’t acceptable. After 8, yes eight, engineer to my house, none of which were interested in the history of the fault and none of which were willing to do the OGEA reset I requested to set the line speed back to 80Mbs to train down to a stable speed, I have pretty much given up on OpenReach.

Enter 887VA. When I started using this router as a bridge two days ago, the sync speed was already 5Mbs up from the OpenReach Modem. I decided to hammer the connection using Iperf and monitor it for errors. I set iperf away all night at the maximum speed of the line and checked it in the morning. There were a total of 7 CRCs and no drop outs. Result. I shutdown “controller vdsl 0” and brought it back up to find another 1.3Mbs sync speed. I repeated this procedure again the next night and yet again gained another 0.9 Mbs sync speed, bringing me to just under 60Mbs.

Another benefit of using the 887VA is the fact I can see my full line stats. Bonus.

I’m going to continue to try and increase my line speed over the next week and see how high I can get it. If only I could use the “del noise-margin” command.

 

Sky Fibre Unlimited – pfsense

I took the day off work today to wait in the house for an OpenReach engine to switch me to FTTC from Sky. The engineer turned up at the door at 8:10am… Perfect! Up and running by 8:30am… Sky router in the cupboard and pfsense doing the hard work by 8:45am.

The only complaint I have about the whole order process was that I couldn’t upgrade my order. By that I mean I ordered the 40-10Mb/s package initially and then called back to change it to the 80-20Mb/s package. The lovely lady on the phone said “no problem!” As it transpires, however, I cannot actually upgrade until I have had the lower package for a month. Gutted!

I know Sky don’t like people using their own routers / firewalls with their internet service but frankly, I don’t give a shit! Their router is utter pants. A quick iPerf to a known high speed network and I found the throughput on the Sky router was approximately 34.2Mb/s download and 7.6Mb/s upload. After switching to my pfsense box I was getting a consistent 39.4Mb/s download and 9.2Mb/s upload. Case closed!

Now. How did I get it working with pfsense? I’ll show you. Just follow the steps below.

1. Connect to your Sky router either via WiFi or Ethernet. Make sure its plugged in and switched on as well. Obviously.
2.Open your web browser and type in the routers IP address. The default is http://192.168.0.1.
3.Click on the Maintenance link at the top of the page. It will ask you to login. The default username is “admin” and password is “sky” without quotes.
4. Scroll down the page until you find the “LAN Port” section. You will see the following.


5. Copy the Mac Address into notepad for use later. Make sure it is the LAN Mac Address that you use otherwise you will fail.
6. Head to http://www.cm9.net/skypass/ and click the button for F@ST2504 once you have read and accept the T&C’s.
7. Input the Mac Address from notepad to the LAN MAC Address field and your Default WPA Key in the other field. The WPA key is the “Your Password” section on the little slip of paper inside the router box. It is also printed on the back of the router.
8. Copy and paste bother the username and password to notepad for later use.
9. Connect to your pfsense box and login.
10. Go to Interfaces.
11. Fill in the information as follows. Type: Set to DHCP. Mac Address: Copy and paste the LAN Mac from notepad. Hostname: <username>|<password> as copied from the cm9 site.

12. Click “Save”
13. Click Apply Changes.
14. Plug your OpenReach Modem (Lan 1 port) into your pfsense box (WAN port).

That’ it! Simple eh?

I believe the hostname field is DHCP option 61. Providing your router supports this option i don’t see why this wouldn’t work with any other “cable” router or firewall.

1: The Hardware and Topology

I know it’s been a while since I initially announced this project, but unfortunately this is a “side project” and my day job needs to take priority.

First of all I thought i would show you the hardware that we will be using for this project, along with the topology that we intend to deploy in order to provide our tenants with a second to none service.

First of all lets talk about the topology. I have created a diagram to depict this, available here, but I will attempt to explain the theory behind each section.

First of all we get our Internet connection from our ISP’s data centre via two disparate gigabit private fibre circuits that are fed from two separate BT exchanges. These circuits terminate into two separate “presentation” switches on our site to create a resilient Internet connection.

It is from these switches that our corporate firewalls are fed, and also where we will be connecting our pfSense box. We currently have 32 public IP’s that can be bound to by any device plugged into these switches, providing it passes the security checks when it is plugged in of course.

On the tenant side of the pfSense box we will link to a switch (or two). To prevent each tenant from seeing each other , and to prevent to them from binding to each others external IP’s we intend to use the PPPoE Server feature of pfSense for authentication. This will also allow us to track how much bandwidth each tenant uses.

Here are a few pics of the server we will be running pfSense from.

IBM x3550

To the rear of the server you can see six gigabit LAN interfaces (the seventh is an on board management port and cannot be used for networking). Just to the right there are two power supplies.

For testing we are using a Cisco C2960PD-8TT-L switch which is an eight port 10/100 switch with one Gigabit uplink port. The switch is powered via POE on the uplink port and is completely silent.

We are still unsure of the configuration we will be using on our pfSense box at the moment but over the next few weeks we will be testing various setup, each of which will be documented by me on this blog.

If you have any suggestions on this project then please share them in the comments!

 

Installing / Configuring and Administering pfSense as a multi-tenant firewall

I am about to embark on a mission… A mission to provide uncontested but limited Internet connectivity to our tenants. To do this I have decided to deploy pfSense, and I will be documenting each step for both our reference here at work, and in the hope that it will help somebody do something similar in the future.

To start with, we needed a specification of what we need the system to do. Here it is.

  • The firewall must serve multiple tenants (up to 50+)
  • The firewall must give each tenant their own external IP
  • The firewall must prevent each of the tenants from seeing each others’ networks
  • The firewall must allow us to limit the amount of bandwidth each tenant can utilize (otherwise they have free reign of our dual redundant gigabit fibre connections)
  • The firewall must allow us to filter out certain traffic such as p2p
  • The firewall must allow us to set data caps for each tenant
  • The firewall must let us create a DMZ for each tenant if required
  • The firewall must allow us to configure network services for each tenant (DHCP, DNS, etc)
  • The firewall must allow each tenant to have their own VPN connection if required
  • The firewall must allow us to report on bandwidth utilization and data transfer usage on a per-tenant basis

This may seem a tall order for one box, but with pfsense it is absolutely possible providing the hardware is capable of it. for our firewall we are going to re-deploy one of our old servers which was decommissioned during our virtualization project. The server used to be one of our domain controllers and it performed well while it was in service. I believe it will perform well as firewall as well. Its spec is below.

  • IBM x3550 1u Server
  • 2x Dual core Xeon processors
  • 4GB Ram
  • 2 x 76GB SAS disks in a RAID 1 (mirrored) configuration
  • 2x On board Intel Pro/1000 Gigabit NIC’s
  • 1x Dual port Intel Pro/1000 Gigabit NIC
  • N+1 Power supplies

As you can see the server isn’t wanting when it comes to specs for the purpose it will be used for. It was slightly higher speced but parts have since been “pinched” for other projects. If this project goes well then we will be looking to build another similar firewall using our other domain controller of the same spec and cluster them for both resilience and load balancing.

I will be starting this project this afternoon so check back for updates, step-by-step guides and images of the entire process during “Project FireServer”.

Part 1 – The Hardware and Topology ->>>

Xbox Live and PlayStation Network with pfSense

Many people sem to be having a problem using pfSense with Xbox Live or PlayStation Network to game online. I have both and both of them are working fine through pfSense, without opening up UPnP up to all devices on the network. This also works with the game that seems to cause the most issues… Call of Duty: Modern Warfare 3.

So here is what you need to do to make it work.

  1. Assign Static DHCP mappings to the console(s)
  2. Enable UPnP and restrict it to the console(s)
  3. Modify Outbound NAT rules for the console(s)

Each step should be repeated for each console. I should probably point out that the WAN interface on my setup is called EXTERNAL and the LAN interface is called TRUSTED.

1. Assign Static DHCP mappings to the console(s)

For this step the MAC address of the console(s) will be handy. Login to your pfSense box and go to Status > DHCP Leases in the navigation bar. Find the line that contains the MAC address of your console and click the icon to add a static mapping.

The MAC address field should contain the MAC address of the console you are configuring. IP Address is the IP that will be assigned to the console and must be outside the DCHP range of your network. Hostname can be set to PS3 or Xbox depending on the console you are configuring and Description is optional.

Click on save to save the mapping.

Click Apply Changes to set the change in stone.

Repeat this step for the other console if required.

2. Enable UPnP and restrict it to the console(s)

Go to Services > UPnP & NAT-PMP on the navigation bar.

Enable the following options.

  • Enable UPnP and NAT-PMP
  • Allow UPnP Port Mapping
  • Allow NAT-PMP Port Mapping
  • By default deny access to UPnP & NAT-PMP?

Make sure you select the Interface that your console(s) are connected to.

You can enable the “Log Packets” option to troubleshoot if you like.

Enter “allow 88-65535 192.168.100.7/32 88-65535” into the User specified permissions box(es), one for each console. replace 192.169.100.7/32 with the IP address of the console you are configuring. The /32 limits the subnet to a single IP address and is important.

Click change.

3. Modify Outbound NAT rules for the console(s)

Click on Firewall > NAT in the navigation bar and select the the Outbound TAB. Change your NAT type from “Automatic Outbound NAT” to “Manual Outbound ANT”. Click Save.

Click the icon at the top of the table to create a new outbound NAT rule.

In the Source: Address box enter the IP address of the console you are configuring. Select 32 from the drop-down menu next to the address.In the Translation section check the box called Static Port. Enter a description if you wish but it is not required.

Click Save.

Repeat this step for each console if required.

In the Outbound NAT table select the check box next to the row(s) you have just created and click the icon next  to the line containing the “Auto created rule for TRUSTED to EXTERNAL ” row in the table.

Click theApply Changes button.

You should now be good to go.

My setup consists of  the following for reference.

  • pfSense 2.0 WARP running on a Watch Guard Firebox 700
  • Xbox 360 slim running latest firmware
  • PS3 slim running latest firmware

Let me know in the comments if you have any problems.