1: The Hardware and Topology

I know it’s been a while since I initially announced this project, but unfortunately this is a “side project” and my day job needs to take priority.

First of all I thought i would show you the hardware that we will be using for this project, along with the topology that we intend to deploy in order to provide our tenants with a second to none service.

First of all lets talk about the topology. I have created a diagram to depict this, available here, but I will attempt to explain the theory behind each section.

First of all we get our Internet connection from our ISP’s data centre via two disparate gigabit private fibre circuits that are fed from two separate BT exchanges. These circuits terminate into two separate “presentation” switches on our site to create a resilient Internet connection.

It is from these switches that our corporate firewalls are fed, and also where we will be connecting our pfSense box. We currently have 32 public IP’s that can be bound to by any device plugged into these switches, providing it passes the security checks when it is plugged in of course.

On the tenant side of the pfSense box we will link to a switch (or two). To prevent each tenant from seeing each other , and to prevent to them from binding to each others external IP’s we intend to use the PPPoE Server feature of pfSense for authentication. This will also allow us to track how much bandwidth each tenant uses.

Here are a few pics of the server we will be running pfSense from.

IBM x3550

To the rear of the server you can see six gigabit LAN interfaces (the seventh is an on board management port and cannot be used for networking). Just to the right there are two power supplies.

For testing we are using a Cisco C2960PD-8TT-L switch which is an eight port 10/100 switch with one Gigabit uplink port. The switch is powered via POE on the uplink port and is completely silent.

We are still unsure of the configuration we will be using on our pfSense box at the moment but over the next few weeks we will be testing various setup, each of which will be documented by me on this blog.

If you have any suggestions on this project then please share them in the comments!


3 thoughts on “1: The Hardware and Topology”

  1. Hi Dan,

    Interesting! I stumbled across this while looking to see what kind of systems other people have rolled out to accomplish this. I have set up and maintain a pfSense system for a business centre that creates separate networks in a way that is similar to how you’ve described, albeit on a smaller scale.

    The system uses virtual IP addresses on the WAN interface. I’ve set up multiple VLAN interfaces, each with separate DNS relay/DHCP, NAT etc which use a simple VLAN enabled switch as a “breakout box” with clusters of ports allocated to individual tenants as their own private LANs. The firewall rules successfully stop the possiblity of tenants having access to each other’s subnets. The traffic shaping works fairly well but is incredibly clunky to manage. The traffic reporting is also far from ideal. I was considering either running NTOP or running a separate server running Zabbix purely for details network monitoring. I also plan on investigating a bridged interface/s with traffic shaping rules to allow tenants the ability to use a public IP address themselves connected to their own firewall.

    In the past I got quite far building a custom Linux-based server to pretty much do the same, using the “tc” command for traffic shaping and with a custom built web GUI. If only there were more hours in the day!

    I look forward to seeing how you progress with your installation! 🙂


    1. Hi Alex,

      Since my last post we urgently needed the ability to traffic shape for one of our customers. The customer wanted to test their field equipment with various speeds of internet connection to ensure it would work in worst case scenarios.

      Because of the urgency of this I was tasked to get the box in and working without PPPoE authentication and accounting. The original requirements list will still be implemented, eventually, but for now, our box runs with two bridged interfaces and traffic shaping rules.

      This works really well as a traffic shaper and also allows our tenants and customers to use public addresses on their own firewalls / routers. BandwidthD seems to be doing a decent enough job at tracking usage as well. I will blog about our current configuration over the weekend. Hopefully it will help you out when you come to implementing a bridge on your box.



  2. Hi

    I came across your article today its very interesting we have a similar situation were we are deploying to many virtual instances of pfsesne to facilitate separate customers on our rack

    Did you progress any further would be very interested as I would like to multi tentant pfsense, I would glady share any information we gain in testing


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.