Building a network in a Gaelscoil

Background

I'm on the Board of Management of a local Gaelscoil and I'm the "IT Support Guy" for the school.  In May 2017 the Principal got the keys to a newly built 16 classroom building, with associated General Purpose Hall, Learning Support Rooms, Administrative Offices, First Aid Room and storage space, having been in temporary accommodation since 2012.  Yay!  The school's story is here.

I was on the hook to deliver a functioning network to the building, with wireless coverage, print and file services and content-filtered internet connectivity.  Schools built in Ireland in the last 5-10 years are completed to a consistent specification, so we got CAT-5e structured cabling around the building delivered to a 42U cabinet in our DCC (machine) Room and a empty rack (save for the CCTV system which was installed and commissioned by the builders before I took over).  The rest of this post describes how I built the network, some of the design decisions I made and a few of the things I learned along the way.

The Plan

My expectation was that I'd get an empty rack and a labelled patch panel, and that everything else would be on me to source, configure, commission, troubleshoot and generally get working.  I'd been using UniFi APs in the temporary site the year before and found them to be reasonably priced, easy to managed and generally reliable.

I augmented my wish list of features to include

• Separate networks for staff and students
• My own router to separate my network from the one provided by Schools' Broadband
• Remote Management of the infrastructure.

and I was able to do all this on the UniFi platform.

The Build

I couldn't find a reseller for UniFi in Ireland, so the kit had to be ordered from the UK.  My BOM for the network was

• 1 UniFi US-48-750W Switch
• 10 UniFi UAC AP AC Pro Access Points
• 1 UniFi Security Gateway
• 1 UniFi Hybrid Cloud Key

I had a few other incidentals

• Shelving for the rack (I had a lot of loose equipment)
• 100m reel of CAT-5e cabling and a (large) bag of RJ45s
• Rack-mountable PDUs (we got a twin socket in the room for the rack and the CCTV system was already using one of them!)
• Velcro Cable Ties
• Cable trays and tidies for the rack
• Bag of cage nuts
• A workstation-class desktop to act as a general purpose server on the network

which I was able to source (quite reasonably) from random internet vendors.  I relied heavily on Parcel Motel for landing a number of deliveries from sellers who just wouldn't ship to Ireland.

Once the Ubiquiti gear got delivered, I brought the core components and couple of APs home and did the initial config, VLAN and SSID setup and static addressing on my kitchen table.  Once I got into the building, all that would be left to do was to place the APs and provision them on the network.

I never saw a commissioning or test report on the cabling from the builders.  My first job was to test, trace, catalogue, inventory and map all the points (98 of them) terminating on the patch panels.  It took myself and my Dad (who selflessly gave up a long weekend to help me) the first day to walk the building with a cable tester, ladder and couple of walkie-talkies and complete the spreadsheet of points and room names that became authoritative.

I found that all the connections were electrically sound, but they had been tested before the face-plates were installed on the wall sockets.  A couple were upside down or the plate was cracked, so simple repairs and replacements were needed.

This completed inventory turned out to be very useful later: there were gaps in the table that were filled when (for example) the builders came back to put the Building Management System (BMS) onto the network.

Once the cabling was tested, getting the switch into the rack and the first couple of APs on the network went smoothly.  There are only a few wired ports in use on the network (printers, the server and the Secretary's PC) so the network itself was straight forward to commission.

Internet connectivity was not so straight forward, however.

The builders had run cabling from the demarcation drop in a utility closet to the patch panel for our circuits, but the service provider was chronically incapable of connecting the lines for us in the exchange.  It literally took them 3 (monumentally frustrating) months to do it.  In the meantime, the school had one PSTN line ported from the temporary building we'd been using the year before.

My "fix" for this was to get a 4G Router (here on Amazon), pop in an all-you-can-eat data-only SIM for €20 month, put it near a window in an empty classroom (for coverage) and patch it back to WAN interface on the security gateway.  Bit creaky (I described at the time as "a massive Roman aqueduct, capable of supplying a city, being fed from a leaky garden hose"), but it we were on the internet!  Content filtering was a must for the school, so I tweaked the DNS to use Family Shield from OpenDNS and that held things together until the telecoms company finally got their act together.

Here's how it looked in service

The Outcome

The network was up and running when the students turned up on day one.  The next concern was keeping it running reliably without sinking a lot of my personal time into it (or generating frustrations for the teaching staff if it was flaky).  I have a full-time job that involves a bit of travel: I couldn't be running to the school every couple of weeks to fix some random problem.

One year in (I'm very late writing this post up!) it has been minimal effort on my part to keep the network running.  Once the ISP issues were resolved, we had reliable, content filtered internet connectivity.  I've been back to do jobs, but most of those have been around printer issues or some new feature or service to be added to the network.

Here are the before and after pics for the cabinet

Ongoing maintenance

Patching the firmware on the switch, APs, security gateway and cloud controller is straight-forward.  There are apps on iOS and Android that allow me to patch the devices as the firmware is released with a couple of clicks while sitting on my couch watching TV.  Over the past 12 months, it's been reliable and consistent.

Overall, I'm pleased with the UniFi kit and would recommend it.

Lessons Learned

• It's a lot cheaper and cleaner to cut and crimp your own patch leads

Pre-finished and terminated patch cables are surprisingly expensive (or at the least demonstrate shocking value for money),  I saved a chunk of cash by buying 100m of CAT-5e and a bag of RJ-45s.  With my trusty crimping iron and my cheap-n-cheerful cable tester I was able to cut all the patch leads to the right length and get some cool looking callouses on the pads of my fingers :)

• It was time well spent testing and tracking each point deployed (location, good/not good, repair work)

It took a day to get around to every data point in the school, test it and log it on the list. 

This paid off big time as I had hands-on knowledge of where everything was so when stuff I'd made no provision for was to be plugged in, I was able to move pretty fast (see the next point).

• Getting the BMS onto the network was ugly

Part of the school building's design was that the BMS (Building Management System) would be controlled by a PC onsite.  The expectation was the school would provide a PC (for the BMS guys to install their software on) and the underlying network to connect it to.  Indeed, there was a network point installed in the cabinet the BMS was in for this very purpose.  

Unfortunately none of this was known to me when I was designing the network. 

My inventory of network points became useful when the BMS technician came on-site to setup the PC.  I'd have been happier if I could have kept the BMS on a separate VLAN, but I also had to provide a PC for the software to run on and that was the secretary's desktop.  Having spare static IPs in the pool and an immediate knowledge of what cables were where took some of the pain out of this. 

Why the BMS isn't running a webserver and allows itself to be configured over HTTPS (rather than a fat Windows PC client) is beyond me.

• Separate your network from Schools' Broadband

The Schools' Broadband team will ship and install a router in the school for free.  They retain control of the appliance, offer support and do the content filtering (all for free!).  They manage the DHCP pool for the network and will give you static and dynamic addresses to work with.  The obvious use-case is giving static IPs to the printer and letting everything else DHCP.  If you use up all the addresses in the pool, they'll give you a bigger address range, but it'll be from a different block: anything dependant on static addressing will be broken. 

Inserting (in my case) the security gateway device and consuming only one address from the Schools' Broadband range, meant I had full control over all addressing on my network and no dependency on the router beyond connectivity.  When I had to change ISPs (from my cellular stop-gap to the permanent wired connection) all I had to do was swap the cables and wait 10 mins for everything to settle down.   

Conclusion

I got to design and build a moderately complicated network to meet a specific need for the school and which (from my perspective at least) meets that need and has been a relatively low burden to maintain to date.

I'll thank the Príomhoide (Principal) for trusting me enough to do this without asking too many questions, my Dad for spending a couple of days holding the ladder and helping with the scut-work in getting it all working and my ever-patient wife for giving me the time I needed to work on this without interruption.

Next Up

There's a need to get a telephone handset installed in every classroom for intra-building communications.  We already have a commercially supported PBX serving the administrative offices, but the desire is that this will be a separate service with no external dialling.  My rough plan (and hopefully the topic of another posting!) is to get a cheap-n-cheerful PBX appliance and 20 VoIP handsets that I can run through the spare ports on the switch and get it all deployed in a couple of days.  The gear's ordered, so stay tuned...