100G Network Howto

June 3, 2024

Safety first.

So yes, I decided to go with a 100 gig LAN in my new office, just because I can. This is my first fiber optic project and it's turning out great. The old tutorial from 2019 was at my prior location, and this new one is at my new suite in the same building.

Update June 3, 2024: I finally got this project back on track after starting it in 2021. The issues were twofold: one, the Wifi was adequate for my employees and I just did a hard line for myself; the other issue was the fiber optic termination. Although I now have my office wired up with the fiber I purchased, terminating fiber connectors was the barrier to proceeding. No one wanted to do just the termination part for a reasonable price, because apparently that is the most tedious and least profitable portion. So I'm going to do it myself; I selected the Corning UniCam system, which comes with a toolkit and supposedly takes less than a minute per LC connector. In that connection, I decided to go with a generic FS.com switch that has SFP+ and QSFP28 ports for now. That will be a lot cheaper overall than the Cisco Catalyst or Nvidia Spectrum, which I was considering but which are obscenely priced. I'm going to have 10 gig fiber to each endpoint and 100G to my workstation. I guess this means most of the office will still be 10G, but I will have up to eight 100G devices as a supplement and for future expansion. I also found an access point that takes a SFP connector, which means I can have fiber to the access point and the Wifi will be blazing fast.

Cable selection

I went with the highest quality option, which is Corning plenum-rated single mode fiber in a six-stranded configuration. That way I have two pairs to each endpoint and one extra pair for future use. Overkill for most people, but I want to ensure the highest speeds and best transmission for a video-intensive project that I'm working on.

Pulling Cable

This was the hard part. The old walls don't have conduit in them, so I was running around with different methods trying to figure out how to get the cable pulled through essentially an empty space. There are literally holes in the wall above the ceiling, and boxes at the bottom, with no strings or other means of running the cable-- at least in the old walls.

In the new wall I believe I saw a string for pulling cable, and I assume there is conduit, but that doesn't help me when my priority is my desktop computer in my office and that box seemed impossible to pull wire into. I tried running new conduit, but it seemed to be going sideways into empty space. I also managed to get the new conduit stuck in the wall.

I tried running just the fiber, but it was getting caught on something. I tried just fish tape (metal wire for pulling cables), but it got caught on the electrical conduit for the next box over and I barely got it out. The solution ended up being very simple: I took a fishing weight and some fishing line, and I simply lowered the weight through the space directly above the box. Here I have the first cable coming out of the wall box. Success!

Terminating Connectors

Terminating LC connectors on single mode fiber (SMF) was honestly beyond my capability until I found the Corning UniCam system, which has a cleaver with built-in go/no-go LED indicators.

Router and Switch Selection

Obviously, there are a lot of 100G router and switch options. My internet is a 1G fiber connection from Cogent, so I don't care about the uplink port on the switch. What I care about is the LAN. I tentatively selected the Cisco Catalyst 9500 series, which offers a combination of 100G and 10/25/40G ports. I need 100G from my workstation to the IT closet, and everyone else can have 10G I think. I ended up deciding that for budgetary reasons, as well as avoiding overkill, that I will use a generic FS.com switch for now. I selected the S5860-48SC, which has 48 SFP+ ports for 10G and 8 QSFP28 ports for up to 100G. The switch worked perfectly out of the box, although the web configuration interface could use some improvements. It was exactly as advertised and I like the redundant power and hot-swappable fans. At $4,000 for this unit, I'm not expecting perfection. It's very cool that I can set each port to be an uplink or access port, and visually see which ports are active. I also discovered that I can hook up the administrator port to another switch and access the control panel from the same LAN that the switch is running. This means I can set a static mapping in my router and access the switch remotely if needed using my existing firewall/NAT port forwarding rules. Very cool.

Performance

To be determined.

Cost

1000 feet of top quality fiber optic cable: $330

Tools and accessories: $200

48 LC connectors and Corning toolkit: $4,500

FS.com S5860-48SC switch with Broadcom chip: $4,000

Optical transceivers: $1,500

Total: $10,830

The Verdict

Coming soon.

top

Last updated: June 03, 2024 07:39:21