Tag: Design

Categories
HomeLab

Homelab – Services and Software

In the final installment of this series I would like to go thru some of the software used in my homelab and how they are used.

The following few posts I will go thru the Setup and Building of all the solutions mentioned here.

I started with ESXi 7U2 deployed on all 4 of my Hosts. My host used as a Management cluster will host the below.

  • Windows 10 VM: Used as a Jump Box to manage my lab from. All the software and tools will be installed here. everything can be done from my Laptop as well but I prefer this route to keep my lab portable / standard even when my own tools change.
  • Windows Server 2022: This will be my primary Active Directory Server. DNS will also run on this DC.
  • Windows Server 2022: This is used as my Microsoft Certificate Authority
  • Virtual Center Server Appliance: Virtual Center to manage my Lab from.

All 3 servers are hosted on the internal SSD and set to Auto start on host startup.

I then Setup my Synology NAS. I used 4 x 4TB WD Red Drives configured in RAID10. My NAS Hosts the below Services

  • NTP for my LAB
  • iSCSI Storage for VM’s
  • NFS used for Kubernetes and also ISO images. I also have a NFS Share which Hosts the VMware Content Library.
  • Active Backup for Business used for backups of All Critical VM’s.
  • Web Server Used for distribution my Proxy .wpad file.
  • Internet Proxy Server

Next up was the 3 Node vSAN Cluster, Also using ESXi 7U2. these were added to my virtual center using dns names.

I moved all networking to two separate Virtual Distributed Switches one used for normal traffic and the second for vsan.

Now it was time to get some redundancy up for my Domain controller adding a second Windows 2022 server.

At this point we have all the core components in place to start building our business. Initially I will also, at this point, build myself a few Templates for Ubuntu and Server 2022 just to make my life easier later on, I will in a later post show how this can be automated.

To make the lab more functional I need some additional tools. First of which would be VMware Lifecycle manger to assist with the deployments of all other tools. I then Go ahead from there to deploy;

  • vRealize Operations manager – Monitoring and Troubleshooting
  • vRealize Log Insight – Central Log Management
  • vRealize Automation – Automating everything
  • vRealize Saltstack Config – Configuration Management for servers

Once vRealize Automation is setup and usefull I deployed 6 Ubuntu VM’s to be be used as Docker hosts for the below

  • Services Host:
    • Portainer for Docker Management GUI
    • UptimeKuma for Internet and Services Monitoring
  • NGINX Proxy Manager for reverse Proxy and SSL Certs
  • Harbor Container Registry

The other 3 Docker VM’s are used to play around with different applications

And finally time to get some Kubernetes Clusters up. I used Ranchers K3s to build a 3 Node cluster for “testing” and also spun up a Tanzu Community Edition cluster for “testing”.

That is my Homelab. It should allow for any testing I need to do for Work or Home use. I use various Tools at different times as I learn new things or hear about a project I find interesting. If this was a Business it would be ready to get departments to deploy their applications. There are still security tools missing and few tools used in Enterprises which does not make sense for me to run in my Homelab due to their massive resource requirements or dependencies.

Categories
HomeLab

Homelab – Network Design

For my Homelab Network Conceptual Design I split my Home into two Zones, each behind its own firewall. On the Home side I would segregate Devices in to 3 Categories.

  • Dodgy – This would mean devices I do not not trust at all, so any IOT device which I do not have control over. They would only be allowed to talk to the internet and use my PiHole for DNS.
  • Dodgy but have to live with it – This would be device that need either multicast traffic to other networks or reside on a open vlan to function. These include services like Plex or even Phillips Hue. The network would be Controlled with minimum access allowed to my Secure Network.
  • Secure – This is where my Laptops, Phones and NAS Reside.

Lab Network

For my Lab network I would try to replicate a Corporate network as close as possible. I would segregate and group traffic based on function or broadcast domains. I would also create two internal DMZ VLANs that hairpin thru the firewall. One would be used for External Services and one for Internal services(Testing before moving it external)

I allocated Subnets in the 10.70.0.0/16 range to my Homelab. I would then sub assign these in groups of 10 Class C subnets to each Larger Group ex. 10.70.10.0/24 to 10-70.19.0/24 would be allocated to Management VLANs. I would then go down in each of the App/DB/Web Groups and allocate /26 networks to each of Dev, Test and Prod. I would also reserve some smaller scopes for dedicated VLAN’s for stuff like TKG or vRealize.

In a proper network there would be firewall rules in place to separate all of the above but I am a bit too lazy to do that yet. IP addressing is handled by the UniFi Dream Machine and the Scopes are defined to distribute the DNS IP’s of my Microsoft DNS Servers as well as NTP which is running on my Synology NAS.

DMZ

As far as the two DMZ’s go they are properly Firewalled off from the rest of the environment with outbound rules only created on a per IP and Port basis as needed. There is a Port Forward Rule on the Dream Machine which will forward port 443 traffic to a Instances of NGINX running as a Reverse Proxy in the DMZ, that means all traffic into my Lab will always originate from that Point. I am also running some other Load balancers/ WAF/Reverse Proxy solutions in the DMZ for Testing.

The Internal DMZ has the same Firewall rules but the inbound traffic will be from my Home/Lab and use internal DNS Entries and Certificates. this allows me to test firewall rules and solutions internally before exposing them to the internet.

In my Lab I do not use any DNS filtering like PiHole or Adguard but I do still block outbound DNS to ensure all services use my internal DNS. All server do have Internet access just for ease of used. I do have a Proxy server running on my Synology NAS but this is currently not used anymore. My DNS is still setup to supply the necessary details on some DHCP scopes but I have moved away from the proxy Solution.

Storage

iSCSI and NFS Storage is running over the Management network currently as I only have two very unutilized VM’s using iSCSI. NFS is mostly used to store ISO images and as Shared storage for my Kubernetes Clusters. My vSAN Network us running on a single non routed subnet local to my Cisco SG300. There is two uplinks per hosts but only a single switch, which luckily never restarted on its own. This is to be changed when doing my 10GB SFP+ upgrade.

Backups are also running over the management networks to my Synology NAS.

I have a VLAN created for Remote access VPN with the intension of expanding my Homelab into either Azure or AWS but currently the cost of running the VPN devices in either cloud is a bit too high.

Up Next will be a Summary of what will be running in My Lab

Categories
HomeLab

Homelab – Physical Design

The design of my new home network design was guided by the requirement for Internet stability and Segregation between HomeLab and Home Network as far as possible.

Home Networking Design

I decided to have my internet come into my Firewall for the Home Lab. This was to ensure that when I do decide to Expose services to the internet my Attack surface for my Home would be Reduced due to the 2nd firewall between my Lab and Home Network. Part of the risk I still have would be Man in the Middle attacks which reside in my HomeLab. To mitigate this I would run all my Home Network traffic thru a VPN service with the connection established on my Internal Firewall. An added benefit is that all Lab service are Internet Side of my Internal Firewall and I do not need to VPN into my Lab.

The second Firewall is a Dream Router sitting downstream from the Dream Machine in my Lab

My Home network has Wifi From the Dream Router Covering half of the house and the other half covered by a Unifi Nano HD. Every room has a Network Point cabled from the Central Switch which reside in the same Closet as the Dream Router, from there I use the small unifi mini switches to my devices. I prefer to used Cabled over Wifi where possible

Storage

My NAS is used as a Backup location for Our PC’s as well as a Media Server using Plex. I also use my NAS as a Intermediary between Cloud Storage we use in Dropbox/Google Drive/OneDrive and a long term backup in AWS Glacier. We have our most important Data in each respective cloud as well as on my NAS locally. a Subset of Really Important Data (Family Photos etc..) are being backed up to AWS Glacier.

DNS

DNS for my Home is Provided by 2 Raspberry Pi’s running PiHole. Both are running in Docker with domain forwarding setup for my Lab Domain to enable me to resolve hostnames in my lab from my home network. DNS is supplied by DHCP from the Unifi Dream Machine and all traffic on port 53 are blocked in my network except to my own DNS servers.

I run a small 4 Node K3S cluster on a set of Raspberry Pi’s on my Internal network. Not sure what to do with them yet as I tend to break them every Time I touch them.

HomeLab Networking Design

From my Dream Machine in the Lab I Use one Physical port for my Downstream Dream Router and my Home. Two ports from the Dream Machine are used as Uplinks to my two 8 Port Unifi Switches and the Last port is the Uplink to my Cisco SG300 used for Out of Band Networking.

There is ISL between the two Unifi Switches with RSTP value lower than the uplink to the Dream Machine. This was done as the Switch in the Dream Machine does not seem to support Jumbo Frames.

Each host and my NAS is Patched to each Unifi Switch. Additionally the three Workload Nodes have two Nic’s Patched to the Cisco Out of Band Switch which is used for vSAN traffic.

All caballing is done using flat Cat6 cables and at this time is still a real mess.

Future Plans for my Homelab Networking Design

I have a 10GBe upgrade planned as Budget opens up and Time is allocated. I will be replacing the Cisco SG300 with a set of 4 port Microtik 10GB Sfp+ switches. Which will have its Own ISL and uplink to the two Unifi switches. My vSAN and vMotion traffic will run over the 10GB network an I will keep my VM’s on the 1GB.

Conclusion

Ensuring that I prevent any Dependencies on my Homelab from my Home network upped the WAF(Wife Acceptance Factor) a lot. I currently only need to schedule a maintenance window if I need to update the Firmware or Software on the Unifi Kit. The Dream Machine does have a Small network drop when modifying or creating VLAN’s but on the wired connection this is only a single ping. But for the WiFi this causes a disconnect. Because the Downstream Dream Router is Wired there is nearly no impact. I bravely Tested this during one of my Wife’s Video Calls and the fact that I am writing this now, is proof that it works.

Next up we will look at how it all pieces together.

Categories
HomeLab

Homelab – Requirements

After Multiple bad Homelab builds I sat down this time to look at what I Really need and also what fits my Budget and most importantly what my Family Will accept a a Minimum Viable Product.

As my Homelab will form part of my home network and there will inevitably be some crossover between the two I would need to ensure that anything used by my Family are Both Secure and Reliable while my homelab could be a bit more in flux (aka Broken).My primary use case for my Lab is to learn new Technologies and also test solutions before implementing them at work. a Lot of my design considerations revolved around creating a Lab to resemble a enterprise network as close as I could without having a six figure budget. The Technologies I will focus on for my design will include

  • ESXi as a Hypervisor using vSAN and iSCSI as Storage Solutions.
  • Distributed Switches to start with to be enhanced using NSX-t.
  • Microsoft Active Directory for DNS/Users/Certificates to be extended to Azure AD later on.
  • Ubuntu Linux for all services where possible.
  • Docker for Applications instead of dedicated VM’s.
  • Kubernetes for Applications based of Rancher or Tanzu Clusters instead of Docker.
  • Monitoring thru vRealize Operations and Grafana/Prometheus.
  • Logging into vRealize LogInsight and Splunk.
  • Configuration Management thru Saltstack and Ansible with Terraform for any other use cases.
  • User Frontend thru vRealize Automation.

As for budget I set myself a Growth Budget of $2000 plus whatever I could get for my old lab hardware and clearing out any old Gear and stuff I had laying around.

My List of requirements was.

  • Separation between Services used for Home/Family and Lab.
  • Ability to Expose services to the Internet.
  • Network Segregation.
  • Redundancy for Storage and Networking in Lab.
  • Redundancy for Home Services where Possible.
  • Mix of Redundant iSCSI/NFS as a possible solution for Kubernetes Storage.
  • vSAN for vSphere Environment.
  • Microsoft AD Environment.
  • Internal and External Certificate Support.
  • Selfhosted where possible

Constraints

And as all designs there must always be some Constraints as well.

  • Low Power Consumption and low Noise
  • Limited Backup Availability.
  • As close to VMware HCL as I can get on Hardware.
  • Changes to my Lab should not impact Family
  • New Under Warranty hardware where possible

I also decided that some compromises would need to be made. Unfortunately I do not have a unlimited budget and the hardware should have a usable lifespan of about 2 years. I tend to sell my old hardware while they are still a bit relevant. I found that this gives me the lowest cost of ownership and also the ability to have a fairly modern lab.

Part of any lab build will be the cost for software. Now most of the software out there you could use on a trial basis but that requires a constant rebuild of solutions. For my Software Requirements I went for the below.

  • Microsoft – I got a Microsoft Visual Studio Dev Essentials Subscription. This works out to about $45pm. My reasoning behind it is that I get all this back in Azure Credits which I could use to expand my Azure Cloud Experience. This would then Cover all my Microsoft licensing, from SQL to Server to desktops.
  • VMware – VMUG advantage is the solution here. With nearly every On prem Product covered at $200pa this was a No Brainer for me.
  • Linux – I decided to go for Ubuntu as far as possible.

With Work from Home now the norm in our House Internet/WiFi stability is critical, not to mention the User anger I need to face when Youtube just thinks about buffering.

In the next part I will go over the Hardware Choices and Pricing for my Homelab.

Exit mobile version