Introduction

Warning

These docs contain information that relates to my setup. They may or may not work for you.

Overview

This repository is my home Kubernetes cluster in a declarative state. Flux watches the kubernetes folder and will make the changes to the cluster based on the YAML manifests.

Core Components

backube/volsync and backube/snapscheduler: Backup and recovery of persistent volume claims.
cilium/cilium: Kubernetes CNI.
jetstack/cert-manager: Creates SSL certificates for services in my Kubernetes cluster.
kubernetes-sigs/external-dns: Automatically manages DNS records from my cluster in CloudFlare.
kubernetes/ingress-nginx: Ingress controller to expose HTTP traffic to pods over DNS.
mozilla/sops: Manages secrets for Kubernetes, Ansible and Terraform.
rancher/system-upgrade-controller: Handles Kubernetes and Talos upgrades automatically.
rook/rook: Distributed block storage for peristent storage.
siderolabs/talos: The Kubernetes Operating System.

🤖 Automation

Github Actions for checking code formatting and running periodic jobs
Renovate keeps the application charts and container images up-to-date

☁️ Cloud Dependencies

1Password for managing secrets via external-secrets.
AWS SES for sending emails.
Cloudflare tunnels for exposing services & creating certificates & managing domains.
Cloudflare R2 for daily backups.
Pushover for sending alerts.

Total cloud costs yearly is approximately ~$150/year.

Directories

This Git repository contains the following directories.

📁 ansible         # Ansible playbooks for various systems managed outside the cluster
📁 hacks           # Contains random scripts
📁 kubernetes      # Kubernetes cluster defined as code
├─📁 bootstrap     # Flux installation to bootstrap cluster
├─📁 flux           # Main Flux configuration of repository
└─📁 apps          # Apps deployed into my cluster grouped by namespace
📁 talos           # Contains the configuration for Talos operating system
📁 terraform       # Contains Cloudflare & Google Compute infrastructure applied automatically by Flux tf-controller

My mash-mash setup of random hardware I managed to acquire. I also have a few SBC RaspberryPi/ODROIDs lying around, but haven't had a lot of good luck running k3s on them, so sticking to amd64 based machines for now.

Device	Count	OS Disk Size	Data Disk Size	Ram	Operating System	Purpose
HP EliteDesk 800 G1	1	512GB SSD	512GB	8GB	Talos	control-plane
HP EliteDesk 800 G3	2	256/512GB SSD	512GB	16GB	Talos	control-plane
HP EliteDesk 800 G4	2	256 SSD	256GB	16/32GB	Talos	worker
i5-2500K/R7 370 (old repurposed gaming computer)	1	120GB SSD	N/A	24GB	Talos	worker
i5-6700K/GTX1080 (repurposed gaming computer)	1	256GB SSD	N/A	32GB	Talos	worker
Synology DS920+	1	N/A	2x8TB & 2x4TB	20GB	DSM 7.1.1	NAS
CyberPower CP1500AVRLCD	1	N/A	N/A	N/A	N/A	UPS
Ubiquiti EdgeRouter 10X	1	N/A	N/A	512MB	EdgeOS	Router
Ubiquiti UAP-AC-Lite	1	N/A	N/A	N/A	N/A	WiFi AP
PiKVM V4 Mini	1	N/A	N/A	N/A	PiKVM	KVM
TESmart HDMI KVM Switch 8 Ports	1	N/A	N/A	N/A	N/A	KVM Switch
TP-Link TL-SG1024D 24 Port 1Gbps Switch	1	N/A	N/A	N/A	N/A	Network Switch