= persica cluster =

This is a cluster of three identical nodes, named `persica1/2/3`

 * Alma Linux 9.1 x64
 * Dell Optiplex 9020 Micro
  * Intel Core i5-4590T @ 2.00 GHz
  * 16gb DDR3-1600
  * 128gb SSD

<<TableOfContents>>

== k8s notes ==

 * Make a simple 3-node cluster
 * Single-node control plane will run externally, on illustrious
 * Use kubeadm to build the cluster: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
 * Selected containerd as the container runtime
 * Will use Flannel as the networking plugin
 * Allocated IPs:
  * persica1 / 192.168.1.31
  * persica2 / 192.168.1.32
  * persica3 / 192.168.1.33
 * Ingress: undecided so far
 * Cgroup driver: let's use systemd
 * k8s version: whatever is latest right now (2023-04-04)

== Build notes ==

=== Per node ===

 * Set BIOS to full UEFI mode, no legacy
 * We'll be using DHCP, so find the MAC address so we can give it a consistent IP address when it boots
 * Add the MAC address and IP assignment to dnsmasq on calico (a pihole box)
  * `/etc/dnsmasq.d/02-pihole-dhcp-persica-cluster.conf`
  * Something like this {{{
dhcp-host=98:90:96:BE:89:52,set:persica,192.168.1.31,persica1,5m
# one dhcp-host line per host
dhcp-boot=tag:persica,grub/grubx64.efi,illustrious.thighhighs.top,192.168.1.12
}}}
  * Run `pihole restartdns` after making changes
 * PXE boot for kickstart install, which will hit calico for DHCP, then illustrious for the boot image and kickstart config
 * tftpd-hpa is running on illustrious
  * Upstream repo mirror: https://repo.almalinux.org/almalinux/9/BaseOS/x86_64/os/EFI/BOOT/
  * Drop that content in `/srv/tftp/` {{{
root@illustrious:/srv/tftp# tree
.
├── BOOTX64.EFI
├── default.efi
├── grub
│   ├── grub.cfg
│   ├── grub.cfg-01-98-90-96-be-89-52
│   └── grubx64.efi
├── images
│   └── Alma-9.1
│       ├── initrd.img
│       └── vmlinuz
├── ipxe.efi
└── shimx64.efi
}}}
 * kickstart file served from `/data/www/illustrious/ks`: https://illustrious.thighhighs.top/ks/persica1.ks.cfg
 * KS references:
  * Reference manual: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/performing_an_advanced_rhel_9_installation/kickstart-commands-and-options-reference_installing-rhel-as-an-experienced-user#keyboard-required_kickstart-commands-for-system-configuration
  * Generator tool: https://access.redhat.com/labs/kickstartconfig/
 * k8s doesn't play well with swap so we need to disable it. Provision a minimal swap volume of 1gb, then disable it later

This was useful for figuring out the TFTP stuff for the first time: https://askubuntu.com/questions/1183487/grub2-efi-boot-via-pxe-load-config-file-automatically

Paths are hardcoded into the `grubx64.efi` binary, meaning HDD and PXE versions aren't the same. Make sure you put all the grub stuff in a `grub/` directory. Check the `$prefix` to see where it's searching: 

=== Ansible management after kickstart build ===

I should ansible'ise everything, making minimal assumptions about the kickstart part of the process.

I'm keeping a simple ansible repo in `~/git/persica-ansible/`