calico

Radxa Rock Pi S, a teeny little ARM computer with PoE and 100Mb ethernet

Ubuntu 20.04 (custom Radxa 4.4 kernel)

Linux calico.thighhighs.top 4.4.143-65-rockchip-g58431d38f8f3 #1 SMP PREEMPT Sat Aug 14 09:31:07 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Located at home

Contents

calico
Incomplete notes
1. Wireguard
  1. Fix the wireguard-dkms package
2. Pivpn

Build notes

I used this host to write the general notes for Rock Pi S hardware.

Image and setup SSH keys

Image the SD card and let it boot, it'll get on the network with DHCP
Copy your SSH key, the password is rock
```
ssh-copy-id rock@IP
```
Login again, now it'll use your SSH key
```
ssh rock@IP
```
Set a strong random password, this will be used for both rock and root
```
passwd
```
Sudo up and set the same password for root
```
sudo -i
passwd
```
Record the new password somewhere safe
Lock the rock account now, note that this still permits key access
```
usermod -L rock
```

Grab the rock user's authorized_keys so root can use it

mkdir -m 0700 /root/.ssh
cp /home/rock/.ssh/authorized_keys /root/.ssh/
chown root:root /root/.ssh/authorized_keys ; chmod 0600 /root/.ssh/authorized_keys

Regenerate SSH host keys, we don't know what was installed with the OS image
```
rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server
```
- You could do ssh-keygen -A as an alternative, but it'll generate DSA keys as well which we don't want
Logout completely
Delete the entries from your known_hosts file
SSH again as root@IP, accepting the new keys. It'll use your SSH key instead of asking for password now.

Basic environment stuff

Set hostname:

hostnamectl set-hostname calico.thighhighs.top

Update hostname in /etc/hosts
Uncomment the IPv6 entries in /etc/hosts as well

Set timezone

timedatectl set-timezone Australia/Sydney

Set editor

echo "export EDITOR=vim" > /etc/profile.d/editor-vim.sh

Disable HashKnownHosts

echo -e "Host *\n    HashKnownHosts no" > /etc/ssh/ssh_config.d/99-global.conf

Configure screen

curl -o ~/.screenrc https://gist.githubusercontent.com/barneydesmond/d16c5201ed9d2280251dfca7c620bb86/raw/.screenrc

Configure top by entering this cheatcode
```
z x c b s 1.5 <Enter>
e <zero> 1 W q
```

Disable wifi and bluetooth, we don't need them and it slows down boot

systemctl disable wpa_supplicant.service --now
systemctl disable bluetooth.service --now
systemctl disable rtl8723ds-btfw-load.service --now

echo -e "# Don't load the WLAN+BT module, we don't need it\nblacklist rtl8723ds" > /etc/modprobe.d/blacklist-radios.conf
update-initramfs -u

Install useful packages

apt update
apt install -y vim screen bash-completion lsof tcpdump netcat strace nmap less bsdmainutils tzdata whiptail netbase wget curl python-is-python3 net-tools ack jq make elinks nmap whois ethtool bind9-dnsutils apt-utils man-db

Do a full upgrade then reboot
```
apt full-upgrade
reboot
```

Configure networking

What we want:

Static IPv4 addressing
Autoconfig dynamic IPv6 addressing
- Global stable IPv6 addresses (I guess)
Add a locally-defined static IPv6 address, that other hosts can refer to via DNS etc
DNS resolvers will be manually defined

We'll use netplan to do this, as it greatly simplifies getting what we want without needing to faff around with config in multiple places.

Disable IPv6 privacy addresses, they're enabled by default on Ubuntu
```
sed -r -i 's/tempaddr = 2/tempaddr = 0/' /etc/sysctl.d/10-ipv6-privacy.conf
systemctl restart procps
```
This is a nifty site for testing: http://ip.bieringer.net/ - Look at EUI64_SCOPE and see if it's random/privacy/global. Global is probably what we want for servers.
Install netplan
```
apt install -y netplan.io
```

Remove network-manager, we want to use networkd instead

apt purge network-manager networkmanager-patch
rm -rf /etc/NetworkManager/
apt autoremove

Write the network config in /etc/netplan/10-thighhighs.yaml

network:
    version: 2
    renderer: networkd

    ethernets:
        eth0:
            critical: true
            dhcp-identifier: mac
            dhcp4: false
            dhcp6: true
            dhcp6-overrides:
                use-dns: false
            ipv6-privacy: false
            addresses:
                - "192.168.1.26/24"
                # :1:26 for the .1.26 IPv4, ca6c == 51820, the default Wireguard port
                - "2404:e80:42e3:0:26:0:0:ca6c/64"
            routes:
                - to: 0.0.0.0/0
                  via: 192.168.1.1
                  on-link: true
            nameservers:
                addresses:
                    - 192.168.1.20
                    - 192.168.1.24
                    - fe80::e65f:1ff:fe1c:c6ea
                    - fe80::ba27:ebff:fe8c:f4f8
                search:
                    - thighhighs.top

Sanity check the generated config, hope it doesn't complain
```
netplan generate
netplan apply
```
Reboot and cross your fingers

Save a known-good image for convenience

On another system with an SD card reader, take an image of the system after shrinking the filesystem

e2fsck -f /dev/mmcblk0p2
resize2fs -M /dev/mmcblk0p2

# Use cfdisk or parted to shrink the partition to a bit larger than the FS, has just been reported.
# In this case it's just over 1GiB, so I'll shrink the partition to 1.1GiB.

# Now take the image, capture a bit more than the size of the partitions.
# boot+root partitions are ~1.22GiB (1254MiB) here, so I'll capture 1300MiB just to be sure.
dd bs=1M count=1300 if=/dev/mmcblk0 | pv -br | gzip --fast > 2021-12-13_calico_img_clean_os.img.gz

If you ever need to restore this image, make sure to run resize-assistant afterwards. As well as growing the FS, it needs to locate the backup GPT table at the end of the disk. At the very least you need to run sgdisk -e /dev/mmcblk0 and then partprobe to clean that up.

If you want to expand the filesystem manually:

Boot the image you just restored to the SD card
sgdisk -e /dev/mmcblk0
partprobe
Use parted or cfdisk to expand the 2nd partition to the desired size (or the whole disk)
resize2fs /dev/mmcblk0p2

Pihole

Straightforward basic install, no conflict with other installed services.

curl -sSL https://install.pi-hole.net | bash
Cloudflare upstream
Web interface enabled, full query logging and display
Pi-hole DNS (IPv4): 192.168.1.26
Pi-hole DNS (IPv6): 2404:e80:42e3:0:1:26:0:ca6c

Admin UI at https://calico.thighhighs.top/admin/

Update our network config in /etc/netplan/10-thighhighs.yaml and use localhost resolvers only. This gives us the sum of what pihole/dnsmasq knows from local static configs, plus whatever is forwarded to Cloudflare.

   1 --- 10-thighhighs.yaml.orig     2021-12-13 16:38:46.731548048 +1100
   2 +++ 10-thighhighs.yaml  2021-12-13 16:38:58.290878698 +1100
   3 @@ -21,9 +21,7 @@
   4                    on-link: true
   5              nameservers:
   6                  addresses:
   7 -                    - 192.168.1.20
   8 -                    - 192.168.1.24
   9 -                    - fe80::e65f:1ff:fe1c:c6ea
  10 -                    - fe80::ba27:ebff:fe8c:f4f8
  11 +                    - 127.0.0.1
  12 +                    - ::1
  13                  search:
  14                      - thighhighs.top

Can add TLS \o/ https://discourse.pi-hole.net/t/enabling-https-for-your-pi-hole-web-interface/5771/17

Enable IPv6 upstreams, and Respond only on interface eth0, in http://calico.thighhighs.top/admin/settings.php?tab=dns

This is important as the default setting won't answer queries from other LAN subnets (eg. VPN, IOT segments).

Now is a good time to import the config from the previous install.

Firewall

As per https://docs.pi-hole.net/main/prerequisites/ I've installed ufw and locked things down.

Limit and fail2ban would be good to do as well: https://www.raspberrypi.org/documentation/configuration/security.md

apt install -y ufw
ufw allow from 192.168.1.0/24 to any app OpenSSH
ufw allow from 2404:e80:42e3:0::/64 to any app OpenSSH
ufw enable

# Pihole stuff - https://docs.pi-hole.net/main/prerequisites/#ufw
ufw allow http
ufw allow https
ufw allow domain
ufw allow 67/udp
ufw allow 67/tcp
ufw allow 546:547/udp

Incomplete notes

These need to be cleaned up and confirmed to be good.

Wireguard

We need to make it compile first, then we can use Pivpn as a tool to manage it.

Fix the wireguard-dkms package

Try installing it

apt install wireguard-dkms

Install fails because the module doesn't build. This turns out to be a gcc9 problem.

Described here: https://github.com/openzfs/zfs/issues/8329
Elaborated upon on this kernel commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/? id=0b999ae3614d09d97a1575936bcee884f912b10e

In short, gcc-9 is more strict about this aliasing thing, and throws a warning. That warning is treated as an error because kernel stuff is important, and that causes the DKMS build to bomb out.

Fix 1: fix the wireguard-dkms package or the kernel headers
Fix 2: compile with gcc-8 instead

Fix 1 sounds hard, let's make it work with gcc-8 then. Using an idea from here: https://github.com/dell/dkms/issues/124#issuecomment-681704633

apt install gcc-8

# Fiddle with /usr/src/wireguard-1.0.20201112/dkms.conf and add this at the end.
# This is just the same as the normal MAKE[0] defn, but we've added CC=gcc-8
MAKE[0]="make CC=gcc-8 -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build"

Let apt try to complete the installation now:

apt install

Now it completes!

Pivpn

While I've done wireguard manually before, a scripted tool is just kinda nicer (and I trust them enough to use it).

Clone the repo:

mkdir -p ~/git
cd ~/git/
git clone https://github.com/pivpn/pivpn.git
cd pivpn/

Tweak the auto install script like so:

   1 diff --git a/auto_install/install.sh b/auto_install/install.sh
   2 index debdf78..aebe9ee 100755
   3 --- a/auto_install/install.sh
   4 +++ b/auto_install/install.sh
   5 @@ -466,7 +466,9 @@ preconfigurePackages(){
   6                 # On Debian (and Ubuntu), we can only reliably assume the headers package for amd64: linux-image-amd64
   7                 [[ $PLAT == 'Debian' && $DPKG_ARCH == 'amd64' ]] ||
   8                 # On Ubuntu, additionally the WireGuard package needs to be available, since we didn't test mixing Ubuntu repositories.
   9 -               [[ $PLAT == 'Ubuntu' && $DPKG_ARCH == 'amd64' && -n $AVAILABLE_WIREGUARD ]]
  10 +               [[ $PLAT == 'Ubuntu' && $DPKG_ARCH == 'amd64' && -n $AVAILABLE_WIREGUARD ]] ||
  11 +               # We've dealt with this on our Ubuntu install
  12 +               [[ $PLAT == 'Ubuntu' && $DPKG_ARCH == 'arm64' && -n $AVAILABLE_WIREGUARD ]]
  13         then
  14                 WIREGUARD_SUPPORT=1
  15         fi
  16 @@ -1294,7 +1296,9 @@ installWireGuard(){
  17                 PIVPN_DEPS=(wireguard-tools qrencode)
  18  
  19                 if [ "$WIREGUARD_BUILTIN" -eq 0 ]; then
  20 -                       PIVPN_DEPS+=(linux-headers-generic wireguard-dkms)
  21 +                       # Not safe for rockpi, they use their own headers
  22 +                       #PIVPN_DEPS+=(linux-headers-generic wireguard-dkms)
  23 +                       PIVPN_DEPS+=(wireguard-dkms)
  24                 fi
  25  
  26                 installDependentPackages PIVPN_DEPS[@]

Then run it and follow the prompts. I need to show unsupported NICs because eth0 doesn't register as being "UP" for some reason.

./auto_install/install.sh --show-unsupported-nics

Use these settings:

It'll use these settings:
    pivpnNET="10.6.0.0/24"
    vpnGw="10.6.0.1"
    pivpnPORT=51820
    # use the pihole servers
    pivpnDNS1="192.168.1.26"
    pivpnDNS2="192.168.1.27"
    pivpnHOST = vpn.thighhighs.top

Useful(?) links

Navigation