I grabbed a couple of these, one with NAND flash and one without. Both have Wifi/BT/POE support, and I bought the POE hats because that's a damn good idea.

https://shop.allnetchina.cn/collections/frontpage/products/rock-pi-s?variant=29067635458150

<<TableOfContents>>

= Official docs links =
 * About the onboard SD NAND storage: https://wiki.radxa.com/RockpiS/hardware/SDNAND
 * How to flash stuff directly to the NAND: https://wiki.radxa.com/RockpiS/dev/sdnand-install
  * How to prepare the `rkdeveloptool` binary for your system: https://wiki.radxa.com/RockpiS/dev/otg

= Setup =

I'm using their Ubuntu image here, it's "focal" (20.04 LTS).

== Initial image and packages ==

 * Image the SD card and boot it as normal, get a console either with adb or SSH
  * Default SSH creds are rock//rock, there's no root password set but you can sudo up
  * SSH is enabled by default
 * Login as rock, sudo to root
 * Set hostname: `hostnamectl set-hostname wag1.thighhighs.top`
 * Update hostname in /etc/hosts
 * Uncomment the IPv6 entries in /etc/hosts as well
 * Regenerate SSH host keys {{{
rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server

# As an alternative, though this will generate DSA keys as well
ssh-keygen -A
}}}
 * Packages {{{
apt update
apt install -y vim screen locales bash-completion lsof tcpdump netcat strace nmap less bsdmainutils tzdata whiptail netbase
#dpkg-reconfigure locales
apt full-upgrade
reboot
}}}
 * Delete the entries from your known_hosts then SSH again as rock@host, accepting new keys

Fix your keys
 * ssh-copy-id rock@host
 * ssh rock@host  # login again
 * passwd  # set a strong random password, this will be used for both rock and root
 * sudo -i
 * passwd  # set the same for root now
 * record the new password somewhere
 * Lock the rock account now: usermod -L rock  # this still permits key access
 * Grab the authorized_keys so root can use it
  * mkdir -m 0700 /root/.ssh
  * cp /home/rock/.ssh/authorized_keys /root/.ssh/
  * chown root:root /root/.ssh/authorized_keys ; chmod 0600 /root/.ssh/authorized_keys
 * Logout as rock, login again as root this time


== Network config ==

 * Disable IPv6 privacy addresses {{{
# It's enabled by default on Ubuntu focal
sed -r -i 's/tempaddr = 2/tempaddr = 0/' /etc/sysctl.d/10-ipv6-privacy.conf
systemctl restart procps

# This is a nifty site for testing: http://ip.bieringer.net/
# Look at EUI64_SCOPE and see if it's random/privacy/global. Global is what we want for servers (probably).
}}}

== More config ==

 * Set timezone {{{
timedatectl set-timezone Australia/Sydney
}}}
 * Set editor {{{
echo "export EDITOR=vim" > /etc/profile.d/editor-vim.sh
}}}
 * Python {{{
apt install python-is-python3
}}}
 * Disable HashKnownHosts {{{
echo -e "Host *\n    HashKnownHosts no" > /etc/ssh/ssh_config.d/99-global.conf
}}}
 * Configure screen and top {{{
curl -o ~/.screenrc https://gist.githubusercontent.com/barneydesmond/d16c5201ed9d2280251dfca7c620bb86/raw/.screenrc
curl -o ~/.config/procps/toprc https://gist.githubusercontent.com/barneydesmond/d16c5201ed9d2280251dfca7c620bb86/raw/.toprc
}}}
 * More packages {{{
apt install wget curl net-tools ack jq make mlocate elinks nmap whois ethtool bind9-dnsutils apt-utils man-db
updatedb
reboot
}}}


== Faff with networking ==

We'd like static IP but dynamic IPv6 {{{
apt install netplan.io
}}}

Criteria is:
 * Static IPv4 addressing
 * Autoconfig IPv6 addressing
  * Global static IPv6 addresses (I guess)
 * Add a locally-defined static IPv6 address, that other hosts can refer to via DNS etc
 * DNS resolvers will be manually defined
 * Use networkd instead of network-manager, remove unneeded packages {{{
apt purge network-manager networkmanager-patch
apt autoremove
}}}

This'll do, it goes in `/etc/netplan/10-thighhighs.yaml`
{{{
network:
    version: 2
    renderer: networkd

    ethernets:
        eth0:
            critical: true
            dhcp-identifier: mac
            dhcp4: false
            dhcp6: true
            dhcp6-overrides:
                use-dns: false
            ipv6-privacy: false
            addresses:
                - "192.168.1.26/24"
                # :1:26 for the .1.26 IPv4, ca6c == 51820, the default Wireguard port
                - "2404:e80:42e3:0:26:0:0:ca6c/64"
            routes:
                - to: 0.0.0.0/0
                  via: 192.168.1.1
                  on-link: true
            nameservers:
                addresses:
                    - 192.168.1.20
                    - 192.168.1.24
                    - fe80::e65f:1ff:fe1c:c6ea
                    - fe80::ba27:ebff:fe8c:f4f8
                search:
                    - thighhighs.top
}}}

== Disable wifi and bluetooth ==

We don't need them and it slows down boot.
{{{
systemctl disable wpa_supplicant.service --now
systemctl disable bluetooth.service --now
}}}


== Save an image ==

Now take an image of the system after shrinking the filesystem
{{{
e2fsck -f /dev/mmcblk0p2
resize2fs /dev/mmcblk0p2 2G
# use cfdisk to resize the partition to 2.4G (as a generous example)
dd bs=4M count=600 if=/dev/mmcblk0 | pv -br | gzip --fast > 2021-12-09_calico_img_pre_pihole.img.gz
}}}


= Pihole =

Straightforward basic install, no conflict with other installed services.

 * `curl -sSL https://install.pi-hole.net | bash`
 * Cloudflare upstream
 * Web interface enabled, full query logging and display
 * Pi-hole DNS (IPv4): 192.168.1.26
 * Pi-hole DNS (IPv6): 2404:e80:42e3:0:1:26:0:ca6c

Admin UI at https://calico.thighhighs.top/admin/

Should probably put cloudflare resolvers into the systemwide resolver set, meaning we don't see our own records though.
 * 1.1.1.1
 * 1.0.0.1
 * 2606:4700:4700::1111
 * 2606:4700:4700::1001

Can add TLS \o/ https://discourse.pi-hole.net/t/enabling-https-for-your-pi-hole-web-interface/5771/17


= Firewall =

As per https://docs.pi-hole.net/main/prerequisites/ I've installed ufw and locked things down.

Limit and fail2ban would be good to do as well: https://www.raspberrypi.org/documentation/configuration/security.md

{{{
apt install ufw
ufw allow ssh
ufw enable

# Pihole stuff - https://docs.pi-hole.net/main/prerequisites/#ufw
ufw allow http
ufw allow https
ufw allow domain
ufw allow 67/udp
ufw allow 67/tcp
ufw allow 546:547/udp
}}}


= Wireguard =

We need to make it compile first, then we can use Pivpn as a tool to manage it.

== Fix the wireguard-dkms package ==

Try installing it {{{
apt install wireguard-dkms
}}}

Install fails because the module doesn't build. This turns out to be a gcc9 problem.
 * Described here: https://github.com/openzfs/zfs/issues/8329
 * Elaborated upon on this kernel commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?         id=0b999ae3614d09d97a1575936bcee884f912b10e

In short, gcc-9 is more strict about this aliasing thing, and throws a warning. That warning is treated as an error because kernel stuff is important, and that causes the DKMS build to bomb out.

 * Fix 1: fix the wireguard-dkms package or the kernel headers
 * Fix 2: compile with gcc-8 instead

Fix 1 sounds hard, let's make it work with gcc-8 then. Using an idea from here: https://github.com/dell/dkms/issues/124#issuecomment-681704633

{{{
apt install gcc-8

# Fiddle with /usr/src/wireguard-1.0.20201112/dkms.conf and add this at the end.
# This is just the same as the normal MAKE[0] defn, but we've added CC=gcc-8
MAKE[0]="make CC=gcc-8 -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build"
}}}

Let apt try to complete the installation now: {{{
apt install
}}}

Now it completes!


== Pivpn ==

While I've done wireguard manually before, a scripted tool is just kinda nicer (and I trust them enough to use it).

Clone the repo: {{{#!sh
mkdir -p ~/git
cd ~/git/
git clone https://github.com/pivpn/pivpn.git
cd pivpn/
}}}

Tweak the auto install script like so: {{{#!diff
diff --git a/auto_install/install.sh b/auto_install/install.sh
index debdf78..aebe9ee 100755
--- a/auto_install/install.sh
+++ b/auto_install/install.sh
@@ -466,7 +466,9 @@ preconfigurePackages(){
                # On Debian (and Ubuntu), we can only reliably assume the headers package for amd64: linux-image-amd64
                [[ $PLAT == 'Debian' && $DPKG_ARCH == 'amd64' ]] ||
                # On Ubuntu, additionally the WireGuard package needs to be available, since we didn't test mixing Ubuntu repositories.
-               [[ $PLAT == 'Ubuntu' && $DPKG_ARCH == 'amd64' && -n $AVAILABLE_WIREGUARD ]]
+               [[ $PLAT == 'Ubuntu' && $DPKG_ARCH == 'amd64' && -n $AVAILABLE_WIREGUARD ]] ||
+               # We've dealt with this on our Ubuntu install
+               [[ $PLAT == 'Ubuntu' && $DPKG_ARCH == 'arm64' && -n $AVAILABLE_WIREGUARD ]]
        then
                WIREGUARD_SUPPORT=1
        fi
@@ -1294,7 +1296,9 @@ installWireGuard(){
                PIVPN_DEPS=(wireguard-tools qrencode)
 
                if [ "$WIREGUARD_BUILTIN" -eq 0 ]; then
-                       PIVPN_DEPS+=(linux-headers-generic wireguard-dkms)
+                       # Not safe for rockpi, they use their own headers
+                       #PIVPN_DEPS+=(linux-headers-generic wireguard-dkms)
+                       PIVPN_DEPS+=(wireguard-dkms)
                fi
 
                installDependentPackages PIVPN_DEPS[@]
}}}

Then run it and follow the prompts. I need to show unsupported NICs because eth0 doesn't register as being "UP" for some reason. {{{
./auto_install/install.sh --show-unsupported-nics
}}}

Use these settings: {{{
It'll use these settings:
    pivpnNET="10.6.0.0/24"
    vpnGw="10.6.0.1"
    pivpnPORT=51820
    # use the pihole servers
    pivpnDNS1="192.168.1.26"
    pivpnDNS2="192.168.1.27"
    pivpnHOST = vpn.thighhighs.top
}}}


= System inspection =

I installed their provided image of Debian buster, balena Etcher'd straight onto a spare SD card and inserted. Used adb shell to get initial connectivity to set it up and inspect things.

The root filesystem is all of ~500 MiB, which is great for compactness and speed. It auto-grows on first boot by the looks of it.
{{{
[   11.091476] EXT4-fs (mmcblk0p2): resizing filesystem from 199161 to 7835148 blocks
[   11.518063] EXT4-fs (mmcblk0p2): resized filesystem to 7835148
}}}

== Disk usage ==
{{{
root@rockpis:/# df -hl
Filesystem      Size  Used Avail Use% Mounted on
udev            210M     0  210M   0% /dev
tmpfs            43M  296K   43M   1% /run
/dev/mmcblk0p2   30G  511M   28G   2% /
tmpfs           213M     0  213M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           213M     0  213M   0% /sys/fs/cgroup
}}}

== Block devices ==
 * mmcblk0 is the SD card
 * mmcblk1 is the onboard NAND flash
{{{
root@rockpis:/# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0     179:0    0   30G  0 disk 
├─mmcblk0p1 179:1    0  112M  0 part 
└─mmcblk0p2 179:2    0 29.9G  0 part /
mmcblk1     179:32   0  3.6G  0 disk 
└─mmcblk1p1 179:33   0  3.6G  0 part 
}}}

== CPU ==
{{{
root@rockpis:/# lscpu 
Architecture:        aarch64
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           1
Vendor ID:           ARM
Model:               2
Model name:          Cortex-A35
Stepping:            r0p2
CPU max MHz:         1296.0000
CPU min MHz:         408.0000
BogoMIPS:            48.00
Flags:               fp asimd aes pmull sha1 sha2 crc32
}}}

== Network interfaces ==
{{{
root@rockpis:/# ifconfig 
eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 4e:43:df:6b:85:ff  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6  bytes 752 (752.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 26  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 2  bytes 106 (106.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 106 (106.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

p2p0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 1a:77:e9:6d:75:84  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether e6:a6:66:59:15:ed  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
}}}