neru has modest hardware specs but I'm not planning to use her as a router. She's a small, low-power appliance that can run containers and takes a USB stick for extra storage.
neru is a hexS 2025 model:
Homepage: https://mikrotik.com/product/hex_s_2025
- Product code: E60iUGS
Contents
What runs here
- TFTP serving up grub and grub config
- nginx in a container serving kernels and kickstart configs, referred to by grub config
Setting up a new client for PXE booting
- Configure DHCP lease on appropriate server (likely hoshino nowadays)
- Lease needs to be in a suitable Network
- Network points to neru (192.168.1.25) as next server
Boot file name is grub/grubx64-improved.efi
- Create grub config on neru
Mount neru's filesystem on a client: sshfs -o reconnect furinkan@neru.thighhighs.top:/usb1p1/srv /mnt/neru-usb/
Create the client-specific config file in grub/grub.cfg-01-MA-CM-AC-MA-CM-AC, copying an existing one as a template and editing it
Create the kickstart config in ks/hostname.ks.cfg, copying an existing one as a template and editing it
You should now be ready to boot and fully automatically install your client!
Features and uses
- 5x 1G RJ45 ports
- 1x 2.5G SFP port
- PoE powered
- arm CPU, meaning we can run containers
Hardware
- CPU: EN7562CT
Switch chip: EN7523 (https://help.mikrotik.com/docs/spaces/ROS/pages/15302988/Switch+Chip+Features)
A surprisingly decent switch chip for such a cheap device, it does hardware STP and VLAN filtering sensibly.
Config
Notes from when I set her up as a PXE server, running TFTP (native) and HTTP (nginx in a container), serving files from the USB stick.
Served up here: http://neru.thighhighs.top:8080/
Config dump
Taken from a /export on the CLI.
It might look a bit unusual because there's no default bridge, we're only using ether1 as the WAN port, and NATing traffic to/from the container NIC running on there.
# This is for the container NICs (veth) to attach to, and it works just like a normal LAN segment.
# The router is the gateway and has the .1 address on the bridge, and the clients on the segment
# get .2, .3, .4 etc as usual. You masquerade-NAT traffic from the containers to the outside
# world, and DNAT any inbound connections that need to go to a container listening port.
#
# Mikrotik/ROS doesn't have host-mode networking for containers, but you can emulate it by giving
# the veth interface an address on the "public" network, basically a sibling to the router's
# outbound interface. Join both to a bridge (be careful with the STP settings), and now you have
# a container directly on the network, no NAT needed. The only caveat is that you need an IP for
# each container, rather than them all sharing the host's IP address.
#
# In this case, I'm happy enough with NAT and it works just fine.
/interface bridge
add name=containers priority=0xd000 protocol-mode=mstp vlan-filtering=yes
# Virtual NIC that the container will own. We give it an IP address that we'll NAT from/to
/interface veth
add address=172.25.0.2/24 container-mac-address=2E:9E:C1:16:16:92 dhcp=no gateway=172.25.0.1 gateway6="" mac-address=2E:9E:C1:16:16:91 name=nginx
# Add the container veth to the bridge
/interface bridge port
add bridge=containers interface=nginx
# These VLANs live directly on ether1 because it's all that matters, we're not using the default bridge.
/interface vlan
add interface=ether1 name=VLAN10_GENERAL vlan-id=10
add interface=ether1 name=VLAN11_MGMT vlan-id=11
# We don't really need interface lists, but it's kinda handy in case we need them later.
# For example we might have a bunch of containers and want them to talk to each other,
# but not necessarily with the outside world.
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface list member
add interface=ether1 list=WAN
add interface=VLAN11_MGMT list=LAN
add interface=VLAN10_GENERAL list=LAN
/user group
add name=mikrodash policy=read,test,api,!local,!telnet,!ssh,!ftp,!reboot,!write,!policy,!winbox,!password,!web,!sniff,!sensitive,!romon,!rest-api
/container
add domain-name=thighhighs.top env="HOME=/usr/share/nginx/html" hostname=neru interface=nginx layer-dir="" logging=yes memory-high=128.0MiB mount=\
/usb1p1/srv:/usr/share/nginx/html:ro,/usb1p1/nginx-conf.d:/etc/nginx/conf.d:ro name=nginx remote-image=arm32v5/nginx:trixie root-dir=/usb1p1/container-roots/nginx start-on-boot=yes
/container config
set memory-high=256.0MiB registry-url=https://registry-1.docker.io tmpdir=/usb1p1/tmp
/disk
add mount-point-template=usb1p1 parent=usb1 partition-number=1 partition-offset=16384 partition-size=61524131840 type=partition
/disk settings
set auto-media-interface=*8
/ip neighbor discovery-settings
set discover-interface-list=all lldp-mac-phy-config=yes lldp-max-frame-size=yes lldp-vlan-info=yes
# Note that the VLAN 10 interface gets its address via DHCP, only the mgmt address is static.
# We need to make sure our NAT rules handle this correctly, don't hard-code addresses.
/ip address
add address=192.168.11.9/24 interface=VLAN11_MGMT network=192.168.11.0
add address=172.25.0.1/24 interface=containers network=172.25.0.0
/ip dhcp-client
add interface=VLAN10_GENERAL name=client1
# Can be queried as a nameserver, probably to serve the containers (not that they would need to, I think)
/ip dns
set allow-remote-requests=yes
# This is really standard stuff, with tight rules to only accept exactly what we want.
# The INPUT chain is boring, the FORWARD chain is where the fun happens (just a little).
#
# There's an implicit Accept at the end of chain. That means that if a connection arrives and:
# - it's not destined for the local host (this router's mgmt or general IP address)
# - that means it would need to be forwarded
# - but it's been dstnat'ed already
# then it's okay to forward it along. Otherwise it's getting dropped. That means we won't
# route traffic between the mgmt and general networks, but we will forward traffic through
# to the container subnet.
#
# Note that if you went with host-style networking, and the veth has an IP address on the
# public network, I'm not sure if that counts as hitting the Input or Forward chain.
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Mikrotik Winbox" dst-port=8291 in-interface=VLAN11_MGMT protocol=tcp
add action=accept chain=input comment=SSH dst-port=22 in-interface-list=LAN protocol=tcp
add action=accept chain=input comment="TFTP server" dst-port=69 in-interface-list=LAN log-prefix="udp/69 TFTP" protocol=udp
add action=accept chain=input comment="SNMP server" dst-port=161 in-interface-list=LAN protocol=udp
add action=accept chain=input comment="tcp/8728 Mikrotik api" dst-port=8728 in-interface=VLAN11_MGMT protocol=tcp
add action=accept chain=input comment="tcp/8729 Mikrotik api-ssl" dst-port=8729 in-interface=VLAN11_MGMT protocol=tcp
add action=accept chain=input comment="udp/5678 Neighbour Discovery" dst-address=255.255.255.255 dst-port=5678 log-prefix="udp/5678 Neighbour Discovery" protocol=udp
add action=drop chain=input comment="Drop everything else"
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=forward comment="Allow containers outbound DNS (udp)" connection-state=new dst-port=53 in-interface=containers out-interface=VLAN10_GENERAL protocol=udp
add action=accept chain=forward comment="Allow containers outbound DNS (tcp)" connection-state=new dst-port=53 in-interface=containers out-interface=VLAN10_GENERAL protocol=tcp
add action=accept chain=forward comment="Allow containers outbound HTTP" connection-state=new dst-port=80 in-interface=containers out-interface=VLAN10_GENERAL protocol=tcp
add action=accept chain=forward comment="Allow containers outbound HTTPS" connection-state=new dst-port=443 in-interface=containers out-interface=VLAN10_GENERAL protocol=tcp
add action=drop chain=forward comment="Drop anything not DSTNATed, nginx implicit accepted" connection-nat-state=!dstnat connection-state=new log=yes log-prefix="dropped forward"
# Heaps simple: masquerade (SNAT) traffic from containers to the outside network
# (which is kinda incorrectly called "LAN" here), and DNAT inbound connections if
# they match a listening container.
/ip firewall nat
add action=masquerade chain=srcnat in-interface=containers out-interface=VLAN10_GENERAL
add action=dst-nat chain=dstnat dst-port=8080 in-interface-list=LAN protocol=tcp to-addresses=172.25.0.2 to-ports=80
# The TFTP server is a bit fiddly on Mikrotik, and people report not being able
# to get it work, despite being absolutely trivial to configure.
#
# This config maps all filenames directly through to `/usb1p1/srv/` on the USB stick. I've made it
# writeable because that's useful sometimes, and I trust the clients enough on my network.
#
# Note that the TFTP server can only handle files up to 64MiB in size, it'll error out if
# the transfer gets that far. That's why we need to run an HTTP server in the first place,
# because initrd files are a couple hundred MiB. Also it's waaaaaay faster, you'll
# appreciate the time savings.
/ip tftp
add read-only=no real-filename=usb1p1/srv/ req-filename=.*
/ip tftp settings
set max-block-size=8192
# This is mostly stock standard defaults, I don't really run IPv6 on my network.
/ipv6 address
add address=fdfd:2501:2762:11::7 interface=VLAN11_MGMT
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
# I should clean these up but I'm lazy.
/ipv6 firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" dst-port=33434-33534 protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
add action=fasttrack-connection chain=forward comment="defconf: fasttrack6" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
# Basic system stuff
/snmp
set contact=Furinkan enabled=yes location="Conferta Ave"
/system clock
set time-zone-name=Australia/Sydney
/system identity
set name=neru
/system ntp client
set enabled=yes
/system ntp client servers
add address=pool.ntp.org
add address=au.pool.ntp.org
# This is nice, means it doesn't beep after booting up every time. It's fine, but I want to do
# frequent RouterOS updates without annoying the family, and the beep is LOUD.
/system routerboard settings
set silent-boot=yes
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
Original setup exploration
I have aris, she's an original hex S with microSD card slot and USB. She has an mmips CPU, which means no container support 
But I have neru, she's a hex S 2025 model, no microSD but with USB. I'd prefer microSD because I have spares, but beggars can't be choosers. I really don't want stuff sticking out, so it'll have to be a micro-profile USB stick.
I've already messed with this before, I've enabled container mode (https://help.mikrotik.com/docs/spaces/ROS/pages/93749258/Device-mode#Devicemode-Enablingdevice-modefeature) and installed the container package.
pop in the usb stick (Sandisk Ultra Fit 64gb USB 3.2 gen1 5Gbps), it already has a partition and FS. Set the mountpoint and format it.
/disk/set usb1-part1 mount-point-template=usb1p1 /disk/settings/set auto-media-sharing=no auto-smb-sharing=no /disk/format usb1-part1 file-system=ext4 label=neru-data
Follow the network setup outline, I've picked an IP range that shouldn't conflict with any of my stuff. The suggested docker network settings are fine in most cases though: https://help.mikrotik.com/docs/spaces/ROS/pages/84901929/Container#Container-RunningPi-hole
/interface/bridge/add name=containers priority=0xD000 protocol-mode=mstp vlan-filtering=yes /ip/address/add address=172.25.0.1/24 interface=containers /interface/veth/add name=netbooting address=172.25.0.2/24 gateway=172.25.0.1 /interface/bridge/port/add bridge=containers interface=netbooting /ip/firewall/nat/add chain=srcnat action=masquerade src-address=172.25.0.0/24
Setup container stuff
/file/add type=directory name=usb1p1/containers /file/add type=directory name=usb1p1/srv /file/add type=directory name=usb1p1/tmp /container/config/set tmpdir=usb1p1/tmp /container/config/set registry-url=https://registry-1.docker.io /container/config/set memory-high=256MiB /container/add auto-restart-interval=1m domain-name=thighhighs.top hostname=neru interface=netbooting logging=yes memory-high=128.0MiB mount=/usb1p1/srv:/var/www:ro name=caddy remote-image=library/caddy root-dir=/usb1p1/container-roots/caddy start-on-boot=yes workdir=/srv
The container dies immediately after start, log shows "exited with signal 4", which is a SIGILL. I'm almost certain this is an architecture limitation. Sure enough...
> For devices with EN7562CT CPU like the hEX Refresh, only arm32v5 container images are supported
I bet it's more than just that. I need a caddy image that supports v5 specifically. It's probably pulled a v7 or v8 image.
Okay caddy is too much fucking work. I'd have to build it myself on a different machine, and the Dockerfile for it only uses alpine, which isn't supported on arm32v5. I'll just use nginx, I only need static fileserving anyway so it's all moot.
https://hub.docker.com/r/arm32v5/nginx
/container/add name=nginx remote-image=arm32v5/nginx:trixie check-certificate=yes interface=netbooting mount=/usb1p1/srv:/usr/share/nginx/html:ro hostname=neru domain-name=thighhighs.top memory-high=128.0MiB root-dir=/usb1p1/container-roots/nginx start-on-boot=yes logging=yes /container/start nginx /ip/firewall/nat/add chain=dstnat dst-address=192.168.1.25 dst-port=8080 protocol=tcp action=dst-nat to-addresses=172.25.0.2 to-ports=80
Configure it:
root@abc3def4c047:/etc/nginx/conf.d# cat default.conf
server {
listen 80;
server_name localhost;
#access_log /var/log/nginx/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}