makiatto is a 10Gbps switch which will handle traffic for high bandwidth devices. Right now that's the NAS ([[../fenny| fenny]]), and will include my workstation once I get 10G connectivity to that room as well. makiatto is connected to [[../sabrina| sabrina]] via a 2x10G LAG, though for practical purposes there's nothing connected to sabrina that will need that much bandwidth, even in aggregate. makiatto is a Mikrotik CRS312-4C+8XG-RM: * the homepage: https://mikrotik.com/product/crs312_4c_8xg_rm The CRS312 has the option to run SwitchOS, but I'm sticking with RouterOS because it integrates well with everything else I'm using. <> = Features and uses = * 8x 10G RJ45 ports * 4x 10G RJ45-or-SFP+ combo ports * 1x 100M mgmt port attached directly to the CPU = Hardware = Marvell 98DX8212 switch chip, which governs what features you can lean on and how they're handled. Mikrotik has notes about L3 Hardware Offload: https://help.mikrotik.com/docs/spaces/ROS/pages/62390319/L3+Hardware+Offloading#L3HardwareOffloading-CRS3xx%2CCRS5xx%3ASwitchDX8000andDX4000Series And some notes about the switch chips specifically in the higher end models: https://help.mikrotik.com/docs/spaces/ROS/pages/30474317/CRS3xx+CRS5xx+CCR2116+CCR2216+switch+chip+features#CRS3xx,CRS5xx,CCR2116,CCR2216switchchipfeatures-Models Of particular interest to us is hardware offloading of bridge VLAN filtering (with some listed [[https://help.mikrotik.com/docs/spaces/ROS/pages/328068/Bridging+and+Switching#BridgingandSwitching-BridgeVLANFiltering| caveats]]), which means performance won't dip even if I need to do some VLAN faffery. = Config = Some notes from when I received it. We'll use sfpplus3/4 on both sabrina and makiatto and form a bonded trunk link. == Prepare sabrina == * remove sfpplus3/4 from bridge on sabrina * create bond-makiatto on sabrina * MTU 9000 * slaves sfpplus 3/4 * for the mode let's try balance-xor as it can use layer3+4 hashing: "Only 802.3ad and balance-xor bonding modes are hardware offloaded, other bonding modes will use the CPU's resources. The built-in switch chip will always use Layer2+Layer3+Layer4 for a transmit hash policy, changing the transmit hash policy manually will have no effect" * select hash policy 3+4 from the options anyway * LACP rate 1sec (though we're not using LACP?) * add the bond to the bridge on sabrina {{{ /interface bridge port add bridge=bridge comment="makiatto uplink" interface=bond-makiatto }}} == Bootstrap makiatto == I can't get onto makiatto's serial console for some reason, so I'll just use the Mikrotik default IP address method, with a USB ethernet dongle on my laptop for convenience. * assign IP/mask to bridge {{{ /ip/address/add address=192.168.1.3/24 interface=bridge network=192.168.1.0 /ip/dns/set servers=192.168.1.26 /ip/route/add dst-address=0.0.0.0/0 gateway=192.168.1.1 }}} * connect any ethernet port on sabrina to makiatto as a proto uplink * get on winbox now as it's easier * do general system config * set new admin password * create new user account for myself with secure password * login again as myself * disable admin account * add ssh key {{{ /user/ssh-keys/add user=furinkan key="ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIISjFHfP6t3LLM9D7uWroYCrSdcZqN17tl5eQ/eJlKgq furinkan@wa-chan 2022-03-21" }}} * add oxidized user {{{ /user/add name=oxidized group=read password="upz.udf5jeq0wtr*NCK" address=192.168.3.0/26 }}} * general settings {{{ /snmp/set contact=Furinkan enabled=yes location="Conferta Ave" /system/clock/set time-zone-name=Australia/Sydney /system/identity/set name=makiatto /system/ntp/client/servers/add address=pool.ntp.org /system/ntp/client/set enabled=yes }}} * do we need to do hardware config now? Actually not really, the fans are pretty reasonable out-of-the-box. {{{ [furinkan@makiatto] > /system/health/print Columns: NAME, VALUE, TYPE # NAME VALUE TYPE 0 switch-temperature 52 C 1 phy-temperature 54 C 2 fan1-speed 0 RPM 3 fan2-speed 0 RPM 4 fan3-speed 0 RPM 5 fan4-speed 0 RPM 6 psu1-state ok 7 psu2-state fail [furinkan@makiatto] > /system/health/settings/print fan-full-speed-temp: 65C fan-target-temp: 58C fan-min-speed-percent: 0% fan-control-interval: 30s }}} This is nice, it actually spins down to zero RPM when it's cool. I'll leave those settings as-is, they're good. * set the MTU on everything to be 9000/9092 now {{{ /interface/ethernet/set [ find default-name~"combo[1-4]" ] l2mtu=9098 mtu=9000 /interface/ethernet/set [ find default-name~"ether[1-8]" ] l2mtu=9098 mtu=9000 }}} == Setup the bond interface on makiatto == * remove combo3-4 from the bridge {{{ /interface/bridge/port/remove [ find interface=combo3 ] /interface/bridge/port/remove [ find interface=combo4 ] }}} * create the bond and add to the bridge {{{ /interface/bonding/add name=bond-sabrina mode=balance-xor lacp-rate=1sec mtu=9000 slaves=combo3,combo4 transmit-hash-policy=layer-3-and-4 /interface/bridge/port/add bridge=bridge comment="sabrina uplink" interface=bond-sabrina /interface/bridge/port/comment [ find interface~"combo[1-4]" ] comment="" /interface/bridge/port/comment [ find interface~"ether[1-9]" ] comment="" }}} * connect first uplink port between sabrina and makiatto * connect second uplink port between sabrina and makiatto Looks good! Does the bridge's MTU go up if we remove ether9 (mgmt port) from the bridge? That port has a max MTU of 2028 and max speed of 100M. {{{ /interface/bridge/port/remove [ find interface=ether9 ] }}} It does! Up to 9098 now, baby! Let's give ether9 the default static IP address and be done with it. {{{ /ip/address/add address=192.168.88.1/24 interface=ether9 }}} == Housekeeping == Check for updates and apply latest stable Router OS version. makiatto is currently running RouterOS 7.12.2 and the latest is 7.17 at the time of writing (2025-01-20) == Prepare LACP bond ports for fenny == fenny has 10G RJ45 ports, so we'll connect her to ether7-8 on makiatto. {{{ /interface/bridge/port/remove [ find interface~"ether[78]" ] /interface/bonding/add name=bond-fenny mode=802.3ad lacp-rate=1sec mtu=9000 slaves=ether7,ether8 /interface/bridge/port/add bridge=bridge comment="fenny NAS" interface=bond-fenny }}} = Fun ideas = Like the CRS320, the CRS312 also has lots of free real estate on the front. Why not put a custom waifu decal on it? See [[../sabrina#Fun_ideas| sabrina]] for context.