makiatto is a 10Gbps switch which will handle traffic for high bandwidth devices. Right now that's the NAS (fenny), and will include my workstation once I get 10G connectivity to that room as well. makiatto is connected to sabrina via a 2x10G LAG, though for practical purposes there's nothing connected to sabrina that will need that much bandwidth, even in aggregate.

makiatto is a Mikrotik CRS312-4C+8XG-RM:

• the homepage: https://mikrotik.com/product/crs312_4c_8xg_rm

The CRS312 has the option to run SwitchOS, but I'm sticking with RouterOS because it integrates well with everything else I'm using.

Features and uses

• 8x 10G RJ45 ports
• 4x 10G RJ45-or-SFP+ combo ports
• 1x 100M mgmt port attached directly to the CPU

Hardware

Marvell 98DX8212 switch chip, which governs what features you can lean on and how they're handled. Mikrotik has notes about L3 Hardware Offload: https://help.mikrotik.com/docs/spaces/ROS/pages/62390319/L3+Hardware+Offloading#L3HardwareOffloading-CCR2xxx,CRS3xx,CRS5xx:Switch98DX8xxxand98DX4xxxSeries

And some notes about the switch chips specifically in the higher end models: https://help.mikrotik.com/docs/spaces/ROS/pages/30474317/CRS3xx+CRS5xx+CCR2116+CCR2216+switch+chip+features#CRS3xx,CRS5xx,CCR2116,CCR2216switchchipfeatures-Models

Of particular interest to us is hardware offloading of bridge VLAN filtering (with some listed caveats), which means performance won't dip even if I need to do some VLAN faffery.

Config

Some notes from when I received it. We'll use sfpplus3/4 on both sabrina and makiatto and form a bonded trunk link.

Prepare sabrina

• remove sfpplus3/4 from bridge on sabrina
• create bond-makiatto on sabrina
  • MTU 9000
  • slaves sfpplus 3/4
  • for the mode let's try balance-xor as it can use layer3+4 hashing: "Only 802.3ad and balance-xor bonding modes are hardware offloaded, other bonding modes will use the CPU's resources. The built-in switch chip will always use Layer2+Layer3+Layer4 for a transmit hash policy, changing the transmit hash policy manually will have no effect"
  • select hash policy 3+4 from the options anyway
  • LACP rate 1sec (though we're not using LACP?)

• add the bond to the bridge on sabrina

  /interface bridge port
  add bridge=bridge comment="makiatto uplink" interface=bond-makiatto

Bootstrap makiatto

I can't get onto makiatto's serial console for some reason, so I'll just use the Mikrotik default IP address method, with a USB ethernet dongle on my laptop for convenience.

• assign IP/mask to bridge

  /ip/address/add address=192.168.1.3/24 interface=bridge network=192.168.1.0
  /ip/dns/set servers=192.168.1.26
  /ip/route/add dst-address=0.0.0.0/0 gateway=192.168.1.1

• connect any ethernet port on sabrina to makiatto as a proto uplink
• get on winbox now as it's easier
• do general system config
  • set new admin password
  • create new user account for myself with secure password
  • login again as myself
  • disable admin account

  • add ssh key

    /user/ssh-keys/add user=furinkan key="ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIISjFHfP6t3LLM9D7uWroYCrSdcZqN17tl5eQ/eJlKgq furinkan@wa-chan 2022-03-21"

  • add oxidized user

    /user/add name=oxidized group=read password="upz.udf5jeq0wtr*NCK" address=192.168.3.0/26

  • general settings

    /snmp/set contact=Furinkan enabled=yes location="Conferta Ave"
    /system/clock/set time-zone-name=Australia/Sydney
    /system/identity/set name=makiatto
    /system/ntp/client/servers/add address=pool.ntp.org
    /system/ntp/client/set enabled=yes

  • do we need to do hardware config now? Actually not really, the fans are pretty reasonable out-of-the-box.

    [furinkan@makiatto] > /system/health/print
    Columns: NAME, VALUE, TYPE
    #  NAME                VALUE  TYPE
    0  switch-temperature  52     C
    1  phy-temperature     54     C
    2  fan1-speed          0      RPM
    3  fan2-speed          0      RPM
    4  fan3-speed          0      RPM
    5  fan4-speed          0      RPM
    6  psu1-state          ok
    7  psu2-state          fail
    
    [furinkan@makiatto] > /system/health/settings/print
        fan-full-speed-temp: 65C
            fan-target-temp: 58C
      fan-min-speed-percent: 0%
       fan-control-interval: 30s

    This is nice, it actually spins down to zero RPM when it's cool. I'll leave those settings as-is, they're good.

• set the MTU on everything to be 9000/9092 now

  /interface/ethernet/set [ find default-name~"combo[1-4]" ] l2mtu=9098 mtu=9000
  /interface/ethernet/set [ find default-name~"ether[1-8]" ] l2mtu=9098 mtu=9000

Setup the bond interface on makiatto

• remove combo3-4 from the bridge

  /interface/bridge/port/remove [ find interface=combo3 ]
  /interface/bridge/port/remove [ find interface=combo4 ]

• create the bond and add to the bridge

  /interface/bonding/add name=bond-sabrina mode=balance-xor lacp-rate=1sec mtu=9000 slaves=combo3,combo4 transmit-hash-policy=layer-3-and-4
  /interface/bridge/port/add bridge=bridge comment="sabrina uplink" interface=bond-sabrina
  /interface/bridge/port/comment [ find interface~"combo[1-4]" ] comment=""
  /interface/bridge/port/comment [ find interface~"ether[1-9]" ] comment=""

• connect first uplink port between sabrina and makiatto
• connect second uplink port between sabrina and makiatto

Looks good!

Does the bridge's MTU go up if we remove ether9 (mgmt port) from the bridge? That port has a max MTU of 2028 and max speed of 100M.

/interface/bridge/port/remove [ find interface=ether9 ]

It does! Up to 9098 now, baby!

Let's give ether9 the default static IP address and be done with it.

/ip/address/add address=192.168.88.1/24 interface=ether9

Housekeeping

Check for updates and apply latest stable Router OS version.

makiatto is currently running RouterOS 7.12.2 and the latest is 7.17 at the time of writing (2025-01-20)

Prepare LACP bond ports for fenny

fenny has 10G RJ45 ports, so we'll connect her to ether7-8 on makiatto.

/interface/bridge/port/remove [ find interface~"ether[78]" ]
/interface/bonding/add name=bond-fenny mode=802.3ad lacp-rate=1sec mtu=9000 slaves=ether7,ether8
/interface/bridge/port/add bridge=bridge comment="fenny NAS" interface=bond-fenny

Fun ideas

Like the CRS320, the CRS312 also has lots of free real estate on the front. Why not put a custom waifu decal on it? See sabrina for context.