Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 28744 0 --:--:-- --:--:-- --:--:-- 29555
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.687 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.687/0.687/0.687/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.488 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.488/0.488/0.488/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.636 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.636/0.636/0.636/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.556 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.556/0.556/0.556/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50992 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 149 MBytes 1.25 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 124 MBytes 1.04 Gbits/sec 1 1.61 MBytes [ 5] 2.00-3.00 sec 148 MBytes 1.24 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 148 MBytes 1.24 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 130 MBytes 1.09 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 140 MBytes 1.17 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 141 MBytes 1.18 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 139 MBytes 1.16 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 129 MBytes 1.08 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 138 MBytes 1.15 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.35 GBytes 1.16 Gbits/sec 1 sender [ 5] 0.00-10.00 sec 1.35 GBytes 1.16 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
06/03/2026-17:59:53.823749 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50976 -> 40.0.0.2:5001 06/03/2026-17:59:53.824752 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50992 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 50103 0 --:--:-- --:--:-- --:--:-- 53200
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.603 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.603/0.603/0.603/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.540 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.540/0.540/0.540/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=4.03 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 4.033/4.033/4.033/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=1.96 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.960/1.960/1.960/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 47078 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 147 MBytes 1.23 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 184 MBytes 1.54 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 154 MBytes 1.29 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 134 MBytes 1.12 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 170 MBytes 1.43 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 188 MBytes 1.57 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 182 MBytes 1.53 Gbits/sec 1 1.20 MBytes [ 5] 7.00-8.00 sec 195 MBytes 1.64 Gbits/sec 0 1.32 MBytes [ 5] 8.00-9.00 sec 151 MBytes 1.27 Gbits/sec 0 1.41 MBytes [ 5] 9.00-10.00 sec 175 MBytes 1.47 Gbits/sec 0 1.48 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.64 GBytes 1.41 Gbits/sec 1 sender [ 5] 0.00-10.00 sec 1.64 GBytes 1.41 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
06/03/2026-18:00:33.343341 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47064 -> 40.0.0.2:5001 06/03/2026-18:00:33.345319 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47078 -> 40.0.0.2:5001
Step 10: Run the command system journal show | cat on DUT0 and check whether the output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jun 03 18:00:23.000066 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Jun 03 18:00:23.000187 osdx systemd-timedated[870021]: Changed local time to Wed 2026-06-03 18:00:23 UTC Jun 03 18:00:23.001786 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'set date 2026-06-03 18:00:23'. Jun 03 18:00:23.002313 osdx systemd-journald[262779]: Time jumped backwards, rotating. Jun 03 18:00:23.356964 osdx systemd-journald[262779]: Runtime Journal (/run/log/journal/140771393e044d28bd27951346e92000) is 1.9M, max 13.8M, 11.9M free. Jun 03 18:00:23.358446 osdx systemd-journald[262779]: Received client request to rotate journal, rotating. Jun 03 18:00:23.358693 osdx systemd-journald[262779]: Vacuuming done, freed 0B of archived journals from /run/log/journal/140771393e044d28bd27951346e92000. Jun 03 18:00:23.368098 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'system journal clear'. Jun 03 18:00:23.668592 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'system coredump delete all'. Jun 03 18:00:24.002090 osdx OSDxCLI[869522]: User 'admin' entered the configuration menu. Jun 03 18:00:24.179894 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jun 03 18:00:24.247684 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jun 03 18:00:24.348479 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jun 03 18:00:24.425713 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jun 03 18:00:24.542731 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jun 03 18:00:24.616417 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jun 03 18:00:24.741842 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'show working'. Jun 03 18:00:24.828663 osdx ubnt-cfgd[870054]: inactive Jun 03 18:00:24.867960 osdx INFO[870061]: FRR daemons did not change Jun 03 18:00:24.910326 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jun 03 18:00:24.935882 osdx WARNING[870103]: No supported link modes on interface eth1 Jun 03 18:00:24.938842 osdx modulelauncher[870103]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Jun 03 18:00:24.938861 osdx modulelauncher[870103]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Jun 03 18:00:24.940738 osdx modulelauncher[870103]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Jun 03 18:00:24.940751 osdx modulelauncher[870103]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jun 03 18:00:24.978344 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jun 03 18:00:25.032585 osdx WARNING[870178]: No supported link modes on interface eth0 Jun 03 18:00:25.034684 osdx modulelauncher[870178]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Jun 03 18:00:25.034705 osdx modulelauncher[870178]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Jun 03 18:00:25.036422 osdx modulelauncher[870178]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Jun 03 18:00:25.036433 osdx modulelauncher[870178]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jun 03 18:00:25.050612 osdx (udev-worker)[870194]: Network interface NamePolicy= disabled on kernel command line. Jun 03 18:00:25.093465 osdx (udev-worker)[870207]: Network interface NamePolicy= disabled on kernel command line. Jun 03 18:00:25.433836 osdx cfgd[1899]: [869522]Completed change to active configuration Jun 03 18:00:25.455833 osdx OSDxCLI[869522]: User 'admin' committed the configuration. Jun 03 18:00:25.482482 osdx OSDxCLI[869522]: User 'admin' left the configuration menu. Jun 03 18:00:28.209443 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jun 03 18:00:28.323539 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:28.460132 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:29.205732 osdx file_operation[870388]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jun 03 18:00:29.234916 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jun 03 18:00:29.381181 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jun 03 18:00:29.547824 osdx OSDxCLI[869522]: User 'admin' entered the configuration menu. Jun 03 18:00:29.612723 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 03 18:00:29.713949 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 03 18:00:29.779271 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 03 18:00:29.874142 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 03 18:00:29.929975 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 03 18:00:30.029530 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 03 18:00:30.135013 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 03 18:00:30.218868 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 03 18:00:30.325373 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 03 18:00:30.392849 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 03 18:00:30.524927 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 03 18:00:30.646199 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 03 18:00:30.745046 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 03 18:00:30.824979 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 03 18:00:30.916126 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 03 18:00:31.033465 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 03 18:00:31.125814 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 03 18:00:31.273193 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 03 18:00:31.346723 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 03 18:00:31.449433 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 03 18:00:31.538257 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'show working'. Jun 03 18:00:31.624375 osdx ubnt-cfgd[870441]: inactive Jun 03 18:00:31.718689 osdx INFO[870489]: FRR daemons did not change Jun 03 18:00:31.979047 osdx systemd[1]: Reloading. Jun 03 18:00:32.030335 osdx systemd-sysv-generator[870538]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jun 03 18:00:32.174716 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 03 18:00:32.199513 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 03 18:00:32.489205 osdx INFO[870521]: Rules successfully loaded Jun 03 18:00:32.496906 osdx cfgd[1899]: [869522]Completed change to active configuration Jun 03 18:00:32.497829 osdx OSDxCLI[869522]: User 'admin' committed the configuration. Jun 03 18:00:32.517315 osdx OSDxCLI[869522]: User 'admin' left the configuration menu. Jun 03 18:00:32.723287 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:32.825815 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:33.355966 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=59507 DF PROTO=TCP SPT=47064 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 03 18:00:33.356939 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19129 DF PROTO=TCP SPT=47078 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 03 18:00:43.533063 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 11: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51636 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 235 MBytes 1.97 Gbits/sec 38 1.21 MBytes [ 5] 1.00-2.00 sec 196 MBytes 1.65 Gbits/sec 0 1.33 MBytes [ 5] 2.00-3.00 sec 209 MBytes 1.75 Gbits/sec 0 1.42 MBytes [ 5] 3.00-4.00 sec 150 MBytes 1.26 Gbits/sec 0 1.49 MBytes [ 5] 4.00-5.00 sec 198 MBytes 1.66 Gbits/sec 0 1.54 MBytes [ 5] 5.00-6.00 sec 185 MBytes 1.55 Gbits/sec 0 1.54 MBytes [ 5] 6.00-7.00 sec 259 MBytes 2.17 Gbits/sec 0 1.54 MBytes [ 5] 7.00-8.00 sec 241 MBytes 2.02 Gbits/sec 0 1.54 MBytes [ 5] 8.00-9.00 sec 222 MBytes 1.87 Gbits/sec 67 1.13 MBytes [ 5] 9.00-10.00 sec 218 MBytes 1.82 Gbits/sec 0 1.26 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.06 GBytes 1.77 Gbits/sec 105 sender [ 5] 0.00-10.00 sec 2.06 GBytes 1.77 Gbits/sec receiver iperf Done.
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
06/03/2026-18:00:33.343341 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47064 -> 40.0.0.2:5001 06/03/2026-18:00:33.345319 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47078 -> 40.0.0.2:5001 06/03/2026-18:00:49.627853 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51626 -> 40.0.0.2:5001 06/03/2026-18:00:49.628865 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51636 -> 40.0.0.2:5001
Step 14: Run the command system journal show | cat on DUT0 and check whether the output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jun 03 18:00:23.000066 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Jun 03 18:00:23.000187 osdx systemd-timedated[870021]: Changed local time to Wed 2026-06-03 18:00:23 UTC Jun 03 18:00:23.001786 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'set date 2026-06-03 18:00:23'. Jun 03 18:00:23.002313 osdx systemd-journald[262779]: Time jumped backwards, rotating. Jun 03 18:00:23.356964 osdx systemd-journald[262779]: Runtime Journal (/run/log/journal/140771393e044d28bd27951346e92000) is 1.9M, max 13.8M, 11.9M free. Jun 03 18:00:23.358446 osdx systemd-journald[262779]: Received client request to rotate journal, rotating. Jun 03 18:00:23.358693 osdx systemd-journald[262779]: Vacuuming done, freed 0B of archived journals from /run/log/journal/140771393e044d28bd27951346e92000. Jun 03 18:00:23.368098 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'system journal clear'. Jun 03 18:00:23.668592 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'system coredump delete all'. Jun 03 18:00:24.002090 osdx OSDxCLI[869522]: User 'admin' entered the configuration menu. Jun 03 18:00:24.179894 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jun 03 18:00:24.247684 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jun 03 18:00:24.348479 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jun 03 18:00:24.425713 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jun 03 18:00:24.542731 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jun 03 18:00:24.616417 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jun 03 18:00:24.741842 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'show working'. Jun 03 18:00:24.828663 osdx ubnt-cfgd[870054]: inactive Jun 03 18:00:24.867960 osdx INFO[870061]: FRR daemons did not change Jun 03 18:00:24.910326 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jun 03 18:00:24.935882 osdx WARNING[870103]: No supported link modes on interface eth1 Jun 03 18:00:24.938842 osdx modulelauncher[870103]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Jun 03 18:00:24.938861 osdx modulelauncher[870103]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Jun 03 18:00:24.940738 osdx modulelauncher[870103]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Jun 03 18:00:24.940751 osdx modulelauncher[870103]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jun 03 18:00:24.978344 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jun 03 18:00:25.032585 osdx WARNING[870178]: No supported link modes on interface eth0 Jun 03 18:00:25.034684 osdx modulelauncher[870178]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Jun 03 18:00:25.034705 osdx modulelauncher[870178]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Jun 03 18:00:25.036422 osdx modulelauncher[870178]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Jun 03 18:00:25.036433 osdx modulelauncher[870178]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Jun 03 18:00:25.050612 osdx (udev-worker)[870194]: Network interface NamePolicy= disabled on kernel command line. Jun 03 18:00:25.093465 osdx (udev-worker)[870207]: Network interface NamePolicy= disabled on kernel command line. Jun 03 18:00:25.433836 osdx cfgd[1899]: [869522]Completed change to active configuration Jun 03 18:00:25.455833 osdx OSDxCLI[869522]: User 'admin' committed the configuration. Jun 03 18:00:25.482482 osdx OSDxCLI[869522]: User 'admin' left the configuration menu. Jun 03 18:00:28.209443 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jun 03 18:00:28.323539 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:28.460132 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:29.205732 osdx file_operation[870388]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jun 03 18:00:29.234916 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jun 03 18:00:29.381181 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jun 03 18:00:29.547824 osdx OSDxCLI[869522]: User 'admin' entered the configuration menu. Jun 03 18:00:29.612723 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 03 18:00:29.713949 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 03 18:00:29.779271 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 03 18:00:29.874142 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 03 18:00:29.929975 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 03 18:00:30.029530 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 03 18:00:30.135013 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 03 18:00:30.218868 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 03 18:00:30.325373 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 03 18:00:30.392849 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 03 18:00:30.524927 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 03 18:00:30.646199 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 03 18:00:30.745046 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 03 18:00:30.824979 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 03 18:00:30.916126 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 03 18:00:31.033465 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 03 18:00:31.125814 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 03 18:00:31.273193 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 03 18:00:31.346723 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 03 18:00:31.449433 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 03 18:00:31.538257 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'show working'. Jun 03 18:00:31.624375 osdx ubnt-cfgd[870441]: inactive Jun 03 18:00:31.718689 osdx INFO[870489]: FRR daemons did not change Jun 03 18:00:31.979047 osdx systemd[1]: Reloading. Jun 03 18:00:32.030335 osdx systemd-sysv-generator[870538]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jun 03 18:00:32.174716 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 03 18:00:32.199513 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 03 18:00:32.489205 osdx INFO[870521]: Rules successfully loaded Jun 03 18:00:32.496906 osdx cfgd[1899]: [869522]Completed change to active configuration Jun 03 18:00:32.497829 osdx OSDxCLI[869522]: User 'admin' committed the configuration. Jun 03 18:00:32.517315 osdx OSDxCLI[869522]: User 'admin' left the configuration menu. Jun 03 18:00:32.723287 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:32.825815 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 03 18:00:33.355966 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=59507 DF PROTO=TCP SPT=47064 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 03 18:00:33.356939 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19129 DF PROTO=TCP SPT=47078 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 03 18:00:43.533063 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Jun 03 18:00:43.658133 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'system journal show | cat'. Jun 03 18:00:43.853183 osdx OSDxCLI[869522]: User 'admin' entered the configuration menu. Jun 03 18:00:43.946636 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 03 18:00:44.029096 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 03 18:00:44.143723 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 03 18:00:44.265772 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 03 18:00:44.352432 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 03 18:00:44.578120 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 03 18:00:44.730882 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 03 18:00:44.835199 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 03 18:00:44.942771 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 03 18:00:45.081802 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 03 18:00:45.199958 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 03 18:00:45.332956 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 03 18:00:45.443443 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 03 18:00:45.584216 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 03 18:00:45.693017 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 03 18:00:45.963714 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 03 18:00:46.066701 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 03 18:00:46.213291 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 03 18:00:46.271570 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 03 18:00:46.379655 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 03 18:00:46.433779 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Jun 03 18:00:46.544154 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Jun 03 18:00:46.634980 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Jun 03 18:00:46.778694 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Jun 03 18:00:46.855101 osdx OSDxCLI[869522]: User 'admin' added a new cfg line: 'show changes'. Jun 03 18:00:46.955511 osdx ubnt-cfgd[870662]: inactive Jun 03 18:00:47.015907 osdx INFO[870685]: FRR daemons did not change Jun 03 18:00:47.237168 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Jun 03 18:00:49.166183 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Jun 03 18:00:49.166317 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Jun 03 18:00:49.166351 osdx systemd[1]: suricata@FW.service: Consumed 2.134s CPU time. Jun 03 18:00:49.190625 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 03 18:00:49.207381 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 03 18:00:49.398722 osdx INFO[870712]: Rules successfully loaded Jun 03 18:00:49.404474 osdx cfgd[1899]: [869522]Completed change to active configuration Jun 03 18:00:49.405022 osdx OSDxCLI[869522]: User 'admin' committed the configuration. Jun 03 18:00:49.420793 osdx OSDxCLI[869522]: User 'admin' left the configuration menu. Jun 03 18:00:49.630338 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=49602 DF PROTO=TCP SPT=51626 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jun 03 18:00:49.630405 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=35625 DF PROTO=TCP SPT=51636 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jun 03 18:00:53.036982 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Jun 03 18:00:59.807557 osdx OSDxCLI[869522]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 8531 0 --:--:-- --:--:-- --:--:-- 8580
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.439 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.439/0.439/0.439/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.690 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.690/0.690/0.690/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.838 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.838/0.838/0.838/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.843 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.843/0.843/0.843/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 52790 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 216 MBytes 1.81 Gbits/sec 0 1.62 MBytes [ 5] 1.00-2.00 sec 144 MBytes 1.21 Gbits/sec 0 1.62 MBytes [ 5] 2.00-3.00 sec 169 MBytes 1.42 Gbits/sec 0 1.62 MBytes [ 5] 3.00-4.00 sec 141 MBytes 1.18 Gbits/sec 0 1.62 MBytes [ 5] 4.00-5.00 sec 192 MBytes 1.61 Gbits/sec 0 1.62 MBytes [ 5] 5.00-6.00 sec 200 MBytes 1.68 Gbits/sec 15 1.24 MBytes [ 5] 6.00-7.00 sec 152 MBytes 1.28 Gbits/sec 0 1.35 MBytes [ 5] 7.00-8.00 sec 164 MBytes 1.37 Gbits/sec 0 1.44 MBytes [ 5] 8.00-9.00 sec 189 MBytes 1.58 Gbits/sec 0 1.51 MBytes [ 5] 9.00-10.01 sec 228 MBytes 1.89 Gbits/sec 1 1.54 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1.75 GBytes 1.50 Gbits/sec 16 sender [ 5] 0.00-10.02 sec 1.75 GBytes 1.50 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
06/03/2026-18:01:32.413455 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52786 -> 40.0.0.2:5001 06/03/2026-18:01:32.414974 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52790 -> 40.0.0.2:5001
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=52790 dport=5001 packets=1299240 bytes=1948494757 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52790 packets=140300 bytes=7288912 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=569 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=569 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=129 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=129 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=52786 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=52786 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=846 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=846 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=568 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=568 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 11: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 46748 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 248 MBytes 2.08 Gbits/sec 0 1.55 MBytes [ 5] 1.00-2.00 sec 230 MBytes 1.93 Gbits/sec 64 1.21 MBytes [ 5] 2.00-3.01 sec 216 MBytes 1.80 Gbits/sec 0 1.33 MBytes [ 5] 3.01-4.00 sec 241 MBytes 2.05 Gbits/sec 0 1.46 MBytes [ 5] 4.00-5.00 sec 215 MBytes 1.80 Gbits/sec 0 1.54 MBytes [ 5] 5.00-6.00 sec 236 MBytes 1.98 Gbits/sec 0 1.54 MBytes [ 5] 6.00-7.00 sec 196 MBytes 1.65 Gbits/sec 0 1.54 MBytes [ 5] 7.00-8.00 sec 209 MBytes 1.75 Gbits/sec 0 1.54 MBytes [ 5] 8.00-9.00 sec 229 MBytes 1.92 Gbits/sec 0 1.54 MBytes [ 5] 9.00-10.00 sec 264 MBytes 2.21 Gbits/sec 0 1.54 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.23 GBytes 1.92 Gbits/sec 64 sender [ 5] 0.00-10.00 sec 2.23 GBytes 1.91 Gbits/sec receiver iperf Done.
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
06/03/2026-18:01:32.413455 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52786 -> 40.0.0.2:5001 06/03/2026-18:01:32.414974 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:52790 -> 40.0.0.2:5001 06/03/2026-18:01:47.232566 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46742 -> 40.0.0.2:5001 06/03/2026-18:01:47.233719 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:46748 -> 40.0.0.2:5001
Step 14: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=46742 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=46742 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=46748 dport=5001 packets=1653775 bytes=2480162593 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=46748 packets=215213 bytes=11183428 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 13167 0 --:--:-- --:--:-- --:--:-- 13333
Step 2: Run the command file show running://drop-performance.rules on DUT0 and expect the following output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=1.03 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.026/1.026/1.026/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.477 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.477/0.477/0.477/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.650 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.650/0.650/0.650/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.941 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.941/0.941/0.941/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/03/2026-18:02:25.947723 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56770 -> 40.0.0.2:5000
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=849 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=849 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=574 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=574 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=132 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=132 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=56770 dport=5000 packets=8 bytes=646 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=56770 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=573 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=573 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 5 flow entries have been shown.
Step 11: Run the command traffic policy FW_PLAN show on DUT0 and check whether the output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 8 210 522 2 - 4 4 312 312 ------------------------------------------------------------------ Total 8 8 522 522 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 11 445 898 2 - 6 6 453 453 ------------------------------------------------------------------ Total 11 11 898 898
Note
Testing with another conntrack mark.
Step 12: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 13: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 14: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/03/2026-18:02:25.947723 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56770 -> 40.0.0.2:5000 06/03/2026-18:02:32.942367 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56770 -> 40.0.0.2:5000 06/03/2026-18:02:34.517313 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41700 -> 40.0.0.2:5000
Step 15: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=41700 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=41700 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 16: Run the command traffic policy FW_PLAN show on DUT0 and check whether the output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 445 775 2 - 5 5 330 330 --------------------------------------------------------------------- Total 10 10 775 775
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 45038 0 --:--:-- --:--:-- --:--:-- 53200
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.467 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.467/0.467/0.467/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=9.63 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 9.628/9.628/9.628/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.655 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.655/0.655/0.655/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=1.03 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.032/1.032/1.032/0.000 ms
Step 8: Initiate a background bandwidth test from DUT2 to DUT1. Control is returned, allowing other tasks to be performed while the test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
06/03/2026-18:03:00.654662 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33268 -> 40.0.0.2:5001 06/03/2026-18:03:00.659827 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33274 -> 40.0.0.2:5001
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33268 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33268 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33274 dport=5001 packets=7477 bytes=11210257 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33274 packets=800 bytes=41620 [ASSURED] [OFFLOAD, packets=7464 bytes=11195056 packets=798 bytes=41508] mark=129834765 use=2 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=852 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=852 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=579 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=579 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=135 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=135 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=578 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=578 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Step 11: Stop the current bandwidth test between DUT2 and DUT1.
Step 12: Initiate a background bandwidth test from DUT2 to DUT1. Control is returned, allowing other tasks to be performed while the test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
06/03/2026-18:03:00.654662 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33268 -> 40.0.0.2:5001 06/03/2026-18:03:00.659827 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33274 -> 40.0.0.2:5001 06/03/2026-18:03:01.360277 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33280 -> 40.0.0.2:5001 06/03/2026-18:03:01.364938 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43484 -> 40.0.0.2:5001 06/03/2026-18:03:01.375358 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43484 -> 40.0.0.2:5001 06/03/2026-18:03:01.387407 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43484 -> 40.0.0.2:5001
Step 14: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33280 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33280 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=33268 dport=5001 packets=11 bytes=746 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33268 packets=10 bytes=544 [ASSURED] [OFFLOAD, packets=2 bytes=105 packets=5 bytes=275] mark=129834765 use=2 icmp 1 28 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=852 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=852 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 28 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=579 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=579 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 28 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=135 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=135 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=43484 dport=5001 packets=15 bytes=20696 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43484 packets=1 bytes=32 [OFFLOAD, packets=11 bytes=16236 packets=0 bytes=0] mark=129834765 use=2 icmp 1 28 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=578 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=578 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 7 flow entries have been shown.
Step 15: Stop the current bandwidth test between DUT2 and DUT1.
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 34447 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run the command file show running://drop-performance.rules on DUT0 and expect the following output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.247 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.247/0.247/0.247/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.496 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.496/0.496/0.496/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.992 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.992/0.992/0.992/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=5.76 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 5.762/5.762/5.762/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/03/2026-18:03:23.212618 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44274 -> 40.0.0.2:5000
Step 10: Run the command service firewall FW show early-drop-stats eth1 on DUT0 and check whether the output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 44274 5000 yes 201 0 8 660 40.0.0.2 20.0.0.2 5000 44274 yes 201 0 0 0
Step 11: Run the command interfaces ethernet eth1 monitor xdp-stats times 1 on DUT0 and expect the following output:
Show output
Period of 0.250138s ending at 1780509806.901699 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 15 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
06/03/2026-18:03:23.212618 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44274 -> 40.0.0.2:5000 06/03/2026-18:03:27.366598 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:45987 -> 40.0.0.2:5001
Step 14: Run the command service firewall FW show early-drop-stats eth1 on DUT0 and check whether the output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5001 45987 no 201 0 0 0 20.0.0.2 40.0.0.2 45987 5001 no 201 0 0 0 20.0.0.2 40.0.0.2 44274 5000 yes 201 0 11 834 40.0.0.2 20.0.0.2 5000 44274 yes 201 0 0 0
Step 15: Run the command interfaces ethernet eth1 monitor xdp-stats times 1 on DUT0 and expect the following output:
Show output
Period of 0.250122s ending at 1780509811.025739 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 36 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)