Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 55474 0 --:--:-- --:--:-- --:--:-- 66500
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.583 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.583/0.583/0.583/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.414 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.414/0.414/0.414/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.788 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.788/0.788/0.788/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=1.68 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.679/1.679/1.679/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 37162 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 139 MBytes 1.17 Gbits/sec 101 1.15 MBytes [ 5] 1.00-2.00 sec 136 MBytes 1.14 Gbits/sec 0 1.29 MBytes [ 5] 2.00-3.00 sec 132 MBytes 1.11 Gbits/sec 0 1.39 MBytes [ 5] 3.00-4.00 sec 140 MBytes 1.17 Gbits/sec 0 1.47 MBytes [ 5] 4.00-5.00 sec 136 MBytes 1.14 Gbits/sec 0 1.53 MBytes [ 5] 5.00-6.00 sec 120 MBytes 1.01 Gbits/sec 0 1.54 MBytes [ 5] 6.00-7.00 sec 141 MBytes 1.18 Gbits/sec 0 1.54 MBytes [ 5] 7.00-8.00 sec 142 MBytes 1.20 Gbits/sec 0 1.54 MBytes [ 5] 8.00-9.00 sec 139 MBytes 1.16 Gbits/sec 0 1.54 MBytes [ 5] 9.00-10.00 sec 144 MBytes 1.21 Gbits/sec 0 1.54 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.34 GBytes 1.15 Gbits/sec 101 sender [ 5] 0.00-10.00 sec 1.34 GBytes 1.15 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
09/16/2026-23:47:22.532398 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37150 -> 40.0.0.2:5001 09/16/2026-23:47:22.533395 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37162 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 82943 0 --:--:-- --:--:-- --:--:-- 88666
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.553 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.553/0.553/0.553/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.544 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.544/0.544/0.544/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=3.79 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 3.794/3.794/3.794/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.709 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.709/0.709/0.709/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 43912 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 249 MBytes 2.08 Gbits/sec 0 1.60 MBytes [ 5] 1.00-2.00 sec 208 MBytes 1.75 Gbits/sec 0 1.60 MBytes [ 5] 2.00-3.00 sec 214 MBytes 1.79 Gbits/sec 0 1.60 MBytes [ 5] 3.00-4.00 sec 248 MBytes 2.08 Gbits/sec 0 1.60 MBytes [ 5] 4.00-5.00 sec 221 MBytes 1.86 Gbits/sec 0 1.60 MBytes [ 5] 5.00-6.00 sec 204 MBytes 1.71 Gbits/sec 0 1.60 MBytes [ 5] 6.00-7.00 sec 122 MBytes 1.03 Gbits/sec 32 1.14 MBytes [ 5] 7.00-8.00 sec 204 MBytes 1.71 Gbits/sec 0 1.27 MBytes [ 5] 8.00-9.00 sec 201 MBytes 1.69 Gbits/sec 0 1.37 MBytes [ 5] 9.00-10.00 sec 188 MBytes 1.57 Gbits/sec 0 1.45 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.01 GBytes 1.73 Gbits/sec 32 sender [ 5] 0.00-10.38 sec 2.01 GBytes 1.66 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
09/16/2026-23:47:59.592259 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43900 -> 40.0.0.2:5001 09/16/2026-23:47:59.593360 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43912 -> 40.0.0.2:5001
Step 10: Run the command system journal show | cat on DUT0 and check whether the output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Sep 16 23:47:50.427941 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Sep 16 23:47:50.000238 osdx systemd-timedated[517692]: Changed local time to Wed 2026-09-16 23:47:50 UTC Sep 16 23:47:50.001700 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'set date 2026-09-16 23:47:50'. Sep 16 23:47:50.002932 osdx systemd-journald[303514]: Time jumped backwards, rotating. Sep 16 23:47:50.334499 osdx systemd-journald[303514]: Runtime Journal (/run/log/journal/7ab9d0c6c88d486ab30f6cca192546c5) is 3.4M, max 13.8M, 10.3M free. Sep 16 23:47:50.334923 osdx systemd-journald[303514]: Received client request to rotate journal, rotating. Sep 16 23:47:50.334965 osdx systemd-journald[303514]: Vacuuming done, freed 0B of archived journals from /run/log/journal/7ab9d0c6c88d486ab30f6cca192546c5. Sep 16 23:47:50.344999 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'system journal clear'. Sep 16 23:47:50.572933 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'system coredump delete all'. Sep 16 23:47:50.778551 osdx OSDxCLI[517175]: User 'admin' entered the configuration menu. Sep 16 23:47:50.899665 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Sep 16 23:47:50.953937 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Sep 16 23:47:51.064523 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Sep 16 23:47:51.132334 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Sep 16 23:47:51.229113 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Sep 16 23:47:51.291061 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Sep 16 23:47:51.397865 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'show working'. Sep 16 23:47:51.459718 osdx ubnt-cfgd[517725]: inactive Sep 16 23:47:51.496905 osdx INFO[517738]: FRR daemons did not change Sep 16 23:47:51.526955 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Sep 16 23:47:51.546416 osdx WARNING[517780]: No supported link modes on interface eth1 Sep 16 23:47:51.547736 osdx modulelauncher[517780]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Sep 16 23:47:51.547748 osdx modulelauncher[517780]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Sep 16 23:47:51.548848 osdx modulelauncher[517780]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Sep 16 23:47:51.548855 osdx modulelauncher[517780]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Sep 16 23:47:51.578933 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Sep 16 23:47:51.625472 osdx WARNING[517855]: No supported link modes on interface eth0 Sep 16 23:47:51.626799 osdx modulelauncher[517855]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Sep 16 23:47:51.626815 osdx modulelauncher[517855]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Sep 16 23:47:51.628328 osdx modulelauncher[517855]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Sep 16 23:47:51.628338 osdx modulelauncher[517855]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Sep 16 23:47:51.636360 osdx (udev-worker)[517869]: Network interface NamePolicy= disabled on kernel command line. Sep 16 23:47:51.976323 osdx cfgd[1899]: [517175]Completed change to active configuration Sep 16 23:47:51.976881 osdx OSDxCLI[517175]: User 'admin' committed the configuration. Sep 16 23:47:51.997745 osdx OSDxCLI[517175]: User 'admin' left the configuration menu. Sep 16 23:47:54.677524 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Sep 16 23:47:54.769705 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:54.851048 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:55.512254 osdx file_operation[518066]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Sep 16 23:47:55.539612 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Sep 16 23:47:55.710420 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Sep 16 23:47:55.862023 osdx OSDxCLI[517175]: User 'admin' entered the configuration menu. Sep 16 23:47:55.928610 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Sep 16 23:47:56.040223 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Sep 16 23:47:56.102745 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Sep 16 23:47:56.206193 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Sep 16 23:47:56.267629 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Sep 16 23:47:56.406192 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Sep 16 23:47:56.533940 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Sep 16 23:47:56.617163 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Sep 16 23:47:56.757965 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Sep 16 23:47:56.833954 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Sep 16 23:47:56.941203 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Sep 16 23:47:57.012113 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Sep 16 23:47:57.151841 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Sep 16 23:47:57.231229 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Sep 16 23:47:57.327206 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Sep 16 23:47:57.396231 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Sep 16 23:47:57.482343 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Sep 16 23:47:57.623830 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Sep 16 23:47:57.679597 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Sep 16 23:47:57.780663 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Sep 16 23:47:57.887010 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'show working'. Sep 16 23:47:58.026214 osdx ubnt-cfgd[518119]: inactive Sep 16 23:47:58.145842 osdx INFO[518167]: FRR daemons did not change Sep 16 23:47:58.441628 osdx systemd[1]: Reloading. Sep 16 23:47:58.506949 osdx systemd-sysv-generator[518221]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Sep 16 23:47:58.663350 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Sep 16 23:47:58.686244 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Sep 16 23:47:58.971737 osdx INFO[518200]: Rules successfully loaded Sep 16 23:47:58.979294 osdx cfgd[1899]: [517175]Completed change to active configuration Sep 16 23:47:58.979889 osdx OSDxCLI[517175]: User 'admin' committed the configuration. Sep 16 23:47:58.995071 osdx OSDxCLI[517175]: User 'admin' left the configuration menu. Sep 16 23:47:59.157127 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:59.240115 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:59.594953 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=8820 DF PROTO=TCP SPT=43900 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Sep 16 23:47:59.595042 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=34405 DF PROTO=TCP SPT=43912 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Sep 16 23:48:08.891491 osdx systemd[1]: Starting logrotate.service - Rotate log files... Sep 16 23:48:08.934775 osdx systemd[1]: logrotate.service: Deactivated successfully. Sep 16 23:48:08.934941 osdx systemd[1]: Finished logrotate.service - Rotate log files. Sep 16 23:48:10.287410 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 11: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 51352 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 257 MBytes 2.16 Gbits/sec 0 1.60 MBytes [ 5] 1.00-2.00 sec 222 MBytes 1.87 Gbits/sec 0 1.60 MBytes [ 5] 2.00-3.00 sec 205 MBytes 1.72 Gbits/sec 7 1.13 MBytes [ 5] 3.00-4.00 sec 226 MBytes 1.90 Gbits/sec 0 1.26 MBytes [ 5] 4.00-5.00 sec 214 MBytes 1.79 Gbits/sec 0 1.38 MBytes [ 5] 5.00-6.00 sec 276 MBytes 2.32 Gbits/sec 0 1.51 MBytes [ 5] 6.00-7.00 sec 289 MBytes 2.42 Gbits/sec 0 1.54 MBytes [ 5] 7.00-8.00 sec 315 MBytes 2.64 Gbits/sec 0 1.54 MBytes [ 5] 8.00-9.00 sec 326 MBytes 2.74 Gbits/sec 0 1.54 MBytes [ 5] 9.00-10.00 sec 312 MBytes 2.62 Gbits/sec 0 1.54 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.58 GBytes 2.22 Gbits/sec 7 sender [ 5] 0.00-10.00 sec 2.58 GBytes 2.22 Gbits/sec receiver iperf Done.
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
09/16/2026-23:47:59.592259 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43900 -> 40.0.0.2:5001 09/16/2026-23:47:59.593360 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43912 -> 40.0.0.2:5001 09/16/2026-23:48:15.539819 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51346 -> 40.0.0.2:5001 09/16/2026-23:48:15.540771 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51352 -> 40.0.0.2:5001
Step 14: Run the command system journal show | cat on DUT0 and check whether the output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Sep 16 23:47:50.427941 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Sep 16 23:47:50.000238 osdx systemd-timedated[517692]: Changed local time to Wed 2026-09-16 23:47:50 UTC Sep 16 23:47:50.001700 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'set date 2026-09-16 23:47:50'. Sep 16 23:47:50.002932 osdx systemd-journald[303514]: Time jumped backwards, rotating. Sep 16 23:47:50.334499 osdx systemd-journald[303514]: Runtime Journal (/run/log/journal/7ab9d0c6c88d486ab30f6cca192546c5) is 3.4M, max 13.8M, 10.3M free. Sep 16 23:47:50.334923 osdx systemd-journald[303514]: Received client request to rotate journal, rotating. Sep 16 23:47:50.334965 osdx systemd-journald[303514]: Vacuuming done, freed 0B of archived journals from /run/log/journal/7ab9d0c6c88d486ab30f6cca192546c5. Sep 16 23:47:50.344999 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'system journal clear'. Sep 16 23:47:50.572933 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'system coredump delete all'. Sep 16 23:47:50.778551 osdx OSDxCLI[517175]: User 'admin' entered the configuration menu. Sep 16 23:47:50.899665 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Sep 16 23:47:50.953937 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Sep 16 23:47:51.064523 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Sep 16 23:47:51.132334 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Sep 16 23:47:51.229113 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Sep 16 23:47:51.291061 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Sep 16 23:47:51.397865 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'show working'. Sep 16 23:47:51.459718 osdx ubnt-cfgd[517725]: inactive Sep 16 23:47:51.496905 osdx INFO[517738]: FRR daemons did not change Sep 16 23:47:51.526955 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Sep 16 23:47:51.546416 osdx WARNING[517780]: No supported link modes on interface eth1 Sep 16 23:47:51.547736 osdx modulelauncher[517780]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Sep 16 23:47:51.547748 osdx modulelauncher[517780]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Sep 16 23:47:51.548848 osdx modulelauncher[517780]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off -- Sep 16 23:47:51.548855 osdx modulelauncher[517780]: Command '/sbin/ethtool -s eth1 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Sep 16 23:47:51.578933 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Sep 16 23:47:51.625472 osdx WARNING[517855]: No supported link modes on interface eth0 Sep 16 23:47:51.626799 osdx modulelauncher[517855]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Sep 16 23:47:51.626815 osdx modulelauncher[517855]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Sep 16 23:47:51.628328 osdx modulelauncher[517855]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off -- Sep 16 23:47:51.628338 osdx modulelauncher[517855]: Command '/sbin/ethtool -s eth0 autoneg on advertise Pause off Asym_Pause off --' returned non-zero exit status 75. Sep 16 23:47:51.636360 osdx (udev-worker)[517869]: Network interface NamePolicy= disabled on kernel command line. Sep 16 23:47:51.976323 osdx cfgd[1899]: [517175]Completed change to active configuration Sep 16 23:47:51.976881 osdx OSDxCLI[517175]: User 'admin' committed the configuration. Sep 16 23:47:51.997745 osdx OSDxCLI[517175]: User 'admin' left the configuration menu. Sep 16 23:47:54.677524 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Sep 16 23:47:54.769705 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:54.851048 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:55.512254 osdx file_operation[518066]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Sep 16 23:47:55.539612 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Sep 16 23:47:55.710420 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Sep 16 23:47:55.862023 osdx OSDxCLI[517175]: User 'admin' entered the configuration menu. Sep 16 23:47:55.928610 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Sep 16 23:47:56.040223 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Sep 16 23:47:56.102745 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Sep 16 23:47:56.206193 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Sep 16 23:47:56.267629 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Sep 16 23:47:56.406192 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Sep 16 23:47:56.533940 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Sep 16 23:47:56.617163 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Sep 16 23:47:56.757965 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Sep 16 23:47:56.833954 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Sep 16 23:47:56.941203 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Sep 16 23:47:57.012113 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Sep 16 23:47:57.151841 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Sep 16 23:47:57.231229 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Sep 16 23:47:57.327206 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Sep 16 23:47:57.396231 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Sep 16 23:47:57.482343 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Sep 16 23:47:57.623830 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Sep 16 23:47:57.679597 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Sep 16 23:47:57.780663 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Sep 16 23:47:57.887010 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'show working'. Sep 16 23:47:58.026214 osdx ubnt-cfgd[518119]: inactive Sep 16 23:47:58.145842 osdx INFO[518167]: FRR daemons did not change Sep 16 23:47:58.441628 osdx systemd[1]: Reloading. Sep 16 23:47:58.506949 osdx systemd-sysv-generator[518221]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Sep 16 23:47:58.663350 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Sep 16 23:47:58.686244 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Sep 16 23:47:58.971737 osdx INFO[518200]: Rules successfully loaded Sep 16 23:47:58.979294 osdx cfgd[1899]: [517175]Completed change to active configuration Sep 16 23:47:58.979889 osdx OSDxCLI[517175]: User 'admin' committed the configuration. Sep 16 23:47:58.995071 osdx OSDxCLI[517175]: User 'admin' left the configuration menu. Sep 16 23:47:59.157127 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:59.240115 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Sep 16 23:47:59.594953 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=8820 DF PROTO=TCP SPT=43900 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Sep 16 23:47:59.595042 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=34405 DF PROTO=TCP SPT=43912 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Sep 16 23:48:08.891491 osdx systemd[1]: Starting logrotate.service - Rotate log files... Sep 16 23:48:08.934775 osdx systemd[1]: logrotate.service: Deactivated successfully. Sep 16 23:48:08.934941 osdx systemd[1]: Finished logrotate.service - Rotate log files. Sep 16 23:48:10.287410 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Sep 16 23:48:10.431908 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'system journal show | cat'. Sep 16 23:48:10.588861 osdx OSDxCLI[517175]: User 'admin' entered the configuration menu. Sep 16 23:48:10.645513 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Sep 16 23:48:10.739287 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Sep 16 23:48:10.789811 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Sep 16 23:48:10.888308 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Sep 16 23:48:10.940349 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Sep 16 23:48:11.036199 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Sep 16 23:48:11.134341 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Sep 16 23:48:11.201744 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Sep 16 23:48:11.293321 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Sep 16 23:48:11.368967 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Sep 16 23:48:11.472258 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Sep 16 23:48:11.541911 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Sep 16 23:48:11.638379 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Sep 16 23:48:11.710806 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Sep 16 23:48:11.817533 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Sep 16 23:48:11.898612 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Sep 16 23:48:12.015880 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Sep 16 23:48:12.088494 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Sep 16 23:48:12.226671 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Sep 16 23:48:12.290026 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Sep 16 23:48:12.395355 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Sep 16 23:48:12.459403 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Sep 16 23:48:12.564512 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Sep 16 23:48:12.621484 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Sep 16 23:48:12.738095 osdx OSDxCLI[517175]: User 'admin' added a new cfg line: 'show changes'. Sep 16 23:48:12.818128 osdx ubnt-cfgd[518344]: inactive Sep 16 23:48:12.882138 osdx INFO[518367]: FRR daemons did not change Sep 16 23:48:13.184216 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Sep 16 23:48:15.019712 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Sep 16 23:48:15.019845 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Sep 16 23:48:15.019881 osdx systemd[1]: suricata@FW.service: Consumed 1.889s CPU time. Sep 16 23:48:15.051378 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Sep 16 23:48:15.075298 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Sep 16 23:48:15.281622 osdx INFO[518395]: Rules successfully loaded Sep 16 23:48:15.290468 osdx cfgd[1899]: [517175]Completed change to active configuration Sep 16 23:48:15.290959 osdx OSDxCLI[517175]: User 'admin' committed the configuration. Sep 16 23:48:15.307124 osdx OSDxCLI[517175]: User 'admin' left the configuration menu. Sep 16 23:48:15.543025 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=63915 DF PROTO=TCP SPT=51346 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Sep 16 23:48:15.543112 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=19531 DF PROTO=TCP SPT=51352 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Sep 16 23:48:20.041481 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Sep 16 23:48:25.696058 osdx OSDxCLI[517175]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 85806 0 --:--:-- --:--:-- --:--:-- 88666
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.415 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.415/0.415/0.415/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.421 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.421/0.421/0.421/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.553 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.553/0.553/0.553/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.492 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.492/0.492/0.492/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 59622 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 292 MBytes 2.45 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 221 MBytes 1.86 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 216 MBytes 1.81 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 230 MBytes 1.93 Gbits/sec 92 1.23 MBytes [ 5] 4.00-5.00 sec 240 MBytes 2.01 Gbits/sec 0 1.36 MBytes [ 5] 5.00-6.00 sec 260 MBytes 2.18 Gbits/sec 0 1.49 MBytes [ 5] 6.00-7.00 sec 221 MBytes 1.86 Gbits/sec 0 1.54 MBytes [ 5] 7.00-8.00 sec 156 MBytes 1.31 Gbits/sec 0 1.54 MBytes [ 5] 8.00-9.00 sec 228 MBytes 1.91 Gbits/sec 0 1.54 MBytes [ 5] 9.00-10.00 sec 230 MBytes 1.93 Gbits/sec 0 1.54 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.24 GBytes 1.93 Gbits/sec 92 sender [ 5] 0.00-10.00 sec 2.24 GBytes 1.92 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
09/16/2026-23:48:54.794556 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59606 -> 40.0.0.2:5001 09/16/2026-23:48:54.795497 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59622 -> 40.0.0.2:5001
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=510 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=510 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=511 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=511 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=59622 dport=5001 packets=1661618 bytes=2491848229 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59622 packets=208148 bytes=10820296 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=154 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=154 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=59606 dport=5001 packets=14 bytes=1195 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59606 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=1043 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=1043 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 11: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50758 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 263 MBytes 2.21 Gbits/sec 0 1.62 MBytes [ 5] 1.00-2.00 sec 261 MBytes 2.19 Gbits/sec 0 1.62 MBytes [ 5] 2.00-3.00 sec 222 MBytes 1.87 Gbits/sec 0 1.62 MBytes [ 5] 3.00-4.00 sec 249 MBytes 2.09 Gbits/sec 0 1.62 MBytes [ 5] 4.00-5.00 sec 250 MBytes 2.10 Gbits/sec 0 1.62 MBytes [ 5] 5.00-6.00 sec 241 MBytes 2.02 Gbits/sec 0 1.62 MBytes [ 5] 6.00-7.00 sec 134 MBytes 1.12 Gbits/sec 1 1.41 KBytes [ 5] 7.00-8.00 sec 124 MBytes 1.04 Gbits/sec 64 1.18 MBytes [ 5] 8.00-9.00 sec 304 MBytes 2.55 Gbits/sec 0 1.36 MBytes [ 5] 9.00-10.00 sec 299 MBytes 2.51 Gbits/sec 0 1.51 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.29 GBytes 1.97 Gbits/sec 65 sender [ 5] 0.00-10.00 sec 2.29 GBytes 1.97 Gbits/sec receiver iperf Done.
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
09/16/2026-23:48:54.794556 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59606 -> 40.0.0.2:5001 09/16/2026-23:48:54.795497 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59622 -> 40.0.0.2:5001 09/16/2026-23:49:09.617498 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50746 -> 40.0.0.2:5001 09/16/2026-23:49:09.618532 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50758 -> 40.0.0.2:5001
Step 14: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=50758 dport=5001 packets=1699907 bytes=2549366129 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50758 packets=281207 bytes=14613268 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=50746 dport=5001 packets=16 bytes=1301 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50746 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 4495 0 --:--:-- --:--:-- --:--:-- 4545
Step 2: Run the command file show running://drop-performance.rules on DUT0 and expect the following output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.823 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.823/0.823/0.823/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.825 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.825/0.825/0.825/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.632 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.632/0.632/0.632/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.785 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.785/0.785/0.785/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
09/16/2026-23:49:44.567368 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43254 -> 40.0.0.2:5000
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=1046 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=1046 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=516 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=516 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=515 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=515 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=157 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=157 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=43254 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=43254 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 5 flow entries have been shown.
Step 11: Run the command traffic policy FW_PLAN show on DUT0 and check whether the output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 8 210 522 2 - 4 4 312 312 ------------------------------------------------------------------ Total 8 8 522 522 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 11 445 898 2 - 6 6 453 453 ------------------------------------------------------------------ Total 11 11 898 898
Note
Testing with another conntrack mark.
Step 12: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 13: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 14: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
09/16/2026-23:49:44.567368 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43254 -> 40.0.0.2:5000 09/16/2026-23:49:50.708264 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43254 -> 40.0.0.2:5000 09/16/2026-23:49:52.253947 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43260 -> 40.0.0.2:5000
Step 15: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=43260 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=43260 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 16: Run the command traffic policy FW_PLAN show on DUT0 and check whether the output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 445 775 2 - 5 5 330 330 --------------------------------------------------------------------- Total 10 10 775 775
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 43237 0 --:--:-- --:--:-- --:--:-- 44333
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.512 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.512/0.512/0.512/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.651 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.651/0.651/0.651/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.936 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.936/0.936/0.936/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.747 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.747/0.747/0.747/0.000 ms
Step 8: Initiate a background bandwidth test from DUT2 to DUT1. Control is returned, allowing other tasks to be performed while the test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
09/16/2026-23:50:18.791450 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42286 -> 40.0.0.2:5001 09/16/2026-23:50:18.792510 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42292 -> 40.0.0.2:5001
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=520 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=520 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=42286 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42286 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=42292 dport=5001 packets=19180 bytes=28762805 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42292 packets=2387 bytes=124312 [ASSURED] [OFFLOAD, packets=19167 bytes=28747604 packets=2385 bytes=124200] mark=129834765 use=2 icmp 1 29 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=521 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=521 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=160 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=160 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=1049 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=1049 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Step 11: Stop the current bandwidth test between DUT2 and DUT1.
Step 12: Initiate a background bandwidth test from DUT2 to DUT1. Control is returned, allowing other tasks to be performed while the test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
09/16/2026-23:50:18.791450 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42286 -> 40.0.0.2:5001 09/16/2026-23:50:18.792510 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42292 -> 40.0.0.2:5001 09/16/2026-23:50:19.416408 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42298 -> 40.0.0.2:5001 09/16/2026-23:50:19.417607 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:38403 -> 40.0.0.2:5001 09/16/2026-23:50:19.428785 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:38403 -> 40.0.0.2:5001 09/16/2026-23:50:19.442169 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:38403 -> 40.0.0.2:5001
Step 14: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 28 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=520 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=520 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=42286 dport=5001 packets=12 bytes=798 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42286 packets=9 bytes=492 [ASSURED] [OFFLOAD, packets=3 bytes=157 packets=4 bytes=211] mark=129834765 use=4 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=38403 dport=5001 packets=17 bytes=23648 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=38403 packets=1 bytes=32 [OFFLOAD, packets=13 bytes=19188 packets=0 bytes=0] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=42298 dport=5001 packets=7 bytes=555 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42298 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 icmp 1 28 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=521 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=521 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=160 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=160 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 28 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=1049 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=1049 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 7 flow entries have been shown.
Step 15: Stop the current bandwidth test between DUT2 and DUT1.
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 70846 0 --:--:-- --:--:-- --:--:-- 97k
Step 2: Run the command file show running://drop-performance.rules on DUT0 and expect the following output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.485 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.485/0.485/0.485/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.767 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.767/0.767/0.767/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.06 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.056/1.056/1.056/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.645 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.645/0.645/0.645/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
09/16/2026-23:50:43.388443 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43660 -> 40.0.0.2:5000
Step 10: Run the command service firewall FW show early-drop-stats eth1 on DUT0 and check whether the output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 43660 yes 201 0 0 0 20.0.0.2 40.0.0.2 43660 5000 yes 201 0 7 602
Step 11: Run the command interfaces ethernet eth1 monitor xdp-stats times 1 on DUT0 and expect the following output:
Show output
Period of 0.250127s ending at 1789602647.040686 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 15 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
09/16/2026-23:50:43.388443 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43660 -> 40.0.0.2:5000 09/16/2026-23:50:47.216444 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:32887 -> 40.0.0.2:5001
Step 14: Run the command service firewall FW show early-drop-stats eth1 on DUT0 and check whether the output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5001 32887 no 201 0 0 0 20.0.0.2 40.0.0.2 32887 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 43660 yes 201 0 0 0 20.0.0.2 40.0.0.2 43660 5000 yes 201 0 11 834
Step 15: Run the command interfaces ethernet eth1 monitor xdp-stats times 1 on DUT0 and expect the following output:
Show output
Period of 0.250125s ending at 1789602650.805807 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 35 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)