Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 20982 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.533 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.533/0.533/0.533/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.421 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.421/0.421/0.421/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57514 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 147 MBytes 1.23 Gbits/sec 27 1.81 MBytes [ 5] 1.00-2.00 sec 152 MBytes 1.28 Gbits/sec 0 1.97 MBytes [ 5] 2.00-3.00 sec 152 MBytes 1.28 Gbits/sec 0 2.10 MBytes [ 5] 3.00-4.00 sec 148 MBytes 1.24 Gbits/sec 38 1.57 MBytes [ 5] 4.00-5.00 sec 142 MBytes 1.20 Gbits/sec 0 1.65 MBytes [ 5] 5.00-6.00 sec 146 MBytes 1.23 Gbits/sec 0 1.71 MBytes [ 5] 6.00-7.00 sec 148 MBytes 1.24 Gbits/sec 0 1.75 MBytes [ 5] 7.00-8.00 sec 156 MBytes 1.31 Gbits/sec 0 1.80 MBytes [ 5] 8.00-9.00 sec 148 MBytes 1.24 Gbits/sec 0 1.85 MBytes [ 5] 9.00-10.00 sec 159 MBytes 1.33 Gbits/sec 0 1.92 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.46 GBytes 1.26 Gbits/sec 65 sender [ 5] 0.00-10.01 sec 1.46 GBytes 1.25 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/06/2026-16:08:57.869550 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57500 -> 40.0.0.2:5001 04/06/2026-16:08:57.870422 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57514 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 21717 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.546 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.546/0.546/0.546/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.412 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.412/0.412/0.412/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 43606 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 320 MBytes 2.68 Gbits/sec 5 1.92 MBytes [ 5] 1.00-2.00 sec 268 MBytes 2.24 Gbits/sec 37 1.46 MBytes [ 5] 2.00-3.00 sec 301 MBytes 2.53 Gbits/sec 0 1.60 MBytes [ 5] 3.00-4.00 sec 294 MBytes 2.46 Gbits/sec 182 1.25 MBytes [ 5] 4.00-5.00 sec 311 MBytes 2.61 Gbits/sec 0 1.42 MBytes [ 5] 5.00-6.00 sec 298 MBytes 2.50 Gbits/sec 0 1.57 MBytes [ 5] 6.00-7.00 sec 310 MBytes 2.60 Gbits/sec 0 1.70 MBytes [ 5] 7.00-8.00 sec 281 MBytes 2.36 Gbits/sec 83 1.27 MBytes [ 5] 8.00-9.00 sec 298 MBytes 2.50 Gbits/sec 0 1.43 MBytes [ 5] 9.00-10.00 sec 321 MBytes 2.69 Gbits/sec 0 1.59 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.93 GBytes 2.52 Gbits/sec 307 sender [ 5] 0.00-10.00 sec 2.93 GBytes 2.52 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/06/2026-16:09:32.763548 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43604 -> 40.0.0.2:5001 04/06/2026-16:09:32.764456 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43606 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Apr 06 16:09:25.334035 osdx systemd-journald[1969]: Runtime Journal (/run/log/journal/8555dfc266884f30a3544832475c4d6c) is 1.7M, max 13.8M, 12.0M free. Apr 06 16:09:25.334485 osdx systemd-journald[1969]: Received client request to rotate journal, rotating. Apr 06 16:09:25.334520 osdx systemd-journald[1969]: Vacuuming done, freed 0B of archived journals from /run/log/journal/8555dfc266884f30a3544832475c4d6c. Apr 06 16:09:25.344771 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'system journal clear'. Apr 06 16:09:25.572501 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'system coredump delete all'. Apr 06 16:09:25.827001 osdx OSDxCLI[321632]: User 'admin' entered the configuration menu. Apr 06 16:09:25.905677 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Apr 06 16:09:25.983966 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Apr 06 16:09:26.043768 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Apr 06 16:09:26.138177 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Apr 06 16:09:26.200920 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Apr 06 16:09:26.290936 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Apr 06 16:09:26.355494 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'show working'. Apr 06 16:09:26.447807 osdx ubnt-cfgd[322097]: inactive Apr 06 16:09:26.474442 osdx INFO[322103]: FRR daemons did not change Apr 06 16:09:26.538447 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Apr 06 16:09:26.558944 osdx WARNING[322142]: No supported link modes on interface eth1 Apr 06 16:09:26.560239 osdx modulelauncher[322142]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Apr 06 16:09:26.560250 osdx modulelauncher[322142]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Apr 06 16:09:26.561405 osdx modulelauncher[322142]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Apr 06 16:09:26.561415 osdx modulelauncher[322142]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Apr 06 16:09:26.590446 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Apr 06 16:09:26.632513 osdx WARNING[322221]: No supported link modes on interface eth0 Apr 06 16:09:26.633952 osdx modulelauncher[322221]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Apr 06 16:09:26.633962 osdx modulelauncher[322221]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Apr 06 16:09:26.635125 osdx modulelauncher[322221]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Apr 06 16:09:26.635132 osdx modulelauncher[322221]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Apr 06 16:09:26.644626 osdx (udev-worker)[322237]: Network interface NamePolicy= disabled on kernel command line. Apr 06 16:09:26.797589 osdx cfgd[1666]: [321632]Completed change to active configuration Apr 06 16:09:26.810042 osdx OSDxCLI[321632]: User 'admin' committed the configuration. Apr 06 16:09:26.826342 osdx OSDxCLI[321632]: User 'admin' left the configuration menu. Apr 06 16:09:28.853901 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Apr 06 16:09:28.932730 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Apr 06 16:09:29.034797 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Apr 06 16:09:29.517028 osdx file_operation[322400]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Apr 06 16:09:29.544743 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Apr 06 16:09:29.690720 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Apr 06 16:09:29.847231 osdx OSDxCLI[321632]: User 'admin' entered the configuration menu. Apr 06 16:09:29.907723 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 06 16:09:30.002210 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 06 16:09:30.082807 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 06 16:09:30.173499 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 06 16:09:30.238838 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 06 16:09:30.341963 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 06 16:09:30.430605 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 06 16:09:30.500838 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 06 16:09:30.605065 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 06 16:09:30.657254 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 06 16:09:30.749103 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 06 16:09:30.814156 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 06 16:09:30.897795 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 06 16:09:30.952093 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 06 16:09:31.048290 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 06 16:09:31.109192 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 06 16:09:31.194796 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 06 16:09:31.250144 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 06 16:09:31.343085 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 06 16:09:31.395877 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 06 16:09:31.519359 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'show working'. Apr 06 16:09:31.597361 osdx ubnt-cfgd[322447]: inactive Apr 06 16:09:31.671080 osdx INFO[322492]: FRR daemons did not change Apr 06 16:09:31.927307 osdx systemd[1]: Reloading. Apr 06 16:09:31.974450 osdx systemd-sysv-generator[322548]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Apr 06 16:09:32.094768 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 06 16:09:32.115306 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 06 16:09:32.333842 osdx INFO[322529]: Rules successfully loaded Apr 06 16:09:32.334544 osdx cfgd[1666]: [321632]Completed change to active configuration Apr 06 16:09:32.337087 osdx OSDxCLI[321632]: User 'admin' committed the configuration. Apr 06 16:09:32.396619 osdx OSDxCLI[321632]: User 'admin' left the configuration menu. Apr 06 16:09:32.766471 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=9877 DF PROTO=TCP SPT=43604 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 06 16:09:32.766539 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=9587 DF PROTO=TCP SPT=43606 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 06 16:09:42.952194 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33354 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 338 MBytes 2.84 Gbits/sec 13 1.92 MBytes [ 5] 1.00-2.00 sec 330 MBytes 2.77 Gbits/sec 48 1.50 MBytes [ 5] 2.00-3.00 sec 345 MBytes 2.89 Gbits/sec 0 1.66 MBytes [ 5] 3.00-4.00 sec 355 MBytes 2.98 Gbits/sec 0 1.81 MBytes [ 5] 4.00-5.00 sec 332 MBytes 2.79 Gbits/sec 0 1.94 MBytes [ 5] 5.00-6.00 sec 340 MBytes 2.85 Gbits/sec 141 1.52 MBytes [ 5] 6.00-7.00 sec 354 MBytes 2.97 Gbits/sec 0 1.68 MBytes [ 5] 7.00-8.00 sec 350 MBytes 2.94 Gbits/sec 0 1.83 MBytes [ 5] 8.00-9.00 sec 339 MBytes 2.84 Gbits/sec 0 1.96 MBytes [ 5] 9.00-10.00 sec 322 MBytes 2.70 Gbits/sec 396 1.51 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.33 GBytes 2.86 Gbits/sec 598 sender [ 5] 0.00-10.00 sec 3.32 GBytes 2.85 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/06/2026-16:09:32.763548 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43604 -> 40.0.0.2:5001 04/06/2026-16:09:32.764456 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43606 -> 40.0.0.2:5001 04/06/2026-16:09:48.102008 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33344 -> 40.0.0.2:5001 04/06/2026-16:09:48.103223 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33354 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Apr 06 16:09:25.334035 osdx systemd-journald[1969]: Runtime Journal (/run/log/journal/8555dfc266884f30a3544832475c4d6c) is 1.7M, max 13.8M, 12.0M free. Apr 06 16:09:25.334485 osdx systemd-journald[1969]: Received client request to rotate journal, rotating. Apr 06 16:09:25.334520 osdx systemd-journald[1969]: Vacuuming done, freed 0B of archived journals from /run/log/journal/8555dfc266884f30a3544832475c4d6c. Apr 06 16:09:25.344771 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'system journal clear'. Apr 06 16:09:25.572501 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'system coredump delete all'. Apr 06 16:09:25.827001 osdx OSDxCLI[321632]: User 'admin' entered the configuration menu. Apr 06 16:09:25.905677 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Apr 06 16:09:25.983966 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Apr 06 16:09:26.043768 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Apr 06 16:09:26.138177 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Apr 06 16:09:26.200920 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Apr 06 16:09:26.290936 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Apr 06 16:09:26.355494 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'show working'. Apr 06 16:09:26.447807 osdx ubnt-cfgd[322097]: inactive Apr 06 16:09:26.474442 osdx INFO[322103]: FRR daemons did not change Apr 06 16:09:26.538447 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Apr 06 16:09:26.558944 osdx WARNING[322142]: No supported link modes on interface eth1 Apr 06 16:09:26.560239 osdx modulelauncher[322142]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Apr 06 16:09:26.560250 osdx modulelauncher[322142]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Apr 06 16:09:26.561405 osdx modulelauncher[322142]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Apr 06 16:09:26.561415 osdx modulelauncher[322142]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Apr 06 16:09:26.590446 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Apr 06 16:09:26.632513 osdx WARNING[322221]: No supported link modes on interface eth0 Apr 06 16:09:26.633952 osdx modulelauncher[322221]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Apr 06 16:09:26.633962 osdx modulelauncher[322221]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Apr 06 16:09:26.635125 osdx modulelauncher[322221]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Apr 06 16:09:26.635132 osdx modulelauncher[322221]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Apr 06 16:09:26.644626 osdx (udev-worker)[322237]: Network interface NamePolicy= disabled on kernel command line. Apr 06 16:09:26.797589 osdx cfgd[1666]: [321632]Completed change to active configuration Apr 06 16:09:26.810042 osdx OSDxCLI[321632]: User 'admin' committed the configuration. Apr 06 16:09:26.826342 osdx OSDxCLI[321632]: User 'admin' left the configuration menu. Apr 06 16:09:28.853901 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Apr 06 16:09:28.932730 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Apr 06 16:09:29.034797 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Apr 06 16:09:29.517028 osdx file_operation[322400]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Apr 06 16:09:29.544743 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Apr 06 16:09:29.690720 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Apr 06 16:09:29.847231 osdx OSDxCLI[321632]: User 'admin' entered the configuration menu. Apr 06 16:09:29.907723 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 06 16:09:30.002210 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 06 16:09:30.082807 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 06 16:09:30.173499 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 06 16:09:30.238838 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 06 16:09:30.341963 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 06 16:09:30.430605 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 06 16:09:30.500838 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 06 16:09:30.605065 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 06 16:09:30.657254 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 06 16:09:30.749103 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 06 16:09:30.814156 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 06 16:09:30.897795 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 06 16:09:30.952093 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 06 16:09:31.048290 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 06 16:09:31.109192 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 06 16:09:31.194796 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 06 16:09:31.250144 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 06 16:09:31.343085 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 06 16:09:31.395877 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 06 16:09:31.519359 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'show working'. Apr 06 16:09:31.597361 osdx ubnt-cfgd[322447]: inactive Apr 06 16:09:31.671080 osdx INFO[322492]: FRR daemons did not change Apr 06 16:09:31.927307 osdx systemd[1]: Reloading. Apr 06 16:09:31.974450 osdx systemd-sysv-generator[322548]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Apr 06 16:09:32.094768 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 06 16:09:32.115306 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 06 16:09:32.333842 osdx INFO[322529]: Rules successfully loaded Apr 06 16:09:32.334544 osdx cfgd[1666]: [321632]Completed change to active configuration Apr 06 16:09:32.337087 osdx OSDxCLI[321632]: User 'admin' committed the configuration. Apr 06 16:09:32.396619 osdx OSDxCLI[321632]: User 'admin' left the configuration menu. Apr 06 16:09:32.766471 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=9877 DF PROTO=TCP SPT=43604 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 06 16:09:32.766539 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=9587 DF PROTO=TCP SPT=43606 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 06 16:09:42.952194 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Apr 06 16:09:43.056621 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'system journal show | cat'. Apr 06 16:09:43.237154 osdx OSDxCLI[321632]: User 'admin' entered the configuration menu. Apr 06 16:09:43.303766 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 06 16:09:43.400999 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 06 16:09:43.451532 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 06 16:09:43.546541 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 06 16:09:43.617625 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 06 16:09:43.858838 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 06 16:09:43.950302 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 06 16:09:44.033801 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 06 16:09:44.137015 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 06 16:09:44.197341 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 06 16:09:44.287575 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 06 16:09:44.343041 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 06 16:09:44.435887 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 06 16:09:44.491229 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 06 16:09:44.578572 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 06 16:09:44.644325 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 06 16:09:44.758828 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 06 16:09:44.916839 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 06 16:09:45.002629 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 06 16:09:45.097033 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 06 16:09:45.166935 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Apr 06 16:09:45.268466 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Apr 06 16:09:45.331497 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Apr 06 16:09:45.432180 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Apr 06 16:09:45.497239 osdx OSDxCLI[321632]: User 'admin' added a new cfg line: 'show changes'. Apr 06 16:09:45.595933 osdx ubnt-cfgd[322674]: inactive Apr 06 16:09:45.641572 osdx INFO[322694]: FRR daemons did not change Apr 06 16:09:45.868397 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Apr 06 16:09:47.099782 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Apr 06 16:09:47.099895 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Apr 06 16:09:47.099925 osdx systemd[1]: suricata@FW.service: Consumed 1.288s CPU time. Apr 06 16:09:47.118715 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 06 16:09:47.134756 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 06 16:09:47.311229 osdx INFO[322721]: Rules successfully loaded Apr 06 16:09:47.311747 osdx cfgd[1666]: [321632]Completed change to active configuration Apr 06 16:09:47.851409 osdx OSDxCLI[321632]: User 'admin' committed the configuration. Apr 06 16:09:47.866500 osdx OSDxCLI[321632]: User 'admin' left the configuration menu. Apr 06 16:09:48.102451 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=13537 DF PROTO=TCP SPT=33344 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Apr 06 16:09:48.106448 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=9585 DF PROTO=TCP SPT=33354 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Apr 06 16:09:55.030970 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Apr 06 16:09:59.127773 osdx OSDxCLI[321632]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 4512 0 --:--:-- --:--:-- --:--:-- 4607
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.568 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.568/0.568/0.568/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.538 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.538/0.538/0.538/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 39358 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 291 MBytes 2.44 Gbits/sec 47 1.86 MBytes [ 5] 1.00-2.00 sec 331 MBytes 2.78 Gbits/sec 0 2.02 MBytes [ 5] 2.00-3.00 sec 328 MBytes 2.75 Gbits/sec 0 2.14 MBytes [ 5] 3.00-4.00 sec 346 MBytes 2.90 Gbits/sec 0 2.24 MBytes [ 5] 4.00-5.00 sec 306 MBytes 2.57 Gbits/sec 10 1.69 MBytes [ 5] 5.00-6.00 sec 340 MBytes 2.85 Gbits/sec 0 1.83 MBytes [ 5] 6.00-7.00 sec 345 MBytes 2.89 Gbits/sec 0 1.97 MBytes [ 5] 7.00-8.00 sec 349 MBytes 2.93 Gbits/sec 78 1.49 MBytes [ 5] 8.00-9.00 sec 342 MBytes 2.87 Gbits/sec 0 1.64 MBytes [ 5] 9.00-10.00 sec 361 MBytes 3.02 Gbits/sec 0 1.80 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.26 GBytes 2.80 Gbits/sec 135 sender [ 5] 0.00-10.00 sec 3.26 GBytes 2.80 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/06/2026-16:10:20.400148 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39350 -> 40.0.0.2:5001 04/06/2026-16:10:20.401059 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39358 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=57 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=57 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=39358 dport=5001 packets=2418301 bytes=3627438481 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=39358 packets=386936 bytes=20112628 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=39350 dport=5001 packets=16 bytes=1301 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=39350 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=202 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=202 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50144 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 357 MBytes 3.00 Gbits/sec 36 1.92 MBytes [ 5] 1.00-2.00 sec 359 MBytes 3.01 Gbits/sec 44 1.49 MBytes [ 5] 2.00-3.00 sec 372 MBytes 3.12 Gbits/sec 102 1.18 MBytes [ 5] 3.00-4.00 sec 374 MBytes 3.14 Gbits/sec 0 1.39 MBytes [ 5] 4.00-5.00 sec 370 MBytes 3.10 Gbits/sec 0 1.57 MBytes [ 5] 5.00-6.00 sec 350 MBytes 2.94 Gbits/sec 0 1.73 MBytes [ 5] 6.00-7.00 sec 384 MBytes 3.22 Gbits/sec 0 1.89 MBytes [ 5] 7.00-8.00 sec 385 MBytes 3.23 Gbits/sec 0 2.03 MBytes [ 5] 8.00-9.00 sec 352 MBytes 2.96 Gbits/sec 0 2.16 MBytes [ 5] 9.00-10.00 sec 348 MBytes 2.92 Gbits/sec 102 1.23 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.57 GBytes 3.06 Gbits/sec 284 sender [ 5] 0.00-10.00 sec 3.56 GBytes 3.06 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/06/2026-16:10:20.400148 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39350 -> 40.0.0.2:5001 04/06/2026-16:10:20.401059 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39358 -> 40.0.0.2:5001 04/06/2026-16:10:35.183400 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50134 -> 40.0.0.2:5001 04/06/2026-16:10:35.184231 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50144 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=50134 dport=5001 packets=16 bytes=1302 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50134 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=50144 dport=5001 packets=2643497 bytes=3965232433 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50144 packets=454025 bytes=23606020 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 8596 0 --:--:-- --:--:-- --:--:-- 8695
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.518 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.518/0.518/0.518/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.482 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.482/0.482/0.482/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/06/2026-16:11:08.488834 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57534 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=205 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=205 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=60 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=60 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=57534 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=57534 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/06/2026-16:11:08.488834 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57534 -> 40.0.0.2:5000 04/06/2026-16:11:14.793919 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57534 -> 40.0.0.2:5000 04/06/2026-16:11:16.278598 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37432 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=37432 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=37432 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 409 740 2 - 5 5 331 331 --------------------------------------------------------------------- Total 10 10 740 740
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 25976 0 --:--:-- --:--:-- --:--:-- 32250
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.570 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.570/0.570/0.570/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.435 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.435/0.435/0.435/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/06/2026-16:11:39.643039 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53522 -> 40.0.0.2:5001 04/06/2026-16:11:39.644151 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53528 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53528 dport=5001 packets=63495 bytes=95233857 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53528 packets=8153 bytes=425100 [ASSURED] [OFFLOAD, packets=63482 bytes=95218656 packets=8151 bytes=424988] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=63 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=63 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53522 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53522 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=208 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=208 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 12942 0 --:--:-- --:--:-- --:--:-- 13300
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
04/06/2026-16:11:39.643039 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53522 -> 40.0.0.2:5001 04/06/2026-16:11:39.644151 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53528 -> 40.0.0.2:5001 04/06/2026-16:11:41.531681 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53542 -> 40.0.0.2:5001 04/06/2026-16:11:41.533519 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43796 -> 40.0.0.2:5001 04/06/2026-16:11:41.544544 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43796 -> 40.0.0.2:5001 04/06/2026-16:11:41.556496 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:43796 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53542 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53542 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=43796 dport=5001 packets=11 bytes=14792 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43796 packets=1 bytes=32 [OFFLOAD, packets=7 bytes=10332 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 46630 0 --:--:-- --:--:-- --:--:-- 50000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.575 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.575/0.575/0.575/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.462 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.462/0.462/0.462/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/06/2026-16:12:00.823526 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48894 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 48894 5000 yes 201 0 8 673 40.0.0.2 20.0.0.2 5000 48894 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250139s ending at 1775491924.502965 XDP_DROP 8 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 26847127 pkts ( 0 pps) 34360958 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
04/06/2026-16:12:00.823526 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48894 -> 40.0.0.2:5000 04/06/2026-16:12:04.654074 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:37011 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 37011 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5001 37011 no 201 0 0 0 20.0.0.2 40.0.0.2 48894 5000 yes 201 0 11 847 40.0.0.2 20.0.0.2 5000 48894 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250130s ending at 1775491928.278178 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 26847187 pkts ( 0 pps) 34360963 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)