Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 20146 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.596 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.596/0.596/0.596/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.516 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.516/0.516/0.516/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50130 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 91.5 MBytes 768 Mbits/sec 14 1.80 MBytes [ 5] 1.00-2.00 sec 96.2 MBytes 807 Mbits/sec 66 1.37 MBytes [ 5] 2.00-3.00 sec 88.8 MBytes 744 Mbits/sec 0 1.45 MBytes [ 5] 3.00-4.00 sec 90.0 MBytes 755 Mbits/sec 0 1.51 MBytes [ 5] 4.00-5.00 sec 91.2 MBytes 765 Mbits/sec 0 1.55 MBytes [ 5] 5.00-6.00 sec 96.2 MBytes 807 Mbits/sec 0 1.58 MBytes [ 5] 6.00-7.00 sec 71.2 MBytes 598 Mbits/sec 0 1.59 MBytes [ 5] 7.00-8.00 sec 66.2 MBytes 556 Mbits/sec 0 1.60 MBytes [ 5] 8.00-9.00 sec 53.8 MBytes 451 Mbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 46.2 MBytes 388 Mbits/sec 0 1.64 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 792 MBytes 664 Mbits/sec 80 sender [ 5] 0.00-10.02 sec 790 MBytes 662 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/02/2026-10:37:45.446197 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50122 -> 40.0.0.2:5001 06/02/2026-10:37:45.447273 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50130 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 54753 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.666 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.666/0.666/0.666/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.454 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.454/0.454/0.454/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33058 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 290 MBytes 2.44 Gbits/sec 36 1.35 MBytes [ 5] 1.00-2.00 sec 228 MBytes 1.91 Gbits/sec 0 1.47 MBytes [ 5] 2.00-3.00 sec 262 MBytes 2.20 Gbits/sec 0 1.59 MBytes [ 5] 3.00-4.00 sec 275 MBytes 2.31 Gbits/sec 0 1.70 MBytes [ 5] 4.00-5.00 sec 360 MBytes 3.02 Gbits/sec 0 1.84 MBytes [ 5] 5.00-6.00 sec 316 MBytes 2.65 Gbits/sec 0 1.95 MBytes [ 5] 6.00-7.00 sec 311 MBytes 2.61 Gbits/sec 57 1.52 MBytes [ 5] 7.00-8.00 sec 295 MBytes 2.47 Gbits/sec 0 1.64 MBytes [ 5] 8.00-9.00 sec 305 MBytes 2.56 Gbits/sec 0 1.77 MBytes [ 5] 9.00-10.00 sec 335 MBytes 2.81 Gbits/sec 139 1.40 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.91 GBytes 2.50 Gbits/sec 232 sender [ 5] 0.00-10.00 sec 2.91 GBytes 2.50 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/02/2026-10:38:17.948665 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33056 -> 40.0.0.2:5001 06/02/2026-10:38:17.949596 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33058 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jun 02 10:38:10.319984 osdx systemd-journald[148515]: Runtime Journal (/run/log/journal/a0363f0a73514b24b35c4ba7ae73dc6e) is 1.7M, max 13.8M, 12.0M free. Jun 02 10:38:10.321176 osdx systemd-journald[148515]: Received client request to rotate journal, rotating. Jun 02 10:38:10.321226 osdx systemd-journald[148515]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a0363f0a73514b24b35c4ba7ae73dc6e. Jun 02 10:38:10.324093 osdx sudo[488212]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 02 10:38:10.332728 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'system journal clear'. Jun 02 10:38:10.567972 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'system coredump delete all'. Jun 02 10:38:10.805587 osdx OSDxCLI[487793]: User 'admin' entered the configuration menu. Jun 02 10:38:10.891248 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jun 02 10:38:10.972849 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jun 02 10:38:11.041151 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jun 02 10:38:11.138475 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jun 02 10:38:11.206675 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jun 02 10:38:11.311255 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jun 02 10:38:11.386799 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'show working'. Jun 02 10:38:11.502311 osdx ubnt-cfgd[488243]: inactive Jun 02 10:38:11.535717 osdx INFO[488253]: FRR daemons did not change Jun 02 10:38:11.569187 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jun 02 10:38:11.649183 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jun 02 10:38:11.666803 osdx (udev-worker)[488377]: Network interface NamePolicy= disabled on kernel command line. Jun 02 10:38:11.707026 osdx (udev-worker)[488384]: Network interface NamePolicy= disabled on kernel command line. Jun 02 10:38:11.830826 osdx cfgd[1665]: [487793]Completed change to active configuration Jun 02 10:38:11.842316 osdx OSDxCLI[487793]: User 'admin' committed the configuration. Jun 02 10:38:11.859221 osdx OSDxCLI[487793]: User 'admin' left the configuration menu. Jun 02 10:38:13.931552 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jun 02 10:38:14.009909 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 02 10:38:14.088217 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 02 10:38:14.527726 osdx sudo[488541]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 02 10:38:14.588771 osdx file_operation[488544]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jun 02 10:38:14.607137 osdx sudo[488551]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 02 10:38:14.608915 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jun 02 10:38:14.740143 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jun 02 10:38:14.888167 osdx OSDxCLI[487793]: User 'admin' entered the configuration menu. Jun 02 10:38:14.954205 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 02 10:38:15.049449 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 02 10:38:15.107840 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 02 10:38:15.203662 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 02 10:38:15.261564 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 02 10:38:15.360203 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 02 10:38:15.450871 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 02 10:38:15.525173 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 02 10:38:15.609555 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 02 10:38:15.667067 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 02 10:38:15.754433 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 02 10:38:15.815034 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 02 10:38:15.914754 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 02 10:38:16.002216 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 02 10:38:16.086981 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 02 10:38:16.229967 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 02 10:38:16.286352 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 02 10:38:16.410953 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 02 10:38:16.495755 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 02 10:38:16.579042 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 02 10:38:16.662713 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'show working'. Jun 02 10:38:16.789424 osdx ubnt-cfgd[488599]: inactive Jun 02 10:38:16.865846 osdx INFO[488636]: FRR daemons did not change Jun 02 10:38:17.089939 osdx systemd[1]: Reloading. Jun 02 10:38:17.157185 osdx systemd-sysv-generator[488689]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jun 02 10:38:17.293472 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 02 10:38:17.325281 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 02 10:38:17.528545 osdx INFO[488671]: Rules successfully loaded Jun 02 10:38:17.529318 osdx cfgd[1665]: [487793]Completed change to active configuration Jun 02 10:38:17.531400 osdx OSDxCLI[487793]: User 'admin' committed the configuration. Jun 02 10:38:17.550721 osdx OSDxCLI[487793]: User 'admin' left the configuration menu. Jun 02 10:38:17.949185 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=41186 DF PROTO=TCP SPT=33056 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 02 10:38:17.953194 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=62048 DF PROTO=TCP SPT=33058 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 02 10:38:28.196601 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 42732 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 248 MBytes 2.08 Gbits/sec 81 1.90 MBytes [ 5] 1.00-2.00 sec 326 MBytes 2.74 Gbits/sec 164 1.48 MBytes [ 5] 2.00-3.00 sec 314 MBytes 2.63 Gbits/sec 61 1.21 MBytes [ 5] 3.00-4.00 sec 270 MBytes 2.26 Gbits/sec 0 1.36 MBytes [ 5] 4.00-5.00 sec 304 MBytes 2.55 Gbits/sec 0 1.52 MBytes [ 5] 5.00-6.00 sec 304 MBytes 2.55 Gbits/sec 79 1.17 MBytes [ 5] 6.00-7.00 sec 269 MBytes 2.25 Gbits/sec 0 1.32 MBytes [ 5] 7.00-8.00 sec 284 MBytes 2.38 Gbits/sec 0 1.47 MBytes [ 5] 8.00-9.00 sec 284 MBytes 2.38 Gbits/sec 0 1.60 MBytes [ 5] 9.00-10.00 sec 304 MBytes 2.55 Gbits/sec 0 1.73 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.84 GBytes 2.44 Gbits/sec 385 sender [ 5] 0.00-10.00 sec 2.84 GBytes 2.43 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/02/2026-10:38:17.948665 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33056 -> 40.0.0.2:5001 06/02/2026-10:38:17.949596 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33058 -> 40.0.0.2:5001 06/02/2026-10:38:33.489218 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42726 -> 40.0.0.2:5001 06/02/2026-10:38:33.490134 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42732 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jun 02 10:38:10.319984 osdx systemd-journald[148515]: Runtime Journal (/run/log/journal/a0363f0a73514b24b35c4ba7ae73dc6e) is 1.7M, max 13.8M, 12.0M free. Jun 02 10:38:10.321176 osdx systemd-journald[148515]: Received client request to rotate journal, rotating. Jun 02 10:38:10.321226 osdx systemd-journald[148515]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a0363f0a73514b24b35c4ba7ae73dc6e. Jun 02 10:38:10.324093 osdx sudo[488212]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 02 10:38:10.332728 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'system journal clear'. Jun 02 10:38:10.567972 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'system coredump delete all'. Jun 02 10:38:10.805587 osdx OSDxCLI[487793]: User 'admin' entered the configuration menu. Jun 02 10:38:10.891248 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jun 02 10:38:10.972849 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jun 02 10:38:11.041151 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jun 02 10:38:11.138475 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jun 02 10:38:11.206675 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jun 02 10:38:11.311255 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jun 02 10:38:11.386799 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'show working'. Jun 02 10:38:11.502311 osdx ubnt-cfgd[488243]: inactive Jun 02 10:38:11.535717 osdx INFO[488253]: FRR daemons did not change Jun 02 10:38:11.569187 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jun 02 10:38:11.649183 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jun 02 10:38:11.666803 osdx (udev-worker)[488377]: Network interface NamePolicy= disabled on kernel command line. Jun 02 10:38:11.707026 osdx (udev-worker)[488384]: Network interface NamePolicy= disabled on kernel command line. Jun 02 10:38:11.830826 osdx cfgd[1665]: [487793]Completed change to active configuration Jun 02 10:38:11.842316 osdx OSDxCLI[487793]: User 'admin' committed the configuration. Jun 02 10:38:11.859221 osdx OSDxCLI[487793]: User 'admin' left the configuration menu. Jun 02 10:38:13.931552 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jun 02 10:38:14.009909 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 02 10:38:14.088217 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 02 10:38:14.527726 osdx sudo[488541]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 02 10:38:14.588771 osdx file_operation[488544]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jun 02 10:38:14.607137 osdx sudo[488551]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 02 10:38:14.608915 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jun 02 10:38:14.740143 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jun 02 10:38:14.888167 osdx OSDxCLI[487793]: User 'admin' entered the configuration menu. Jun 02 10:38:14.954205 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 02 10:38:15.049449 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 02 10:38:15.107840 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 02 10:38:15.203662 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 02 10:38:15.261564 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 02 10:38:15.360203 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 02 10:38:15.450871 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 02 10:38:15.525173 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 02 10:38:15.609555 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 02 10:38:15.667067 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 02 10:38:15.754433 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 02 10:38:15.815034 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 02 10:38:15.914754 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 02 10:38:16.002216 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 02 10:38:16.086981 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 02 10:38:16.229967 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 02 10:38:16.286352 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 02 10:38:16.410953 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 02 10:38:16.495755 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 02 10:38:16.579042 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 02 10:38:16.662713 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'show working'. Jun 02 10:38:16.789424 osdx ubnt-cfgd[488599]: inactive Jun 02 10:38:16.865846 osdx INFO[488636]: FRR daemons did not change Jun 02 10:38:17.089939 osdx systemd[1]: Reloading. Jun 02 10:38:17.157185 osdx systemd-sysv-generator[488689]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jun 02 10:38:17.293472 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 02 10:38:17.325281 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 02 10:38:17.528545 osdx INFO[488671]: Rules successfully loaded Jun 02 10:38:17.529318 osdx cfgd[1665]: [487793]Completed change to active configuration Jun 02 10:38:17.531400 osdx OSDxCLI[487793]: User 'admin' committed the configuration. Jun 02 10:38:17.550721 osdx OSDxCLI[487793]: User 'admin' left the configuration menu. Jun 02 10:38:17.949185 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=41186 DF PROTO=TCP SPT=33056 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 02 10:38:17.953194 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=62048 DF PROTO=TCP SPT=33058 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 02 10:38:28.196601 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Jun 02 10:38:28.302236 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'system journal show | cat'. Jun 02 10:38:28.517884 osdx OSDxCLI[487793]: User 'admin' entered the configuration menu. Jun 02 10:38:28.582847 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 02 10:38:28.688978 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 02 10:38:28.753007 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 02 10:38:28.867239 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 02 10:38:28.929231 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 02 10:38:29.029417 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 02 10:38:29.118509 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 02 10:38:29.194542 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 02 10:38:29.294859 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 02 10:38:29.376540 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 02 10:38:29.446239 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 02 10:38:29.549156 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 02 10:38:29.619236 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 02 10:38:29.876147 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 02 10:38:29.931721 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 02 10:38:30.049562 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 02 10:38:30.122521 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 02 10:38:30.240670 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 02 10:38:30.296282 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 02 10:38:30.395090 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 02 10:38:30.463492 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Jun 02 10:38:30.560131 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Jun 02 10:38:30.627678 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Jun 02 10:38:30.732118 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Jun 02 10:38:30.800455 osdx OSDxCLI[487793]: User 'admin' added a new cfg line: 'show changes'. Jun 02 10:38:30.906578 osdx ubnt-cfgd[488819]: inactive Jun 02 10:38:30.951490 osdx INFO[488835]: FRR daemons did not change Jun 02 10:38:31.221695 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Jun 02 10:38:32.314705 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Jun 02 10:38:32.314812 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Jun 02 10:38:32.314842 osdx systemd[1]: suricata@FW.service: Consumed 1.121s CPU time. Jun 02 10:38:32.357854 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 02 10:38:32.381072 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 02 10:38:32.562113 osdx INFO[488860]: Rules successfully loaded Jun 02 10:38:32.562864 osdx cfgd[1665]: [487793]Completed change to active configuration Jun 02 10:38:33.226094 osdx OSDxCLI[487793]: User 'admin' committed the configuration. Jun 02 10:38:33.255871 osdx OSDxCLI[487793]: User 'admin' left the configuration menu. Jun 02 10:38:33.493199 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=42196 DF PROTO=TCP SPT=42726 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jun 02 10:38:33.493266 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=57353 DF PROTO=TCP SPT=42732 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jun 02 10:38:40.033554 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Jun 02 10:38:43.739484 osdx OSDxCLI[487793]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 54591 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.542 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.542/0.542/0.542/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.491 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.491/0.491/0.491/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 40286 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 273 MBytes 2.29 Gbits/sec 46 1.91 MBytes [ 5] 1.00-2.00 sec 288 MBytes 2.41 Gbits/sec 0 2.08 MBytes [ 5] 2.00-3.00 sec 301 MBytes 2.53 Gbits/sec 262 1.59 MBytes [ 5] 3.00-4.00 sec 318 MBytes 2.66 Gbits/sec 0 1.73 MBytes [ 5] 4.00-5.00 sec 326 MBytes 2.74 Gbits/sec 0 1.86 MBytes [ 5] 5.00-6.00 sec 292 MBytes 2.45 Gbits/sec 123 1.43 MBytes [ 5] 6.00-7.00 sec 355 MBytes 2.98 Gbits/sec 0 1.58 MBytes [ 5] 7.00-8.00 sec 298 MBytes 2.50 Gbits/sec 0 1.70 MBytes [ 5] 8.00-9.00 sec 309 MBytes 2.59 Gbits/sec 18 1.33 MBytes [ 5] 9.00-10.00 sec 306 MBytes 2.57 Gbits/sec 0 1.49 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec 449 sender [ 5] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/02/2026-10:39:05.078841 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40272 -> 40.0.0.2:5001 06/02/2026-10:39:05.079885 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40286 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=40272 dport=5001 packets=16 bytes=1298 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=40272 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=40286 dport=5001 packets=2219764 bytes=3329623501 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=40286 packets=285622 bytes=14848036 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=116 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=116 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=372 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=372 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 34296 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 273 MBytes 2.29 Gbits/sec 174 1.83 MBytes [ 5] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 98 1.42 MBytes [ 5] 2.00-3.00 sec 310 MBytes 2.60 Gbits/sec 0 1.57 MBytes [ 5] 3.00-4.00 sec 315 MBytes 2.64 Gbits/sec 0 1.71 MBytes [ 5] 4.00-5.00 sec 314 MBytes 2.63 Gbits/sec 0 1.83 MBytes [ 5] 5.00-6.00 sec 305 MBytes 2.56 Gbits/sec 335 1.42 MBytes [ 5] 6.00-7.00 sec 344 MBytes 2.88 Gbits/sec 0 1.58 MBytes [ 5] 7.00-8.00 sec 381 MBytes 3.20 Gbits/sec 0 1.74 MBytes [ 5] 8.00-9.00 sec 316 MBytes 2.65 Gbits/sec 261 1.39 MBytes [ 5] 9.00-10.00 sec 346 MBytes 2.90 Gbits/sec 0 1.56 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.11 GBytes 2.67 Gbits/sec 868 sender [ 5] 0.00-10.00 sec 3.11 GBytes 2.67 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/02/2026-10:39:05.078841 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40272 -> 40.0.0.2:5001 06/02/2026-10:39:05.079885 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40286 -> 40.0.0.2:5001 06/02/2026-10:39:19.907552 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34286 -> 40.0.0.2:5001 06/02/2026-10:39:19.908806 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34296 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=34286 dport=5001 packets=16 bytes=1301 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34286 packets=13 bytes=1016 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=34296 dport=5001 packets=2306388 bytes=3459566397 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=34296 packets=338035 bytes=17580676 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 32776 0 --:--:-- --:--:-- --:--:-- 33333
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.562 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.562/0.562/0.562/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.468 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.468/0.468/0.468/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/02/2026-10:39:54.996176 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56878 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=119 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=119 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=375 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=375 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=56878 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=56878 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/02/2026-10:39:54.996176 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56878 -> 40.0.0.2:5000 06/02/2026-10:40:01.197084 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:56878 -> 40.0.0.2:5000 06/02/2026-10:40:02.714209 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53612 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=53612 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=53612 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 22388 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.583 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.583/0.583/0.583/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.501 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.501/0.501/0.501/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/02/2026-10:40:25.705065 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53028 -> 40.0.0.2:5001 06/02/2026-10:40:25.705919 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53044 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=122 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=122 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53044 dport=5001 packets=53222 bytes=79822909 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53044 packets=7353 bytes=384376 [ASSURED] [OFFLOAD, packets=53209 bytes=79807708 packets=7351 bytes=384264] mark=129834765 use=2 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=378 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=378 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53028 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53028 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 23955 0 --:--:-- --:--:-- --:--:-- 26600
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
06/02/2026-10:40:25.705065 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53028 -> 40.0.0.2:5001 06/02/2026-10:40:25.705919 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53044 -> 40.0.0.2:5001 06/02/2026-10:40:27.575189 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53060 -> 40.0.0.2:5001 06/02/2026-10:40:27.576573 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:57247 -> 40.0.0.2:5001 06/02/2026-10:40:27.587277 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:57247 -> 40.0.0.2:5001 06/02/2026-10:40:27.599298 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:57247 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53060 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53060 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=57247 dport=5001 packets=17 bytes=23648 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=57247 packets=1 bytes=32 [OFFLOAD, packets=13 bytes=19188 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 37929 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.624 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.624/0.624/0.624/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.633 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.633/0.633/0.633/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/02/2026-10:40:46.181688 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37950 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 37950 5000 yes 201 0 8 673 40.0.0.2 20.0.0.2 5000 37950 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250135s ending at 1780396849.914141 XDP_DROP 9 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 22040318 pkts ( 12 pps) 28977206 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
06/02/2026-10:40:46.181688 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37950 -> 40.0.0.2:5000 06/02/2026-10:40:50.054397 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:53425 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 37950 5000 yes 201 0 11 847 20.0.0.2 40.0.0.2 53425 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5001 53425 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 37950 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250130s ending at 1780396853.807492 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 22040375 pkts ( 0 pps) 28977211 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)