Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23433 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.578 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.578/0.578/0.578/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.442 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.442/0.442/0.442/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 47704 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 83.0 MBytes 696 Mbits/sec 0 2.52 MBytes [ 5] 1.00-2.00 sec 93.8 MBytes 786 Mbits/sec 76 1.27 MBytes [ 5] 2.00-3.00 sec 130 MBytes 1.09 Gbits/sec 0 1.35 MBytes [ 5] 3.00-4.00 sec 152 MBytes 1.28 Gbits/sec 0 1.41 MBytes [ 5] 4.00-5.00 sec 156 MBytes 1.31 Gbits/sec 0 1.48 MBytes [ 5] 5.00-6.00 sec 152 MBytes 1.28 Gbits/sec 0 1.56 MBytes [ 5] 6.00-7.00 sec 151 MBytes 1.27 Gbits/sec 0 1.63 MBytes [ 5] 7.00-8.00 sec 166 MBytes 1.39 Gbits/sec 0 1.70 MBytes [ 5] 8.00-9.00 sec 155 MBytes 1.30 Gbits/sec 0 1.76 MBytes [ 5] 9.00-10.00 sec 154 MBytes 1.29 Gbits/sec 0 1.83 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.36 GBytes 1.17 Gbits/sec 76 sender [ 5] 0.00-10.01 sec 1.36 GBytes 1.17 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/10/2025-21:12:48.882575 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47698 -> 40.0.0.2:5001 10/10/2025-21:12:48.883515 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47704 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 48826 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.567 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.567/0.567/0.567/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.515 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.515/0.515/0.515/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 58258 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 136 MBytes 1.14 Gbits/sec 10 1.36 MBytes [ 5] 1.00-2.00 sec 194 MBytes 1.63 Gbits/sec 0 1.49 MBytes [ 5] 2.00-3.00 sec 251 MBytes 2.11 Gbits/sec 0 1.59 MBytes [ 5] 3.00-4.00 sec 259 MBytes 2.17 Gbits/sec 0 1.67 MBytes [ 5] 4.00-5.00 sec 289 MBytes 2.42 Gbits/sec 0 1.73 MBytes [ 5] 5.00-6.00 sec 292 MBytes 2.45 Gbits/sec 0 1.84 MBytes [ 5] 6.00-7.00 sec 268 MBytes 2.24 Gbits/sec 0 1.94 MBytes [ 5] 7.00-8.00 sec 308 MBytes 2.58 Gbits/sec 44 1.50 MBytes [ 5] 8.00-9.00 sec 352 MBytes 2.96 Gbits/sec 0 1.66 MBytes [ 5] 9.00-10.00 sec 350 MBytes 2.94 Gbits/sec 0 1.80 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec 54 sender [ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/10/2025-21:13:21.525749 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58244 -> 40.0.0.2:5001 10/10/2025-21:13:21.527609 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58258 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Oct 10 21:13:13.000060 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Oct 10 21:13:13.000297 osdx systemd-timedated[374649]: Changed local time to Fri 2025-10-10 21:13:13 UTC Oct 10 21:13:13.000683 osdx systemd-journald[246736]: Time jumped backwards, rotating. Oct 10 21:13:13.001521 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'set date 2025-10-10 21:13:13'. Oct 10 21:13:13.325047 osdx systemd-journald[246736]: Runtime Journal (/run/log/journal/d6792964fe7547c9a60a3d774aa97dac) is 1.9M, max 13.8M, 11.8M free. Oct 10 21:13:13.328679 osdx systemd-journald[246736]: Received client request to rotate journal, rotating. Oct 10 21:13:13.328750 osdx systemd-journald[246736]: Vacuuming done, freed 0B of archived journals from /run/log/journal/d6792964fe7547c9a60a3d774aa97dac. Oct 10 21:13:13.336220 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'system journal clear'. Oct 10 21:13:13.543611 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'system coredump delete all'. Oct 10 21:13:13.801549 osdx OSDxCLI[374222]: User 'admin' entered the configuration menu. Oct 10 21:13:13.894802 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Oct 10 21:13:13.970082 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Oct 10 21:13:14.023680 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Oct 10 21:13:14.111244 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Oct 10 21:13:14.167978 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Oct 10 21:13:14.258104 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Oct 10 21:13:14.325763 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'show working'. Oct 10 21:13:14.416714 osdx ubnt-cfgd[374683]: inactive Oct 10 21:13:14.448091 osdx INFO[374689]: FRR daemons did not change Oct 10 21:13:14.472692 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Oct 10 21:13:14.488244 osdx WARNING[374728]: No supported link modes on interface eth1 Oct 10 21:13:14.489727 osdx modulelauncher[374728]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Oct 10 21:13:14.489738 osdx modulelauncher[374728]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Oct 10 21:13:14.490876 osdx modulelauncher[374728]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Oct 10 21:13:14.490885 osdx modulelauncher[374728]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Oct 10 21:13:14.516679 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Oct 10 21:13:14.556272 osdx WARNING[374807]: No supported link modes on interface eth0 Oct 10 21:13:14.557656 osdx modulelauncher[374807]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Oct 10 21:13:14.557668 osdx modulelauncher[374807]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Oct 10 21:13:14.558799 osdx modulelauncher[374807]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Oct 10 21:13:14.558806 osdx modulelauncher[374807]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Oct 10 21:13:14.568197 osdx (udev-worker)[374823]: Network interface NamePolicy= disabled on kernel command line. Oct 10 21:13:14.735756 osdx cfgd[1464]: [374222]Completed change to active configuration Oct 10 21:13:14.749685 osdx OSDxCLI[374222]: User 'admin' committed the configuration. Oct 10 21:13:14.773490 osdx OSDxCLI[374222]: User 'admin' left the configuration menu. Oct 10 21:13:16.728776 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Oct 10 21:13:16.803986 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Oct 10 21:13:16.912822 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Oct 10 21:13:17.551451 osdx file_operation[374986]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Oct 10 21:13:17.573421 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Oct 10 21:13:17.710487 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Oct 10 21:13:17.845598 osdx OSDxCLI[374222]: User 'admin' entered the configuration menu. Oct 10 21:13:17.903912 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 10 21:13:17.997507 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 10 21:13:18.053079 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 10 21:13:18.158879 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 10 21:13:18.230249 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 10 21:13:18.333449 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 10 21:13:18.428786 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 10 21:13:18.510636 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 10 21:13:18.616041 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 10 21:13:18.666489 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 10 21:13:18.929220 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 10 21:13:18.985485 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 10 21:13:19.098617 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 10 21:13:19.164856 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 10 21:13:19.270865 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 10 21:13:19.350251 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 10 21:13:19.449476 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 10 21:13:19.536758 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 10 21:13:19.607413 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 10 21:13:19.705428 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 10 21:13:19.798550 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'show working'. Oct 10 21:13:19.884629 osdx ubnt-cfgd[375037]: inactive Oct 10 21:13:19.947630 osdx INFO[375074]: FRR daemons did not change Oct 10 21:13:20.171464 osdx systemd[1]: Reloading. Oct 10 21:13:20.244696 osdx systemd-sysv-generator[375130]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Oct 10 21:13:20.385197 osdx systemd[1]: Starting logrotate.service - Rotate log files... Oct 10 21:13:20.391781 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 10 21:13:20.412065 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 10 21:13:20.422582 osdx systemd[1]: logrotate.service: Deactivated successfully. Oct 10 21:13:20.422749 osdx systemd[1]: Finished logrotate.service - Rotate log files. Oct 10 21:13:20.625530 osdx INFO[375111]: Rules successfully loaded Oct 10 21:13:20.626101 osdx cfgd[1464]: [374222]Completed change to active configuration Oct 10 21:13:20.628095 osdx OSDxCLI[374222]: User 'admin' committed the configuration. Oct 10 21:13:21.159401 osdx OSDxCLI[374222]: User 'admin' left the configuration menu. Oct 10 21:13:21.528695 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=62502 DF PROTO=TCP SPT=58244 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 10 21:13:21.528772 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=5116 DF PROTO=TCP SPT=58258 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 10 21:13:31.720007 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50372 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 203 MBytes 1.70 Gbits/sec 53 1.85 MBytes [ 5] 1.00-2.00 sec 321 MBytes 2.69 Gbits/sec 0 2.02 MBytes [ 5] 2.00-3.00 sec 320 MBytes 2.68 Gbits/sec 234 1.55 MBytes [ 5] 3.00-4.00 sec 302 MBytes 2.54 Gbits/sec 0 1.68 MBytes [ 5] 4.00-5.00 sec 331 MBytes 2.78 Gbits/sec 0 1.82 MBytes [ 5] 5.00-6.00 sec 302 MBytes 2.54 Gbits/sec 293 1.43 MBytes [ 5] 6.00-7.00 sec 331 MBytes 2.78 Gbits/sec 0 1.59 MBytes [ 5] 7.00-8.00 sec 330 MBytes 2.77 Gbits/sec 0 1.73 MBytes [ 5] 8.00-9.00 sec 304 MBytes 2.55 Gbits/sec 34 1.35 MBytes [ 5] 9.00-10.00 sec 326 MBytes 2.74 Gbits/sec 0 1.52 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.00 GBytes 2.58 Gbits/sec 614 sender [ 5] 0.00-10.00 sec 3.00 GBytes 2.57 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/10/2025-21:13:21.525749 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58244 -> 40.0.0.2:5001 10/10/2025-21:13:21.527609 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58258 -> 40.0.0.2:5001 10/10/2025-21:13:36.340208 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50368 -> 40.0.0.2:5001 10/10/2025-21:13:36.341242 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50372 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Oct 10 21:13:13.000060 osdx systemd[1]: Started systemd-timedated.service - Time & Date Service. Oct 10 21:13:13.000297 osdx systemd-timedated[374649]: Changed local time to Fri 2025-10-10 21:13:13 UTC Oct 10 21:13:13.000683 osdx systemd-journald[246736]: Time jumped backwards, rotating. Oct 10 21:13:13.001521 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'set date 2025-10-10 21:13:13'. Oct 10 21:13:13.325047 osdx systemd-journald[246736]: Runtime Journal (/run/log/journal/d6792964fe7547c9a60a3d774aa97dac) is 1.9M, max 13.8M, 11.8M free. Oct 10 21:13:13.328679 osdx systemd-journald[246736]: Received client request to rotate journal, rotating. Oct 10 21:13:13.328750 osdx systemd-journald[246736]: Vacuuming done, freed 0B of archived journals from /run/log/journal/d6792964fe7547c9a60a3d774aa97dac. Oct 10 21:13:13.336220 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'system journal clear'. Oct 10 21:13:13.543611 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'system coredump delete all'. Oct 10 21:13:13.801549 osdx OSDxCLI[374222]: User 'admin' entered the configuration menu. Oct 10 21:13:13.894802 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Oct 10 21:13:13.970082 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Oct 10 21:13:14.023680 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Oct 10 21:13:14.111244 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Oct 10 21:13:14.167978 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Oct 10 21:13:14.258104 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Oct 10 21:13:14.325763 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'show working'. Oct 10 21:13:14.416714 osdx ubnt-cfgd[374683]: inactive Oct 10 21:13:14.448091 osdx INFO[374689]: FRR daemons did not change Oct 10 21:13:14.472692 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Oct 10 21:13:14.488244 osdx WARNING[374728]: No supported link modes on interface eth1 Oct 10 21:13:14.489727 osdx modulelauncher[374728]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on Oct 10 21:13:14.489738 osdx modulelauncher[374728]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. Oct 10 21:13:14.490876 osdx modulelauncher[374728]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- Oct 10 21:13:14.490885 osdx modulelauncher[374728]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Oct 10 21:13:14.516679 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Oct 10 21:13:14.556272 osdx WARNING[374807]: No supported link modes on interface eth0 Oct 10 21:13:14.557656 osdx modulelauncher[374807]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on Oct 10 21:13:14.557668 osdx modulelauncher[374807]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. Oct 10 21:13:14.558799 osdx modulelauncher[374807]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- Oct 10 21:13:14.558806 osdx modulelauncher[374807]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. Oct 10 21:13:14.568197 osdx (udev-worker)[374823]: Network interface NamePolicy= disabled on kernel command line. Oct 10 21:13:14.735756 osdx cfgd[1464]: [374222]Completed change to active configuration Oct 10 21:13:14.749685 osdx OSDxCLI[374222]: User 'admin' committed the configuration. Oct 10 21:13:14.773490 osdx OSDxCLI[374222]: User 'admin' left the configuration menu. Oct 10 21:13:16.728776 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Oct 10 21:13:16.803986 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Oct 10 21:13:16.912822 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Oct 10 21:13:17.551451 osdx file_operation[374986]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Oct 10 21:13:17.573421 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Oct 10 21:13:17.710487 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Oct 10 21:13:17.845598 osdx OSDxCLI[374222]: User 'admin' entered the configuration menu. Oct 10 21:13:17.903912 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 10 21:13:17.997507 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 10 21:13:18.053079 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 10 21:13:18.158879 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 10 21:13:18.230249 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 10 21:13:18.333449 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 10 21:13:18.428786 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 10 21:13:18.510636 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 10 21:13:18.616041 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 10 21:13:18.666489 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 10 21:13:18.929220 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 10 21:13:18.985485 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 10 21:13:19.098617 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 10 21:13:19.164856 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 10 21:13:19.270865 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 10 21:13:19.350251 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 10 21:13:19.449476 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 10 21:13:19.536758 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 10 21:13:19.607413 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 10 21:13:19.705428 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 10 21:13:19.798550 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'show working'. Oct 10 21:13:19.884629 osdx ubnt-cfgd[375037]: inactive Oct 10 21:13:19.947630 osdx INFO[375074]: FRR daemons did not change Oct 10 21:13:20.171464 osdx systemd[1]: Reloading. Oct 10 21:13:20.244696 osdx systemd-sysv-generator[375130]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Oct 10 21:13:20.385197 osdx systemd[1]: Starting logrotate.service - Rotate log files... Oct 10 21:13:20.391781 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 10 21:13:20.412065 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 10 21:13:20.422582 osdx systemd[1]: logrotate.service: Deactivated successfully. Oct 10 21:13:20.422749 osdx systemd[1]: Finished logrotate.service - Rotate log files. Oct 10 21:13:20.625530 osdx INFO[375111]: Rules successfully loaded Oct 10 21:13:20.626101 osdx cfgd[1464]: [374222]Completed change to active configuration Oct 10 21:13:20.628095 osdx OSDxCLI[374222]: User 'admin' committed the configuration. Oct 10 21:13:21.159401 osdx OSDxCLI[374222]: User 'admin' left the configuration menu. Oct 10 21:13:21.528695 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=62502 DF PROTO=TCP SPT=58244 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 10 21:13:21.528772 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=5116 DF PROTO=TCP SPT=58258 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 10 21:13:31.720007 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Oct 10 21:13:31.824321 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'system journal show | cat'. Oct 10 21:13:31.976215 osdx OSDxCLI[374222]: User 'admin' entered the configuration menu. Oct 10 21:13:32.033335 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 10 21:13:32.131028 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 10 21:13:32.182965 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 10 21:13:32.275216 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 10 21:13:32.326023 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 10 21:13:32.424745 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 10 21:13:32.501721 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 10 21:13:32.573568 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 10 21:13:32.671258 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 10 21:13:32.727142 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 10 21:13:32.821894 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 10 21:13:32.876754 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 10 21:13:32.971298 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 10 21:13:33.027557 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 10 21:13:33.117694 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 10 21:13:33.181115 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 10 21:13:33.275832 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 10 21:13:33.332914 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 10 21:13:33.425446 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 10 21:13:33.481508 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 10 21:13:33.578564 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Oct 10 21:13:33.633863 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Oct 10 21:13:33.771263 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Oct 10 21:13:33.838993 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Oct 10 21:13:33.967880 osdx OSDxCLI[374222]: User 'admin' added a new cfg line: 'show changes'. Oct 10 21:13:34.074842 osdx ubnt-cfgd[375257]: inactive Oct 10 21:13:34.119058 osdx INFO[375273]: FRR daemons did not change Oct 10 21:13:34.355644 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Oct 10 21:13:35.851693 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Oct 10 21:13:35.851800 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Oct 10 21:13:35.851832 osdx systemd[1]: suricata@FW.service: Consumed 1.561s CPU time. Oct 10 21:13:35.868973 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 10 21:13:35.885126 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 10 21:13:36.079008 osdx INFO[375300]: Rules successfully loaded Oct 10 21:13:36.079521 osdx cfgd[1464]: [374222]Completed change to active configuration Oct 10 21:13:36.081618 osdx OSDxCLI[374222]: User 'admin' committed the configuration. Oct 10 21:13:36.104765 osdx OSDxCLI[374222]: User 'admin' left the configuration menu. Oct 10 21:13:36.340690 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=12527 DF PROTO=TCP SPT=50368 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Oct 10 21:13:36.344690 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=43312 DF PROTO=TCP SPT=50372 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Oct 10 21:13:43.032921 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Oct 10 21:13:46.627646 osdx OSDxCLI[374222]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 22715 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.545 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.545/0.545/0.545/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.431 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.431/0.431/0.431/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33888 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 140 MBytes 1.18 Gbits/sec 34 1.28 MBytes [ 5] 1.00-2.00 sec 258 MBytes 2.16 Gbits/sec 0 1.40 MBytes [ 5] 2.00-3.00 sec 286 MBytes 2.40 Gbits/sec 0 1.54 MBytes [ 5] 3.00-4.00 sec 271 MBytes 2.28 Gbits/sec 0 1.67 MBytes [ 5] 4.00-5.00 sec 270 MBytes 2.26 Gbits/sec 0 1.77 MBytes [ 5] 5.00-6.00 sec 284 MBytes 2.38 Gbits/sec 0 1.88 MBytes [ 5] 6.00-7.00 sec 296 MBytes 2.49 Gbits/sec 130 1.45 MBytes [ 5] 7.00-8.00 sec 342 MBytes 2.87 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 340 MBytes 2.85 Gbits/sec 0 1.75 MBytes [ 5] 9.00-10.00 sec 352 MBytes 2.96 Gbits/sec 0 1.90 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.77 GBytes 2.38 Gbits/sec 164 sender [ 5] 0.00-10.00 sec 2.77 GBytes 2.38 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/10/2025-21:14:10.390293 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33878 -> 40.0.0.2:5001 10/10/2025-21:14:10.391155 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33888 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=33888 dport=5001 packets=2056840 bytes=3085249701 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33888 packets=299151 bytes=15546740 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=379 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=379 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=33878 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33878 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=79 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=79 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 33548 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 264 MBytes 2.22 Gbits/sec 50 1.87 MBytes [ 5] 1.00-2.00 sec 349 MBytes 2.93 Gbits/sec 0 2.03 MBytes [ 5] 2.00-3.00 sec 340 MBytes 2.85 Gbits/sec 32 1.53 MBytes [ 5] 3.00-4.00 sec 314 MBytes 2.63 Gbits/sec 37 1.21 MBytes [ 5] 4.00-5.00 sec 318 MBytes 2.66 Gbits/sec 0 1.38 MBytes [ 5] 5.00-6.00 sec 361 MBytes 3.03 Gbits/sec 0 1.56 MBytes [ 5] 6.00-7.00 sec 356 MBytes 2.99 Gbits/sec 0 1.72 MBytes [ 5] 7.00-8.00 sec 331 MBytes 2.78 Gbits/sec 0 1.86 MBytes [ 5] 8.00-9.00 sec 360 MBytes 3.02 Gbits/sec 0 2.00 MBytes [ 5] 9.00-10.00 sec 125 MBytes 1.05 Gbits/sec 1 1.41 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.04 GBytes 2.62 Gbits/sec 120 sender [ 5] 0.00-10.21 sec 3.04 GBytes 2.56 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/10/2025-21:14:10.390293 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33878 -> 40.0.0.2:5001 10/10/2025-21:14:10.391155 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33888 -> 40.0.0.2:5001 10/10/2025-21:14:25.168551 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33540 -> 40.0.0.2:5001 10/10/2025-21:14:25.169360 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33548 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=33548 dport=5001 packets=2257871 bytes=3386794705 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33548 packets=378013 bytes=19648220 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=33540 dport=5001 packets=17 bytes=1354 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=33540 packets=15 bytes=1132 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 3024 0 --:--:-- --:--:-- --:--:-- 3030
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.520 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.520/0.520/0.520/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.506 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.506/0.506/0.506/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/10/2025-21:14:58.532000 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33978 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=382 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=382 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=33978 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=33978 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=82 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=82 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 5 10 409 778 2 - 5 5 369 369 ------------------------------------------------------------------ Total 10 10 778 778
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/10/2025-21:14:58.532000 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33978 -> 40.0.0.2:5000 10/10/2025-21:15:04.827101 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:33978 -> 40.0.0.2:5000 10/10/2025-21:15:06.301249 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48434 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=48434 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=48434 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 5 10 409 740 2 - 5 5 331 331 --------------------------------------------------------------------- Total 10 10 740 740
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 19507 0 --:--:-- --:--:-- --:--:-- 21500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.531 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.531/0.531/0.531/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.433 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.433/0.433/0.433/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/10/2025-21:15:32.982392 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43550 -> 40.0.0.2:5001 10/10/2025-21:15:32.983362 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43560 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=43560 dport=5001 packets=17624 bytes=26428805 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43560 packets=2010 bytes=104528 [ASSURED] [OFFLOAD, packets=17611 bytes=26413604 packets=2008 bytes=104416] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=85 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=85 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=385 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=385 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=43550 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43550 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 776 0 --:--:-- --:--:-- --:--:-- 777
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
10/10/2025-21:15:32.982392 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43550 -> 40.0.0.2:5001 10/10/2025-21:15:32.983362 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43560 -> 40.0.0.2:5001 10/10/2025-21:15:35.666851 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:43566 -> 40.0.0.2:5001 10/10/2025-21:15:35.668116 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41572 -> 40.0.0.2:5001 10/10/2025-21:15:35.679221 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41572 -> 40.0.0.2:5001 10/10/2025-21:15:35.691193 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:41572 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=43566 dport=5001 packets=7 bytes=555 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=43566 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=2 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=41572 dport=5001 packets=21 bytes=29552 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=41572 packets=1 bytes=32 [OFFLOAD, packets=17 bytes=25092 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 33721 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.608 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.608/0.608/0.608/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.486 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.486/0.486/0.486/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/10/2025-21:16:01.979482 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57958 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 57958 yes 201 0 0 0 20.0.0.2 40.0.0.2 57958 5000 yes 201 0 8 673
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250139s ending at 1760130965.548132 XDP_DROP 9 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 22018813 pkts ( 0 pps) 28456151 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
10/10/2025-21:16:01.979482 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57958 -> 40.0.0.2:5000 10/10/2025-21:16:05.682041 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:44060 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 44060 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 57958 yes 201 0 0 0 20.0.0.2 40.0.0.2 57958 5000 yes 201 0 12 905 40.0.0.2 20.0.0.2 5001 44060 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250140s ending at 1760130969.391605 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 22018873 pkts ( 0 pps) 28456156 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)