Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 50879 0 --:--:-- --:--:-- --:--:-- 53200
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.333 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.333/0.333/0.333/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.339 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.339/0.339/0.339/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.404 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.404/0.404/0.404/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.635 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.635/0.635/0.635/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50098 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 60.8 MBytes 510 Mbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 56.2 MBytes 472 Mbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 57.5 MBytes 482 Mbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 56.2 MBytes 472 Mbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 55.0 MBytes 461 Mbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 58.8 MBytes 493 Mbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 55.0 MBytes 461 Mbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 56.2 MBytes 472 Mbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 56.2 MBytes 472 Mbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 55.0 MBytes 461 Mbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 567 MBytes 476 Mbits/sec 0 sender [ 5] 0.00-10.01 sec 565 MBytes 473 Mbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
05/19/2026-19:31:40.058616 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50094 -> 40.0.0.2:5001 05/19/2026-19:31:40.059281 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50098 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 94932 0 --:--:-- --:--:-- --:--:-- 129k
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.249 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.249/0.249/0.249/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.241 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.241/0.241/0.241/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.702 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.702/0.702/0.702/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.717 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.717/0.717/0.717/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 35674 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 357 MBytes 2.99 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 382 MBytes 3.21 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 362 MBytes 3.04 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 359 MBytes 3.01 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 364 MBytes 3.05 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 358 MBytes 3.00 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 350 MBytes 2.94 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 368 MBytes 3.08 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 381 MBytes 3.20 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 401 MBytes 3.37 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.60 GBytes 3.09 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 3.59 GBytes 3.09 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
05/19/2026-19:32:10.042018 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35668 -> 40.0.0.2:5001 05/19/2026-19:32:10.042793 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35674 -> 40.0.0.2:5001
Step 10: Run the command system journal show | cat on DUT0 and check whether the output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
May 19 19:32:02.282628 osdx systemd-journald[505760]: Runtime Journal (/run/log/journal/d1b141b298644f3ea5560bad25bf4943) is 1.9M, max 13.8M, 11.8M free. May 19 19:32:02.285692 osdx systemd-journald[505760]: Received client request to rotate journal, rotating. May 19 19:32:02.285747 osdx systemd-journald[505760]: Vacuuming done, freed 0B of archived journals from /run/log/journal/d1b141b298644f3ea5560bad25bf4943. May 19 19:32:02.292634 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'system journal clear'. May 19 19:32:02.490550 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'system coredump delete all'. May 19 19:32:02.706996 osdx OSDxCLI[1019250]: User 'admin' entered the configuration menu. May 19 19:32:02.788453 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. May 19 19:32:02.874113 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. May 19 19:32:02.983216 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. May 19 19:32:03.045762 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. May 19 19:32:03.139292 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. May 19 19:32:03.189115 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. May 19 19:32:03.283697 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'show working'. May 19 19:32:03.363310 osdx ubnt-cfgd[1019772]: inactive May 19 19:32:03.396702 osdx INFO[1019779]: FRR daemons did not change May 19 19:32:03.425684 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 May 19 19:32:03.447435 osdx WARNING[1019821]: No supported link modes on interface eth1 May 19 19:32:03.448823 osdx modulelauncher[1019821]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on May 19 19:32:03.448838 osdx modulelauncher[1019821]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. May 19 19:32:03.450000 osdx modulelauncher[1019821]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- May 19 19:32:03.450010 osdx modulelauncher[1019821]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. May 19 19:32:03.485693 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 May 19 19:32:03.527708 osdx WARNING[1019896]: No supported link modes on interface eth0 May 19 19:32:03.529000 osdx modulelauncher[1019896]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on May 19 19:32:03.529011 osdx modulelauncher[1019896]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. May 19 19:32:03.530202 osdx modulelauncher[1019896]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- May 19 19:32:03.530209 osdx modulelauncher[1019896]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. May 19 19:32:03.539786 osdx (udev-worker)[1019912]: Network interface NamePolicy= disabled on kernel command line. May 19 19:32:03.860618 osdx cfgd[1918]: [1019250]Completed change to active configuration May 19 19:32:03.890149 osdx OSDxCLI[1019250]: User 'admin' committed the configuration. May 19 19:32:03.913261 osdx OSDxCLI[1019250]: User 'admin' left the configuration menu. May 19 19:32:06.156441 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. May 19 19:32:06.229412 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:06.335660 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:06.834107 osdx file_operation[1020106]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// May 19 19:32:06.854991 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. May 19 19:32:06.976381 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'file show running://test-performance.rules'. May 19 19:32:07.123945 osdx OSDxCLI[1019250]: User 'admin' entered the configuration menu. May 19 19:32:07.179342 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. May 19 19:32:07.273843 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. May 19 19:32:07.323128 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. May 19 19:32:07.415006 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. May 19 19:32:07.465513 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. May 19 19:32:07.561033 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. May 19 19:32:07.642414 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. May 19 19:32:07.710257 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. May 19 19:32:07.794807 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. May 19 19:32:07.845203 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. May 19 19:32:07.939158 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. May 19 19:32:07.994231 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. May 19 19:32:08.083414 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. May 19 19:32:08.137249 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. May 19 19:32:08.228929 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. May 19 19:32:08.285138 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. May 19 19:32:08.372473 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. May 19 19:32:08.425388 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. May 19 19:32:08.517871 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. May 19 19:32:08.570157 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. May 19 19:32:08.684157 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'show working'. May 19 19:32:08.758773 osdx ubnt-cfgd[1020154]: inactive May 19 19:32:08.835544 osdx INFO[1020202]: FRR daemons did not change May 19 19:32:09.100849 osdx systemd[1]: Reloading. May 19 19:32:09.129690 osdx systemd-sysv-generator[1020252]: stat() failed on /etc/init.d/README, ignoring: No such file or directory May 19 19:32:09.250006 osdx systemd[1]: Starting logrotate.service - Rotate log files... May 19 19:32:09.253600 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... May 19 19:32:09.271874 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. May 19 19:32:09.278208 osdx systemd[1]: logrotate.service: Deactivated successfully. May 19 19:32:09.278342 osdx systemd[1]: Finished logrotate.service - Rotate log files. May 19 19:32:09.451802 osdx INFO[1020234]: Rules successfully loaded May 19 19:32:09.457364 osdx cfgd[1918]: [1019250]Completed change to active configuration May 19 19:32:09.457811 osdx OSDxCLI[1019250]: User 'admin' committed the configuration. May 19 19:32:09.531524 osdx OSDxCLI[1019250]: User 'admin' left the configuration menu. May 19 19:32:09.621551 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:09.726748 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:10.045988 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=53058 DF PROTO=TCP SPT=35668 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d May 19 19:32:10.053021 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=21679 DF PROTO=TCP SPT=35674 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d May 19 19:32:20.187568 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 11: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 38796 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 359 MBytes 3.01 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 369 MBytes 3.09 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 371 MBytes 3.11 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 361 MBytes 3.03 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 369 MBytes 3.09 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 379 MBytes 3.18 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 381 MBytes 3.20 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 360 MBytes 3.02 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 375 MBytes 3.15 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 380 MBytes 3.19 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.62 GBytes 3.11 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 3.62 GBytes 3.11 Gbits/sec receiver iperf Done.
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
05/19/2026-19:32:10.042018 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35668 -> 40.0.0.2:5001 05/19/2026-19:32:10.042793 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35674 -> 40.0.0.2:5001 05/19/2026-19:32:24.863197 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38784 -> 40.0.0.2:5001 05/19/2026-19:32:24.863918 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38796 -> 40.0.0.2:5001
Step 14: Run the command system journal show | cat on DUT0 and check whether the output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
May 19 19:32:02.282628 osdx systemd-journald[505760]: Runtime Journal (/run/log/journal/d1b141b298644f3ea5560bad25bf4943) is 1.9M, max 13.8M, 11.8M free. May 19 19:32:02.285692 osdx systemd-journald[505760]: Received client request to rotate journal, rotating. May 19 19:32:02.285747 osdx systemd-journald[505760]: Vacuuming done, freed 0B of archived journals from /run/log/journal/d1b141b298644f3ea5560bad25bf4943. May 19 19:32:02.292634 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'system journal clear'. May 19 19:32:02.490550 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'system coredump delete all'. May 19 19:32:02.706996 osdx OSDxCLI[1019250]: User 'admin' entered the configuration menu. May 19 19:32:02.788453 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. May 19 19:32:02.874113 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. May 19 19:32:02.983216 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. May 19 19:32:03.045762 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. May 19 19:32:03.139292 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. May 19 19:32:03.189115 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. May 19 19:32:03.283697 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'show working'. May 19 19:32:03.363310 osdx ubnt-cfgd[1019772]: inactive May 19 19:32:03.396702 osdx INFO[1019779]: FRR daemons did not change May 19 19:32:03.425684 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 May 19 19:32:03.447435 osdx WARNING[1019821]: No supported link modes on interface eth1 May 19 19:32:03.448823 osdx modulelauncher[1019821]: osdx.utils.xos cmd error: /sbin/ethtool -A eth1 autoneg on May 19 19:32:03.448838 osdx modulelauncher[1019821]: Command '/sbin/ethtool -A eth1 autoneg on' returned non-zero exit status 76. May 19 19:32:03.450000 osdx modulelauncher[1019821]: osdx.utils.xos cmd error: /sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off -- May 19 19:32:03.450010 osdx modulelauncher[1019821]: Command '/sbin/ethtool -s eth1 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. May 19 19:32:03.485693 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 May 19 19:32:03.527708 osdx WARNING[1019896]: No supported link modes on interface eth0 May 19 19:32:03.529000 osdx modulelauncher[1019896]: osdx.utils.xos cmd error: /sbin/ethtool -A eth0 autoneg on May 19 19:32:03.529011 osdx modulelauncher[1019896]: Command '/sbin/ethtool -A eth0 autoneg on' returned non-zero exit status 76. May 19 19:32:03.530202 osdx modulelauncher[1019896]: osdx.utils.xos cmd error: /sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off -- May 19 19:32:03.530209 osdx modulelauncher[1019896]: Command '/sbin/ethtool -s eth0 autoneg on advertise Asym_Pause off Pause off --' returned non-zero exit status 75. May 19 19:32:03.539786 osdx (udev-worker)[1019912]: Network interface NamePolicy= disabled on kernel command line. May 19 19:32:03.860618 osdx cfgd[1918]: [1019250]Completed change to active configuration May 19 19:32:03.890149 osdx OSDxCLI[1019250]: User 'admin' committed the configuration. May 19 19:32:03.913261 osdx OSDxCLI[1019250]: User 'admin' left the configuration menu. May 19 19:32:06.156441 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. May 19 19:32:06.229412 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:06.335660 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:06.834107 osdx file_operation[1020106]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// May 19 19:32:06.854991 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. May 19 19:32:06.976381 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'file show running://test-performance.rules'. May 19 19:32:07.123945 osdx OSDxCLI[1019250]: User 'admin' entered the configuration menu. May 19 19:32:07.179342 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. May 19 19:32:07.273843 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. May 19 19:32:07.323128 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. May 19 19:32:07.415006 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. May 19 19:32:07.465513 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. May 19 19:32:07.561033 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. May 19 19:32:07.642414 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. May 19 19:32:07.710257 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. May 19 19:32:07.794807 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. May 19 19:32:07.845203 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. May 19 19:32:07.939158 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. May 19 19:32:07.994231 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. May 19 19:32:08.083414 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. May 19 19:32:08.137249 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. May 19 19:32:08.228929 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. May 19 19:32:08.285138 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. May 19 19:32:08.372473 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. May 19 19:32:08.425388 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. May 19 19:32:08.517871 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. May 19 19:32:08.570157 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. May 19 19:32:08.684157 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'show working'. May 19 19:32:08.758773 osdx ubnt-cfgd[1020154]: inactive May 19 19:32:08.835544 osdx INFO[1020202]: FRR daemons did not change May 19 19:32:09.100849 osdx systemd[1]: Reloading. May 19 19:32:09.129690 osdx systemd-sysv-generator[1020252]: stat() failed on /etc/init.d/README, ignoring: No such file or directory May 19 19:32:09.250006 osdx systemd[1]: Starting logrotate.service - Rotate log files... May 19 19:32:09.253600 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... May 19 19:32:09.271874 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. May 19 19:32:09.278208 osdx systemd[1]: logrotate.service: Deactivated successfully. May 19 19:32:09.278342 osdx systemd[1]: Finished logrotate.service - Rotate log files. May 19 19:32:09.451802 osdx INFO[1020234]: Rules successfully loaded May 19 19:32:09.457364 osdx cfgd[1918]: [1019250]Completed change to active configuration May 19 19:32:09.457811 osdx OSDxCLI[1019250]: User 'admin' committed the configuration. May 19 19:32:09.531524 osdx OSDxCLI[1019250]: User 'admin' left the configuration menu. May 19 19:32:09.621551 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:09.726748 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. May 19 19:32:10.045988 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=53058 DF PROTO=TCP SPT=35668 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d May 19 19:32:10.053021 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=21679 DF PROTO=TCP SPT=35674 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d May 19 19:32:20.187568 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. May 19 19:32:20.281086 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'system journal show | cat'. May 19 19:32:20.457845 osdx OSDxCLI[1019250]: User 'admin' entered the configuration menu. May 19 19:32:20.512811 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. May 19 19:32:20.608272 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. May 19 19:32:20.656820 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. May 19 19:32:20.752412 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. May 19 19:32:20.803582 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. May 19 19:32:20.899080 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. May 19 19:32:20.980829 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. May 19 19:32:21.046809 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. May 19 19:32:21.139305 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. May 19 19:32:21.190171 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. May 19 19:32:21.284011 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. May 19 19:32:21.337209 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. May 19 19:32:21.429379 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. May 19 19:32:21.485443 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. May 19 19:32:21.574694 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. May 19 19:32:21.628145 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. May 19 19:32:21.721211 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. May 19 19:32:22.173327 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. May 19 19:32:22.224189 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. May 19 19:32:22.319647 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. May 19 19:32:22.371103 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. May 19 19:32:22.464935 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. May 19 19:32:22.516868 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. May 19 19:32:22.615275 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. May 19 19:32:22.676205 osdx OSDxCLI[1019250]: User 'admin' added a new cfg line: 'show changes'. May 19 19:32:22.770852 osdx ubnt-cfgd[1020378]: inactive May 19 19:32:22.818325 osdx INFO[1020401]: FRR daemons did not change May 19 19:32:23.111054 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... May 19 19:32:24.356353 osdx systemd[1]: suricata@FW.service: Deactivated successfully. May 19 19:32:24.356465 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. May 19 19:32:24.356499 osdx systemd[1]: suricata@FW.service: Consumed 1.183s CPU time. May 19 19:32:24.386499 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... May 19 19:32:24.413315 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. May 19 19:32:24.632550 osdx INFO[1020428]: Rules successfully loaded May 19 19:32:24.638777 osdx cfgd[1918]: [1019250]Completed change to active configuration May 19 19:32:24.639352 osdx OSDxCLI[1019250]: User 'admin' committed the configuration. May 19 19:32:24.655319 osdx OSDxCLI[1019250]: User 'admin' left the configuration menu. May 19 19:32:24.865700 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=4741 DF PROTO=TCP SPT=38784 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff May 19 19:32:24.865759 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:21:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=18707 DF PROTO=TCP SPT=38796 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff May 19 19:32:32.032963 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. May 19 19:32:35.012800 osdx OSDxCLI[1019250]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 48987 0 --:--:-- --:--:-- --:--:-- 53200
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.470 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.470/0.470/0.470/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.373 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.373/0.373/0.373/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.710 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.710/0.710/0.710/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.450 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.450/0.450/0.450/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 55122 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 388 MBytes 3.26 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 412 MBytes 3.46 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 395 MBytes 3.31 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 424 MBytes 3.55 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 381 MBytes 3.20 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 392 MBytes 3.29 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 396 MBytes 3.32 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 394 MBytes 3.30 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 395 MBytes 3.31 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 369 MBytes 3.09 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.85 GBytes 3.31 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 3.85 GBytes 3.31 Gbits/sec receiver iperf Done.
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
05/19/2026-19:32:59.482756 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55110 -> 40.0.0.2:5001 05/19/2026-19:32:59.483424 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55122 -> 40.0.0.2:5001
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=55122 dport=5001 packets=2857794 bytes=4286567357 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=55122 packets=534260 bytes=27771268 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=615 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=615 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=55110 dport=5001 packets=16 bytes=1298 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=55110 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=616 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=616 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=530 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=530 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=111 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=111 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 11: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect the following output on
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 42160 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 406 MBytes 3.41 Gbits/sec 0 1.61 MBytes [ 5] 1.00-2.00 sec 394 MBytes 3.30 Gbits/sec 0 1.61 MBytes [ 5] 2.00-3.00 sec 382 MBytes 3.21 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 368 MBytes 3.08 Gbits/sec 0 1.61 MBytes [ 5] 4.00-5.00 sec 378 MBytes 3.17 Gbits/sec 0 1.61 MBytes [ 5] 5.00-6.00 sec 381 MBytes 3.20 Gbits/sec 0 1.61 MBytes [ 5] 6.00-7.00 sec 401 MBytes 3.37 Gbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 408 MBytes 3.42 Gbits/sec 0 1.61 MBytes [ 5] 8.00-9.00 sec 391 MBytes 3.28 Gbits/sec 0 1.61 MBytes [ 5] 9.00-10.00 sec 409 MBytes 3.43 Gbits/sec 0 1.61 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.83 GBytes 3.29 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 3.82 GBytes 3.28 Gbits/sec receiver iperf Done.
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
05/19/2026-19:32:59.482756 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55110 -> 40.0.0.2:5001 05/19/2026-19:32:59.483424 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55122 -> 40.0.0.2:5001 05/19/2026-19:33:14.222502 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42154 -> 40.0.0.2:5001 05/19/2026-19:33:14.223357 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:42160 -> 40.0.0.2:5001
Step 14: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=42160 dport=5001 packets=2836747 bytes=4254979049 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42160 packets=518862 bytes=26971640 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=42154 dport=5001 packets=16 bytes=1298 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=42154 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 41893 0 --:--:-- --:--:-- --:--:-- 50000
Step 2: Run the command file show running://drop-performance.rules on DUT0 and expect the following output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.218 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.218/0.218/0.218/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.324 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.324/0.324/0.324/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.489 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.489/0.489/0.489/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.378 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.378/0.378/0.378/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
05/19/2026-19:33:45.299186 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37782 -> 40.0.0.2:5000
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=621 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=621 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=114 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=114 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=533 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=533 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=37782 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=37782 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=620 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=620 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 5 flow entries have been shown.
Step 11: Run the command traffic policy FW_PLAN show on DUT0 and check whether the output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 8 210 522 2 - 4 4 312 312 ------------------------------------------------------------------ Total 8 8 522 522 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 10 356 809 2 - 6 6 453 453 ------------------------------------------------------------------ Total 10 10 809 809
Note
Testing with another conntrack mark.
Step 12: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 13: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 14: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
05/19/2026-19:33:45.299186 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37782 -> 40.0.0.2:5000 05/19/2026-19:33:51.445287 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37782 -> 40.0.0.2:5000 05/19/2026-19:33:52.893040 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48848 -> 40.0.0.2:5000
Step 15: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=48848 dport=5000 packets=7 bytes=557 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=48848 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 16: Run the command traffic policy FW_PLAN show on DUT0 and check whether the output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 9 356 686 2 - 5 5 330 330 --------------------------------------------------------------------- Total 9 9 686 686
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/test-performance.rules running:// force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 266 100 266 0 0 46880 0 --:--:-- --:--:-- --:--:-- 53200
Step 2: Run the command file show running://test-performance.rules on DUT0 and expect the following output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance TCP traffic"; bypass; flow: established, to_server; sid: 40;) alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.431 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.431/0.431/0.431/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.421 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.421/0.421/0.421/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.676 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.676/0.676/0.676/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.658 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.658/0.658/0.658/0.000 ms
Step 8: Initiate a background bandwidth test from DUT2 to DUT1. Control is returned, allowing other tasks to be performed while the test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance TCP traffic).+$Show output
05/19/2026-19:34:16.664470 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51974 -> 40.0.0.2:5001 05/19/2026-19:34:16.665237 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51980 -> 40.0.0.2:5001
Step 10: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=51980 dport=5001 packets=58318 bytes=87471253 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51980 packets=14716 bytes=765288 [ASSURED] [OFFLOAD, packets=58305 bytes=87456052 packets=14714 bytes=765176] mark=129834765 use=2 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=536 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=536 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=51974 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51974 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=2 icmp 1 29 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=626 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=626 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=625 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=625 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=117 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=117 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Step 11: Stop the current bandwidth test between DUT2 and DUT1.
Step 12: Initiate a background bandwidth test from DUT2 to DUT1. Control is returned, allowing other tasks to be performed while the test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
05/19/2026-19:34:16.664470 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51974 -> 40.0.0.2:5001 05/19/2026-19:34:16.665237 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51980 -> 40.0.0.2:5001 05/19/2026-19:34:17.268344 [**] [1:40:0] Skipping test network performance TCP traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:51988 -> 40.0.0.2:5001 05/19/2026-19:34:17.269202 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:36932 -> 40.0.0.2:5001 05/19/2026-19:34:17.280256 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:36932 -> 40.0.0.2:5001
Step 14: Run the command system conntrack show on DUT0 and check whether the output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=536 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=536 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=51974 dport=5001 packets=10 bytes=694 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51974 packets=9 bytes=480 [ASSURED] [OFFLOAD, packets=3 bytes=157 packets=4 bytes=211] mark=129834765 use=3 icmp 1 28 src=20.0.0.1 dst=20.0.0.2 type=8 code=0 id=626 packets=1 bytes=84 src=20.0.0.2 dst=20.0.0.1 type=0 code=0 id=626 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=51988 dport=5001 packets=7 bytes=555 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=51988 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 icmp 1 28 src=40.0.0.1 dst=40.0.0.2 type=8 code=0 id=625 packets=1 bytes=84 src=40.0.0.2 dst=40.0.0.1 type=0 code=0 id=625 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=36932 dport=5001 packets=9 bytes=11840 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=36932 packets=1 bytes=32 [OFFLOAD, packets=5 bytes=7380 packets=0 bytes=0] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=117 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=117 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 7 flow entries have been shown.
Step 15: Stop the current bandwidth test between DUT2 and DUT1.
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run the command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force on DUT0 and expect the following output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 53533 0 --:--:-- --:--:-- --:--:-- 66666
Step 2: Run the command file show running://drop-performance.rules on DUT0 and expect the following output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping the IP address 40.0.0.2 from DUT0:
admin@DUT0$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=64 time=0.328 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.328/0.328/0.328/0.000 ms
Step 5: Ping the IP address 20.0.0.2 from DUT0:
admin@DUT0$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=64 time=0.295 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.295/0.295/0.295/0.000 ms
Step 6: Ping the IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.751 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.751/0.751/0.751/0.000 ms
Step 7: Ping the IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.479 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.479/0.479/0.479/0.000 ms
Step 8: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 9: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
05/19/2026-19:34:37.087420 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38414 -> 40.0.0.2:5000
Step 10: Run the command service firewall FW show early-drop-stats eth1 on DUT0 and check whether the output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 38414 yes 201 0 0 0 20.0.0.2 40.0.0.2 38414 5000 yes 201 0 6 544
Step 11: Run the command interfaces ethernet eth1 monitor xdp-stats times 1 on DUT0 and expect the following output:
Show output
Period of 0.250102s ending at 1779219280.704950 XDP_DROP 7 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 15 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect the following output on
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 13: Run the command service firewall FW show logging fast | tail on DUT0 and check whether the output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
05/19/2026-19:34:37.087420 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38414 -> 40.0.0.2:5000 05/19/2026-19:34:40.835138 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:56781 -> 40.0.0.2:5001
Step 14: Run the command service firewall FW show early-drop-stats eth1 on DUT0 and check whether the output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 38414 yes 201 0 0 0 20.0.0.2 40.0.0.2 56781 5001 no 201 0 0 0 20.0.0.2 40.0.0.2 38414 5000 yes 201 0 10 776 40.0.0.2 20.0.0.2 5001 56781 no 201 0 0 0
Step 15: Run the command interfaces ethernet eth1 monitor xdp-stats times 1 on DUT0 and expect the following output:
Show output
Period of 0.250147s ending at 1779219284.427749 XDP_DROP 10 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 35 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)