Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 27004 0 --:--:-- --:--:-- --:--:-- 32250
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.679 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.679/0.679/0.679/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.555 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.555/0.555/0.555/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 40350 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 105 MBytes 881 Mbits/sec 132 1.75 MBytes [ 5] 1.00-2.00 sec 97.5 MBytes 817 Mbits/sec 18 1.34 MBytes [ 5] 2.00-3.00 sec 97.5 MBytes 818 Mbits/sec 0 1.43 MBytes [ 5] 3.00-4.00 sec 98.8 MBytes 828 Mbits/sec 0 1.50 MBytes [ 5] 4.00-5.00 sec 95.0 MBytes 797 Mbits/sec 0 1.55 MBytes [ 5] 5.00-6.00 sec 93.8 MBytes 786 Mbits/sec 0 1.59 MBytes [ 5] 6.00-7.00 sec 93.8 MBytes 786 Mbits/sec 0 1.61 MBytes [ 5] 7.00-8.00 sec 91.2 MBytes 765 Mbits/sec 0 1.62 MBytes [ 5] 8.00-9.00 sec 65.0 MBytes 545 Mbits/sec 0 1.64 MBytes [ 5] 9.00-10.00 sec 50.0 MBytes 419 Mbits/sec 0 1.66 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 888 MBytes 744 Mbits/sec 150 sender [ 5] 0.00-10.02 sec 886 MBytes 742 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/20/2025-15:07:41.524338 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40344 -> 40.0.0.2:5001 06/20/2025-15:07:41.525427 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40350 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 823 0 --:--:-- --:--:-- --:--:-- 826
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.566 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.566/0.566/0.566/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.537 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.537/0.537/0.537/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 37542 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 140 MBytes 1.18 Gbits/sec 38 1.90 MBytes [ 5] 1.00-2.00 sec 156 MBytes 1.31 Gbits/sec 165 1.45 MBytes [ 5] 2.00-3.00 sec 294 MBytes 2.46 Gbits/sec 0 1.58 MBytes [ 5] 3.00-4.00 sec 286 MBytes 2.40 Gbits/sec 0 1.70 MBytes [ 5] 4.00-5.00 sec 259 MBytes 2.17 Gbits/sec 223 1.27 MBytes [ 5] 5.00-6.00 sec 279 MBytes 2.34 Gbits/sec 0 1.42 MBytes [ 5] 6.00-7.00 sec 292 MBytes 2.45 Gbits/sec 0 1.56 MBytes [ 5] 7.00-8.00 sec 305 MBytes 2.56 Gbits/sec 0 1.70 MBytes [ 5] 8.00-9.00 sec 291 MBytes 2.44 Gbits/sec 0 1.82 MBytes [ 5] 9.00-10.00 sec 300 MBytes 2.52 Gbits/sec 94 1.43 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.54 GBytes 2.18 Gbits/sec 520 sender [ 5] 0.00-10.00 sec 2.54 GBytes 2.18 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/20/2025-15:08:18.208626 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37534 -> 40.0.0.2:5001 06/20/2025-15:08:18.209645 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37542 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jun 20 15:08:09.379798 osdx systemd-journald[210303]: Runtime Journal (/run/log/journal/a9538f26f6924eb58d28105803b55de8) is 1.9M, max 15.3M, 13.3M free. Jun 20 15:08:09.383024 osdx systemd-journald[210303]: Received client request to rotate journal, rotating. Jun 20 15:08:09.383099 osdx systemd-journald[210303]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a9538f26f6924eb58d28105803b55de8. Jun 20 15:08:09.384522 osdx sudo[349803]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 20 15:08:09.392222 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'system journal clear'. Jun 20 15:08:09.610164 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'system coredump delete all'. Jun 20 15:08:09.874143 osdx OSDxCLI[349374]: User 'admin' entered the configuration menu. Jun 20 15:08:09.961083 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jun 20 15:08:10.045010 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jun 20 15:08:10.107972 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jun 20 15:08:10.199606 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jun 20 15:08:10.268060 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jun 20 15:08:10.356065 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jun 20 15:08:10.437672 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'show working'. Jun 20 15:08:10.540970 osdx ubnt-cfgd[349832]: inactive Jun 20 15:08:10.574869 osdx INFO[349842]: FRR daemons did not change Jun 20 15:08:10.595022 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jun 20 15:08:10.667030 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jun 20 15:08:10.685653 osdx (udev-worker)[349964]: Network interface NamePolicy= disabled on kernel command line. Jun 20 15:08:10.727306 osdx (udev-worker)[349970]: Network interface NamePolicy= disabled on kernel command line. Jun 20 15:08:10.832591 osdx cfgd[1460]: [349374]Completed change to active configuration Jun 20 15:08:10.950044 osdx OSDxCLI[349374]: User 'admin' committed the configuration. Jun 20 15:08:10.970400 osdx OSDxCLI[349374]: User 'admin' left the configuration menu. Jun 20 15:08:13.370304 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jun 20 15:08:13.447746 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 20 15:08:13.565117 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 20 15:08:14.110189 osdx sudo[350120]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 20 15:08:14.170668 osdx file_operation[350123]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jun 20 15:08:14.361319 osdx sudo[350130]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 20 15:08:14.366582 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jun 20 15:08:14.593515 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jun 20 15:08:14.751897 osdx OSDxCLI[349374]: User 'admin' entered the configuration menu. Jun 20 15:08:14.824225 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 20 15:08:14.954253 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 20 15:08:15.023771 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 20 15:08:15.133180 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 20 15:08:15.189610 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 20 15:08:15.337934 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 20 15:08:15.443677 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 20 15:08:15.525905 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 20 15:08:15.596405 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 20 15:08:15.698225 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 20 15:08:15.759245 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 20 15:08:15.862977 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 20 15:08:16.015876 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 20 15:08:16.158663 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 20 15:08:16.359282 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 20 15:08:16.463361 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 20 15:08:16.523686 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 20 15:08:16.644406 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 20 15:08:16.704526 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 20 15:08:16.801931 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 20 15:08:16.914922 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'show working'. Jun 20 15:08:17.015170 osdx ubnt-cfgd[350174]: inactive Jun 20 15:08:17.138175 osdx INFO[350211]: FRR daemons did not change Jun 20 15:08:17.403539 osdx systemd[1]: Reloading. Jun 20 15:08:17.443024 osdx systemd-sysv-generator[350264]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jun 20 15:08:17.567340 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 20 15:08:17.584913 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 20 15:08:17.789393 osdx INFO[350246]: Rules successfully loaded Jun 20 15:08:17.789936 osdx cfgd[1460]: [349374]Completed change to active configuration Jun 20 15:08:17.791909 osdx OSDxCLI[349374]: User 'admin' committed the configuration. Jun 20 15:08:17.808304 osdx OSDxCLI[349374]: User 'admin' left the configuration menu. Jun 20 15:08:18.211200 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=11648 DF PROTO=TCP SPT=37534 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 20 15:08:18.211281 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=12857 DF PROTO=TCP SPT=37542 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 20 15:08:28.380540 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 47638 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 300 MBytes 2.52 Gbits/sec 1 1.95 MBytes [ 5] 1.00-2.00 sec 311 MBytes 2.61 Gbits/sec 9 1.48 MBytes [ 5] 2.00-3.00 sec 271 MBytes 2.28 Gbits/sec 0 1.61 MBytes [ 5] 3.00-4.00 sec 292 MBytes 2.45 Gbits/sec 0 1.74 MBytes [ 5] 4.00-5.00 sec 270 MBytes 2.26 Gbits/sec 0 1.85 MBytes [ 5] 5.00-6.00 sec 268 MBytes 2.24 Gbits/sec 0 1.95 MBytes [ 5] 6.00-7.00 sec 301 MBytes 2.53 Gbits/sec 4 1.52 MBytes [ 5] 7.00-8.00 sec 269 MBytes 2.25 Gbits/sec 39 1.19 MBytes [ 5] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 0 1.35 MBytes [ 5] 9.00-10.00 sec 284 MBytes 2.38 Gbits/sec 0 1.49 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.78 GBytes 2.39 Gbits/sec 53 sender [ 5] 0.00-10.00 sec 2.78 GBytes 2.39 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/20/2025-15:08:18.208626 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37534 -> 40.0.0.2:5001 06/20/2025-15:08:18.209645 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:37542 -> 40.0.0.2:5001 06/20/2025-15:08:33.075003 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47624 -> 40.0.0.2:5001 06/20/2025-15:08:33.076028 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:47638 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Jun 20 15:08:09.379798 osdx systemd-journald[210303]: Runtime Journal (/run/log/journal/a9538f26f6924eb58d28105803b55de8) is 1.9M, max 15.3M, 13.3M free. Jun 20 15:08:09.383024 osdx systemd-journald[210303]: Received client request to rotate journal, rotating. Jun 20 15:08:09.383099 osdx systemd-journald[210303]: Vacuuming done, freed 0B of archived journals from /run/log/journal/a9538f26f6924eb58d28105803b55de8. Jun 20 15:08:09.384522 osdx sudo[349803]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 20 15:08:09.392222 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'system journal clear'. Jun 20 15:08:09.610164 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'system coredump delete all'. Jun 20 15:08:09.874143 osdx OSDxCLI[349374]: User 'admin' entered the configuration menu. Jun 20 15:08:09.961083 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Jun 20 15:08:10.045010 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Jun 20 15:08:10.107972 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Jun 20 15:08:10.199606 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Jun 20 15:08:10.268060 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Jun 20 15:08:10.356065 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Jun 20 15:08:10.437672 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'show working'. Jun 20 15:08:10.540970 osdx ubnt-cfgd[349832]: inactive Jun 20 15:08:10.574869 osdx INFO[349842]: FRR daemons did not change Jun 20 15:08:10.595022 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Jun 20 15:08:10.667030 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Jun 20 15:08:10.685653 osdx (udev-worker)[349964]: Network interface NamePolicy= disabled on kernel command line. Jun 20 15:08:10.727306 osdx (udev-worker)[349970]: Network interface NamePolicy= disabled on kernel command line. Jun 20 15:08:10.832591 osdx cfgd[1460]: [349374]Completed change to active configuration Jun 20 15:08:10.950044 osdx OSDxCLI[349374]: User 'admin' committed the configuration. Jun 20 15:08:10.970400 osdx OSDxCLI[349374]: User 'admin' left the configuration menu. Jun 20 15:08:13.370304 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Jun 20 15:08:13.447746 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Jun 20 15:08:13.565117 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Jun 20 15:08:14.110189 osdx sudo[350120]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 20 15:08:14.170668 osdx file_operation[350123]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Jun 20 15:08:14.361319 osdx sudo[350130]: pam_limits(sudo:session): invalid line '@200:215 hard maxlogins ' - skipped Jun 20 15:08:14.366582 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Jun 20 15:08:14.593515 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Jun 20 15:08:14.751897 osdx OSDxCLI[349374]: User 'admin' entered the configuration menu. Jun 20 15:08:14.824225 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 20 15:08:14.954253 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 20 15:08:15.023771 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 20 15:08:15.133180 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 20 15:08:15.189610 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 20 15:08:15.337934 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 20 15:08:15.443677 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 20 15:08:15.525905 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 20 15:08:15.596405 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 20 15:08:15.698225 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 20 15:08:15.759245 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 20 15:08:15.862977 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 20 15:08:16.015876 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 20 15:08:16.158663 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 20 15:08:16.359282 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 20 15:08:16.463361 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 20 15:08:16.523686 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 20 15:08:16.644406 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 20 15:08:16.704526 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 20 15:08:16.801931 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 20 15:08:16.914922 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'show working'. Jun 20 15:08:17.015170 osdx ubnt-cfgd[350174]: inactive Jun 20 15:08:17.138175 osdx INFO[350211]: FRR daemons did not change Jun 20 15:08:17.403539 osdx systemd[1]: Reloading. Jun 20 15:08:17.443024 osdx systemd-sysv-generator[350264]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Jun 20 15:08:17.567340 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 20 15:08:17.584913 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 20 15:08:17.789393 osdx INFO[350246]: Rules successfully loaded Jun 20 15:08:17.789936 osdx cfgd[1460]: [349374]Completed change to active configuration Jun 20 15:08:17.791909 osdx OSDxCLI[349374]: User 'admin' committed the configuration. Jun 20 15:08:17.808304 osdx OSDxCLI[349374]: User 'admin' left the configuration menu. Jun 20 15:08:18.211200 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=11648 DF PROTO=TCP SPT=37534 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 20 15:08:18.211281 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=12857 DF PROTO=TCP SPT=37542 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Jun 20 15:08:28.380540 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Jun 20 15:08:28.479078 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'system journal show | cat'. Jun 20 15:08:28.662569 osdx OSDxCLI[349374]: User 'admin' entered the configuration menu. Jun 20 15:08:28.724495 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Jun 20 15:08:28.848881 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Jun 20 15:08:28.907450 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Jun 20 15:08:29.009373 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Jun 20 15:08:29.074645 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Jun 20 15:08:29.186749 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Jun 20 15:08:29.274370 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Jun 20 15:08:29.361340 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Jun 20 15:08:29.417467 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Jun 20 15:08:29.520812 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Jun 20 15:08:29.575587 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Jun 20 15:08:29.676436 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Jun 20 15:08:29.735795 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Jun 20 15:08:29.865616 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Jun 20 15:08:29.966833 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Jun 20 15:08:30.060240 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Jun 20 15:08:30.117334 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Jun 20 15:08:30.218446 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Jun 20 15:08:30.295402 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Jun 20 15:08:30.378702 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Jun 20 15:08:30.438304 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Jun 20 15:08:30.535621 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Jun 20 15:08:30.591830 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Jun 20 15:08:30.694538 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Jun 20 15:08:30.773560 osdx OSDxCLI[349374]: User 'admin' added a new cfg line: 'show changes'. Jun 20 15:08:30.882303 osdx ubnt-cfgd[350390]: inactive Jun 20 15:08:30.922140 osdx INFO[350406]: FRR daemons did not change Jun 20 15:08:31.169110 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Jun 20 15:08:32.556554 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Jun 20 15:08:32.556661 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Jun 20 15:08:32.556692 osdx systemd[1]: suricata@FW.service: Consumed 1.460s CPU time. Jun 20 15:08:32.580114 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Jun 20 15:08:32.603333 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Jun 20 15:08:32.791042 osdx INFO[350431]: Rules successfully loaded Jun 20 15:08:32.791904 osdx cfgd[1460]: [349374]Completed change to active configuration Jun 20 15:08:32.794487 osdx OSDxCLI[349374]: User 'admin' committed the configuration. Jun 20 15:08:32.834283 osdx OSDxCLI[349374]: User 'admin' left the configuration menu. Jun 20 15:08:33.079351 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=51733 DF PROTO=TCP SPT=47624 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jun 20 15:08:33.079429 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=616 DF PROTO=TCP SPT=47638 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Jun 20 15:08:39.031436 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Jun 20 15:08:43.365040 osdx OSDxCLI[349374]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 39716 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.651 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.651/0.651/0.651/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.573 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.573/0.573/0.573/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 36396 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 119 MBytes 1.00 Gbits/sec 28 1.76 MBytes [ 5] 1.00-2.00 sec 115 MBytes 965 Mbits/sec 32 1.34 MBytes [ 5] 2.00-3.00 sec 276 MBytes 2.32 Gbits/sec 0 1.46 MBytes [ 5] 3.00-4.00 sec 292 MBytes 2.45 Gbits/sec 0 1.60 MBytes [ 5] 4.00-5.00 sec 302 MBytes 2.54 Gbits/sec 0 1.73 MBytes [ 5] 5.00-6.00 sec 309 MBytes 2.59 Gbits/sec 0 1.86 MBytes [ 5] 6.00-7.00 sec 282 MBytes 2.37 Gbits/sec 0 1.97 MBytes [ 5] 7.00-8.00 sec 305 MBytes 2.56 Gbits/sec 411 1.52 MBytes [ 5] 8.00-9.00 sec 305 MBytes 2.56 Gbits/sec 0 1.65 MBytes [ 5] 9.00-10.00 sec 305 MBytes 2.56 Gbits/sec 0 1.77 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.55 GBytes 2.19 Gbits/sec 471 sender [ 5] 0.00-10.00 sec 2.55 GBytes 2.19 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/20/2025-15:09:08.963239 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36392 -> 40.0.0.2:5001 06/20/2025-15:09:08.964276 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36396 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=36392 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=36392 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=36396 dport=5001 packets=1891522 bytes=2837269421 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=36396 packets=294416 bytes=15298732 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=409 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=409 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=70 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=70 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 50062 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 276 MBytes 2.32 Gbits/sec 127 1.36 MBytes [ 5] 1.00-2.00 sec 292 MBytes 2.45 Gbits/sec 0 1.51 MBytes [ 5] 2.00-3.00 sec 299 MBytes 2.51 Gbits/sec 0 1.65 MBytes [ 5] 3.00-4.00 sec 311 MBytes 2.61 Gbits/sec 0 1.78 MBytes [ 5] 4.00-5.00 sec 312 MBytes 2.62 Gbits/sec 0 1.91 MBytes [ 5] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 185 1.44 MBytes [ 5] 6.00-7.00 sec 269 MBytes 2.25 Gbits/sec 0 1.59 MBytes [ 5] 7.00-8.00 sec 258 MBytes 2.16 Gbits/sec 0 1.70 MBytes [ 5] 8.00-9.00 sec 294 MBytes 2.46 Gbits/sec 0 1.79 MBytes [ 5] 9.00-10.00 sec 311 MBytes 2.61 Gbits/sec 0 1.91 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.83 GBytes 2.43 Gbits/sec 312 sender [ 5] 0.00-10.01 sec 2.83 GBytes 2.43 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/20/2025-15:09:08.963239 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36392 -> 40.0.0.2:5001 06/20/2025-15:09:08.964276 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:36396 -> 40.0.0.2:5001 06/20/2025-15:09:23.739440 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50050 -> 40.0.0.2:5001 06/20/2025-15:09:23.740463 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:50062 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=50050 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50050 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=50062 dport=5001 packets=2101886 bytes=3152820357 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=50062 packets=323062 bytes=16802736 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 655 0 --:--:-- --:--:-- --:--:-- 655
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.545 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.545/0.545/0.545/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.547 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.547/0.547/0.547/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/20/2025-15:09:56.888506 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57752 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=73 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=73 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=57752 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=57752 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=412 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=412 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/20/2025-15:09:56.888506 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57752 -> 40.0.0.2:5000 06/20/2025-15:10:03.352583 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57752 -> 40.0.0.2:5000 06/20/2025-15:10:04.849963 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:35574 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=35574 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=35574 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 23708 0 --:--:-- --:--:-- --:--:-- 25800
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.549 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.549/0.549/0.549/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.444 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.444/0.444/0.444/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
06/20/2025-15:10:28.327243 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57482 -> 40.0.0.2:5001 06/20/2025-15:10:28.328485 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57486 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=57482 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=57482 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=415 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=415 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=76 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=76 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=57486 dport=5001 packets=24049 bytes=36067753 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=57486 packets=3141 bytes=163340 [ASSURED] [OFFLOAD, packets=24036 bytes=36052552 packets=3139 bytes=163228] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 1135 0 --:--:-- --:--:-- --:--:-- 1136
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
06/20/2025-15:10:28.327243 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57482 -> 40.0.0.2:5001 06/20/2025-15:10:28.328485 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57486 -> 40.0.0.2:5001 06/20/2025-15:10:31.489326 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:53832 -> 40.0.0.2:5001 06/20/2025-15:10:31.490636 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:59302 -> 40.0.0.2:5001 06/20/2025-15:10:31.501730 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:59302 -> 40.0.0.2:5001 06/20/2025-15:10:31.513727 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:59302 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
udp 17 src=20.0.0.2 dst=40.0.0.2 sport=59302 dport=5001 packets=13 bytes=17744 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=59302 packets=1 bytes=32 [OFFLOAD, packets=9 bytes=13284 packets=0 bytes=0] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=53832 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=53832 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 36757 0 --:--:-- --:--:-- --:--:-- 40000
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.707 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.707/0.707/0.707/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.576 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.576/0.576/0.576/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
06/20/2025-15:10:51.631655 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40346 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 40346 5000 yes 201 0 8 673 40.0.0.2 20.0.0.2 5000 40346 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250364s ending at 1750432255.383416 XDP_DROP 9 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 14 pkts ( 4 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
06/20/2025-15:10:51.631655 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:40346 -> 40.0.0.2:5000 06/20/2025-15:10:55.540593 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:42846 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 40346 5000 yes 201 0 11 847 20.0.0.2 40.0.0.2 42846 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 40346 yes 201 0 0 0 40.0.0.2 20.0.0.2 5001 42846 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250286s ending at 1750432259.269449 XDP_DROP 11 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 34 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)