Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 51210 0 --:--:-- --:--:-- --:--:-- 64500
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.554 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.554/0.554/0.554/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.520 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.520/0.520/0.520/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 49266 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 90.2 MBytes 757 Mbits/sec 135 1.84 MBytes [ 5] 1.00-2.00 sec 106 MBytes 891 Mbits/sec 0 2.02 MBytes [ 5] 2.00-3.00 sec 104 MBytes 870 Mbits/sec 0 2.16 MBytes [ 5] 3.00-4.00 sec 96.2 MBytes 807 Mbits/sec 4 1.61 MBytes [ 5] 4.00-5.00 sec 90.0 MBytes 755 Mbits/sec 0 1.69 MBytes [ 5] 5.00-6.00 sec 90.0 MBytes 755 Mbits/sec 0 1.76 MBytes [ 5] 6.00-7.00 sec 88.8 MBytes 744 Mbits/sec 0 1.80 MBytes [ 5] 7.00-8.00 sec 90.0 MBytes 755 Mbits/sec 284 1.32 MBytes [ 5] 8.00-9.00 sec 53.8 MBytes 451 Mbits/sec 0 1.39 MBytes [ 5] 9.00-10.00 sec 48.8 MBytes 409 Mbits/sec 0 1.45 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 858 MBytes 720 Mbits/sec 423 sender [ 5] 0.00-10.01 sec 856 MBytes 717 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/10/2025-18:34:35.613808 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49260 -> 40.0.0.2:5001 04/10/2025-18:34:35.615013 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49266 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
The test is extended by using other packet marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 268 0 --:--:-- --:--:-- --:--:-- 268
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.553 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.553/0.553/0.553/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.539 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.539/0.539/0.539/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 48732 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 103 MBytes 864 Mbits/sec 197 1.61 MBytes [ 5] 1.00-2.00 sec 119 MBytes 996 Mbits/sec 17 1.23 MBytes [ 5] 2.00-3.00 sec 169 MBytes 1.42 Gbits/sec 0 1.32 MBytes [ 5] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec 0 1.46 MBytes [ 5] 4.00-5.00 sec 294 MBytes 2.46 Gbits/sec 0 1.60 MBytes [ 5] 5.00-6.00 sec 288 MBytes 2.41 Gbits/sec 0 1.72 MBytes [ 5] 6.00-7.00 sec 260 MBytes 2.18 Gbits/sec 0 1.83 MBytes [ 5] 7.00-8.00 sec 252 MBytes 2.12 Gbits/sec 0 1.93 MBytes [ 5] 8.00-9.00 sec 264 MBytes 2.21 Gbits/sec 0 2.02 MBytes [ 5] 9.00-10.00 sec 285 MBytes 2.39 Gbits/sec 15 1.57 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.26 GBytes 1.94 Gbits/sec 229 sender [ 5] 0.00-10.00 sec 2.26 GBytes 1.94 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/10/2025-18:35:13.348386 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48720 -> 40.0.0.2:5001 04/10/2025-18:35:13.349491 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48732 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Apr 10 18:35:02.394194 osdx systemd-journald[55338]: Runtime Journal (/run/log/journal/5c505a9749274d37b6c4605b7d8c5dbf) is 1.9M, max 15.3M, 13.4M free. Apr 10 18:35:02.398030 osdx systemd-journald[55338]: Received client request to rotate journal, rotating. Apr 10 18:35:02.398109 osdx systemd-journald[55338]: Vacuuming done, freed 0B of archived journals from /run/log/journal/5c505a9749274d37b6c4605b7d8c5dbf. Apr 10 18:35:02.406120 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'system journal clear'. Apr 10 18:35:02.775828 osdx osdx-coredump[182090]: Deleting all coredumps in /opt/vyatta/etc/config/coredump... Apr 10 18:35:02.783499 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'system coredump delete all'. Apr 10 18:35:03.315734 osdx OSDxCLI[181647]: User 'admin' entered the configuration menu. Apr 10 18:35:03.397264 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Apr 10 18:35:03.485981 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Apr 10 18:35:03.541332 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Apr 10 18:35:03.640719 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Apr 10 18:35:03.699653 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Apr 10 18:35:03.795373 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Apr 10 18:35:03.873322 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'show working'. Apr 10 18:35:03.969692 osdx ubnt-cfgd[182112]: inactive Apr 10 18:35:05.000182 osdx INFO[182124]: FRR daemons did not change Apr 10 18:35:05.026034 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Apr 10 18:35:05.106034 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Apr 10 18:35:05.132425 osdx (udev-worker)[182247]: Network interface NamePolicy= disabled on kernel command line. Apr 10 18:35:05.275876 osdx cfgd[1682]: [181647]Completed change to active configuration Apr 10 18:35:05.287497 osdx OSDxCLI[181647]: User 'admin' committed the configuration. Apr 10 18:35:05.304660 osdx OSDxCLI[181647]: User 'admin' left the configuration menu. Apr 10 18:35:08.255844 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Apr 10 18:35:08.338140 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Apr 10 18:35:08.422197 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Apr 10 18:35:08.905893 osdx file_operation[182403]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Apr 10 18:35:09.410376 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Apr 10 18:35:09.570763 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Apr 10 18:35:09.794224 osdx OSDxCLI[181647]: User 'admin' entered the configuration menu. Apr 10 18:35:09.869175 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 10 18:35:09.983896 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 10 18:35:10.040213 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 10 18:35:10.167653 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 10 18:35:10.236700 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 10 18:35:10.354935 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 10 18:35:10.451701 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 10 18:35:10.546277 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 10 18:35:10.606994 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 10 18:35:10.700545 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 10 18:35:10.758566 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 10 18:35:10.886110 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 10 18:35:10.953853 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 10 18:35:11.056974 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 10 18:35:11.129541 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 10 18:35:11.245483 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 10 18:35:11.329291 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 10 18:35:11.413967 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 10 18:35:11.503378 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 10 18:35:11.593536 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 10 18:35:11.759378 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'show working'. Apr 10 18:35:11.856358 osdx ubnt-cfgd[182454]: inactive Apr 10 18:35:12.168748 osdx INFO[182491]: FRR daemons did not change Apr 10 18:35:12.425901 osdx systemd[1]: Reloading. Apr 10 18:35:12.526023 osdx systemd-sysv-generator[182544]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Apr 10 18:35:12.646378 osdx systemd[1]: Starting logrotate.service - Rotate log files... Apr 10 18:35:12.649437 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 10 18:35:12.669518 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 10 18:35:12.670849 osdx systemd[1]: logrotate.service: Deactivated successfully. Apr 10 18:35:12.671018 osdx systemd[1]: Finished logrotate.service - Rotate log files. Apr 10 18:35:12.859583 osdx INFO[182526]: Rules successfully loaded Apr 10 18:35:12.860339 osdx cfgd[1682]: [181647]Completed change to active configuration Apr 10 18:35:12.862197 osdx OSDxCLI[181647]: User 'admin' committed the configuration. Apr 10 18:35:12.888857 osdx OSDxCLI[181647]: User 'admin' left the configuration menu. Apr 10 18:35:13.350104 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=59525 DF PROTO=TCP SPT=48720 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 10 18:35:13.350198 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=55199 DF PROTO=TCP SPT=48732 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 10 18:35:23.783692 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Note
The following steps are just a reiteration of the previous test, but with the difference that the packet mark is an extra mark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 1 mask 3294967295 set service firewall FW stream bypass extra-mark 1 value 3294967295 set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 49908 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 211 MBytes 1.77 Gbits/sec 14 1.81 MBytes [ 5] 1.00-2.00 sec 252 MBytes 2.12 Gbits/sec 0 1.97 MBytes [ 5] 2.00-3.00 sec 211 MBytes 1.77 Gbits/sec 14 1.50 MBytes [ 5] 3.00-4.00 sec 155 MBytes 1.30 Gbits/sec 79 783 KBytes [ 5] 4.00-5.00 sec 259 MBytes 2.17 Gbits/sec 0 1003 KBytes [ 5] 5.00-6.00 sec 271 MBytes 2.28 Gbits/sec 0 1.17 MBytes [ 5] 6.00-7.00 sec 300 MBytes 2.52 Gbits/sec 0 1.34 MBytes [ 5] 7.00-8.00 sec 318 MBytes 2.66 Gbits/sec 0 1.50 MBytes [ 5] 8.00-9.00 sec 325 MBytes 2.73 Gbits/sec 0 1.65 MBytes [ 5] 9.00-10.00 sec 325 MBytes 2.73 Gbits/sec 0 1.79 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.57 GBytes 2.20 Gbits/sec 107 sender [ 5] 0.00-10.00 sec 2.56 GBytes 2.20 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/10/2025-18:35:13.348386 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48720 -> 40.0.0.2:5001 04/10/2025-18:35:13.349491 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48732 -> 40.0.0.2:5001 04/10/2025-18:35:30.151686 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49892 -> 40.0.0.2:5001 04/10/2025-18:35:30.152748 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49908 -> 40.0.0.2:5001
Step 12: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Apr 10 18:35:02.394194 osdx systemd-journald[55338]: Runtime Journal (/run/log/journal/5c505a9749274d37b6c4605b7d8c5dbf) is 1.9M, max 15.3M, 13.4M free. Apr 10 18:35:02.398030 osdx systemd-journald[55338]: Received client request to rotate journal, rotating. Apr 10 18:35:02.398109 osdx systemd-journald[55338]: Vacuuming done, freed 0B of archived journals from /run/log/journal/5c505a9749274d37b6c4605b7d8c5dbf. Apr 10 18:35:02.406120 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'system journal clear'. Apr 10 18:35:02.775828 osdx osdx-coredump[182090]: Deleting all coredumps in /opt/vyatta/etc/config/coredump... Apr 10 18:35:02.783499 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'system coredump delete all'. Apr 10 18:35:03.315734 osdx OSDxCLI[181647]: User 'admin' entered the configuration menu. Apr 10 18:35:03.397264 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Apr 10 18:35:03.485981 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Apr 10 18:35:03.541332 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Apr 10 18:35:03.640719 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Apr 10 18:35:03.699653 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Apr 10 18:35:03.795373 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Apr 10 18:35:03.873322 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'show working'. Apr 10 18:35:03.969692 osdx ubnt-cfgd[182112]: inactive Apr 10 18:35:05.000182 osdx INFO[182124]: FRR daemons did not change Apr 10 18:35:05.026034 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Apr 10 18:35:05.106034 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Apr 10 18:35:05.132425 osdx (udev-worker)[182247]: Network interface NamePolicy= disabled on kernel command line. Apr 10 18:35:05.275876 osdx cfgd[1682]: [181647]Completed change to active configuration Apr 10 18:35:05.287497 osdx OSDxCLI[181647]: User 'admin' committed the configuration. Apr 10 18:35:05.304660 osdx OSDxCLI[181647]: User 'admin' left the configuration menu. Apr 10 18:35:08.255844 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Apr 10 18:35:08.338140 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Apr 10 18:35:08.422197 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Apr 10 18:35:08.905893 osdx file_operation[182403]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Apr 10 18:35:09.410376 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Apr 10 18:35:09.570763 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Apr 10 18:35:09.794224 osdx OSDxCLI[181647]: User 'admin' entered the configuration menu. Apr 10 18:35:09.869175 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 10 18:35:09.983896 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 10 18:35:10.040213 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 10 18:35:10.167653 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 10 18:35:10.236700 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 10 18:35:10.354935 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 10 18:35:10.451701 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 10 18:35:10.546277 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 10 18:35:10.606994 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 10 18:35:10.700545 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 10 18:35:10.758566 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 10 18:35:10.886110 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 10 18:35:10.953853 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 10 18:35:11.056974 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 10 18:35:11.129541 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 10 18:35:11.245483 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 10 18:35:11.329291 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 10 18:35:11.413967 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 10 18:35:11.503378 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 10 18:35:11.593536 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 10 18:35:11.759378 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'show working'. Apr 10 18:35:11.856358 osdx ubnt-cfgd[182454]: inactive Apr 10 18:35:12.168748 osdx INFO[182491]: FRR daemons did not change Apr 10 18:35:12.425901 osdx systemd[1]: Reloading. Apr 10 18:35:12.526023 osdx systemd-sysv-generator[182544]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Apr 10 18:35:12.646378 osdx systemd[1]: Starting logrotate.service - Rotate log files... Apr 10 18:35:12.649437 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 10 18:35:12.669518 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 10 18:35:12.670849 osdx systemd[1]: logrotate.service: Deactivated successfully. Apr 10 18:35:12.671018 osdx systemd[1]: Finished logrotate.service - Rotate log files. Apr 10 18:35:12.859583 osdx INFO[182526]: Rules successfully loaded Apr 10 18:35:12.860339 osdx cfgd[1682]: [181647]Completed change to active configuration Apr 10 18:35:12.862197 osdx OSDxCLI[181647]: User 'admin' committed the configuration. Apr 10 18:35:12.888857 osdx OSDxCLI[181647]: User 'admin' left the configuration menu. Apr 10 18:35:13.350104 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=59525 DF PROTO=TCP SPT=48720 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 10 18:35:13.350198 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=55199 DF PROTO=TCP SPT=48732 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Apr 10 18:35:23.783692 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'. Apr 10 18:35:23.882709 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'system journal show | cat'. Apr 10 18:35:24.079223 osdx OSDxCLI[181647]: User 'admin' entered the configuration menu. Apr 10 18:35:24.164974 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Apr 10 18:35:24.252522 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Apr 10 18:35:24.351142 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Apr 10 18:35:24.449899 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Apr 10 18:35:24.509146 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Apr 10 18:35:24.621415 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Apr 10 18:35:24.742545 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Apr 10 18:35:24.798347 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Apr 10 18:35:24.905126 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Apr 10 18:35:24.975442 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Apr 10 18:35:25.079399 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Apr 10 18:35:25.153410 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Apr 10 18:35:25.225274 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Apr 10 18:35:25.329346 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Apr 10 18:35:25.386488 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Apr 10 18:35:25.498516 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Apr 10 18:35:25.579550 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Apr 10 18:35:25.682879 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Apr 10 18:35:25.758677 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Apr 10 18:35:25.837932 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Apr 10 18:35:25.897558 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 value 3294967295'. Apr 10 18:35:25.996554 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass extra-mark 1 mask 3294967295'. Apr 10 18:35:26.056656 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector FW_SEL_EXTRA_MARK'. Apr 10 18:35:26.185333 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_EXTRA_MARK rule 1 extra-mark 1 value 3294967295'. Apr 10 18:35:26.270292 osdx OSDxCLI[181647]: User 'admin' added a new cfg line: 'show changes'. Apr 10 18:35:26.382285 osdx ubnt-cfgd[182673]: inactive Apr 10 18:35:27.970863 osdx INFO[182689]: FRR daemons did not change Apr 10 18:35:28.265708 osdx systemd[1]: Stopping suricata@FW.service - Suricata client "FW" service... Apr 10 18:35:29.660217 osdx systemd[1]: suricata@FW.service: Deactivated successfully. Apr 10 18:35:29.660324 osdx systemd[1]: Stopped suricata@FW.service - Suricata client "FW" service. Apr 10 18:35:29.660359 osdx systemd[1]: suricata@FW.service: Consumed 1.487s CPU time. Apr 10 18:35:29.674395 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Apr 10 18:35:29.698291 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Apr 10 18:35:29.882683 osdx INFO[182714]: Rules successfully loaded Apr 10 18:35:29.884391 osdx cfgd[1682]: [181647]Completed change to active configuration Apr 10 18:35:29.886320 osdx OSDxCLI[181647]: User 'admin' committed the configuration. Apr 10 18:35:29.916602 osdx OSDxCLI[181647]: User 'admin' left the configuration menu. Apr 10 18:35:30.154079 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=22250 DF PROTO=TCP SPT=49892 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Apr 10 18:35:30.154153 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:01:de:ad:be:ef:6c:20:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=31482 DF PROTO=TCP SPT=49908 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d EMARK1=0xc46535ff Apr 10 18:35:32.032277 osdx systemd[1]: systemd-timedated.service: Deactivated successfully. Apr 10 18:35:40.331600 osdx OSDxCLI[181647]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then this test is broadened by using other conntrack marks that we have customized for the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 26034 0 --:--:-- --:--:-- --:--:-- 32250
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.576 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.576/0.576/0.576/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.532 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.532/0.532/0.532/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 39806 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 112 MBytes 937 Mbits/sec 47 1.73 MBytes [ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec 50 1.32 MBytes [ 5] 2.00-3.00 sec 210 MBytes 1.76 Gbits/sec 0 1.40 MBytes [ 5] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec 0 1.54 MBytes [ 5] 4.00-5.00 sec 306 MBytes 2.57 Gbits/sec 0 1.68 MBytes [ 5] 5.00-6.00 sec 298 MBytes 2.49 Gbits/sec 0 1.80 MBytes [ 5] 6.00-7.00 sec 329 MBytes 2.76 Gbits/sec 0 1.93 MBytes [ 5] 7.00-8.00 sec 314 MBytes 2.63 Gbits/sec 115 1.50 MBytes [ 5] 8.00-9.00 sec 269 MBytes 2.25 Gbits/sec 0 1.63 MBytes [ 5] 9.00-10.00 sec 264 MBytes 2.21 Gbits/sec 197 1.23 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.44 GBytes 2.09 Gbits/sec 409 sender [ 5] 0.00-10.00 sec 2.43 GBytes 2.09 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/10/2025-18:36:07.021042 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39792 -> 40.0.0.2:5001 04/10/2025-18:36:07.022252 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39806 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=81 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=81 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=39792 dport=5001 packets=16 bytes=1299 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=39792 packets=13 bytes=1020 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=39806 dport=5001 packets=1805887 bytes=2708822881 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=39806 packets=215303 bytes=11192248 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=458 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=458 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Note
The following steps are just a reiteration of the previous test, but with the difference that the conntrack mark used is an extra connmark.
Step 9: Modify the following configuration lines in DUT0 :
set service firewall FW stream bypass extra-mark 2 mask 3294967295 set service firewall FW stream bypass extra-mark 2 set-extra-connmark set service firewall FW stream bypass extra-mark 2 value 3294967295 set traffic policy FW_PLAN rule 2 selector FW_SEL_EXTRA_MARK set traffic selector FW_SEL_EXTRA_MARK rule 1 not extra-connmark 2 value 3294967295
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 55044 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 242 MBytes 2.03 Gbits/sec 29 1.27 MBytes [ 5] 1.00-2.00 sec 331 MBytes 2.78 Gbits/sec 0 1.44 MBytes [ 5] 2.00-3.00 sec 322 MBytes 2.71 Gbits/sec 0 1.60 MBytes [ 5] 3.00-4.00 sec 304 MBytes 2.55 Gbits/sec 0 1.73 MBytes [ 5] 4.00-5.00 sec 281 MBytes 2.36 Gbits/sec 157 1.29 MBytes [ 5] 5.00-6.00 sec 270 MBytes 2.26 Gbits/sec 0 1.44 MBytes [ 5] 6.00-7.00 sec 285 MBytes 2.39 Gbits/sec 0 1.57 MBytes [ 5] 7.00-8.00 sec 289 MBytes 2.42 Gbits/sec 0 1.70 MBytes [ 5] 8.00-9.00 sec 308 MBytes 2.58 Gbits/sec 0 1.82 MBytes [ 5] 9.00-10.00 sec 279 MBytes 2.34 Gbits/sec 109 1.42 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.84 GBytes 2.44 Gbits/sec 295 sender [ 5] 0.00-10.00 sec 2.84 GBytes 2.44 Gbits/sec receiver iperf Done.
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/10/2025-18:36:07.021042 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39792 -> 40.0.0.2:5001 04/10/2025-18:36:07.022252 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:39806 -> 40.0.0.2:5001 04/10/2025-18:36:21.852626 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55042 -> 40.0.0.2:5001 04/10/2025-18:36:21.853591 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:55044 -> 40.0.0.2:5001
Step 12: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*emark2=3294967295.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=55042 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=55042 packets=13 bytes=1018 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=55044 dport=5001 packets=2107842 bytes=3161747413 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=55044 packets=231101 bytes=12009452 [ASSURED] (Sc: not-bypass) mark=129834765 emark2=3294967295 use=1 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Test Bypass-Drop Using Conntrack Marks
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test is aimed at configuring “Capture bypass drop” to avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 1575 0 --:--:-- --:--:-- --:--:-- 1574
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic queue FW_Q elements 1 set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.465 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.465/0.465/0.465/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.482 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.482/0.482/0.482/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/10/2025-18:36:57.195067 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38242 -> 40.0.0.2:5000
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
icmp 1 26 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=84 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=84 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 26 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=461 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=461 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=38242 dport=5000 packets=8 bytes=610 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=38242 packets=4 bytes=217 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 3 flow entries have been shown.
Step 9: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 4 7 210 438 2 - 3 3 228 228 ------------------------------------------------------------------ Total 7 7 438 438 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high ------------------------------------------------------------------ rule selector pkts match pkts eval bytes match bytes eval ------------------------------------------------------------------ 1 FW_SEL_DROP 6 11 499 868 2 - 5 5 369 369 ------------------------------------------------------------------ Total 11 11 868 868
Note
Testing with another conntrack mark.
Step 10: Modify the following configuration lines in DUT0 :
delete service firewall FW bypass action drop set connmark mark set service firewall FW bypass action drop set connmark extra-mark 2 value 3967295294 set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP_EM set traffic selector FW_SEL_DROP_EM rule 1 extra-connmark 2 value 3967295294
Step 11: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 12: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/10/2025-18:36:57.195067 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38242 -> 40.0.0.2:5000 04/10/2025-18:37:03.524110 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:38242 -> 40.0.0.2:5000 04/10/2025-18:37:05.163435 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48686 -> 40.0.0.2:5000
Step 13: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*emark2=3967295294.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=48686 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=48686 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=0 emark2=3967295294 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 14: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP_EM\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 4 7 210 376 2 - 3 3 166 166 --------------------------------------------------------------------- Total 7 7 376 376 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP_EM 6 11 499 830 2 - 5 5 331 331 --------------------------------------------------------------------- Total 11 11 830 830
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 38914 0 --:--:-- --:--:-- --:--:-- 43000
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.610 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.610/0.610/0.610/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.480 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.480/0.480/0.480/0.000 ms
Step 6: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
04/10/2025-18:37:38.712452 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48738 -> 40.0.0.2:5001 04/10/2025-18:37:38.713334 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48744 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=464 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=464 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=48744 dport=5001 packets=75151 bytes=112720753 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48744 packets=10237 bytes=533028 [ASSURED] [OFFLOAD, packets=75141 bytes=112710052 packets=10235 bytes=532916] mark=129834765 use=2 icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=87 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=87 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=48738 dport=5001 packets=7 bytes=537 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48738 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=1 bytes=52 packets=4 bytes=211] mark=129834765 use=3 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Step 9: Stop the current bandwidth test between DUT2 and DUT1
Step 10: Run command file copy http://10.215.168.1/~robot/test-performance-udp.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 133 100 133 0 0 407 0 --:--:-- --:--:-- --:--:-- 407
Step 11: Run command file show running://test-performance-udp.rules at DUT0 and expect this output:
Show output
alert udp any any -> any 5001 (msg: "Skipping test network performance UDP traffic"; bypass; flow: established, to_server; sid: 41;)
Step 12: Modify the following configuration lines in DUT0 :
set service firewall FW ruleset file 'running://test-performance-udp.rules'
Step 13: Initiate a background bandwidth test from DUT2 to DUT1. The control is returned back allowing to perform another tasks while test is running
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 udp port 5001 parallel 1
Step 14: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance UDP traffic).+$Show output
04/10/2025-18:37:38.712452 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48738 -> 40.0.0.2:5001 04/10/2025-18:37:38.713334 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48744 -> 40.0.0.2:5001 04/10/2025-18:37:41.751042 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:48748 -> 40.0.0.2:5001 04/10/2025-18:37:41.752469 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:48917 -> 40.0.0.2:5001 04/10/2025-18:37:41.763237 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:48917 -> 40.0.0.2:5001 04/10/2025-18:37:41.775306 [**] [1:41:0] Skipping test network performance UDP traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:48917 -> 40.0.0.2:5001
Step 15: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^udp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=48748 dport=5001 packets=8 bytes=607 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48748 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=2 bytes=104 packets=4 bytes=211] mark=129834765 use=3 udp 17 src=20.0.0.2 dst=40.0.0.2 sport=48917 dport=5001 packets=12 bytes=16268 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=48917 packets=1 bytes=32 [OFFLOAD, packets=8 bytes=11808 packets=0 bytes=0] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 2 flow entries have been shown.
Step 16: Stop the current bandwidth test between DUT2 and DUT1
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 73260 0 --:--:-- --:--:-- --:--:-- 97k
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0 :
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.543 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.543/0.543/0.543/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.563 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.563/0.563/0.563/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
04/10/2025-18:38:12.142935 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49294 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 49294 5000 yes 201 0 9 731 40.0.0.2 20.0.0.2 5000 49294 yes 201 0 0 0
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250140s ending at 1744310295.899991 XDP_DROP 9 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 13 pkts ( 0 pps) 1 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
04/10/2025-18:38:12.142935 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:49294 -> 40.0.0.2:5000 04/10/2025-18:38:16.060298 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:44568 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 20.0.0.2 40.0.0.2 49294 5000 yes 201 0 12 905 40.0.0.2 20.0.0.2 5001 44568 no 201 0 0 0 20.0.0.2 40.0.0.2 44568 5001 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 49294 yes 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250193s ending at 1744310299.764851 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 33 pkts ( 0 pps) 2 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)