Bypass Tests
The following scenario shows different configuration alternatives to improve the OSDx firewall performance.
Test Local Bypass
Description
Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 109k 0 --:--:-- --:--:-- --:--:-- 125k
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.861 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.861/0.861/0.861/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.696 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.696/0.696/0.696/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 59404 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 22.2 MBytes 187 Mbits/sec 0 970 KBytes [ 5] 1.00-2.00 sec 30.0 MBytes 251 Mbits/sec 0 2.37 MBytes [ 5] 2.00-3.00 sec 30.0 MBytes 252 Mbits/sec 165 1.82 MBytes [ 5] 3.00-4.00 sec 36.2 MBytes 304 Mbits/sec 0 1.98 MBytes [ 5] 4.00-5.00 sec 95.0 MBytes 797 Mbits/sec 247 1.48 MBytes [ 5] 5.00-6.00 sec 97.5 MBytes 818 Mbits/sec 28 1.11 MBytes [ 5] 6.00-7.00 sec 88.8 MBytes 744 Mbits/sec 0 1.18 MBytes [ 5] 7.00-8.00 sec 95.0 MBytes 797 Mbits/sec 0 1.23 MBytes [ 5] 8.00-9.00 sec 97.5 MBytes 818 Mbits/sec 0 1.27 MBytes [ 5] 9.00-10.00 sec 85.0 MBytes 713 Mbits/sec 0 1.32 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 677 MBytes 568 Mbits/sec 440 sender [ 5] 0.00-10.01 sec 675 MBytes 566 Mbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/09/2024-09:51:32.543108 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59394 -> 40.0.0.2:5001 10/09/2024-09:51:32.544983 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:59404 -> 40.0.0.2:5001
Test Capture Bypass Using Packet Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.
Performance must improve considerably compared to the Local Bypass test.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 6305 0 --:--:-- --:--:-- --:--:-- 6450
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic label BYPASS set traffic policy FW-SKIP rule 1 log prefix SKIP set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS set traffic policy FW-SKIP rule 1 set label BYPASS set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS set traffic selector MARKED-PACKETS rule 1 mark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=1.03 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.027/1.027/1.027/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.827 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.827/0.827/0.827/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 57286 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 49.4 MBytes 414 Mbits/sec 73 1.32 MBytes [ 5] 1.00-2.00 sec 130 MBytes 1.09 Gbits/sec 0 1.47 MBytes [ 5] 2.00-3.00 sec 129 MBytes 1.08 Gbits/sec 55 1.11 MBytes [ 5] 3.00-4.00 sec 150 MBytes 1.26 Gbits/sec 0 1.19 MBytes [ 5] 4.00-5.00 sec 170 MBytes 1.43 Gbits/sec 0 1.25 MBytes [ 5] 5.00-6.00 sec 180 MBytes 1.51 Gbits/sec 0 1.31 MBytes [ 5] 6.00-7.00 sec 150 MBytes 1.26 Gbits/sec 0 1.37 MBytes [ 5] 7.00-8.00 sec 168 MBytes 1.40 Gbits/sec 0 1.42 MBytes [ 5] 8.00-9.00 sec 168 MBytes 1.41 Gbits/sec 0 1.47 MBytes [ 5] 9.00-10.00 sec 158 MBytes 1.32 Gbits/sec 91 1.13 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.42 GBytes 1.22 Gbits/sec 219 sender [ 5] 0.00-10.01 sec 1.41 GBytes 1.21 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/09/2024-09:52:13.369652 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57278 -> 40.0.0.2:5001 10/09/2024-09:52:13.371509 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:57286 -> 40.0.0.2:5001
Step 8: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:
(?m)^.*\[SKIP\-1\].*$Show output
Oct 09 09:52:01.000485 osdx systemd-timedated[314001]: Changed local time to Wed 2024-10-09 09:52:01 UTC Oct 09 09:52:01.003037 osdx systemd-journald[1768]: Time jumped backwards, rotating. Oct 09 09:52:01.003761 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'set date 2024-10-09 09:52:01'. Oct 09 09:52:01.440218 osdx systemd-journald[1768]: Runtime Journal (/run/log/journal/da0729972954483f829d339572dde7c1) is 2.1M, max 15.3M, 13.2M free. Oct 09 09:52:01.443041 osdx systemd-journald[1768]: Received client request to rotate journal, rotating. Oct 09 09:52:01.443121 osdx systemd-journald[1768]: Vacuuming done, freed 0B of archived journals from /run/log/journal/da0729972954483f829d339572dde7c1. Oct 09 09:52:01.460649 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'system journal clear'. Oct 09 09:52:02.004463 osdx osdx-coredump[314018]: Deleting all coredumps in /opt/vyatta/etc/config/coredump... Oct 09 09:52:02.014478 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'system coredump delete all'. Oct 09 09:52:02.812478 osdx OSDxCLI[313663]: User 'admin' entered the configuration menu. Oct 09 09:52:02.945413 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'. Oct 09 09:52:03.031501 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'. Oct 09 09:52:03.150081 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'. Oct 09 09:52:03.250243 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'. Oct 09 09:52:03.380373 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'. Oct 09 09:52:03.470131 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'. Oct 09 09:52:03.592854 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'show working'. Oct 09 09:52:03.731093 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0 Oct 09 09:52:03.847045 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1 Oct 09 09:52:03.892465 osdx (udev-worker)[314145]: Network interface NamePolicy= disabled on kernel command line. Oct 09 09:52:04.151950 osdx cfgd[1434]: [313663]Completed change to active configuration Oct 09 09:52:04.188101 osdx OSDxCLI[313663]: User 'admin' committed the configuration. Oct 09 09:52:04.241262 osdx OSDxCLI[313663]: User 'admin' left the configuration menu. Oct 09 09:52:07.298924 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'ping 10.215.168.1 count 1 size 56 timeout 1'. Oct 09 09:52:07.445783 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'ping 40.0.0.2 count 1 size 56 timeout 1'. Oct 09 09:52:07.566964 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'ping 20.0.0.2 count 1 size 56 timeout 1'. Oct 09 09:52:08.223137 osdx file_operation[314229]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running:// Oct 09 09:52:08.271847 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'. Oct 09 09:52:08.497203 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'file show running://test-performance.rules'. Oct 09 09:52:08.669215 osdx OSDxCLI[313663]: User 'admin' entered the configuration menu. Oct 09 09:52:08.770450 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'. Oct 09 09:52:08.889026 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'. Oct 09 09:52:08.984337 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'. Oct 09 09:52:09.075897 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'. Oct 09 09:52:09.173129 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'. Oct 09 09:52:09.288102 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'. Oct 09 09:52:09.409623 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'. Oct 09 09:52:09.490193 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'. Oct 09 09:52:09.582248 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'. Oct 09 09:52:09.680096 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'. Oct 09 09:52:09.797389 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic label BYPASS'. Oct 09 09:52:09.892988 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'. Oct 09 09:52:10.013429 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'. Oct 09 09:52:10.127842 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'. Oct 09 09:52:10.227791 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'. Oct 09 09:52:10.351551 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL_ENQUEUE rule 1 not label BYPASS'. Oct 09 09:52:10.438435 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL_ENQUEUE'. Oct 09 09:52:10.612722 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'. Oct 09 09:52:10.885950 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'. Oct 09 09:52:10.981667 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'. Oct 09 09:52:11.182021 osdx OSDxCLI[313663]: User 'admin' added a new cfg line: 'show working'. Oct 09 09:52:11.906140 osdx systemd[1]: Reloading. Oct 09 09:52:12.127045 osdx systemd-sysv-generator[314355]: stat() failed on /etc/init.d/README, ignoring: No such file or directory Oct 09 09:52:12.303541 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service... Oct 09 09:52:12.339636 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service. Oct 09 09:52:12.735506 osdx INFO[314339]: Rules successfully loaded Oct 09 09:52:12.736957 osdx cfgd[1434]: [313663]Completed change to active configuration Oct 09 09:52:12.740579 osdx OSDxCLI[313663]: User 'admin' committed the configuration. Oct 09 09:52:12.775817 osdx OSDxCLI[313663]: User 'admin' left the configuration menu. Oct 09 09:52:13.371092 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:11:de:ad:be:ef:6c:30:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=56417 DF PROTO=TCP SPT=57278 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 09 09:52:13.375956 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:11:de:ad:be:ef:6c:30:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=34263 DF PROTO=TCP SPT=57286 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d Oct 09 09:52:14.844297 osdx zebra[1399]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Warning: Failed to connect to the agentx master agent ([NIL]): Oct 09 09:52:20.447885 osdx systemd[1]: Starting logrotate.service - Rotate log files... Oct 09 09:52:20.610731 osdx systemd[1]: logrotate.service: Deactivated successfully. Oct 09 09:52:20.610946 osdx systemd[1]: Finished logrotate.service - Rotate log files. Oct 09 09:52:23.672661 osdx OSDxCLI[313663]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.
Test Capture Bypass Using Conntrack Mark
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.
Performance must improve considerably compared to the Local Bypass test.
Then, the test is broaden by configuring “Capture bypass drop” to also avoid dropped packets from entering the firewall.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 16224 0 --:--:-- --:--:-- --:--:-- 18428
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.778 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.778/0.778/0.778/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.805 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.805/0.805/0.805/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1Expect this output in
DUT2:Connecting to host 40.0.0.2, port 5001 [ 5] local 20.0.0.2 port 44286 connected to 40.0.0.2 port 5001 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 65.8 MBytes 552 Mbits/sec 1 2.38 MBytes [ 5] 1.00-2.00 sec 144 MBytes 1.21 Gbits/sec 246 1.46 MBytes [ 5] 2.00-3.00 sec 188 MBytes 1.57 Gbits/sec 0 1.54 MBytes [ 5] 3.00-4.00 sec 172 MBytes 1.44 Gbits/sec 0 1.60 MBytes [ 5] 4.00-5.00 sec 171 MBytes 1.44 Gbits/sec 16 1.17 MBytes [ 5] 5.00-6.00 sec 149 MBytes 1.25 Gbits/sec 0 1.25 MBytes [ 5] 6.00-7.00 sec 166 MBytes 1.40 Gbits/sec 0 1.31 MBytes [ 5] 7.00-8.00 sec 179 MBytes 1.50 Gbits/sec 0 1.35 MBytes [ 5] 8.00-9.00 sec 185 MBytes 1.55 Gbits/sec 0 1.43 MBytes [ 5] 9.00-10.00 sec 185 MBytes 1.55 Gbits/sec 0 1.49 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.57 GBytes 1.35 Gbits/sec 263 sender [ 5] 0.00-10.00 sec 1.56 GBytes 1.34 Gbits/sec receiver iperf Done.
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/09/2024-09:52:53.678756 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44278 -> 40.0.0.2:5001 10/09/2024-09:52:53.680378 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44286 -> 40.0.0.2:5001
Step 8: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$Show output
tcp 6 19 TIME_WAIT src=20.0.0.2 dst=40.0.0.2 sport=44278 dport=5001 packets=16 bytes=1300 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44278 packets=13 bytes=1019 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=388 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=388 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 udp 17 25 src=127.0.0.1 dst=127.0.0.1 sport=39176 dport=53 packets=1 bytes=62 [UNREPLIED] src=127.0.0.1 dst=127.0.0.1 sport=53 dport=39176 packets=0 bytes=0 mark=0 use=1 udp 17 25 src=127.0.0.1 dst=127.0.0.1 sport=58287 dport=53 packets=1 bytes=62 [UNREPLIED] src=127.0.0.1 dst=127.0.0.1 sport=53 dport=58287 packets=0 bytes=0 mark=0 use=1 tcp 6 9 CLOSE src=20.0.0.2 dst=40.0.0.2 sport=44286 dport=5001 packets=1160967 bytes=1741440833 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=44286 packets=28249 bytes=1462752 [ASSURED] (Sc: not-bypass) mark=129834765 use=1 icmp 1 19 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=36 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=36 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 6 flow entries have been shown.
Step 9: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 4768 0 --:--:-- --:--:-- --:--:-- 4878
Step 10: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 11: Modify the following configuration lines in DUT0:
set service firewall FW bypass action drop set connmark mark 147652983 set service firewall FW ruleset file 'running://drop-performance.rules' set traffic policy FW_PLAN rule 1 action drop set traffic policy FW_PLAN rule 1 selector FW_SEL_DROP set traffic selector FW_SEL_DROP rule 1 connmark 147652983
Step 12: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 13: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/09/2024-09:52:53.678756 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44278 -> 40.0.0.2:5001 10/09/2024-09:52:53.680378 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:44286 -> 40.0.0.2:5001 10/09/2024-09:53:08.699853 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:41968 -> 40.0.0.2:5000
Step 14: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5000.*mark=147652983.*$Show output
tcp 6 29 LAST_ACK src=20.0.0.2 dst=40.0.0.2 sport=41968 dport=5000 packets=9 bytes=700 src=40.0.0.2 dst=20.0.0.2 sport=5000 dport=41968 packets=5 bytes=270 [ASSURED] (Sc: not-bypass) mark=147652983 use=1 conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.
Step 15: Run command traffic policy FW_PLAN show at DUT0 and check if output matches the following regular expressions:
(?m)^1\s+FW_SEL_DROP\s+[1-9].*$Show output
Policy FW_PLAN -- ifc eth1.101 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP 4 5 210 270 2 FW_SEL_ENQUEUE 1 1 60 60 --------------------------------------------------------------------- Total 5 5 270 270 Policy FW_PLAN -- ifc eth1.201 -- hook in prio very-high --------------------------------------------------------------------- rule selector pkts match pkts eval bytes match bytes eval --------------------------------------------------------------------- 1 FW_SEL_DROP 6 9 499 700 2 FW_SEL_ENQUEUE 3 3 201 201 --------------------------------------------------------------------- Total 9 9 700 700
Test Capture And Offload
Description
Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later. In addition, OSDx is instructed to accelerate the flow using internal accelerators.
Performance must improve considerably compared to the previous test, to reach its top value.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 15951 0 --:--:-- --:--:-- --:--:-- 16125
Step 2: Run command file show running://test-performance.rules at DUT0 and expect this output:
Show output
alert tcp any any -> any 5001 (msg: "Skipping test network performance traffic"; bypass; flow: established, to_server; sid: 40;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://test-performance.rules' set service firewall FW stream bypass action accept set conntrack offload-flag set service firewall FW stream bypass mark 129834765 set service firewall FW stream bypass mask 129834765 set service firewall FW stream bypass set-connmark set service firewall FW validator-timeout 20 set system conntrack timeout tcp established 60 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set system offload timeout 60 set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic policy FW_PLAN rule 2 action enqueue FW_Q set traffic policy FW_PLAN rule 2 selector FW_SEL_ENQUEUE set traffic queue FW_Q elements 1 set traffic selector FW_SEL_ENQUEUE rule 1 not connmark 129834765
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.799 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.799/0.799/0.799/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.829 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.829/0.829/0.829/0.000 ms
Step 6: Run command monitor test performance server port 5001 at DUT1.
Step 7: Run command monitor test performance client 40.0.0.2 duration 10 port 5001 at DUT2.
Step 8: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Skipping test network performance traffic).+$Show output
10/09/2024-09:53:40.536135 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58942 -> 40.0.0.2:5001 10/09/2024-09:53:40.538345 [**] [1:40:0] Skipping test network performance traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:58952 -> 40.0.0.2:5001
Step 9: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:
(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.+OFFLOAD.+mark=129834765.*$Show output
icmp 1 29 src=20.0.0.2 dst=40.0.0.2 type=8 code=0 id=39 packets=1 bytes=84 src=40.0.0.2 dst=20.0.0.2 type=0 code=0 id=39 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 icmp 1 29 src=40.0.0.2 dst=20.0.0.2 type=8 code=0 id=391 packets=1 bytes=84 src=20.0.0.2 dst=40.0.0.2 type=0 code=0 id=391 packets=1 bytes=84 (Sc: not-bypass) mark=0 use=1 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=58952 dport=5001 packets=4798 bytes=7191253 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=58952 packets=945 bytes=49244 [ASSURED] [OFFLOAD, packets=4786 bytes=7177552 packets=943 bytes=49132] mark=129834765 use=2 tcp 6 src=20.0.0.2 dst=40.0.0.2 sport=58942 dport=5001 packets=8 bytes=589 src=40.0.0.2 dst=20.0.0.2 sport=5001 dport=58942 packets=7 bytes=376 [ASSURED] [OFFLOAD, packets=5 bytes=388 packets=4 bytes=211] mark=129834765 use=2 conntrack v1.4.7 (conntrack-tools): 4 flow entries have been shown.
Test Traffic Early Dropping
Description
Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.
The contents of the rule file are:
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)
This rule allows the connection to be established and traffic to be dropped later.
Scenario
Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:
Show output
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 200 100 200 0 0 25290 0 --:--:-- --:--:-- --:--:-- 28571
Step 2: Run command file show running://drop-performance.rules at DUT0 and expect this output:
Show output
drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;) drop udp any any -> any 5001 (msg: "Dropping UDP performance test traffic"; sid: 2;)
Step 3: Set the following configuration in DUT0:
set interfaces ethernet eth0 address 10.215.168.64/24 set interfaces ethernet eth0 traffic nat source rule 1 address masquerade set interfaces ethernet eth1 vif 101 address 40.0.0.1/8 set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN set interfaces ethernet eth1 vif 201 address 20.0.0.1/8 set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN set protocols static route 0.0.0.0/0 next-hop 10.215.168.1 set service dns static host-name WAN inet 10.215.168.1 set service firewall FW logging level config set service firewall FW logging outputs fast set service firewall FW mode inline queue FW_Q set service firewall FW ruleset file 'running://drop-performance.rules' set service firewall FW stream bypass action drop set xdp-early-drop eth1 set service firewall FW validator-timeout 20 set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0' set traffic policy FW_PLAN rule 1 action enqueue FW_Q set traffic queue FW_Q elements 1
Step 4: Ping IP address 20.0.0.2 from DUT1:
admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1Show output
PING 20.0.0.2 (20.0.0.2) 56(84) bytes of data. 64 bytes from 20.0.0.2: icmp_seq=1 ttl=63 time=0.908 ms --- 20.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.908/0.908/0.908/0.000 ms
Step 5: Ping IP address 40.0.0.2 from DUT2:
admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1Show output
PING 40.0.0.2 (40.0.0.2) 56(84) bytes of data. 64 bytes from 40.0.0.2: icmp_seq=1 ttl=63 time=0.660 ms --- 40.0.0.2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.660/0.660/0.660/0.000 ms
Step 6: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5000 admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr iperf3: interrupt - the client has terminated admin@osdx$
Step 7: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping TCP performance test traffic).+$Show output
10/09/2024-09:54:07.112023 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34274 -> 40.0.0.2:5000
Step 8: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5000 34274 yes 201 0 0 0 20.0.0.2 40.0.0.2 34274 5000 yes 201 0 9 731
Step 9: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250420s ending at 1728467650.967842 XDP_DROP 10 pkts ( 4 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 2798265 pkts ( 8 pps) 4021349 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)
Step 10: Initiate a bandwidth test from DUT2 to DUT1
admin@DUT1$ monitor test performance server port 5001 admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1Expect this output in
DUT2:^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams iperf3: interrupt - the client has terminated admin@osdx$
Step 11: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:
(?m)^.+(Dropping UDP performance test traffic).+$Show output
10/09/2024-09:54:07.112023 [Drop] [**] [1:1:0] Dropping TCP performance test traffic [**] [Classification: (null)] [Priority: 3] {TCP} 20.0.0.2:34274 -> 40.0.0.2:5000 10/09/2024-09:54:11.253508 [Drop] [**] [1:2:0] Dropping UDP performance test traffic [**] [Classification: (null)] [Priority: 3] {UDP} 20.0.0.2:40977 -> 40.0.0.2:5001
Step 12: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:
yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*Show output
------------------------------------------------------------------------ src dst src port dst port tcp vlan_0 vlan_1 pkts bytes ------------------------------------------------------------------------ 40.0.0.2 20.0.0.2 5001 40977 no 201 0 0 0 40.0.0.2 20.0.0.2 5000 34274 yes 201 0 0 0 20.0.0.2 40.0.0.2 34274 5000 yes 201 0 12 905 20.0.0.2 40.0.0.2 40977 5001 no 201 0 0 0
Step 13: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:
Show output
Pin path: /sys/fs/bpf/eth1 Period of 0.250442s ending at 1728467655.037448 XDP_DROP 12 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_PASS 2798303 pkts ( 0 pps) 4021352 KiB ( 0 Mbits/s) XDP_TX 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s) XDP_REDIRECT 0 pkts ( 0 pps) 0 KiB ( 0 Mbits/s)