Bypass Tests

The following scenario shows different configuration alternatives to improve the OSDx firewall performance.

Builds a scenario with three DUTs in which a performance test is carried out between DUT1 and DUT2, and DUT0 is the router running the firewall. “Local bypass” is set to allow the firewall to internally skips packets belonging to a flow that must be bypassed. The performance test may produce better results than the general tests.

Scenario 

Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:

Step 2: Set the following configuration in DUT0:

set interfaces ethernet eth0 address 10.215.168.64/24
set interfaces ethernet eth0 traffic nat source rule 1 address masquerade
set interfaces ethernet eth1 vif 101 address 40.0.0.1/8
set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN
set interfaces ethernet eth1 vif 201 address 20.0.0.1/8
set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN
set protocols static route 0.0.0.0/0 next-hop 10.215.168.1
set service dns static host-name WAN inet 10.215.168.1
set service firewall FW logging level config
set service firewall FW logging outputs fast
set service firewall FW mode inline queue FW_Q
set service firewall FW ruleset file 'running://test-performance.rules'
set service firewall FW stream bypass
set service firewall FW validator-timeout 20
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'
set traffic policy FW_PLAN rule 1 action enqueue FW_Q
set traffic queue FW_Q elements 1

Step 3: Ping IP address 20.0.0.2 from DUT1:

admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1

Step 4: Ping IP address 40.0.0.2 from DUT2:

admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1

Step 5: Initiate a bandwidth test from DUT2 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1

Step 6: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:

(?m)^.+(Skipping test network performance traffic).+$

Test Capture Bypass 

Description 

Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. “Capture bypass” is set to allow the firewall to mark packets. An external tool can then decide what to do with the flow when the mark is seen. For this example, when packet marks are detected, the traffic is assigned a label, thereby allowing the possibility of classifying traffic. In particular, labeling avoids traffic from entering the firewall.

Performance must improve considerably compared to the previous test.

Scenario 

Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:

Step 2: Set the following configuration in DUT0:

set interfaces ethernet eth0 address 10.215.168.64/24
set interfaces ethernet eth0 traffic nat source rule 1 address masquerade
set interfaces ethernet eth1 vif 101 address 40.0.0.1/8
set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN
set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP
set interfaces ethernet eth1 vif 201 address 20.0.0.1/8
set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN
set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP
set protocols static route 0.0.0.0/0 next-hop 10.215.168.1
set service dns static host-name WAN inet 10.215.168.1
set service firewall FW logging level config
set service firewall FW logging outputs fast
set service firewall FW mode inline queue FW_Q
set service firewall FW ruleset file 'running://test-performance.rules'
set service firewall FW stream bypass mark 129834765
set service firewall FW stream bypass mask 129834765
set service firewall FW validator-timeout 20
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'
set traffic label BYPASS
set traffic policy FW-SKIP rule 1 log prefix SKIP
set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS
set traffic policy FW-SKIP rule 1 set label BYPASS
set traffic policy FW_PLAN rule 1 action enqueue FW_Q
set traffic policy FW_PLAN rule 1 selector FW_SEL
set traffic policy FW_PWAN rule 1 selector FW_SEL
set traffic queue FW_Q elements 1
set traffic selector FW_SEL rule 1 not label BYPASS
set traffic selector MARKED-PACKETS rule 1 mark 129834765

Step 3: Ping IP address 20.0.0.2 from DUT1:

admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1

Step 4: Ping IP address 40.0.0.2 from DUT2:

admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1

Step 5: Initiate a bandwidth test from DUT2 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1

Step 6: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:

(?m)^.+(Skipping test network performance traffic).+$

Step 7: Run command system journal show | cat at DUT0 and check if output matches the following regular expressions:

(?m)^.*\[SKIP\-1\].*$

Show output

Jul 17 17:46:36.281246 osdx systemd-journald[93647]: Runtime Journal (/run/log/journal/7135572a45764d02b8df631348eed5fb) is 2.1M, max 15.3M, 13.2M free.
Jul 17 17:46:36.284822 osdx systemd-journald[93647]: Received client request to rotate journal, rotating.
Jul 17 17:46:36.284878 osdx systemd-journald[93647]: Vacuuming done, freed 0B of archived journals from /run/log/journal/7135572a45764d02b8df631348eed5fb.
Jul 17 17:46:36.291090 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'system journal clear'.
Jul 17 17:46:36.614219 osdx osdx-coredump[398493]: Deleting all coredumps in /opt/vyatta/etc/config/coredump...
Jul 17 17:46:36.622267 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'system coredump delete all'.
Jul 17 17:46:37.108694 osdx OSDxCLI[398169]: User 'admin' entered the configuration menu.
Jul 17 17:46:37.221234 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 address 10.215.168.64/24'.
Jul 17 17:46:37.421831 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth0 traffic nat source rule 1 address masquerade'.
Jul 17 17:46:37.506583 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set protocols static route 0.0.0.0/0 next-hop 10.215.168.1'.
Jul 17 17:46:37.589464 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service dns static host-name WAN inet 10.215.168.1'.
Jul 17 17:46:37.718497 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 address 40.0.0.1/8'.
Jul 17 17:46:37.774052 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 address 20.0.0.1/8'.
Jul 17 17:46:37.890742 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'show working'.
Jul 17 17:46:37.992810 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth0
Jul 17 17:46:38.044815 osdx kernel: 8021q: adding VLAN 0 to HW filter on device eth1
Jul 17 17:46:38.072630 osdx (udev-worker)[398609]: Network interface NamePolicy= disabled on kernel command line.
Jul 17 17:46:38.208702 osdx cfgd[1240]: [398169]Completed change to active configuration
Jul 17 17:46:38.233908 osdx OSDxCLI[398169]: User 'admin' committed the configuration.
Jul 17 17:46:38.249407 osdx OSDxCLI[398169]: User 'admin' left the configuration menu.
Jul 17 17:46:40.252062 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'ping 10.215.168.1      count 1 size 56 timeout 1'.
Jul 17 17:46:40.323727 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'ping 40.0.0.2      count 1 size 56 timeout 1'.
Jul 17 17:46:40.439061 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'ping 20.0.0.2      count 1 size 56 timeout 1'.
Jul 17 17:46:40.931975 osdx file_operation[398687]: using src url: http://10.215.168.1/~robot/test-performance.rules dst url: running://
Jul 17 17:46:40.969528 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'file copy http://10.215.168.1/~robot/test-performance.rules running:// force'.
Jul 17 17:46:41.127613 osdx OSDxCLI[398169]: User 'admin' entered the configuration menu.
Jul 17 17:46:41.199727 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW ruleset file running://test-performance.rules'.
Jul 17 17:46:41.305617 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW mode inline queue FW_Q'.
Jul 17 17:46:41.357826 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW logging outputs fast'.
Jul 17 17:46:41.489200 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW logging level config'.
Jul 17 17:46:41.541631 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW validator-timeout 20'.
Jul 17 17:46:41.644522 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic queue FW_Q elements 1'.
Jul 17 17:46:41.718231 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN'.
Jul 17 17:46:41.804879 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN'.
Jul 17 17:46:41.861611 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mark 129834765'.
Jul 17 17:46:41.957872 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set service firewall FW stream bypass mask 129834765'.
Jul 17 17:46:42.015586 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic label BYPASS'.
Jul 17 17:46:42.116727 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 log prefix SKIP'.
Jul 17 17:46:42.177306 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 selector MARKED-PACKETS'.
Jul 17 17:46:42.280568 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic policy FW-SKIP rule 1 set label BYPASS'.
Jul 17 17:46:42.340044 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic selector MARKED-PACKETS rule 1 mark 129834765'.
Jul 17 17:46:42.431057 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic selector FW_SEL rule 1 not label BYPASS'.
Jul 17 17:46:42.484894 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 selector FW_SEL'.
Jul 17 17:46:42.574029 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic policy FW_PWAN rule 1 selector FW_SEL'.
Jul 17 17:46:42.636429 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 101 traffic policy out FW-SKIP'.
Jul 17 17:46:42.747421 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set interfaces ethernet eth1 vif 201 traffic policy out FW-SKIP'.
Jul 17 17:46:42.803352 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'set traffic policy FW_PLAN rule 1 action enqueue FW_Q'.
Jul 17 17:46:42.951561 osdx OSDxCLI[398169]: User 'admin' added a new cfg line: 'show working'.
Jul 17 17:46:43.457698 osdx systemd[1]: Reloading.
Jul 17 17:46:43.568804 osdx systemd-sysv-generator[398817]: stat() failed on /etc/init.d/README, ignoring: No such file or directory
Jul 17 17:46:43.685298 osdx systemd[1]: Starting suricata@FW.service - Suricata client "FW" service...
Jul 17 17:46:43.706214 osdx systemd[1]: Started suricata@FW.service - Suricata client "FW" service.
Jul 17 17:46:43.899495 osdx INFO[398798]: Rules successfully loaded
Jul 17 17:46:43.901196 osdx cfgd[1240]: [398169]Completed change to active configuration
Jul 17 17:46:43.903191 osdx OSDxCLI[398169]: User 'admin' committed the configuration.
Jul 17 17:46:43.922113 osdx OSDxCLI[398169]: User 'admin' left the configuration menu.
Jul 17 17:46:44.324840 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:11:de:ad:be:ef:6c:30:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=9654 DF PROTO=TCP SPT=32824 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d
Jul 17 17:46:44.324932 osdx kernel: [SKIP-1] ACCEPT IN=eth1.201 OUT=eth1.101 MAC=de:ad:be:ef:6c:11:de:ad:be:ef:6c:30:08:00:45:00:00:59 SRC=20.0.0.2 DST=40.0.0.2 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=60890 DF PROTO=TCP SPT=32838 DPT=5001 WINDOW=502 RES=0x00 ACK PSH URGP=0 MARK=0x7bd1f0d
Jul 17 17:46:46.898213 osdx zebra[1205]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 17 17:46:48.051708 osdx zebra[1205]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 17 17:46:50.711679 osdx zebra[1205]: [RZ3YY-GPH41][EC 100663310] snmp[warning]: Warning: Failed to connect to the agentx master agent ([NIL]):
Jul 17 17:46:53.067874 osdx zebra[1205]: [WPPMZ-G9797] if_zebra_speed_update: eth1.201 old speed: 0 new speed: 4294967295
Jul 17 17:46:53.093566 osdx zebra[1205]: [WPPMZ-G9797] if_zebra_speed_update: eth1.101 old speed: 0 new speed: 4294967295
Jul 17 17:46:54.486243 osdx OSDxCLI[398169]: User 'admin' executed a new command: 'service firewall FW show logging fast | tail'.

Test Simple Capture Bypass 

Description 

Builds a scenario with three DUTs in which a performance test is conducted between DUT1 and DUT2, and DUT0 is the router running the firewall. This test sets the conntrack mark directly, thus skipping all the steps required to set it later.

Performance must improve considerably compared to the previous test.

Scenario 

Step 1: Run command file copy http://10.215.168.1/~robot/test-performance.rules running:// force at DUT0 and expect this output:

Step 2: Set the following configuration in DUT0:

set interfaces ethernet eth0 address 10.215.168.64/24
set interfaces ethernet eth0 traffic nat source rule 1 address masquerade
set interfaces ethernet eth1 vif 101 address 40.0.0.1/8
set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN
set interfaces ethernet eth1 vif 201 address 20.0.0.1/8
set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN
set protocols static route 0.0.0.0/0 next-hop 10.215.168.1
set service dns static host-name WAN inet 10.215.168.1
set service firewall FW logging level config
set service firewall FW logging outputs fast
set service firewall FW mode inline queue FW_Q
set service firewall FW ruleset file 'running://test-performance.rules'
set service firewall FW stream bypass mark 129834765
set service firewall FW stream bypass mask 129834765
set service firewall FW stream bypass set-connmark
set service firewall FW validator-timeout 20
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'
set traffic policy FW_PLAN rule 1 action enqueue FW_Q
set traffic policy FW_PLAN rule 1 selector FW_SEL
set traffic policy FW_PWAN rule 1 selector FW_SEL
set traffic queue FW_Q elements 1
set traffic selector FW_SEL rule 1 not connmark 129834765

Step 3: Ping IP address 20.0.0.2 from DUT1:

admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1

Step 4: Ping IP address 40.0.0.2 from DUT2:

admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1

Step 5: Initiate a bandwidth test from DUT2 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5001 parallel 1

Step 6: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:

(?m)^.+(Skipping test network performance traffic).+$

Step 7: Run command system conntrack show at DUT0 and check if output matches the following regular expressions:

(?m)^tcp\s+.*src=20.0.0.2 dst=40.0.0.2.+dport=5001.*mark=129834765.*$

Test Traffic Early Dropping 

Description 

Builds a scenario with three DUTs and a simple ruleset to drop TCP traffic between DUT1 and DUT2. Such traffic must pass through port 5000 for the rule to match. Later, XDP is queried to check if packets are being dropped at the specified interface.

The contents of the rule file are:

drop tcp any any -> any 5000 (msg: "Dropping TCP performance test traffic"; sid: 1; flow: established, to_server;)

This rule allows the connection to be established and traffic to be dropped later.

Scenario 

Step 1: Run command file copy http://10.215.168.1/~robot/drop-performance.rules running://drop-performance.rules force at DUT0 and expect this output:

Step 2: Set the following configuration in DUT0:

set interfaces ethernet eth0 address 10.215.168.64/24
set interfaces ethernet eth0 traffic nat source rule 1 address masquerade
set interfaces ethernet eth1 vif 101 address 40.0.0.1/8
set interfaces ethernet eth1 vif 101 traffic policy in FW_PLAN
set interfaces ethernet eth1 vif 201 address 20.0.0.1/8
set interfaces ethernet eth1 vif 201 traffic policy in FW_PLAN
set protocols static route 0.0.0.0/0 next-hop 10.215.168.1
set service dns static host-name WAN inet 10.215.168.1
set service firewall FW logging level config
set service firewall FW logging outputs fast
set service firewall FW mode inline queue FW_Q
set service firewall FW ruleset file 'running://drop-performance.rules'
set service firewall FW stream bypass action drop set xdp-early-drop eth1
set service firewall FW validator-timeout 20
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'
set traffic policy FW_PLAN rule 1 action enqueue FW_Q
set traffic queue FW_Q elements 1

Step 3: Ping IP address 20.0.0.2 from DUT1:

admin@DUT1$ ping 20.0.0.2 count 1 size 56 timeout 1

Step 4: Ping IP address 40.0.0.2 from DUT2:

admin@DUT2$ ping 40.0.0.2 count 1 size 56 timeout 1

Step 5: Initiate a bandwidth test from DUT2 to DUT1

admin@DUT1$ monitor test performance server port 5000
admin@DUT2$ monitor test performance client 40.0.0.2 duration 10 port 5000 parallel 1

Step 6: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:

(?m)^.+(Dropping TCP performance test traffic).+$

Step 7: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:

yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*

Step 8: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output:

Step 9: Initiate a bandwidth test from DUT2 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT2$ monitor test performance client 40.0.0.2 duration 30 udp port 5001 parallel 1

Step 10: Run command service firewall FW show logging fast | tail at DUT0 and check if output matches the following regular expressions:

(?m)^.+(Dropping UDP performance test traffic).+$

Step 11: Run command service firewall FW show early-drop-stats eth1 at DUT0 and check if output matches the following regular expressions:

yes\s+201\s+\d+\s+[1-9]\d*\s+[1-9]\d*

Step 12: Run command interfaces ethernet eth1 monitor xdp-stats times 1 at DUT0 and expect this output: