Hash-Policy

These scenarios cover some aspects related to link-aggregation. This particular one covers the hash-policy configuration needed to setup the desired load balancing. In OSDx, this can be achieved using bonding interfaces in either xor-hash or 802.3ad modes, or transmit-load-balance and adaptive-load-balance in static mode.

../../../../_images/topology1.svg

Test Hash-Policy

Description

A bonding interface is configured in DUT0 using xor-hash mode. This scenario shows how to set the hash-policy option to manage load balancing.

Scenario

Step 1: Set the following configuration in DUT0 :

set interfaces bonding bond0 address 192.168.100.10/24
set interfaces bonding bond0 miimon 100
set interfaces bonding bond0 mode xor-hash
set interfaces ethernet eth0 bond-group bond0
set interfaces ethernet eth1 bond-group bond0
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'

Step 2: Set the following configuration in DUT1 :

set interfaces bridge br0 address 192.168.100.20/24
set interfaces bridge br0 stp version rstp
set interfaces ethernet eth0 bridge-group bridge br0
set interfaces ethernet eth1 bridge-group bridge br0
set system login user admin authentication encrypted-password '$6$GSjsCj8gHLv$/VcqU6FLi6CT2Oxn0MJQ2C2tqnRDrYKNF8HIYWJp68nvXvPdFccDsT04.WtigUONbKYrgKg8d6rEs8PjljMkH0'

Step 3: Run command interfaces bonding show at DUT0 and check if output matches the following regular expressions:

bond0\s+192.168.100\.10\/24\s+up\s+up
Show output
------------------------------------------------------------------
Name            IP Address           Admin  Oper  Vrf  Description
------------------------------------------------------------------
bond0  192.168.100.10/24             up     up
       fe80::dcad:beff:feef:6c01/64

Step 4: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Bonding Mode: load balancing (xor)
Show output
Ethernet Channel Bonding Driver: v6.12.90

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 5: Ping IP address 192.168.100.20 from DUT0:

admin@DUT0$ ping 192.168.100.20 count 1 size 56 timeout 1
Show output
PING 192.168.100.20 (192.168.100.20) 56(84) bytes of data.
64 bytes from 192.168.100.20: icmp_seq=1 ttl=64 time=456 ms

--- 192.168.100.20 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 455.799/455.799/455.799/0.000 ms

Step 6: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Transmit Hash Policy: layer2 (0)
Show output
Ethernet Channel Bonding Driver: v6.12.90

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 7: Run command interfaces ethernet clear at DUT0.

Step 8: Initiate a bandwidth test from DUT0 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT0$ monitor test performance client 192.168.100.20 duration 5 port 5001 parallel 10
Expect this output in DUT0:
Connecting to host 192.168.100.20, port 5001
[  5] local 192.168.100.10 port 33696 connected to 192.168.100.20 port 5001
[  7] local 192.168.100.10 port 33706 connected to 192.168.100.20 port 5001
[  9] local 192.168.100.10 port 33718 connected to 192.168.100.20 port 5001
[ 11] local 192.168.100.10 port 33720 connected to 192.168.100.20 port 5001
[ 13] local 192.168.100.10 port 33728 connected to 192.168.100.20 port 5001
[ 15] local 192.168.100.10 port 33734 connected to 192.168.100.20 port 5001
[ 17] local 192.168.100.10 port 33744 connected to 192.168.100.20 port 5001
[ 19] local 192.168.100.10 port 33752 connected to 192.168.100.20 port 5001
[ 21] local 192.168.100.10 port 33768 connected to 192.168.100.20 port 5001
[ 23] local 192.168.100.10 port 33770 connected to 192.168.100.20 port 5001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  49.0 MBytes   411 Mbits/sec   49    235 KBytes
[  7]   0.00-1.00   sec  43.4 MBytes   364 Mbits/sec   24    262 KBytes
[  9]   0.00-1.00   sec  37.2 MBytes   312 Mbits/sec   70    136 KBytes
[ 11]   0.00-1.00   sec  67.4 MBytes   565 Mbits/sec   98    260 KBytes
[ 13]   0.00-1.00   sec  30.3 MBytes   254 Mbits/sec   21    170 KBytes
[ 15]   0.00-1.00   sec  33.1 MBytes   278 Mbits/sec   94    133 KBytes
[ 17]   0.00-1.00   sec  35.1 MBytes   295 Mbits/sec  108    141 KBytes
[ 19]   0.00-1.00   sec  35.2 MBytes   295 Mbits/sec   88    140 KBytes
[ 21]   0.00-1.00   sec  32.8 MBytes   275 Mbits/sec   30    212 KBytes
[ 23]   0.00-1.00   sec  52.4 MBytes   439 Mbits/sec   93    225 KBytes
[SUM]   0.00-1.00   sec   416 MBytes  3.49 Gbits/sec  675
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  45.2 MBytes   379 Mbits/sec    7    283 KBytes
[  7]   1.00-2.00   sec  46.0 MBytes   386 Mbits/sec   14    284 KBytes
[  9]   1.00-2.00   sec  33.3 MBytes   279 Mbits/sec   22    208 KBytes
[ 11]   1.00-2.00   sec  42.1 MBytes   353 Mbits/sec   13    242 KBytes
[ 13]   1.00-2.00   sec  36.0 MBytes   302 Mbits/sec   39    222 KBytes
[ 15]   1.00-2.00   sec  37.3 MBytes   313 Mbits/sec   12    228 KBytes
[ 17]   1.00-2.00   sec  43.0 MBytes   361 Mbits/sec    0    266 KBytes
[ 19]   1.00-2.00   sec  38.2 MBytes   321 Mbits/sec   19    229 KBytes
[ 21]   1.00-2.00   sec  43.3 MBytes   363 Mbits/sec   14    266 KBytes
[ 23]   1.00-2.00   sec  43.9 MBytes   368 Mbits/sec   19    274 KBytes
[SUM]   1.00-2.00   sec   408 MBytes  3.43 Gbits/sec  159
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  23.0 MBytes   193 Mbits/sec    4    286 KBytes
[  7]   2.00-3.00   sec  23.2 MBytes   195 Mbits/sec    4    290 KBytes
[  9]   2.00-3.00   sec  23.1 MBytes   194 Mbits/sec    4    218 KBytes
[ 11]   2.00-3.00   sec  23.4 MBytes   196 Mbits/sec    2    249 KBytes
[ 13]   2.00-3.00   sec  23.3 MBytes   195 Mbits/sec    4    225 KBytes
[ 15]   2.00-3.00   sec  23.2 MBytes   194 Mbits/sec    3    161 KBytes
[ 17]   2.00-3.00   sec  22.7 MBytes   191 Mbits/sec    7    272 KBytes
[ 19]   2.00-3.00   sec  22.9 MBytes   192 Mbits/sec    4    235 KBytes
[ 21]   2.00-3.00   sec  22.9 MBytes   192 Mbits/sec    5    272 KBytes
[ 23]   2.00-3.00   sec  22.9 MBytes   192 Mbits/sec    5    194 KBytes
[SUM]   2.00-3.00   sec   231 MBytes  1.93 Gbits/sec   42
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  30.8 MBytes   259 Mbits/sec   47    181 KBytes
[  7]   3.00-4.00   sec  28.8 MBytes   241 Mbits/sec   43    205 KBytes
[  9]   3.00-4.00   sec  32.7 MBytes   275 Mbits/sec    2    277 KBytes
[ 11]   3.00-4.00   sec  23.3 MBytes   195 Mbits/sec   33    134 KBytes
[ 13]   3.00-4.00   sec  29.5 MBytes   248 Mbits/sec   61    223 KBytes
[ 15]   3.00-4.00   sec  26.0 MBytes   218 Mbits/sec   31    134 KBytes
[ 17]   3.00-4.00   sec  32.3 MBytes   271 Mbits/sec   38    226 KBytes
[ 19]   3.00-4.00   sec  25.1 MBytes   211 Mbits/sec   57    146 KBytes
[ 21]   3.00-4.00   sec  29.2 MBytes   245 Mbits/sec   65    129 KBytes
[ 23]   3.00-4.00   sec  28.2 MBytes   237 Mbits/sec   17    147 KBytes
[SUM]   3.00-4.00   sec   286 MBytes  2.40 Gbits/sec  394
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  38.7 MBytes   324 Mbits/sec   24    185 KBytes
[  7]   4.00-5.00   sec  37.8 MBytes   317 Mbits/sec   66    201 KBytes
[  9]   4.00-5.00   sec  41.9 MBytes   351 Mbits/sec   84    204 KBytes
[ 11]   4.00-5.00   sec  37.0 MBytes   310 Mbits/sec   33    197 KBytes
[ 13]   4.00-5.00   sec  36.7 MBytes   308 Mbits/sec   40    170 KBytes
[ 15]   4.00-5.00   sec  34.3 MBytes   288 Mbits/sec   26    209 KBytes
[ 17]   4.00-5.00   sec  37.9 MBytes   318 Mbits/sec   64    202 KBytes
[ 19]   4.00-5.00   sec  33.6 MBytes   282 Mbits/sec   30    165 KBytes
[ 21]   4.00-5.00   sec  32.5 MBytes   273 Mbits/sec   20    170 KBytes
[ 23]   4.00-5.00   sec  35.0 MBytes   294 Mbits/sec   15    194 KBytes
[SUM]   4.00-5.00   sec   365 MBytes  3.07 Gbits/sec  402
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   187 MBytes   313 Mbits/sec  131             sender
[  5]   0.00-5.00   sec   185 MBytes   310 Mbits/sec                  receiver
[  7]   0.00-5.00   sec   179 MBytes   301 Mbits/sec  151             sender
[  7]   0.00-5.00   sec   178 MBytes   298 Mbits/sec                  receiver
[  9]   0.00-5.00   sec   168 MBytes   282 Mbits/sec  182             sender
[  9]   0.00-5.00   sec   167 MBytes   279 Mbits/sec                  receiver
[ 11]   0.00-5.00   sec   193 MBytes   324 Mbits/sec  179             sender
[ 11]   0.00-5.00   sec   191 MBytes   320 Mbits/sec                  receiver
[ 13]   0.00-5.00   sec   156 MBytes   262 Mbits/sec  165             sender
[ 13]   0.00-5.00   sec   155 MBytes   260 Mbits/sec                  receiver
[ 15]   0.00-5.00   sec   154 MBytes   258 Mbits/sec  166             sender
[ 15]   0.00-5.00   sec   153 MBytes   256 Mbits/sec                  receiver
[ 17]   0.00-5.00   sec   171 MBytes   287 Mbits/sec  217             sender
[ 17]   0.00-5.00   sec   170 MBytes   285 Mbits/sec                  receiver
[ 19]   0.00-5.00   sec   155 MBytes   260 Mbits/sec  198             sender
[ 19]   0.00-5.00   sec   154 MBytes   258 Mbits/sec                  receiver
[ 21]   0.00-5.00   sec   161 MBytes   270 Mbits/sec  134             sender
[ 21]   0.00-5.00   sec   159 MBytes   267 Mbits/sec                  receiver
[ 23]   0.00-5.00   sec   182 MBytes   306 Mbits/sec  149             sender
[ 23]   0.00-5.00   sec   180 MBytes   302 Mbits/sec                  receiver
[SUM]   0.00-5.00   sec  1.67 GBytes  2.86 Gbits/sec  1672             sender
[SUM]   0.00-5.00   sec  1.65 GBytes  2.83 Gbits/sec                  receiver

iperf Done.

Step 9: Run command interfaces ethernet show counters at DUT0 and check if output matches the following regular expressions:

\s+up\s+(?:[1-9]?\d)\s+\d+\s+\d+\s+(?:[1-9]?\d)
\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
Show output
------------------------------------------------------------------------------
Name  Oper  Rx Packets  Rx Bytes  Rx Errors  Tx Packets   Tx Bytes   Tx Errors
------------------------------------------------------------------------------
eth0  up        318477  21037778          0     1263754  1861179491          0
eth1  up             0         0          0           0           0          0
eth2  down           0         0          0           0           0          0
eth3  down           0         0          0           0           0          0

Note

As you can see, traffic is going out through only one of the bonded interfaces.

Step 10: Modify the following configuration lines in DUT0 :

set interfaces bonding bond0 hash-policy layer3+4

Step 11: Run command interfaces bonding show at DUT0 and check if output matches the following regular expressions:

bond0\s+192.168.100\.10\/24\s+up\s+up
Show output
------------------------------------------------------------------
Name            IP Address           Admin  Oper  Vrf  Description
------------------------------------------------------------------
bond0  192.168.100.10/24             up     up
       fe80::dcad:beff:feef:6c01/64

Step 12: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Bonding Mode: load balancing (xor)
Show output
Ethernet Channel Bonding Driver: v6.12.90

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 13: Ping IP address 192.168.100.20 from DUT0:

admin@DUT0$ ping 192.168.100.20 count 1 size 56 timeout 1
Show output
PING 192.168.100.20 (192.168.100.20) 56(84) bytes of data.
64 bytes from 192.168.100.20: icmp_seq=1 ttl=64 time=0.419 ms

--- 192.168.100.20 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.419/0.419/0.419/0.000 ms

Step 14: Run command interfaces bonding bond0 show ports at DUT0 and check if output contains the following tokens:

Transmit Hash Policy: layer3+4 (1)
Show output
Ethernet Channel Bonding Driver: v6.12.90

Bonding Mode: load balancing (xor)
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:01
Slave queue ID: 0

Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: de:ad:be:ef:6c:00
Slave queue ID: 0

Step 15: Run command interfaces ethernet clear at DUT0.

Step 16: Initiate a bandwidth test from DUT0 to DUT1

admin@DUT1$ monitor test performance server port 5001
admin@DUT0$ monitor test performance client 192.168.100.20 duration 5 port 5001 parallel 10
Expect this output in DUT0:
Connecting to host 192.168.100.20, port 5001
[  5] local 192.168.100.10 port 50366 connected to 192.168.100.20 port 5001
[  7] local 192.168.100.10 port 50378 connected to 192.168.100.20 port 5001
[  9] local 192.168.100.10 port 50386 connected to 192.168.100.20 port 5001
[ 11] local 192.168.100.10 port 50400 connected to 192.168.100.20 port 5001
[ 13] local 192.168.100.10 port 50412 connected to 192.168.100.20 port 5001
[ 15] local 192.168.100.10 port 50426 connected to 192.168.100.20 port 5001
[ 17] local 192.168.100.10 port 50436 connected to 192.168.100.20 port 5001
[ 19] local 192.168.100.10 port 50440 connected to 192.168.100.20 port 5001
[ 21] local 192.168.100.10 port 50446 connected to 192.168.100.20 port 5001
[ 23] local 192.168.100.10 port 50448 connected to 192.168.100.20 port 5001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  58.0 MBytes   487 Mbits/sec  209    226 KBytes
[  7]   0.00-1.00   sec  36.4 MBytes   305 Mbits/sec   76    182 KBytes
[  9]   0.00-1.00   sec  32.5 MBytes   272 Mbits/sec   60    168 KBytes
[ 11]   0.00-1.00   sec  34.1 MBytes   286 Mbits/sec  141    123 KBytes
[ 13]   0.00-1.00   sec  45.1 MBytes   378 Mbits/sec   87    161 KBytes
[ 15]   0.00-1.00   sec  31.6 MBytes   265 Mbits/sec   86    209 KBytes
[ 17]   0.00-1.00   sec  46.7 MBytes   392 Mbits/sec  182    221 KBytes
[ 19]   0.00-1.00   sec  35.4 MBytes   297 Mbits/sec   56    208 KBytes
[ 21]   0.00-1.00   sec  36.0 MBytes   302 Mbits/sec   28    194 KBytes
[ 23]   0.00-1.00   sec  35.5 MBytes   298 Mbits/sec  151    144 KBytes
[SUM]   0.00-1.00   sec   391 MBytes  3.28 Gbits/sec  1076
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  40.2 MBytes   338 Mbits/sec   99    178 KBytes
[  7]   1.00-2.00   sec  40.3 MBytes   338 Mbits/sec   39    201 KBytes
[  9]   1.00-2.00   sec  40.0 MBytes   336 Mbits/sec   20    140 KBytes
[ 11]   1.00-2.00   sec  33.6 MBytes   281 Mbits/sec   52    173 KBytes
[ 13]   1.00-2.00   sec  36.8 MBytes   309 Mbits/sec   59    133 KBytes
[ 15]   1.00-2.00   sec  43.4 MBytes   364 Mbits/sec   33    146 KBytes
[ 17]   1.00-2.00   sec  50.0 MBytes   419 Mbits/sec   60    185 KBytes
[ 19]   1.00-2.00   sec  40.1 MBytes   336 Mbits/sec   77    202 KBytes
[ 21]   1.00-2.00   sec  38.6 MBytes   324 Mbits/sec   33    148 KBytes
[ 23]   1.00-2.00   sec  33.2 MBytes   279 Mbits/sec   96    123 KBytes
[SUM]   1.00-2.00   sec   396 MBytes  3.32 Gbits/sec  568
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  42.9 MBytes   360 Mbits/sec   41    197 KBytes
[  7]   2.00-3.00   sec  38.2 MBytes   320 Mbits/sec   59    156 KBytes
[  9]   2.00-3.00   sec  37.1 MBytes   311 Mbits/sec   18    161 KBytes
[ 11]   2.00-3.00   sec  35.1 MBytes   294 Mbits/sec   70    168 KBytes
[ 13]   2.00-3.00   sec  33.8 MBytes   284 Mbits/sec   52    181 KBytes
[ 15]   2.00-3.00   sec  37.0 MBytes   310 Mbits/sec   41    181 KBytes
[ 17]   2.00-3.00   sec  35.3 MBytes   296 Mbits/sec   53    170 KBytes
[ 19]   2.00-3.00   sec  43.2 MBytes   363 Mbits/sec   95    192 KBytes
[ 21]   2.00-3.00   sec  35.6 MBytes   299 Mbits/sec   28    191 KBytes
[ 23]   2.00-3.00   sec  35.3 MBytes   296 Mbits/sec   63    160 KBytes
[SUM]   2.00-3.00   sec   374 MBytes  3.13 Gbits/sec  520
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  39.8 MBytes   333 Mbits/sec   65    141 KBytes
[  7]   3.00-4.00   sec  41.9 MBytes   352 Mbits/sec   49    215 KBytes
[  9]   3.00-4.00   sec  34.9 MBytes   292 Mbits/sec   53    187 KBytes
[ 11]   3.00-4.00   sec  40.3 MBytes   338 Mbits/sec   64    218 KBytes
[ 13]   3.00-4.00   sec  41.9 MBytes   351 Mbits/sec   66    225 KBytes
[ 15]   3.00-4.00   sec  44.2 MBytes   371 Mbits/sec   98    160 KBytes
[ 17]   3.00-4.00   sec  40.6 MBytes   340 Mbits/sec    3    216 KBytes
[ 19]   3.00-4.00   sec  36.0 MBytes   302 Mbits/sec  106    195 KBytes
[ 21]   3.00-4.00   sec  37.8 MBytes   317 Mbits/sec   85    136 KBytes
[ 23]   3.00-4.00   sec  25.8 MBytes   216 Mbits/sec   65    146 KBytes
[SUM]   3.00-4.00   sec   383 MBytes  3.21 Gbits/sec  654
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  37.1 MBytes   311 Mbits/sec   33    170 KBytes
[  7]   4.00-5.00   sec  45.6 MBytes   383 Mbits/sec  106    195 KBytes
[  9]   4.00-5.00   sec  36.1 MBytes   303 Mbits/sec   56    164 KBytes
[ 11]   4.00-5.00   sec  40.3 MBytes   338 Mbits/sec   16    266 KBytes
[ 13]   4.00-5.00   sec  33.8 MBytes   284 Mbits/sec   50    163 KBytes
[ 15]   4.00-5.00   sec  31.6 MBytes   265 Mbits/sec   99    153 KBytes
[ 17]   4.00-5.00   sec  40.0 MBytes   335 Mbits/sec   37    181 KBytes
[ 19]   4.00-5.00   sec  40.8 MBytes   342 Mbits/sec   52    219 KBytes
[ 21]   4.00-5.00   sec  25.2 MBytes   212 Mbits/sec   85    120 KBytes
[ 23]   4.00-5.00   sec  31.1 MBytes   261 Mbits/sec   94    153 KBytes
[SUM]   4.00-5.00   sec   362 MBytes  3.03 Gbits/sec  628
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   218 MBytes   366 Mbits/sec  447             sender
[  5]   0.00-5.01   sec   216 MBytes   362 Mbits/sec                  receiver
[  7]   0.00-5.00   sec   202 MBytes   340 Mbits/sec  329             sender
[  7]   0.00-5.01   sec   201 MBytes   338 Mbits/sec                  receiver
[  9]   0.00-5.00   sec   181 MBytes   303 Mbits/sec  207             sender
[  9]   0.00-5.01   sec   180 MBytes   301 Mbits/sec                  receiver
[ 11]   0.00-5.00   sec   183 MBytes   308 Mbits/sec  343             sender
[ 11]   0.00-5.01   sec   182 MBytes   305 Mbits/sec                  receiver
[ 13]   0.00-5.00   sec   191 MBytes   321 Mbits/sec  314             sender
[ 13]   0.00-5.01   sec   189 MBytes   317 Mbits/sec                  receiver
[ 15]   0.00-5.00   sec   188 MBytes   315 Mbits/sec  357             sender
[ 15]   0.00-5.01   sec   187 MBytes   313 Mbits/sec                  receiver
[ 17]   0.00-5.00   sec   213 MBytes   357 Mbits/sec  335             sender
[ 17]   0.00-5.01   sec   210 MBytes   352 Mbits/sec                  receiver
[ 19]   0.00-5.00   sec   195 MBytes   328 Mbits/sec  386             sender
[ 19]   0.00-5.01   sec   194 MBytes   325 Mbits/sec                  receiver
[ 21]   0.00-5.00   sec   173 MBytes   291 Mbits/sec  259             sender
[ 21]   0.00-5.01   sec   172 MBytes   289 Mbits/sec                  receiver
[ 23]   0.00-5.00   sec   161 MBytes   270 Mbits/sec  469             sender
[ 23]   0.00-5.01   sec   160 MBytes   267 Mbits/sec                  receiver
[SUM]   0.00-5.00   sec  1.86 GBytes  3.20 Gbits/sec  3446             sender
[SUM]   0.00-5.01   sec  1.85 GBytes  3.17 Gbits/sec                  receiver

iperf Done.

Step 17: Run command interfaces ethernet show counters at DUT0 and check if output matches the following regular expressions:

eth0\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
eth1\s+up\s+\d{4,}\s+\d+\s+\d+\s+\d{4,}
Show output
------------------------------------------------------------------------------
Name  Oper  Rx Packets  Rx Bytes  Rx Errors  Tx Packets   Tx Bytes   Tx Errors
------------------------------------------------------------------------------
eth0  up        122237   8094567          0      694535  1051439927          0
eth1  up        133108   8806172          0      679609  1028820084          0
eth2  down           0         0          0           0           0          0
eth3  down           0         0          0           0           0          0

Note

As you can see, this time traffic is going out through both bonded interfaces.