Testing Link Throughput with iPerf

The OOKLA portal speedtest.net has been popular among masses for ages and is ideal to measure your internet bandwidth. But, what about the private enterprise network circuits, where traffic remains within an MPLS cloud and doesn’t break out of ISP perimeter. iPerf serves this very purpose of measuring the throughput of an Ethernet circuit bounded within an MPLS cloud. There are many alternatives to iPerf; for eg: jPerf, trex, netperf are some other alternatives. For this post, we’ll focus on iPerf, precisely iPerf3.

iPerf is a CLI based client-server program which generates either TCP or UDP traffic to test the bandwidth of an Ethernet circuit . It is particularly useful in diagnosing circuit performance related issues by measuring the maximum network throughput a server can handle.

The iPerf package is included in most Linux Distribution repositories. To install it on Debian & Ubuntu use apt -get install iPerf command. CentOS repositories do not have iPerf. Use the EPEL repository, which is a repository used to install third-party software packages on RedHat systems such as RHEL and CentOS:

yum install epel-release 
yum update 
yum install iperf

Before you begin, you will need root access of the Linux server. Once can use sudo su command to run programs as another user, by default the root user.

Let’s consider the below set up for performing iPerf. Our objective is to test the throughput of Ethernet link connected on client CPE.

The two iPerf servers iPerf-01 & iPerf-02 are connected to irb.99 (SVI 99 in Cisco Terminology) on a L3 Core Switch. Two p2p GRE tunnels are created between iPerf servers and CPE Router. For eg:
🔹 Tunnel0 [CPE] ➡️ iPerf-01
🔹 Tunnel1 [CPE] ➡️ iPerf-02
Once the tunnels are built and connectivity is established, the traffic flow will be from one iPerf servers towards another in the tunnel passing through the CPE.

Configuration
1️⃣ Assign irb.99 interface to the client VRF.

harsh_hj@L3-CSW> show configuration | display set | match routing-instances | match irb.99 
set routing-instances <existing VRF> interface irb.99

The above command shows the existing VRF that irb.99 belongs to. Use below commands to remove the interface from existing VRF and add it to desired client VRF.

edit exclusive 
del routing-instances <existing VRF> interface irb.99  
set routing-instances <desired VRF name> interface irb.99

2️⃣ Build GRE Tunnel Pairs
We will be building tunnels as specified above.
🔹 Tunnel0 [CPE] ➡️ iPerf-01
🔹 Tunnel1 [CPE] ➡️ iPerf-02

Note: Tunnel interface IP can be any random private IP of /30 Subnet. Server Interface IP can belong to any one of the below /30 Subnet on irb.99 interface of L3-CSW.

harsh_hj@L3-CSW> show configuration interfaces irb.99 
family inet {
       address 10.153.5.203/30;
       address 10.153.5.211/30;
       address 10.153.5.149/30;     
       address 10.153.5.243/30;
}

Use ifconfig command to view the interface configuration on the iPerf servers.

Add configuration as below:

root@iperf-01:/home/admin# ip tunnel add gre1 mode gre local <Server Interface IP> remote <WAN IP> ttl 255 
root@iperf-01:/home/admin# ip addr add 10.0.0.2/30 dev gre1  root@iperf-01:/home/admin# ip link set gre1 up

10.0.0.2 is the tunnel interface IP here and as stated above this can be any random private IP of /30 subnet.

root@iperf-01:/home/admin# route add 10.0.0.6 gw 10.0.0.1 gre1

It is recommended to add the route at a later stage post P2P Tunnel connectivity is verified. It also helps isolating any issues observed during the test. Similarly configure another iPerf server.

root@iperf-02:/home/admin# ip tunnel add gre1 mode gre local <Server Interface IP> remote <WAN IP> ttl 255 
root@iperf-02:/home/admin# ip addr add 10.0.0.6/30 dev gre1 root@iperf-02:/home/admin# ip link set gre1 up  
root@iperf-02:/home/admin# route add 10.0.0.2 gw 10.0.0.5 gre1

Now configure the other end of the Tunnels on the CPE.

CPE(config)#interface tunnel 0    
CPE(config-if)#tunnel source <WAN IP> 
CPE(config-if)#tunnel destination <Server Interface IP> 
CPE(config-if)#ip address 10.0.0.1 255.255.255.252  
CPE(config)#interface tunnel 1 
CPE(config-if)#tunnel source <WAN IP> 
CPE(config-if)#tunnel destination <Server Interface IP> 
CPE(config-if)#ip address 10.0.0.5 255.255.255.252

If the static routes are not added on servers then at this point you should be able to reach WAN IP or Tunnel IP on CPE from individual iPerf servers. Once the static routes are added on iPerf servers, you should be able to ping iperf-01 to iperf-02 or vice versa.

Performing iPerf Test
On the iperf server you want to test, launch iPerf3 in server mode with command iPerf3 -s. You should see output similar to:

-----------------------------------------------------------  
Server listening on TCP port 5001 TCP window size: 85.3 KByte (default)  
------------------------------------------------------------

On the client iPerf servers, one can use below commands and play with various iPerf parameters to obtain desired results: iperf3 -t 300 -c 10.0.0.2 -P 10. Here -t is the flow rate of the WAN interface on CPE and -P sets the number of parallel streams. The IP used is the tunnel interface of iPerf server. Once the iPerf is completed, output will be as below:

- - - - - - - - - - - - - - - - - - - - - - - - 
[ ID] Interval Transfer Bandwidth Retr 
[ 4] 0.00-300.00 sec 360 MBytes 10.1 Mbits/sec 1195 sender 
[ 4] 0.00-300.00 sec 359 MBytes 10.0 Mbits/sec receiver  
iperf Done. 
root@iperf-02:/home/admin#

iperf3 -c 10.0.0.1 -n 2883584. Here n is the number of Bytes overloaded on the link. If the link is 20Mbps, one can begin with Bytes equivalent to 18Mbps and gradually increase over the bandwidth set.

Note that, these parameters are case sensitive. Click here for complete list of parameters that can be played with