Monitoring network status is an important task in managing servers. There are various tools for monitoring communications, and My Traceroute (MTR) is one of the most powerful. This tool helps the network administrator identify errors and obtain a report of the network’s general status.
This article attempts to investigate MTR, the data it produces, and methods of analyzing and interpreting its generated information.
A Performance Overview of Main Network Diagnostics Tools
Network diagnostics tools like ping, traceroute and MTR use the Control Message Protocol (ICMP) to test communications and the traffic exchanged between two points on the Internet. While pinging an IP, a number of ICMP packets are sent from the source to the destination, and the destination sends a number of ICMP packets back to the source in response. The user can calculate the round trip time (RTT) between these two points based on the responses exchanged between them.
Tools such as MTR and traceroute, however, use the TTL (Time to Live) field in the IP header packet to calculate the hops along the path between the source and the destination. TTL specifies the number of hops that the packet can pass through before expiring. In order to count the number of hops using traceroute or MTR, several small ICMP messages are sent to TTL (value one) first, and the TTL value is gradually increased until the packet finally reaches its destination.
MTR can be regarded as a combination of ping and traceroute. In addition to a general view of the path that the traffic passes through from the source to a specific destination, this tool also provides more information regarding the status, communications and the responsiveness of hops placed between the source and the destination.
How to Install MTR
Unlike ping and MTR, traceroute is not installed by default on most operating systems. Use the following commands according to your operating system to install this tool.
Ubuntu/Debian
sudo apt-get install mtr
CentOS/RHEL/Fedora
yum install mtr
Arch
pacman -S mtr
Windows
You can use WinMTR upstream to install this tool on the Windows operating system.
MAC OS
Use Homebrew and MacPorts to install this tool on the MAC OS X operating system. Use the following command to install with Homebrew:
brew install mtr
And the following command to install with MacPorts:
port install mtr
Producing the MTR Report
MTR can be considered a directional tool as it provides a view of the route taken by the packet from source to destination. The forward and return traffic routes between two points may not be the same, so it is recommended to use MTR on both directions and to collect the resulting information when diagnosing communications’ problems between two points.
Typing the mtr command with a URL or an IP address is a simple method for using MTR, as follows:
mtr example.com
MTR’s biggest advantage compared to traceroute is that its output is constantly updated, so it can monitor changes in network performance over time.
Another method for using MTR is to produce a report using the following command:
mtr --report example.com
One can add the –n option to the mtr command to display each hop’s IP address instead of hostname. In the following command for example, the MTR tool is used to generating a report regarding Google’s public DNS.
mtr -n --report 8.8.8.8 HOST: example Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.0.1 0.0% 10 9.4 7.5 3.1 11.7 2.8 2.|-- 10.89.0.1 0.0% 10 13.1 24.4 11.7 69.9 21.7 3.|-- 173.212.126.117 0.0% 10 22.0 20.7 13.0 26.5 4.5 4.|-- 24.215.102.161 0.0% 10 29.2 28.1 23.4 31.9 2.9 5.|-- 24.215.102.221 0.0% 10 22.0 26.1 22.0 30.1 3.1 6.|-- 24.215.102.9 0.0% 10 25.8 27.2 22.2 33.7 3.5 7.|-- 24.215.101.10 0.0% 10 107.8 52.1 41.5 107.8 19.8 8.|-- 209.85.250.3 0.0% 10 68.0 48.6 42.1 68.0 7.3 9.|-- 8.8.8.8 0.0% 10 42.9 47.3 42.8 56.0 4.2
Each column shown in the output above represents specific information regarding sent packets:
- Host: Represents the hostname or IP address for each hop the packet has passed through.
- Loss: Packet loss percentage at each hop
- Snt: Number of sent packets
- Last: Latency of last sent packet in milliseconds
- Avg: Average latency for all packets in milliseconds
- Best: The lowest round trip time for a packet in milliseconds
- Wrst: The highest round trip time for a packet in milliseconds
- StDev: Standard deviation of latency for each host
MTR Output Analysis
MTR reports are mainly used for investigating packet loss and latency.
Investigating Packet Loss
Packet loss may be caused by issues on one router along the packet’s route to the destination, or ICMP traffic rate limiting by the service provider; however, packet loss does not indicate problems in the latter. For example, packet loss has occurred between hops 1 and 2 in the picture below, which is due to rate limiting on hop 2.
mtr -n --report 8.8.8.8 HOST: example Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.0.1 0.0% 10 9.4 7.5 3.1 11.7 2.8 2.|-- 10.89.0.1 0.0% 10 13.1 24.4 11.7 69.9 21.7 3.|-- 173.212.126.117 60.0% 10 22.0 20.7 13.0 26.5 4.5 4.|-- 24.215.102.161 0.0% 10 29.2 28.1 23.4 31.9 2.9 5.|-- 24.215.102.221 0.0% 10 22.0 26.1 22.0 30.1 3.1 6.|-- 24.215.102.9 0.0% 10 25.8 27.2 22.2 33.7 3.5 7.|-- 24.215.101.10 0.0% 10 107.8 52.1 41.5 107.8 19.8 8.|-- 209.85.250.3 0.0% 10 68.0 48.6 42.1 68.0 7.3 9.|-- 8.8.8.8 0.0% 10 42.9 47.3 42.8 56.0 4.2
Packet loss in more than one hop can indicate problems in routing or a specific router along the route. Remember that packet loss and rate limit may sometimes occur simultaneously. When packet loss occurs in various hops, the value displayed for the last hop is always the real packet loss percentage.
For example, actual packet loss in the output below is the 30 percent value shown for the last hop (hop 9), and rate limit has been used in hops 4, 5 and 6.
mtr -n --report 8.8.8.8 HOST: example Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.0.1 0.0% 10 9.4 7.5 3.1 11.7 2.8 2.|-- 10.89.0.1 0.0% 10 13.1 24.4 11.7 69.9 21.7 3.|-- 173.212.126.117 0.0% 10 22.0 20.7 13.0 26.5 4.5 4.|-- 24.215.102.161 70.0% 10 29.2 28.1 23.4 31.9 2.9 5.|-- 24.215.102.221 70.0% 10 22.0 26.1 22.0 30.1 3.1 6.|-- 24.215.102.9 60.0% 10 25.8 27.2 22.2 33.7 3.5 7.|-- 24.215.101.10 30.0% 10 107.8 52.1 41.5 107.8 19.8 8.|-- 209.85.250.3 30.0% 10 68.0 48.6 42.1 68.0 7.3 9.|-- 8.8.8.8 30.0% 10 42.9 47.3 42.8 56.0 4.2
Note: Remember that packet loss up to 10 percent is normal, but higher numbers point to a problem and require further investigation. We recommend once again using MTR on both the forward and return routes to obtain more accurate information.
If you are using MTR on your own computer and observe significant packet loss in the first hop, the problem is caused by your ISP connection, and if the significant loss is seen in the final hops, the problem is caused by your destination ISP.
Latency Analysis
Latency is a parameter that depends on the distance between the source and the destination and line quality. This value may increase for various reasons, including incorrect router configuration on the destination route, link saturation, etc. The sudden spike in latency, which can indicate a problem with a specific hop’s router, should be taken into consideration while analyzing the MTR output.
In the following output, for example, everything is normal up to hop 6 when the latency suddenly increases, which doesn’t decrease on the next hops. Accordingly, we can conclude that the router has encountered problems in hop 6.
mtr -n --report 8.8.8.8 HOST: example Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.0.1 0.0% 10 9.4 7.5 3.1 11.7 2.8 2.|-- 10.89.0.1 0.0% 10 13.1 24.4 11.7 69.9 21.7 3.|-- 173.212.126.117 0.0% 10 22.0 20.7 13.0 26.5 4.5 4.|-- 24.215.102.161 0.0% 10 29.2 28.1 23.4 31.9 2.9 5.|-- 24.215.102.221 0.0% 10 22.0 26.1 22.0 30.1 3.1 6.|-- 24.215.102.9 0.0% 10 430.4 445.2 426.7 463.6 3.5 7.|-- 24.215.101.10 0.0% 10 432.3 445.2 426.7 463.6 3.8 8.|-- 209.85.250.3 0.0% 10 433.5 445.2 426.7 463.6 7.3 9.|-- 8.8.8.8 0.0% 10 435.9 445.2 426.7 463.6 4.2
The sudden increase in latency is only limited to one hop and decreases again. The sudden increase in that hop is caused by rate limit. Rate limiting affects latency as well as packet loss, and as stated for packet loss in the previous section, one should pay attention to the data regarding the last hops to measure latency.