Troubleshooting Packet Loss on Ports and VXCs

When accessing any network, small units of data called packets are sent and received. When one or more of these packets fail to reach their intended destination, packet loss can manifest in the form of network disruption, slow service, or even total loss of network connectivity.

If you notice packet loss on a Port or VXC, step through these troubleshooting actions to isolate the root cause.

Tip

You can verify Port or VXC status from the Megaport Portal. On the Services page in the Portal, find the service and mouse over its icon. A message displays the status of the service. (The color of the icon also indicates the service status.)

Action Steps
Check interface or CRC errors and packet drops on the device Interface statistics and logs can help identify which end of the cross connect is causing the fault, and the potential solution. For example, an increasing number of incoming errors on a network interface generally rules out that specific SFP (small form pluggable) and indicates a potential issue with other components of the cross connect.
Verify the Tx and Rx optical levels on the device Check the transmitted (Tx) and received (Rx) light levels. This health check enables you to validate physical connectivity. Considerations:
  • If no Rx light is received, the service is down.
  • If you observe degradation of Tx and Rx light levels, the service may be interrupted. Megaport recommends that you check your physical connections.
  • If you are not transmitting or receiving light to/from Megaport, it may be caused by one of the following:
    • Fibre polarity issue – Verify by rolling the fibres at your end.
    • Connectivity issues within your environment or cross connect – Verify by performing physical loopback testing within your environment.
    • Connectivity issues within the Megaport environment – Verify by performing physical loopback testing from your environment towards Megaport.
Verify physical connections with the data center (reseat and replace SFP, clean and replace cables, loopback test) Open a ticket with the data center to perform the following tests:
  1. Check the cross connect for damages or cleaning, if needed.
  2. Ensure that the data center is transmitting adequate light outside of the demarcation point at its end of the connection. The data center should check the light at the demarcation point with a light reading meter.
Verify carrier circuit status (if any) Some cross connects are set through one or multiple carrier network devices before reaching the Megaport network. Verify that the device interfaces in the cross connect path are free of errors and optic light readings are operating correctly.
Validate equipment performance While troubleshooting, Megaport does not have visibility or access outside the Megaport network. To verify that the cause of an issue is within the Megaport network, Megaport Support requires customers to validate the performance of their equipment. This includes ensuring that the hardware specifications and limitations are compatible with Megaport Technical Specifications, and monitoring the network traffic and the workload on the hardware to avoid congestion or degraded performance. To ensure that hardware and your network is operating as expected, we recommend that you validate the performance of the following:
    Hardware
  • Optic (SFP type, speed, and wavelength) and fibre type
  • Port capacity
  • Switch, router, and firewall models
  • Firmware version
    Network
  • Traffic flow
  • Port utilization
  • CPU utilization
  • Configuration
  • Overall network design
If you identify any anomalies, capture the logs, graph details, or any relevant error messages.
Perform traceroute (or other test) to locate the symptom Traceroute is a network diagnostic tool that tracks in real-time the pathway taken by a packet on an IP network from source to destination, reporting the IP addresses of all the routers along the path. Traceroute also records the time taken for each hop the packet makes during its route to the destination.

Perform end-to-end traceroute testing
  • From the host that is originating traffic (A-End), start the traceroute to the destination host (B-End). Then run the traceroute from the destination host to the origin host. Commands and flags may differ by device.
Analyze the results
  • Look for potential asymmetric routing. If the traceroute results are not taking the same path, a traceroute will help pinpoint asymmetric routing somewhere in the network.
  • Are there any places in the traceroute where the response time has significantly increased? If so, are those delays within your network?
    Are there any firewalls or access list rules prohibiting traffic from reaching the destination?

Next steps

If the troubleshooting actions do not resolve your issue, contact support. Before requesting assistance, collect the following information.

  • Troubleshooting results – Provide all the troubleshooting steps you have taken in detail. For example, if loops were placed, note their location and which direction they faced.
  • Source IP address and destination IP address – The source IP address is the IP address of the host that sent the packet. The destination IP address is the IP address of the host that should receive the packet.
  • High-level network diagram – Understanding how your network design is implemented and the connection into the Megaport network helps identify additional focus areas within the troubleshooting process. Provide a network diagram that includes all devices in the path; note each device’s involved IP addresses and VLANs.
  • Ping test results – Provide the output of each ping test performed on the service. Provide all output tests if you have multiple services related to different products (for example, a Port or VXC).
  • Traceroute results – Provide traceroute results, indicating which side of the connection initiated the test and which side was the destination. We recommend that you use the A-End and B-End information from your VXC.

Last update: