When faster isn’t better: How per-packet load balancing throttles your critical traffic

Imagine your network as a highway. Per-packet load balancing splits your data into tiny cars and sends them down multiple lanes, promising faster speeds.

But what if some lanes have hidden potholes and traffic jams? For TCP, the protocol powering most of your critical apps, per-packet load-balancing can backfire spectacularly.

Despite its bandwidth benefits, this approach can cripple TCP throughput in cloud environments. So, how can you avoid becoming a victim of your own network’s “efficiency”?

The hidden trap: How out-of-order packets strangle TCP

But per-packet load balancing tosses this logic out the window by routing packets across paths with mismatched latencies. When packets take different routes, they arrive out of sequence.

TCP mistakes this chaos for packet loss, slamming the brakes on data flow. High-latency paths, such as satellite links, delay acknowledgments (ACKs), tricking TCP into thinking the network is congested.

The result? Shrinking congestion windows and stalled transfers. In extreme cases, throughput plummets by 70% – a death knell for latency-sensitive apps like video calls or cloud databases.

Why your network’s “speed boost” fails TCP

For example, bonding a 50ms terrestrial link with a 500ms satellite path results in ACKs from the satellite arriving too late. TCP’s timers panic, triggering unnecessary retransmissions and throttling speeds to a crawl.

Not all latency differences are created equal. If the difference is less than 10ms, it’s smooth sailing.

However, a 10–50ms difference causes throughput to drop by 20–40% due to frantic retransmissions. When the difference exceeds 50ms, throughput crashes by over 70% as TCP gives up.

What you should do: SD-WAN and smarter TCP

It’s the default for SD-WAN solutions like those offered by Versa, which steer traffic based on real-time latency checks. Steps you should take include:

• Reserve per-packet load balancing for UDP, such as video streaming, which doesn’t care about the order.
• Enforce latency homogeneity by bonding links with similar latency (less than 20ms difference). Versa SD-WAN , for example, treats paths as “equal” only if their latencies differ by less than 10%.
• Upgrade your TCP stack to modern algorithms like TCP BBR, used in Versa’s TCP proxy. BBR maintains 80% throughput even with 100ms latency swings, compared to 30% for older CUBIC TCP.

The bottom line

Per-packet load balancing isn’t evil – it’s just context-sensitive. For TCP-heavy enterprises, the best approach is to default to per-session load balancing, bond links with tight latency controls (ΔRTT <20ms), and deploy SD-WAN for dynamic path selection and TCP optimizations.

By aligning load balancing strategies with protocol quirks, you can dodge the hidden pitfalls and keep your cloud apps running smoothly.

If you would like to get in touch to discuss your SD-WAN deployment, please drop us a line here!