What is Internet Exchange Point (IXP)

Most people know about Internet Service Providers (ISP). People must subscribe to at least one ISP to connect to the Internet. Operating behind the scenes, specialized network infrastructures play a crucial role in ensuring the Internet works correctly and efficiently. This role is the Internet Exchange Point (IXP). In this post, I will explain in greater depth why we need this new role, and what problem this role solves.

Independent ISPs without direct connections

Let me begin with an imaginary example. Here I assume five independent Internet Service Providers (ISPs) operate here in Taiwan. I just give them names: ISP A, ISP B, ISP C, ISP D, and ISP E.

These five ISPs already have their own leased network connections to upstream service providers in the United States. Initially, no other connections are among these ISPs. Since they are all ISPs, they have many clients and servers inside their own network.

Now we must provide full Internet connectivity. That is, any clients in any one of the ISPs can connect to any servers again in any one of the ISPs. Because of the current network connection, huge portions of the traffic must go through their own leased connections to the United States.

Although Internet connectivity is working, the traffic is not economic and efficient. It introduces severe operational, financial, and performance penalties for the network operators and their end-users.

First, traffic among domestic clients and domestic servers is going through extremely expensive international transit links from Taiwan to the United States. As an ISP in Taiwan, if we have a growing number of clients and servers, we must constantly upgrade our expensive inter-continental leased connections.

Second, even if we are willing to sustain the financial cost, the geographic, physical network latency from Taiwan to US is also huge. Traffic from Taiwan to US is going through submarine fiber cable systems across the colossal Pacific Ocean. Round trip from Taiwan to US is above one hundred milliseconds. This is not ideal especially for interactive applications like voice and video conferencing.

To solve this problem, we might consider adding domestic connections in Taiwan to relieve unnecessary traffic loads going to and back from US like a gigantic hairpin.

To understand why this solution works, we can have a closer look at clients in ISP A who are connecting to servers in ISP C.

Without domestic connections, all traffic is going through trans-pacific international transit links. After we add new connections between ISP A and ISP C, the same traffic now goes through new connections instead. Domestic connections are much cheaper and are usually leased at a flat monthly rate. Trans-pacific international transit links are exponentially much more expensive because IP transit fees are typically metered and charged based on sustained bandwidth usage. Domestic network latency in Taiwan is less than 10 milliseconds, which is entirely negligible by comparison.

Adding a direct connection between ISP A and ISP C only solves problems in traffic between A and C. Because we must also accommodate traffic from ISP A to another ISP, for example, ISP D, we must also install new connections between ISP A and ISP D. In the end, as an operator of ISP A, I must add direct connections to any other ISPs.

This is also true for other operators in ISP B, C, D, E. In the end, all connections collectively are like a full mesh connecting any two of these ISPs. Since I assume we have five ISPs in Taiwan, we now need ten connections (5*(5-1)/2) in total to solve any-to-any traffic hairpin problems.

Introducing the IXP to Cure the Full-Mesh Headache

The idea of adding domestic connections indeed solves the international traffic hairpin problem. However, it introduces new challenges. To understand these new challenges, let us look at the issue from ISP A’s perspective.

ISP A is about to install new domestic connections to every other ISP in Taiwan. Because these physical connections must enter other companies’ facilities, ISP A cannot simply ask a telecom provider to install them. It must first negotiate and reach agreements with each ISP on the number of new connections, the connection technology, the installation schedule, and other technical and business details. Even with only four other ISPs, this has already become a major coordination effort.

The problem becomes much larger as the number of ISPs grows. If three new ISPs enter the market, ISP A must repeat the same negotiation process with all three of them. Each new relationship requires another set of discussions, agreements, and technical coordination. If some ISPs are competitors and are unwilling to cooperate, the complexity increases even further.

The scaling problem is also easy to see. Assume we only add one connection between every pair of ISPs. With five ISPs, a full mesh requires ten domestic connections. With twenty ISPs, the number rises to one hundred and ninety connections, calculated as 20 × 19 ÷ 2. With more than one hundred ISPs, the total would exceed 4,950 connections, calculated as 100 × 99 ÷ 2. Such a full-mesh design is too complex to build, manage, understand, and troubleshoot efficiently.

Although adding domestic connections can solve the hairpin problem in theory, it is not practical enough in real life when every ISP must connect directly to every other ISP.

A more practical approach is to introduce a neutral service provider that all ISPs can connect to. We call this neutral provider an Internet Exchange Point, or IXP. Because an IXP provides interconnection services for ISPs, sometimes people describe it as a service provider for service providers.

With an IXP, ISP A no longer needs to negotiate separate connection agreements with every other ISP. Instead, it only needs to establish one agreement and one connection with the IXP. Other ISPs do the same. As a result, local traffic can be exchanged through the IXP without requiring every ISP to maintain direct bilateral connections with every other ISP.

This reduces both administrative work and network complexity. Instead of arranging conferences, contracts, and installation plans with many different companies, ISP A only works with one neutral organization, automatically exchanging routing information through the IXP's central route servers. This saves time, reduces possible business friction among competitors, and makes cooperation much easier.

In an IXP model, each ISP only needs one connection to the IXP, so five ISPs need five connections, twenty ISPs need twenty connections, and one hundred ISPs need one hundred connections. This hub-and-spoke structure is far easier to understand, operate, and troubleshoot, and saves a huge number of unnecessary connections.

For a small and fair fee to connect to the IXP, which is usually negligible compared with expensive international transit costs, ISPs can preserve international bandwidth, reduce latency, and simplify their business relationships through a single neutral provider, the IXP.

Connecting to Multiple IXPs to Eliminate the Single Point of Failure

When all ISPs in Taiwan connect to a single IXP, it solves both the international traffic hairpin problem and the full mesh headache at the same time. However, a new risk arises. This is a typical Single Point of Failure (SPOF) scenario. When the central IXP fails, all domestic traffic stops.

In the practical world, there will be more than one IXP in any country. In Taiwan, we have four independent, active domestic IXPs. If individual ISPs connect to more than one IXP, the independent IXPs provide completely separate traffic pathways. By multi-homing across these distinct platforms, the systemic SPOF risk is effectively eliminated.

Conclusion

Internet Exchange Points (IXPs) enable ISPs to practically leverage domestic connections to increase network performance and conserve expensive international data paths at the same time. ISPs can also eliminate the Single Point of Failure problem by connecting concurrently to more than one independent IXP. Because IXPs serve only other ISPs and not general consumers, we rarely understand this role. IXPs play a crucial role in making our Internet work correctly and efficiently.

One more thing…

I purposely avoided technical information so we can concentrate on clarifying the IXP role itself. We do need a dynamic routing protocol running among networking devices to detect path failure and divert traffic to redundant paths accordingly. That industry standard protocol is the BGP protocol.

In future posts, I will explain how a generic IXP is built, and how an ISP typically connects to an IXP.