Enterprise Networks Unveiled: A Software Engineer's Guide to the Basics (Part 4)

As companies grow, they usually expand beyond a single location. For retail companies this may mean multiple branches. For others, it may mean having standalone data centers and offices. In the last post we followed our API request through the devices it would encounter in a single building. How might that flow change at scale? Let's consider the following network architecture. This architecture introduces multiple locations (also called sites) as well as remote customers and employees. Companies have a few different options when connecting sites to each other. Historically, MPLS (Multiprotocol Label Switching) was a common choice. Fiber links (also known as "dark fiber") are also available. Both of these options can provide companies with a guaranteed bandwidth versus the unpredictability of an internet connection. This can be important to latency sensitive applications. These private connections help extend the private network, with routing helping packets make the pathways available to devices at each site. Site local internet connections have also become more popular. Previous designs may have backhauled all traffic (sometimes called a full tunnel) to the data center. Therefore, any request to the internet from a branch would have to go through the data center to get to a website. With the rise of many Software-as-a-Service (SaaS) applications, adoption of split-tunneling approach grew. For example, a company might let their video chatting application traffic use a site local internet connection rather than sending it across the multi-site connection. An employee might have their internet connection through their cable provider. When working from home, employees may also need access to the private network. This is typically achieved through a client virtual private network (VPN). VPNs can leverage similar encapsulation techniques we've previously discussed to establish an encrypted tunnel to a VPN "head-end" device, typically in a data center (Firewalls may act as this device). This can be either a full tunnel or split tunnel similar to the branch flow mentioned above. In addition to client VPNs, site-to-site VPNs can be established. These can provide an alternative if a private connection fails, or if there is limited traffic between sites, but private connectivity is still required. Due to leveraging the internet as a communication medium both client and site-to-site VPNs cannot make the same latency and bandwidth promises as a private connection. The image below from this guide, shows a sample site-to-site VPN and how the tunnel would overlay the internet links without exposing the private networks. Consumers browsing to your website may also end up communicating with services on your network. As shown in the diagram at the top of the post, there may be a content distribution network (CDN) and/or web application firewall (WAF) between your consumers and the public facing network from your data center. There are a number of vendors in the CDN/WAF space like Cloudflare, Fastly, or Akamai. CDNs help cache content closer to your users to improve performance and reduce load on back-end systems, while WAFs aim to prevent malicious traffic from impacting your services. With all these different flows, network firewalls come in handy to ensure that only the necessary devices can communicate with each other. For example, the branch may only need to communicate to a few services for inventory availability and point of sale systems. The firewall in the data center might allow the CDN and/or WAF to communicate with public services while preventing direct access by other unknown internet callers. This sub-dividing of the internal network is known as segmentation and aims to reduce the impact if any device is compromised. Attackers will attempt "lateral movement", and firewalls can hamper those efforts. Firewalls leverage similar techniques mentioned in the last post to inspect packets. Rules are created to allow or deny certain traffic based on source and destination IP addresses and TCP or UDP ports. The image below from this guide demonstrates this. Some firewalls may also leverage DNS names or have the ability to inspect application-level traffic. Firewalls can also be stateful (more common) or stateless. Stateful-ness allows for return traffic (example: http responses) to pass through the firewall while only needing a rule for the client requests to the http server. This post has provided an example network at scale and the complexity that can emerge with a multi-site deployment. With this knowledge in hand, the next post will do a deeper dive on IP addressing, subnetting, and public vs private addresses and how those pieces factor into network design.

Mar 25, 2025 - 15:54

Enterprise Networks Unveiled: A Software Engineer's Guide to the Basics (Part 4)

As companies grow, they usually expand beyond a single location. For retail companies this may mean multiple branches. For others, it may mean having standalone data centers and offices.

In the last post we followed our API request through the devices it would encounter in a single building. How might that flow change at scale? Let's consider the following network architecture.

This architecture introduces multiple locations (also called sites) as well as remote customers and employees. Companies have a few different options when connecting sites to each other. Historically, MPLS (Multiprotocol Label Switching) was a common choice. Fiber links (also known as "dark fiber") are also available. Both of these options can provide companies with a guaranteed bandwidth versus the unpredictability of an internet connection. This can be important to latency sensitive applications. These private connections help extend the private network, with routing helping packets make the pathways available to devices at each site.

Site local internet connections have also become more popular. Previous designs may have backhauled all traffic (sometimes called a full tunnel) to the data center. Therefore, any request to the internet from a branch would have to go through the data center to get to a website. With the rise of many Software-as-a-Service (SaaS) applications, adoption of split-tunneling approach grew. For example, a company might let their video chatting application traffic use a site local internet connection rather than sending it across the multi-site connection.

An employee might have their internet connection through their cable provider. When working from home, employees may also need access to the private network. This is typically achieved through a client virtual private network (VPN). VPNs can leverage similar encapsulation techniques we've previously discussed to establish an encrypted tunnel to a VPN "head-end" device, typically in a data center (Firewalls may act as this device). This can be either a full tunnel or split tunnel similar to the branch flow mentioned above.

In addition to client VPNs, site-to-site VPNs can be established. These can provide an alternative if a private connection fails, or if there is limited traffic between sites, but private connectivity is still required. Due to leveraging the internet as a communication medium both client and site-to-site VPNs cannot make the same latency and bandwidth promises as a private connection. The image below from this guide, shows a sample site-to-site VPN and how the tunnel would overlay the internet links without exposing the private networks.

Consumers browsing to your website may also end up communicating with services on your network. As shown in the diagram at the top of the post, there may be a content distribution network (CDN) and/or web application firewall (WAF) between your consumers and the public facing network from your data center. There are a number of vendors in the CDN/WAF space like Cloudflare, Fastly, or Akamai. CDNs help cache content closer to your users to improve performance and reduce load on back-end systems, while WAFs aim to prevent malicious traffic from impacting your services.

With all these different flows, network firewalls come in handy to ensure that only the necessary devices can communicate with each other. For example, the branch may only need to communicate to a few services for inventory availability and point of sale systems. The firewall in the data center might allow the CDN and/or WAF to communicate with public services while preventing direct access by other unknown internet callers. This sub-dividing of the internal network is known as segmentation and aims to reduce the impact if any device is compromised. Attackers will attempt "lateral movement", and firewalls can hamper those efforts.

Firewalls leverage similar techniques mentioned in the last post to inspect packets. Rules are created to allow or deny certain traffic based on source and destination IP addresses and TCP or UDP ports. The image below from this guide demonstrates this.

Some firewalls may also leverage DNS names or have the ability to inspect application-level traffic. Firewalls can also be stateful (more common) or stateless. Stateful-ness allows for return traffic (example: http responses) to pass through the firewall while only needing a rule for the client requests to the http server.

This post has provided an example network at scale and the complexity that can emerge with a multi-site deployment. With this knowledge in hand, the next post will do a deeper dive on IP addressing, subnetting, and public vs private addresses and how those pieces factor into network design.