Creating custom CDN

CDN = content delivery network. It is set of servers spread out across the globe that deliver content to visitor as fast as possible. AnyCast DNS connects visitors to closest server.

This tutorial is intended to be used if you are setting up your custom Iconify API and want to take advantage of CDN.

Difference between Iconify API CDN and usual CDN is Iconify API doesn't use origin server. All API requests are processed directly on edge servers.

For additional security Iconify API uses another layer: CloudFlare CDN. Instead of connecting to one origin server, CloudFlare edge servers connect to closest Iconify API CDN server. However this article explains CDN process without using CloudFlare layer.

Why Iconify API cannot use usual CDN services?

Usual CDN services such as Cloudflare, CDN77, KeyCDN and every other service work by caching content.

When visitor connects to CDN, CDN checks if page exists in cache, sends page back to visitor if its cached or connects to origin server if its not cached.

Those CDN services work like proxies. They are great if content can be cached.

So why can't those services be used with Iconify API? Because API cannot be cached. There is unlimited number of URL combinations:

https://api.iconify.design/fa.js?icons=arrow-left,home https://api.iconify.design/fa.js?icons=arrow-left,bars,home https://api.iconify.design/fa.js?icons=align-justify,arrow-left,bars,home https://api.iconify.design/fa.js?icons=align-justify,bars,home

and so on. URLs for almost every request are different, responses are different. It is not something typical CDN can cache.

Because of that Iconify API is hosted on custom CDN that hosts entire API on edge servers instead of connecting to one central server.

Typical CDN process (for content that is not cached):

  • Visitor connects to closest edge server.
  • Edge server checks if it has requested page in cache. If page is cached, it sends page to visitor and that's it. However if page not cached, it connects to origin server to request page,
  • Origin server sends page back to edge server.
  • Edge server caches it and sends page to visitor.

Typical CDN flow

Visitors

Request

Forwarded response

Request

Cached response

CDN

Forwarded request

Response

Origin

Iconify API have no origin servers. Data is generated directly on edge servers, guaranteeing lowest possible latency.

Iconify CDN process:

  • Visitor connects to closest edge server.
  • Edge server generates response and sends it back to visitor.

That's it. Iconify does not use origin server.

Iconify CDN flow

Visitors

Request

Response

Request

Response

CDN

Problem with such setup is CDN providers don't allow users to run scripts on edge servers. All CDN providers are simple (or not so simple in some cases) cache servers. They speed up pages only if page is cached.

Setting up custom CDN without origin server

Iconify API is ran on 8 servers, spread out across the globe. There are servers in Europe, in North America, in South America, in Asia and Oceania. Each server runs a copy of Iconify API. Visitor is always connected to closest server, keeping latency as low as possible.

How to create similar setup for your custom Iconify API CDN (or whatever application you want to host)?

First you need several servers where your scripts are running. They should all respond to your domain (or subdomain) and responses should be identical. Setting up servers it not part of this tutorial. There is nothing special about setting up servers. Any cheap unmanaged VPS provider will do, such as Linode and Vultr.

Main problem is making sure visitors are connected to closest server, not random server. Adding multiple "A" records to your domain won't achieve that. You need AnyCast DNS.

AnyCast DNS is expensive. Very expensive. What if you don't have thousands of dollars to spend? Good news is there is one alternative solution: Amazon's Route53 latency routing.

Route53 latency routing works by redirecting visitor to closest Amazon datacenter.

What if you are not hosting your websites at Amazon datacenters? You can host at datacenters not far from Amazon (same country or state), which is more than good enough to redirect visitors to closest of your servers.

You can find list of AWS regions in AWS documentation.

Setting up Route53

  • First you need AWS account. Sign up at AWS main page.

  • Go to Route53 page (sign in if needed).

  • If you already have domain registered, click "Hosted Zones" link, create new zone. Enter your domain name.

Then you need to add records for each server you have in your CDN. Click "Create Record Set", enter subdomain name in "Name" field, enter IP address in "Value" field, in "Routing Policy" field select "Latency". That last part is very important - that makes it possible to redirect visitors to closest server based on latency. Select region closest to your server's physical location, enter something in "Set ID" field.

If you are running script on AWS service, instead of entering IP select click "Yes" for "Alias" option and select AWS instance.

Click "Create" button to add that record.

Create several more records for your other servers. For each record enter same subdomain, but different IP and different region.

By the time you are finished it should look like this:

Notice there are 3 records for "icons.simplesvg.com" pointing to different IP addresses with different region set up.

That's it. When visiting icons.simplesvg.com user will be redirected to one of 3 IPs: 1.2.3.4, 2.3.4.5, 3.4.5.6 depending on which AWS datacenter is closest. It is precise enough to create custom CDN.

What if you don't want to host your domain at AWS? You'll have to host some other domain and create alias for your main domain to point to secondary domain.

For example, instead of simplesvg.com, host simplesvg.net at AWS, for icons.simplesvg.com (custom API subdomain in example) create CNAME alias pointing to icons.simplesvg.net

How to test it?

To test latency from different countries you can use online services such as MapLatency.

To test if server works you can temporary redirect your browser to that server by adding IP to hosts file (/etc/hosts for *nix and OSX, c:\Windows\System32\etc\hosts for Windows):

1.2.3.4 icons.simplesvg.com

You might need to restart browser. If you are having problems editing files, try powerful code editor like SublimeText.

Then by visiting that domain your browser will direct you to IP you've added to host file instead of usual IP.

Do not forget to remove that line from hosts file after you've done testing.

Cloudflare protection

For additional security or for free SSL certificate you can use Cloudflare to serve as proxy between visitor and your small CDN.

Testing done with api.iconify.design during development when CDN had 7 servers revealed that Cloudflare adds about 10-50ms to loading time from anywhere in the world, so performance impact is minimal, but it might be useful as extra layer of protection against attacks.

Unlike typical CDN flow, this flow assumes you are still using custom CDN, setup as described above in this tutorial. Therefore Cloudflare connects to closest of your servers instead of one server that might be half of the world away. You are benefiting from both custom CDN and Cloudflare protection.

Cloudflare + CDN flow

Visitors

Request

Forwarded response

Request

Forwarded response

CF

Forwarded request

Response

Forwarded request

Response

CDN

This method requires 2 domains: one for Cloudflare, one for AWS. Host your primary domain's zone at Cloudflare, host secondary domain's zone at Route53.

Using Iconify API as example, iconify.design should be hosted at CloudFlare, another-domain.com should be hosted at Route53. api.iconify.design should have CNAME record pointing to api.another-domain.com. Custom API servers should respond to api.another-domain.com

This is a very complex setup, but at the end it might be worth it.