Of course we cannot always share details about our work with customers, but nevertheless it is nice to show our technical achievements and share some of our implemented solutions.
A couple of our customers use the services of Cloudflare as an intermediary reverse proxy before the web traffic hits the actual web server (called origin in Cloudflare terms) where the web application is running.
Even though Cloudflare is officially supported by us (we actually help our customers with the Cloudflare configuration and setup) and in most cases quickly set up, there could be some hidden dragons causing unexpected issues.
So first of all, why would we sometimes recommend switching on Cloudflare in front of our own managed servers? There are a couple of reasons to do that:
At the end it comes down to the traffic. The better known your website or application is, the more traffic you get. This usually applies to automated bots and crawlers but, of course, also to the real users of the website.
The downside with this increased awareness are attacks on your website. These attacks can be targeted attacks to exploit a discovered vulnerability (e.g. an outdated CRM) or large, distributed denial of service attacks (DDoS).
One major pain point is the handling of end user IP addresses, also sometimes referred to as "Visitor IP" or "Remote IP". Due to the fact, that public HTTP requests land at Cloudflare (reverse proxy), Cloudflare sends a new request to the origin server. In this situation the origin server can see the request coming from Cloudflare, from a Cloudflare IP address.
When the web application is configured to handle some requests, based on their source IP address, differently, this won't work anymore. An use case example for this is allowing certain users to an administrative backend of the application.
Another typical example is, when access logs are analyzed for statistics. These logs now only show the Cloudflare IP addresses and falsify the statistics.
To handle this situation, the origin web server needs to be told to read the header "X-Forwarded-For" and use it as "Real IP" instead. This HTTP header is sent in the request from Cloudflare to the Origin server and usually contains the IP address of the original request alongside the Cloudflare proxy server's IP address.
In our infrastructure we enable this on all web servers the requests go through, including our central reverse proxies. This way the application is aware of the original source of the request.
Another known problem is the ability to create a maintenance page. When the web application, for example Wordpress, is directly accessed, a maintenance page can be shown to users and the backend is still accessible to the site admin.
Although this still works when Cloudflare is enabled, it sometimes make sense to cut off the access of the site on Cloudflare, too. But in this scenario the access to the backend is cut as well, leaving the site admin no choice to enable the site on Cloudflare again.
To properly handle this and create a "real maintenance page" on Cloudflare, we have already written a separate blog post. This allows specific IP addresses to access the site normally, while for all other requests a maintenance page/text is showing up. Check out the mentioned blog post for technical details how to create such a maintenance page yourself.
A common problem is a redirect loop, as soon as Cloudflare is activated on your domain and SSL/TLS is enabled. This can be verified using a simple curl request:
ck@infiniroot:~$ curl https://www.example.com -I
HTTP/2 301
date: Wed, 11 Dec 2024 15:06:09 GMT
content-type: text/html
location: https://www.example.com/
cf-cache-status: DYNAMIC
[...]
server: cloudflare
cf-ray: 8f06607bbc66a82f-SYD
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=6211&min_rtt=928&rtt_var=10671&sent=8&recv=10&lost=0&retrans=0&sent_bytes=3429&recv_bytes=829&delivery_rate=4357758&cwnd=254&unsent_bytes=0&cid=14fac1da97da1531&ts=560&x=0"
In this case the request gets a response containing a location header specifying a redirect to https://www.example.com/, the same URL actually requested. This obviously causes a nasty redirect loop.
In most situations the cause of this redirect loop is the SSL/TLS setting on Cloudflare. By default Cloudflare enables an automatic SSL/TLS settings, also called "Flexible".
If you have a proper TLS certificate setup already in place on the origin server/infrastructure, you should change this setting to "Full" or "Full (Strict)".
The redirect loop should be gone after setting the SSL/TLS encryption to "Full" or "Full (Strict)".
ck@infiniroot:~$ curl https://www.example.com -I
HTTP/2 200
date: Wed, 11 Dec 2024 15:14:33 GMT
content-type: text/html; charset=UTF-8
expires: Thu, 19 Nov 1981 08:52:00 GMT
pragma: no-cache
[...]
server: cloudflare
cf-ray: 8f066cca6bd1a967-SYD
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=1199&min_rtt=1015&rtt_var=420&sent=7&recv=8&lost=0&retrans=0&sent_bytes=3428&recv_bytes=829&delivery_rate=3984236&cwnd=254&unsent_bytes=0&cid=725771a2649c32b6&ts=853&x=0"
Cloudflare allows caching of content on their CDN/servers. This can significantly increase the page load speed as these CDN servers are spread across the world and therefore, in many cases, closer to the end users.
However not all applications support caching and not always should cached (stale) content be delivered to end users.
By default Cloudflare enables some kind of caching and automatically enables a (sane) caching rule set. These rules can be appended with "Cache Rules", for example to bypass the Cloudflare cache for specific URI paths or certain requests.
For most websites with content not changing often and not much user interaction (a blog for example), the default Caching rules on Cloudflare work fine. For more complex web applications (such as web shops, websites with user logins or responsive API's), very complex bypass rules need to be created. In these cases it's often advisable to disable the Cache on Cloudflare and use a more advanced caching technology, such as Varnish, on the origin infrastructure.