Routing failure on BT/EE Backbone blocking access to Render.com / Cloudflare

[tomble]

Hi everyone,

I am experiencing a severe routing issue on my EE Home Broadband connection that is intermittently (sometimes totally) blocking access to dashboard.render.com (and associated hosted applications).

As a SaaS business owner who hosts my infrastructure on Render, this routing failure has entirely broken my development workflow and access to my live platform. I have just lost a full business day troubleshooting this infrastructure issue instead of driving actual value for my business, which has been incredibly frustrating.

I have performed extensive differential testing and isolated the issue entirely to the EE/BT core routing network.

The Problem

Any TCP connection over Port 443 (HTTPS) to Render's IP block (216.24.57.x) times out completely on EE Home Broadband.

Diagnostics & Cross-Carrier Testing

• EE Home Broadband: 100% failure rate. Test-NetConnection on port 443 fails with a TCP timeout.

• EE Mobile Hotspot: Degraded/Intermittent. It resolves via a NAT64 gateway (64:ff9b::), fails on the primary target, but occasionally succeeds on a fallback IP.

• Vodafone Mobile Hotspot: 100% success rate. Instantly connects to the destination on port 443.

• Local Settings: I have completely turned off EE Web Protection, Content Lock, and Parental Controls. This is not a local firewall or software block.

Traceroute Evidence (The Smoking Gun)

A tracert from my EE Home Broadband connection shows that traffic traverses the BT core network perfectly fine until it hits the Global Internet Access (GIA) edge network, where it dies immediately at hop 8 (166-49-135-83.gia.bt.net). It never leaves the BT network to be handed off to the destination:

Plaintext

Tracing route to gcp-us-west1-1.origin.onrender.com.cdn.cloudflare.net [216.24.57.7]
over a maximum of 30 hops:

  1     1 ms     1 ms     1 ms  ************
  2     4 ms     3 ms     4 ms  ************
  3     *        *        *     Request timed out.
  4     *        6 ms     *     62.172.102.140
  5     6 ms    14 ms     6 ms  peer7-et-3-0-3.telehouse.ukcore.bt.net [62.6.201.226]
  6     7 ms     7 ms     6 ms  transit1-xe-000.telehouse.ukcore.bt.net [194.72.16.177]
  7     6 ms     6 ms     7 ms  166-49-214-166.gia.bt.net [166.49.214.166]
  8     9 ms     8 ms     8 ms  166-49-135-83.gia.bt.net [166.49.135.83]
  9     *        *        *     Request timed out.
 10     *        *        *     Request timed out.
[...times out continuously up to 30 hops...]

Next Steps

Because this is happening deep within the gia.bt.net transit layer, this appears to be a broken BGP route or an issue at a major peering exchange point between BT/EE and Cloudflare/Render's upstream network.

Could a forum moderator please escalate this traceroute directly to the Tier 2/Tier 3 Network Engineering/NOC teams? Standard customer service handles cannot resolve backbone routing table issues.

I suspect this affects many other internet services hosted on that IP range that Cloudflare/Render uses.

Thank you.

[Mod edit: Personal details redacted. Please do not share personal details as your posts are visible to the public. Thanks!]

[XRaySpeX]

@matthubbert77 : Please explain what those tables of entries are. Are they:

• A tracert sequence with no timings?
• Targets of pings with failures?
• Origins of pings with failures?

[matthubbert77]

It looks to be a test performed by globalping.io - Render support sent them to me to demonstrate how out of 7 attempts, 3 were successful.

I just tried it myself out of curiosity, and sure enough I see intermittent results when I select British Telecommunications. A lot of timeouts.

 

Similar with a tracert. Sometimes they each, often they hang after 166-49-135-83.gia.bt.net

[matthubbert77]

Render sent them to me, I believe they're a test using globalping website to demonstrate how out of 7 attempts, 3 were successful.

Out of curiosity I just tried it myself and sure enough see intermittent results when selecting British Telecommunications, targetting dashboard.render.com. Lots of timeouts.

Similarly with tracert, sometimes it reaches but often it doesn't, hanging after 166-49-135-83.gia.bt.net

[matthubbert77]

I googled the address the traceroutes get hung up on, and it looks like it's starting to pop up in other places too. Here's a recent thread on Plusnet, with someone unable to access a specific site (Pushsafe?) with the same apparent symptoms.

https://community.plus.net/t5/Broadband/I-can-no-longer-access-pushsafer-com-via-Plusnet/td-p/204827... 

[XRaySpeX]

@matthubbert77 : So to be clear they are the results of pinging a specific target, gcp-us-west1-1.origin.onrender.com.cdn.cloudflare.net [216.24.57.251], from various sources, automated somewhat by  globalping.io,

There are often reports of users not being able to reach a site for many numerous varied reasons.

[JimM11]

@matthubbert77 If you need to take it up with someone then see the info link.

https://www.bt.net/peering-information.html

[matthubbert77]

Interestingly, I've been checking all day and not seen the issue yet. I wonder if something changed in the past 12 hours. Using globalping for pings and traceroutes has had 100% success every time, whereas last night, the opposite. Specifically, 166-49-135-83.gia.bt.net isn't showing up in the hops any more. Maybe it's a timing thing, but will continue to monitor.

[JimM11]

@matthubbert77 As it's BT peering and know idea what or way it goes after the hop ceases, unless you have the exact same route followed and all are/can be different at one snapshot in time, then again if it was an Equipment issue, it may have just been reset overnight. My VF connection seemed to be very consistent on the two individual days one on MTR and the other as Tracert.