There seem quite a few RIPE Atlas probes which are giving latency on 150ms + range. Seems like they are downstream or downstream of downstream of Tata Comm’s AS4755 and routing is happening via Singapore!

Two of such traces

This isn’t how regular routing should work since even if both do not announce routes to each other, Jio’s routes are visible at NIXI. E.g NIXI Noida shows 169.149.192.0/18 visible which covers that test IP. Tata AS4755 is probably rejecting that IP.

And if we look at Tata AS6453 routing table for their Chennai PoP, it shows AS_PATH as AS3431 (PCCW Global) who happens to be upstream of Jio’s International network AS64049.

Clearly, that’s just bad routing!

It might be the case Tata is doing intentionally without any evil plans because NIXI might just not have sufficient capacity. Overall ISPs should peer with PNI’s as well as we need IX’es other than just NIXI to promote more peering and promote better interconnect.

Disclaimer: This post and expressed comments are in my personal capacity and my employer has no relation with it.

Overall pretty good and direct. Basically latency value is also as we expect till hop 12 because forward route (i.e from BSNL > Softlayer) is direct from BSNL router on hop 12 but for routers below it they are taking route via US. Return path trace is not showing those routers because BSNL is dropping ICMP.

Reason for problem:

Forward path is terribly bad here because BSNL let usual BGP route selection algorithm to deal with it. Basically BSNL is getting multiple routes for that prefix from Softlayer. One from it’s IP port in India with Tata-VSNL AS4755 and other from it’s port from Tata in Los Angles (Tata AS6453) over IPLC.

So possible routes as per AS paths are:

AS9829 > AS4755 > AS6453 > AS2914 > AS36351

AS9829 > AS6453 > AS2914 > AS36351

Based on default property of BGP, it is picking short AS path i.e 2nd one. In case of #1 BGP session between BSNL AS9829 and Tata-VSNL AS4755 is within India.

Half latency and that’s BSNL router in Delhi/Noida where they are taking drop from Tata. It’s BSNL’s router but sitting on Tata’s IP for BGP session. So this clearly tells that when we see routes from AS9829 to AS4755 Tata-VSNL they are between routers within India.

Now coming back to bad route between BSNL and Softlayer, in that case first few hops are:

Hop 5 has latency of 300ms (usual for India > US routes). Again assuming 209.58.85.5 is coming from /30 and as per usual BSNL practice next IP in that subnet i.e 209.58.85.6 would be on BSNL’s side, let’s ping 209.58.85.6:

So clearly BSNL on 218.248.255.99 is placed in India and is having a BGP session with Tata AS6453 router in Los Angeles. This is over an IPLC circuit of Tata Communications.

Possible fix…

Following an amazing quote – “Never call it a problem unless you have the solution!“

So problem here is not really via Tata’s network. They are just selling bandwidth in form of two products – IP Transit & IPLC. It’s BSNL’s wrong idea of using IPLC carelessly. Likely BSNL won’t care or put much effort in fixing it.

There can be a possible fix from Softlayer side. If they blackhole prefix announcement to BSNL AS9829 via Tata AS6453, BSNL will never pick their IPLC (or even IP) route. Instead they will just pick route via any other upstream like Airtel or Reliance Globalcom.

Let’s look at relationship of Tata AS6453 with PCCW Global (upstream for Softlayer)

Thus if Softlayer could get it’s upstream providers (like PCCW in this specific case) to use 65009:9829 – this will ensure that route learnt by Tata AS6453 from PCCW Global AS3491 is NOT exported to BSNL AS9829. Thus BSNL will instead get route via Bharti Airtel AS9498 or Reliance AS18101.

I just sent this detailed info as email to Softlayer and BSNL. And oh yes – I don’t know why hostgator.in is hosted in Softlayer Singapore anyways. They provide hosting in India out of Ctrls datacenter. Why they host their own home site in Singapore is something beyond my understanding!

With hopes that your packets to Singapore are not routing via US, time for me to get back to my “cramming” for exams. 🙂