Traceroute Isn't Real

I think the "The Worst Diagnostics In The World" section is a bit simplistic about what traceroute does tell you. It can tell you lots of thing beyond "you can reach all the way". Specifically, it can tell you at least some of the networks and locations your packet went through and it can tell you how far it definitely got. These are extremely powerful tools as they rule out lots of problems. It's useful to be able to hand an ISP a "look, I can reach X location in your network and then the traceroute died" and they can't wonder "are you sure your firewall isn't blocking it?"

It's still a super-common tool for communicating issues between networking teams at various ASes. That the author's ISP thought they were too small to provide reasonable support to is not a strike against traceroute. Rather, it's a strike against that ISP.

I wish MTR (My Traceroute) was standard in all operating systems. It offers a number of benefits over Traceroute. MTR essentially combines the functionalities of traceroute and ping, providing a more comprehensive and dynamic view of network paths and performance.

MTR runs continuously, gathering real-time stats that reveal both packet loss and latency trends over time. MTR provides minimum, average, and maximum response times, plus the standard deviation. This is especially useful for troubleshooting intermittent issues or spotting latency spikes.

Of course, MTR isn’t perfect and still faces some of the same challenges as traceroute, like dealing with ICMP rate-limiting, load-balanced paths, or certain network setups that obscure hops. But overall, it provides a richer, more nuanced view, making it a preferred tool for network diagnostics and troubleshooting.

> Traceroute, as far as the industry is concerned, does not exist.

> Look it up. There is no RFC. There are no ports for traceroute, no rules in firewalls to accommodate it, no best practices for network operators. Why is that?

I know this is getting into semantics but this argument is ridiculous. Everything that isn’t explicitly specified in an RFC and has its own protocol doesn’t exist to the industry? Who thinks like that? It’s using behavior of the system to get a result, how does that mean it doesn’t exist? If I do a Speedtest by sending traffic over the internet, does my program not exist because there is no SpeedTest Protocol with its own port, and no RFC has ever been written about it?

This is a nice rant, but telling people not to use this tool because it has many flaws isn't very helpful. AFAIK, there's not a better tool out there to use; the alternative is hoping things will get better by magic and the passage of time (which, to be fair, is somewhat effective; but not ideal if you've got things to do)

Yes, traceroute doesn't address it's hard to get in touch with someone who can help. Sure, anything to do with ICMP probably has to deal with rate limiting (and the two people are tracing so the packet loss is 50% effect is real, and frustrating). But when I've had network problems and a contact who is willing to help, they really want a traceroute or mtr to help narrow down where the problem is.

The trick is finding the right settings to get a mtr that shows what you need to show. My big problem that I needed mtrs for was server a talking to server b over several hops with 2 or 4 way aggregation on each hop. Most of the paths are clean and I can see 0% loss, but there's one link in there with say 10% loss. Default settings will not get you anything useful; you've got to test many 5-tuples (dst host, src host, protocol, dst port, src port) to find one that shows loss and one that doesn't, and then send an mtr from those. You may want to run mtrs in the reverse direction too. You'll need to have a slow probe rate for the mtrs you share, to avoid/reduce the rate limiting issues.

If you can't count on the far side destination definitively responding to pings, your mtrs are going to be too messy to share, unfortunately.

If there's MPLS in the loop, there's an extension to get data from that too, and sometimes it works.

Traceroute isn't real, I like this, sounds like something I'd say. But it is real, it's definitely real, it's installed on my laptop, it's probably installed on yours. I know fairly well how it works, understand many of its short comings, but I still love a traceroute, as does absolutely everyone even most people who truly understand it, or it really wouldn't exist. I am a visual understander, a picture paints a thousand words, even a distorted unreliable picture with a handful of details that are sometimes useful hiding amongst a great deal of irrelevant nonsense is sometimes better than nothing at all.

It's not an ugly hack it's a beautiful elegant solution to the problem of not knowing how your traffic is mostly probably being routed.

The challenge with any tool that uses the probe traffic other than the traffic of interest is that the results may be specific to the probe traffic and completely different from the one you care about.

In theory, iOAM (https://datatracker.ietf.org/doc/rfc9326/) is a much more robust mechanism.

In practice, internet works on the least common denominator, which means that Traceroute (which is a clever hack on top of the ICMP TTL exceeded behavior, a required internet standard) is often the best one can have, if at all. (And if not, then one has to resort to uglier hacks)

That said - one should not underestimate how much info one can dig out by varying TTL/hop count, changing the 5-tuple (source and destination address and ports + protocol), and tweaking the packet rate.

And the dismissive attitude about “absolutely impossible to do anything with this info unless you are Fortune 500” is wrong. For a counter example of cooperation between the “people of the internet”, here’s a nice presentation:

https://youtu.be/G_Ir_gRlst0?feature=shared

As one can derive from the above - it’s absolutely possible, just that the level of SNR required to be reacted to is rather high, well above “my Traceroute is not showing what I think it should be showing”. Which, given the population of the internet, isn’t entirely unreasonable.

No, but it's actually worse than that.

Not only is it contingent on your intermediaries actually responding to your packet with the diagnostic information you want, it assumes that the diagnostic response will also be able to get back to you. If, for instance, your links are failing over super frequently or you have something hilarious happen like the response packet ALSO having a too-low TTL, you may not get a response as you expect.

But wait, there's more! Precisely because of that stepping-increase of TTL, by necessity, it must send as many TTLs as necessary to reach the endpoint. That means one packet per TTL. Remember what I said about links flapping? There is no guarantee that any two packets will or even should go the same route, for any number of reasons, some potentially even legitimate. In some situations you may see different hops between hosts that aren't actually even physically connected!

And I love MTR, but it can handle some of these issues really... interestingly. I seem to semi-regularly see it in a state where it's showing a bunch of packet drops, but really I just have to refresh the display because some state or another got desynchronized.

That said, on simple paths that don't change a whole lot, it's great. A very clever way to expose information you might not otherwise ordinarily have that might even be key to resolving any given issue. You just have to remember just how surprisingly much of networking is made up.

Link to PDF version of the "A Practical Guide to (Correctly) Troubleshooting with Traceroute" presentation (referenced from the post) that doesn't require registering with scribd to download:

https://archive.nanog.org/sites/default/files/traceroute-201...

For all that it mentions common misconceptions, the article is still wrong in at least 3 ways:

First, traceroutes can, if you control both endpoints, place bounds on where a network error is.

Second, traceroute is useful if there are three endpoints and you control at least 2 of them.

Thirdly, you do in fact know something about other people's networks by the mere fact that you've traversed the network before at different times.

I appreciate this, especially for Richard Steenbergen's traceroute presentation which I need to dig into. There's a 2020 version of the presentation if anyone is interested, since the Scribd/Slideshare version in the article is from 2014-ish.

video: https://www.youtube.com/watch?v=L0RUI5kHzEQ

slides: https://storage.googleapis.com/site-media-prod/meetings/NANO...

Edit: yes, I fully agree that traceroute is flawed, it's only ever going to give you an incomplete or even misleading piece of the picture and you shouldn't take what you see as gospel. That said, it has its uses especially for networks that you control and to let you know where to maybe start digging - which is all that any tool does.

This guy is almost comically emphatic. He makes his point so firmly that I almost want to wait for him to finish and ask “So, are you saying Traceroute is not always the best utility for network diagnostics?” then watch his hair catch fire.

traceroute is a useful tool for (amongst other things) determining where a system is physically. It doesn't always give you enough information.

Traceroute to news.ycombinator.com:

  3   96.108.68.141 (po-200-xar01.maynard.ma.boston.comcast.net)  12.105ms  11.724ms  11.931ms 

  4   96.108.68.141 (po-200-xar01.maynard.ma.boston.comcast.net)  13.011ms  10.727ms  19.861ms 

  5   162.151.52.34 (be-501-ar01.needham.ma.boston.comcast.net)  13.988ms  14.721ms  12.921ms 

  6   162.151.52.34 (be-501-ar01.needham.ma.boston.comcast.net)  14.999ms  16.688ms  12.997ms 

  7   4.69.146.65 (ae0.11.bar1.SanDiego1.net.lumen.tech)  76.044ms  79.624ms  78.017ms 

  8   4.69.146.65 (ae0.11.bar1.SanDiego1.net.lumen.tech)  83.962ms  108.675ms  78.987ms 

  9   \*  \*  \*

I recall using this during a sales pitch by some IP Geolocation company that were very proud of their technology. The example that they used, they claimed was in Morristown, NJ. A quick traceroute (from Massachusetts) revealed that the IP was somewhere in the UK as the last hop was close to Heathrow Airport. We did not purchase their solution!

Good article. Reminded me of a time when I was told that a server was taking over 2 minutes to respond and they proved it with traceroute.

They were running "time traceroute host"

Traceroute is often useful because it almost always diagnoses LAN vs. WAN issues. Since most home routers/modems will send TTL Exceeded and so will most edge ISP routers, it is often the quickest way to see where the problem is when there is little or no connectivity.

> There are now things like MPLS, which operate by encapsulating IP […] Without getting much further into how that works: It is completely impossible for it to satisfy the expectations of traceroute.

The author probably should've gotten at least a bit further into how that works, because it is absolutely possible to get an IP traceroute across an MPLS backbone. It works in a slightly curious way - the MPLS payload (original IP packet) is replaced with the ICMP message, but the packet continues to progress forward rather than "turning around"; it only "changes direction" at the end of the MPLS domain. A good number (yes, not all) commercial router vendors implement this.

weird - everything claimed as wrong or bad is, IMO, utterly sensible and robust in the real, mean old world. by the author's logic, TCP's entire logic of congestion control is terrible because it's not eg, a deterministic credit-based mechanism that depends on everyone implementing it perfectly.

I always thought it was a bit humorous Ethernet has a more standardized and defined traceroute functionality than IP as part of https://en.wikipedia.org/wiki/IEEE_802.1ag (typically only found in carrier Ethernet solutions in practice though).

If you run traceroute multiple times - for example throughout the day - and overlay the results, you will have a better idea of the routes that [ICMP or TCP or UDP] packets are taking between src and dest.

It is trivial to add or hide hops to traceroute by just changing the ttl on certain or all packets.

mtr does traceroute.

There's a 3d traceroute in scapy.

Scapy docs > TCP traceroute: https://scapy.readthedocs.io/en/latest/usage.html#tcp-tracer... :

> If you have VPython installed, you also can have a 3D representation of the traceroute.

> One of the "chapters" in my presentation was about traceroute, and it more or less said "Don't use it, because you don't know how, and almost nobody you'll talk to does either, so try your best to ignore them." This is not just my opinion, it's backed up by people much more experienced than me. For a good summary I highly recommend this presentation.

I'm being pedantic but this paragraph was bizarre to read. You are basically telling us we or anyone we know won't know enough about traceroute not to use it but you and many people you know do know enough. It is presumptuous but also inconsistent. Are there people who know, or not?

The article's stance on traceroute is heavy on cynicism but light on practical understanding. Dismissing traceroute outright because it doesn’t solve advanced networking issues is shortsighted. Traceroute wasn’t designed to tackle complex routing issues in BGP or MPLS-dominated environments. It originated in the ARPANET era as a tool for basic IP diagnostics, and while networking has evolved, it still holds practical value for pinpointing straightforward routing issues and helping teams communicate about network reachability.

Traceroute may not be perfect, but the idea that it’s “useless” or fundamentally flawed ignores the tool's actual purpose and its value in controlled network environments or simple troubleshooting scenarios. For instance, traceroute remains effective in diagnosing where an issue shifts from LAN to WAN, as a starting point to eliminate local network problems. In fact, as some commenters pointed out, traceroute often helps ISPs and network engineers by offering visual, actionable clues about where connections are failing or lagging. If you have a network you’re responsible for end-to-end, or you’re troubleshooting with other network teams, traceroute can save time and clarify if there’s an issue within your own network or further upstream.

As for claims that traceroute outputs are too "random" or unreliable to be useful, that’s only the case when people misuse it or fail to understand how it works. No seasoned engineer expects traceroute to provide flawless accuracy, but they know what patterns and data points to look for and how to combine it with other diagnostics. Calling it a "hack" overlooks the simple elegance of leveraging TTL Exceeded responses to map packet paths—a clever solution that’s worked for decades precisely because it provides unique visibility in a non-intrusive way.

The problem isn’t traceroute itself but rather how it’s used and interpreted. Sure, if you’re just firing off traceroute commands without an understanding of what the results mean, you’ll likely draw wrong conclusions. But for those with the knowledge to interpret it properly, it’s a simple, lightweight tool that often provides just enough to kick off troubleshooting.

Instead of writing off traceroute as useless, a more helpful approach would be to recognize its limits, use it within those bounds, and understand that while it won’t solve every networking mystery, it’s still valuable in the right contexts. No diagnostic tool is infallible, and the defeatist attitude of "it’s useless because it’s not perfect" is just unproductive. The goal isn’t to replace traceroute but to use it wisely alongside other tools to get a fuller picture—especially when alternatives aren’t widely implemented or easy to use.

I'll try not to spark any debate and keep it brief—the article is aimed at people who consider themselves "power users." It might also be informative for those who don’t view themselves that way. If you learned what traceroute is from this article, sticking with this info for now is probably fine. But if you already know how it works, you can skip reading. The whole article "isn't real".

It can be used wrong! It's not perfect! It could be better!

Doesn’t matter as it still looks cool when you run it and step away for awhile so all passers-by assume serious business is going on.

It had its use, back then it was mostly intranets, so it was useful, back then there werent much concepts of spoofing, vpn, tor like layering and routing, encapsulation, etc Yes in modern world on public internet it might not be much useful

But still it gives basic overview a starting point for any forensic investigation or debugging network problemns

slams fist on table Irrelevant! I want to have my cool global map and watch lines connecting nodes in real time so I can pretend to be a secret government hackerspy!

(more seriously, when I first read up how traceroute actually works I was laughing for three hours straight)

I strongly disagree the TTL Exceeded message isn't a feature

> Features are things that enable functionality. It doesn't do that.

It does. It enables traceroute. Your entire argument is defeated by the fact it gives you this functionality.

And yeah it's not perfect. Stop making perfect the enemy of good.

In the journey of learning, it is certainly a good point when our understanding grows so great, that we can point to things that are unknown and claim with certainty that they are bullshit.

It is no easy feat, as to denounce a concept as bullshit, we need to effectively prove a negative, which depends on knowing either all of the things adjacent to that subject, or most of them, and coming to the conclusion that it's not like the others.

It's like learning that Astrology and Phrenology are not like Astronomy, or Psychology.

My current suspect for non-real status is Software Architecture, as a subject of study I don't think it holds any merit. And to the extent that it denotes something real, it, they are already covered in other disciplines with clearer peripheries in classical academic curricula and folklore domains.

It's interesting how this "not real" tool has helped me find and fix issues over the years.

I'm confused by something in the "A Practical Guide to (Correctly) Troubleshooting with Traceroute" which is referenced from the article.

Slide 8 titled "Traceroute – What Hops Are You Seeing?" says:

> By convenction, the ICMP is sourced from the ingress interface.

(I assume the author means "the source address of the ICMP message is the address of the ingress interface")

> Random factoid: This behavior is actually non-standard. RFC1812 says the ICMP source MUST be from the egress interface. If obeyed, this would prevent traceroute from working properly.

(I assume by "ICMP source" the author means "the source address of the ICMP message" because I don't see what else it can mean).

To clarify: from the text before that, and the drawing on the slide, the egress interface the author talks about is the egress interface the original message would have taken had its TTL not expired.

Now, I had a look at RFC 1812 (Requirements for IP Version 4 Routers) and I don't see where it says what that slide claims. The closest I can find is section 4.3.2.3 Original Message Header (https://www.rfc-editor.org/rfc/rfc1812#section-4.3.2.3) which says:

> Except where this document specifies otherwise, the IP source address in an ICMP message originated by the router MUST be one of the IP addresses associated with the physical interface over which the ICMP message is transmitted. If the interface has no IP addresses associated with it, the router's router-id (see Section [5.2.5]) is used instead.

To me that reads completely different from the claim on that slide (and also looks like I would have expected).

The author of the presentation seems more knowledgeable about networking details than I am, so it's very well possible that he's right and I'm misunderstanding something. Can anyone shed some light on that?

References:

2016 version of the presentation as linked from the article: https://www.slideshare.net/slideshow/a-practical-guide-to-co...

Updated 2020 version as helpfully linked by 1xdevnet in comment https://news.ycombinator.com/item?id=42056734: https://storage.googleapis.com/site-media-prod/meetings/NANO...

RFC 1812: https://www.rfc-editor.org/rfc/rfc1812

> No, AT&T is not going to push your complaint up the line to XO. Haha. No.

Maybe not my complaint... but I'm sure there's somebody that could do it.

Who would you have to be, to be able to convince AT&T to bother their fiber vendors for you?

Could, say, the NSA get a routing loop fixed within an hour just by shouting into a phone?

funny and smart.

Cloud infrastructure and firewalls alone are a good reason to not use traceroute.