So while adding a robots.txt file to one of my websites for better SEO, I realized that I can look at pretty much any websites' robots.txt file. That got me wondering if there are any cool quotes, hidden art, or anything else companies have put in their robots.txt files. Turns out, there are quite a few.

But first, if you have no clue what a robots.txt file is, let me explain.

What is a Robots.txt File?

This file is used to communicate with web crawlers and bots about which parts of a website they should and shouldn't access. Usually, the file is located at the root of a website (e.g. kennywan.me/robots.txt), uses a simple text format, and is publicly accessible. Because of that public accessibility, some creative companies and developers have turned their robots.txt files into a place to show personality, share ASCII art, add humor, and hide fun messages.

Ascii Art

Interesting Messages

Between all of the fun, quirky messages I've read, the commonality is that they all reference robots in some way or another—which I find very amusing.

Many companies will advertise their careers pages, such as Instacart, which has this comment at the bottom of their robots file: If you're not a bot, we're hiring <url>. It's a clever way to reach the type of curious developers who would potentially be great hires.

Others will have some random interesting comment about some bot, such as Starbucks: Slow an overly aggressive MJ12bot from the UK and Dartmouth College: Roverbot is evil <http://www.roverbot.com>.

Some companies play on their own slogans with a robot twist. Nike says: www.nike.com robots.txt -- just crawl it. Tiffany & Co. does something similar with: www.tiffany.com robots.txt -- where elegance meets the web, crawl with grace. Both are perfectly on-brand.

However, many of my favorites are the fun and quirky robot-themed comments that some places include. For example, Youtube includes this in their robots file:

# Created in the distant future (the year 2000) after
# the robotic uprising of the mid 90's which wiped out all humans.

And Yelp references Isaac Asimov's famous Three Laws of Robotics:

# As always, Asimov's Three Laws are in effect:
# 1. A robot may not injure a human being or, through inaction, allow a human
#    being to come to harm.
# 2. A robot must obey orders given it by human beings except where such
#    orders would conflict with the First Law.
# 3. A robot must protect its own existence as long as such protection does
#    not conflict with the First or Second Law.

Conclusion

If you're curious to see what other companies have hidden in their robots.txt files, it's super easy to check. Just go to any website and add /robots.txt to the end of the URL. You never know what you'll find, whether it's a joke or some ASCII art. But, when you do find something creative, it's a nice reminder that real people with a sense of humor built the site you're visiting.

I've spent way too much time going down this rabbit hole, but it's been super interesting. I love how these little Easter eggs make the technical side of the web feel a bit more human, and inspirted me to add my own little human touch to my websites' robots files. So next time you're building a website, consider adding a little personality to your robots.txt file!