What Is llms.txt? The AI File Everyone Is Still Debating

Definition of llms.txt

An llms.txt is a simple text file (very similar to robots.txt) that helps AI tools like ChatGPT, Claude, and Gemini better understand your website and find your most important content faster.

It acts like a “guidebook” for large language models (LLMs). Instead of making AI models crawl through every page on your website and guess what’s important, an llms.txt file gives precise information on what its key resources are, such as product information, service pages, FAQs, API docs, policies, and knowledge-base content.

You’ve likely seen this topic pop up on SEO forums and LinkedIn: “You need an llms.txt file, or AI will ignore your website.” Many agencies have pondered whether this is something that shifts the needle for their clients – we’ll take a deeper dive into this topic.

What llms.txt Is Actually Trying to Fix

Truth be told, there is a LOT of information on the web. llms.txt is designed to speed up the process of extracting the most useful information as easily as possible for AI systems while side-stepping common problems such as:

• Modern websites can be difficult to read: Most AI crawlers can only understand and parse basic HTML. Even in 2026, they still can’t process content that loads dynamically via JavaScript. Enter llms.txt, which sidesteps this issue through a clean, static Markdown file so that any AI tool can understand it without any problems.
• Too much noise, not enough signal: Contrary to what you might think, when an AI crawler lands on your site, it has no way of knowing what matters most. llms.txt acts like a priority list, showing AI tools first-hand what content matters most.
• Token limits create hard ceilings: AI models work through a token model, meaning it is limited to only processing a certain amount of text at once. This can be an issue for larger websites. llms.txt provides a condensed summary so AI tools get maximum value without hitting those limitations.

You can read the full proposal at llmstxt.org.

The Difference Between Robots.txt and llms.txt

Although on paper an llms.txt file sounds like a robots.txt file for AI platforms, the two actually serve very different purposes.

• A robots.txt file is purely designed to control crawler access for security and privacy purposes. It essentially acts as a fence, informing search engines which parts of the site it can crawl and which areas are off limits. Think of it as a set of rules or boundaries.
• An llms.txt file almost does the complete opposite. It doesn’t block or restrict anything but acts more like an informational map, which assists large language models in finding where a website’s most useful and key pages lie.

Key Differences Compared

Do You Need an llms.txt File to Appear in AI Search Results?

No. While llms.txt remains a hot debate among SEO experts, there is very little evidence that these files are vital to your SEO success as of 2026 for the average Australian small or medium-sized business.

Here’s why we should take the hype over llms.txt with a pinch of salt:

What the Research Shows

So far, only a handful of studies have examined the impact of llms.txt, which include:

The study from ALLMO.ai: It analysed over 94,000 URLs from over 11,000 AI responses between August and December 2025. It tested multiple LLMs, including ChatGPT, Claude, Gemini, Grok, and Perplexity and found no measurable advantage to having an llms.txt file.

The SE Ranking study: The results showed that only 10.13% of 300,000 analysed domains have implemented llms.txt, with the data showing no correlation between having the file and how often a website is cited in AI responses.

The NerdyData study: A code-search report highlighted the low usage rate of the llms.txt file, detecting only 6,777 websites across the entire web that actively deployed an llms.txt file.

There Is No Official Backing Yet From Major AI Companies

No major AI platform has officially confirmed it supports the standard. OpenAI, Anthropic, and Google have all remained silent on whether llms.txt influences ranking, retrieval, or citations in any way.

Because of this, it is still just ‘a proposed standard at the time of writing this post’, and not an official web protocol.

The guidelines published in this text file really only become influential and carry weight when major platforms decide to consistently use them. This widespread adoption is yet to happen.

What Google Says

Even Google has publicly downplayed its importance. Search Advocate John Mueller has confirmed this on multiple platforms, commenting on both Reddit and Bluesky that, as far as he was aware, none of the major AI services was actively using llms.txt files.

That said, the file is not necessarily pointless. It might eventually become more useful as AI search matures and standardisation improves.

History has, however, taught us that many directives that Google has brought out in the past, like metatags, can eventually become ‘phased out’ due to the amount of spam that it was a victim of, which was confirmed by Google.

If llms.txt becomes another unreliable way to rank websites, it may suffer the same fate. As of today, they are already facing criticism due to the level of accuracy they currently suffer from.

Despite the uncertainty around its impact, understanding what the file actually contains is handy if you’re weighing whether you should implement one to your website.

What Does an LLMS.txt File Look Like?

At its core, an llms.txt file is a simple Markdown document. It has elements similar to a typical HTML document, such as a title, a short descriptive text, and H2 headers to add structure and better organise links to a website’s most important resources.

It’s usually split into two main sections: a detailed description and overview of the website at the top, with bullet point links further down pointing to key pages like product guides, FAQs, policies, or other valuable content.

Example of what one looks like (taken from llmstxt.org):

How To Create An LLMS.txt File

While you don’t necessarily need one, ultimately, the choice is yours; after all, it could even future-proof your website should leading LLMs like ChatGPT, Gemini, Perplexity, and Claude start endorsing and supporting the use one day. Luckily, it’s super easy to create your own llms.txt:

The Manual Way

Step 1: Decide What to Include as ‘Important Pages or Files’

Before opening a text editor, think about which pages you think best represent your business to an AI. Here are a few ideas you should consider:

• Product or service pages
• Your most up-to-date blog posts
• Pricing page
• About us page
• Contact page

These give AI tools a clear picture of what your business does and how you help customers.

Step 2: Create Your File

Open a plain text editor (Notepad, VS Code, or TextEdit all work fine) and create a new file named exactly llms.txt. You can format the file using Markdown — here’s our ready-to-use structure you can copy and adapt:

💡 Pro tip: Keep descriptions factual and direct — don’t try to ‘SEO it’ or add any unnecessary marketing fluff. AI tools respond best to plain, informative language.

Step 3: Upload the File to Your Website

Your file needs to be placed in your root domain, so it’s accessible at the following URL: https://yourdomain.com/llms.txt. This will however, depend on your platform:

• WordPress: Upload to your public_html root folder via cPanel File Manager or an FTP client like FileZilla.
• Shopify: Use the Content > Files section in your Shopify Admin to upload the file, then create a URL redirect from /llms.txt to the uploaded file’s CDN URL so it appears accessible from your root domain.
• Wix: Now automatically generates and maintains an llms.txt file for eligible sites. You can find it under SEO & GEO → Tools and settings → llms.txt in your site’s dashboard. You can also opt out of using it in there, although access is currently being rolled out gradually, so not all accounts may have it yet.
• Custom / static site: Add it directly to your root public folder (public, static, or www).

Once uploaded, visit https://yourdomain.com/llms.txt in your browser to make sure it’s live and working correctly.

💡 Pro Tip: Review your file now and then just to keep it fresh and up to date. Remember to remove any links to outdated pages and add new ones as your website grows.

Tools To Help You — Verified llms.txt Generator Tools (2026)

If you’d prefer not getting hands-on, you can use one of the many online tools out there that can crawl your site and generate one for you in minutes. For websites that are looking for CMS integration, tools like Yoast SEO, RankMath, and All-in-One SEO are great options.

Wix also generates the file automatically for eligible sites, while Shopify users can use third-party apps from the App Store.

Some of the better-known generators include:

Who Is Actually Using llms.txt?

While it’s only natural to assume that adoption of the file is heavily in favour of the tech world (including marketers), the actual data record could be skewed by websites that are using generators without even knowing they are doing so. This is thanks to the popularity of CMS systems that auto-include generators (like the SEO plugins found on WordPress).

However, some of the more notable big hitters that have implemented llms.txt include:

• Anthropic — Publishes its own file, while using a dual-file setup (llms.txt and llms-full.txt). It has not confirmed publicly that its crawlers use the standard.
• Cloudflare — Organises its content by specific product categories, allowing AI tools to grab context only for the cloud or security product they need, rather than ingesting the entire site.
• Stripe — Uses it to index their massive payment processing ecosystem and to avoid hitting AI token limits.
• Perplexity — One of the few AI platforms to have published their own llms.txt. They cover this in their developer documentation and API references.
• Zapier — The king of app connections, they use it to organise their extensive integration documentation, making it easier for AI tools to scan through all their connectors.

For a full list, you can visit directory.llmstxt.cloud, which offers an up-to-date sneak peek into the latest websites (currently around 840 websites).

The Real Reason LLMs Might Not Know Much About Your Business

Here’s something that trips a lot of people up. Unlike Google, large language models don’t systematically index your entire website. They don’t crawl page by page, store your content in a database, or pull it up when someone searches for you.

Instead, AI models typically scrape content in real time, focusing on whatever is easiest to find and read. For this reason, your AI visibility isn’t really a file problem (like llms.txt or robots.txt), it’s a continuation of the traditional SEO challenges, such as building quality and helpful content, growing your authority and ranking higher in SERPs.

Currently, the businesses that seem to get cited by AI are those that have invested in a few core practice areas:

• Building a clear entity presence online and front-loading it through schema markup, earning brand mentions and building links from sources that AI already trusts.
• Writing well-structured copy where answers are front-loaded and easy for AI to extract.
• Maintaining consistent quality brand signals across multiple platforms (including good reviews).

These are all things that a well-placed llms.txt file might support, but on its own, it isn’t currently a game-changer.

If you want a deeper look at what actually moves the needle in 2026, we’ve covered more on GEO, entity SEO, and voice search strategy – in this blog post on how an SEO consultant can help you get found.

The Bottom Line

llms.txt is a real thing. It’s just not the game-changer it’s been made out to be. The businesses quietly pulling ahead in AI search results aren’t doing so because of a text file sitting on their server; they’re doing it because of good old-fashioned hard graft and have earned the right to be mentioned. Their content will most likely be comprehensive, authoritative, and well-structured around how AI tools actually think.

Our advice: do the same. Cover your topic properly. And if you’re still not sure where to start, get in touch with the Business Medics team for a free SEO consultation.