What robots.txt does
robots.txt is a small text file at the top of your site (yourwebsite.com/robots.txt) that tells crawlers which pages they may visit. One wrong line can stop Google from crawling your whole site, so it's worth checking after every change.
It controls crawling, not indexing. A page blocked in robots.txt can still show up in Google if other sites link to it, just without a description. To keep a page out of search results, use a noindex tag and let Google crawl it.
What this checker looks for
Rules that block Googlebot from your homepage or whole site, rules that block CSS or JavaScript Google needs to render your pages, lines Google ignores (Crawl-delay, noindex, unknown directives), rules outside a User-agent group, relative sitemap addresses, server errors, files served as web pages, and files over Google's 500 KiB limit.
It reads the file the same way Google does: the most specific User-agent group wins, the longest matching rule wins, and Allow beats Disallow when they're the same length.
When robots.txt is missing or broken
No robots.txt (a 404) is fine: crawlers may visit everything. A server error (5xx) is not: Google treats it as "don't crawl anything" and slows down until it's fixed.