Generate, validate, and test robots.txt rules.
Choose which crawler the rules apply to.
Add Allow or Disallow directives. Paths should usually start with /.
Paste robots.txt content to check common syntax mistakes and risky directives.
Test a URL path against pasted robots.txt rules for a selected crawler.
This tester implements common robots.txt matching behavior including wildcard *,
end-anchor $, longest-match precedence, and Allow winning on equal specificity.