robots.txt Tester

The decision comes from group specificity and path length.

• The longest matching user-agent group wins. The wildcard group is the least specific group.
• The longest matching path wins. When two matches have the same length, Allow beats Disallow.
• A trailing dollar sign anchors the pattern to the end of the path, so a query string is a different match.

The result is not a live crawl log.

• The tester combines product-token matching with RFC 9309 path rules. A live crawler can still behave differently.
• 4xx and 5xx states are not rule lines. The result explains that difference instead of pretending an Allow or Disallow exists.
• A pasted file is not fetched. A remote test accepts only a public HTTP(S) URL.

Submitted addresses are not stored to build a persistent user profile.

A URL check sends a server-side request to the target site. Do not submit local-network, metadata, authenticated or private addresses.