Free Robots.txt Generator Robots.txt Generator Website URL Sitemap URL (Optional) Crawl Delay (seconds) User-Agent All Crawlers (*) Googlebot Bingbot SemrushBot AhrefsBot Disallow Paths (one per line) /wp-admin/ /wp-login.php /cart/ /checkout/ /admin/ /private/ Allow Paths (one per line) /wp-content/uploads/ /assets/ /images/ Generate robots.txt Live Preview Copy Download
