NokofyAuditBot

Crawler of the SEO analysis tool Nokofy Tools.

What it does

It reads a site's public HTML pages and builds a technical SEO report for the site owner — titles, meta descriptions, response statuses, internal links, images and structured data. It is started by a user of our tool, either manually or on a schedule.

It does not submit forms, does not log in, does not execute JavaScript, does not collect personal data and does not keep copies of your pages — only the aggregates the report needs.

How it behaves

• Respects robots.txt — Disallow, Allow and Crawl-delay alike.
• At most 30 requests per minute per site (one every 2 seconds), one at a time — never in parallel.
• At most 500 pages and 25 minutes per analysis.
• On a 429 or 503 response it slows itself down (5s → 15s → 45s) and honours the Retry-After header.
• Requests compressed content (gzip) to keep transfer small.

How to slow us down

Add Crawl-delay to your robots.txt — we honour it without exception:

User-agent: nokofyauditbot
Crawl-delay: 10

How to block us entirely

This stops us immediately, with no need to contact us:

User-agent: nokofyauditbot
Disallow: /

We read robots.txt before every analysis and cache it only briefly, so your change takes effect within minutes.

Parts of the site you want left alone

Individual paths are blocked the same way:

User-agent: nokofyauditbot
Disallow: /admin/
Disallow: /cart

Contact

If the bot causes you trouble, if you want a different pace, or if you believe it is misbehaving — write to podrska@nokofy.com and we will get back to you promptly. Please include the domain and the time from your log so we can identify the exact request.

Nokofy Tools · this page describes the crawler's behaviour as implemented in the code at the time of the latest release.