RefreshlyAudit

If you found this page in your server logs, RefreshlyAudit is our automated website auditor. It visits a site because somebody asked us to check that site — usually its owner, who is paying us for a report on it.

It reads pages the way a search engine does. It never submits forms, never buys anything, never tries to log in, and never changes anything on your site.

How to identify it

Requests come from one of two user agents:

• Audits — RefreshlyAudit/1.0 (+https://refreshly.tech/refreshlyaudit)
• Daily checks — RefreshlyAudit/1.0 (site watch; +https://refreshly.tech/refreshlyaudit)

When a hosting or security service puts a check in front of the site, our browser waits for it to finish the way any browser does. If a site whose owner asked for the audit still blocks it, the audit continues as a standard Chrome whose user agent ends with RefreshlyAudit/1.0 (+https://refreshly.tech/refreshlyaudit). That only happens on the site being audited, never on a competitor site named in an audit.

Both run from AWS in us-east-1. We do not publish a fixed IP range, so treat the user agent as the claim and not the proof: anyone can send any user agent they like. If you need to be certain a request was really ours, email us with the timestamp and we will confirm it.

Well-known addresses

Besides the pages it finds by following links, an audit requests a short, fixed list of addresses that should not be publicly readable, such as /.env, /.git/config and common backup file names, to tell the site owner if one of them is. It is at most forty requests, three at a time, and it stops as soon as a security service starts refusing. If a file is readable we record only that it exists and what kind of file it is, never its contents. These requests are only ever made to the site whose owner asked for the audit, never to a competitor site named in one.

How to block it

We obey robots.txt. To refuse us entirely, add this:

• User-agent: RefreshlyAudit
• Disallow: /

A rule that withholds part of the site is respected too. The audit simply reports on less, and we tell the customer that is what happened rather than working around it.

The well-known-address checks above are not governed by robots.txt, because robots.txt is not an access control. If your firewall blocks them the report says the checks could not be verified. To let them through, allow the RefreshlyAudit user agent.

How often it visits

An audit is a single pass over the site, run when a customer asks for one — typically once, or once a month on a monitoring plan. A site named as a competitor in somebody else’s audit gets less than that: a read-only sample of at most 20 pages, obeying its robots.txt, and never a well-known-address request. The daily check is one request to the home page to see whether the site is answering, alongside a certificate and domain-registration lookup that does not touch your server at all.

If it caused a problem

Tell us and we will stop. Email audit@refreshly.tech with your domain and roughly when it happened, and we will exclude the site and confirm we have done it. You do not need an account with us, and you do not need to prove you own the domain to ask us to go away.