PoliasBot
Have you seen PoliasBot in your website's logs? Here you can find out what it is and how to block it, if you want to.
What is PoliasBot?
PoliasBot collects public web pages for the search index of Polias, a European search engine. It reads pages just as any visitor sees them, so that they can be found with us.
How to recognise it
Its user agent contains PoliasBot, for example:
PoliasBot/0.1 (+https://polias.eu/bot; hallo@polias.eu)
PoliasBot crawls from our main server 162.55.103.77 (polias.eu/bot/ips.json) and from rented extra servers whose addresses change. That is why it is best recognised by its user agent; if in doubt, write to us at hallo@polias.eu.
How it behaves
- It respects
robots.txt(rules forPoliasBot, otherwise for*). - It fetches only a few pages per website and pauses in between.
- It never submits forms and never logs in anywhere.
How to block it
Add this to your website's robots.txt:
User-agent: PoliasBot Disallow: /
Or write to us at hallo@polias.eu and we will remove your website from the index completely. The same address applies for questions or problems with our crawler; we're happy to reply directly.
And Polias-KI?
Alongside the crawler, there is Polias-KI. It only opens individual pages when someone asks a question in the
Polias chat that the page is relevant to, and it doesn't store them in the index. It follows
the robots.txt rules for Polias-KI, otherwise for PoliasBot, otherwise
for *.