For webmasters

Here you will find the way our bot indexes your website and what info it takes. We constantly develop our engine. This list will be updated upon changes made to the algorythm, however, this can be done with a delay

– We are trying to make a bot that will use common SEO tools. If you are not familiar with it, please read some articles

– We index and store all public visible pages oof your website. We do not scan the directory but all links on a webpage

– Bot takes your page name within title tags in header section

– For the page descripition it looks for meta container with description attribute

– For now it stores all the content within body container

– On the search results page it returns page name, it's link and description. If no description provided, then the coontent will be shown to the user

– Crawler visits each page once a day so it won't bother your server at all. It indexes pages based on it's last indexed date: older pages are indexed first

– Currently our crawler doesn't read "robots.txt" file because it doesn't scan directory. It scans the public links only