|
|
GoogleBot and other robots |
|
|
27.03.08 12:00
I have noticed some visits from bots, and Google now even shows my pages quite high - when the searches are specific enough ;)
But it frustrates me to see how many identical pages are indexed, just because some small parameter is different. So I had to learn how to make a robots.txt file. I have also made and submitted a proper sitemap, to guide the GoogleBot in the right direction.
I think I have figured it out now, and look forward to more streamlined robot visits, and a smaller and more correct amout of links from the major search engines.
|
|
|
|