Posts Tagged ‘Googlebots’
Prevent Google using DMOZ site description
Posted by David | Filed under Web Design Tips | Read Full Article
If you are an unhappy webmaster because you can’t seem to change your website description on Google due to what its description is in the DMOZ (Open Directory) then here is something you will find useful!
Generally Google will take either a meta tag description of your site when it displays it in a search engine, [...]
Tags: DMOZ, Googlebots, Open Source Directory
Robots.txt
Posted by David | Filed under Web Design Tips | Read Full Article
A Robots.txt is especially important for those of us with WordPress or similar CMS that have directories we don’t really want Google poking its spider nose into! For example I don’t want Google rummaging through wp-admin/ wp-content/ or wp-includes/ so how can I stop that happening?
Simple - Googlebots along with many other good bots obey a [...]
Tags: Googlebots, Robots, Robots.txt, Search Spiders, Spiders