User-agent: * Disallow: /cgi-bin/ Disallow: /cp/ Disallow: contact.html # spiders can index everything but disallowed directories. Notice that you can block single files like admin.php.