User-agent: * # focusing on all spiders Crawl-delay: 10 Allow: /wp-content/uploads/ Disallow: /awstats/ Disallow: /downloads/ Disallow: /scripts/ Disallow: /wp- Disallow: /trackback/ Disallow: /feed/ Disallow: /comments/ Disallow: /category/ Disallow: /2009/ Disallow: /tag/ # Google Image User-agent: Googlebot-Image Allow: / # Google AdSense User-agent: Mediapartners-Google* Allow: / # Internet Archiver Wayback Machine User-agent: ia_archiver Disallow: / # digg mirror User-agent: duggmirror Disallow: / # Yahoo Pipes User-agent: Yahoo Pipes 1.0 Disallow: /sitemap.xml User-agent: Yahoo Pipes 2.0 Disallow: /sitemap.xml Sitemap: http://www.nextgenerationscience.com/sitemap.xml