# robots.txt file for www.nichd.nih.gov User-Agent: * Disallow: /_layouts/ Disallow: /Documents/Forms/ Disallow: /Pages/Forms/ Disallow: /ReusableContent/ Disallow: /Reports%20List/ Disallow: /Search/ Disallow: /SiteCollectionDocuments/ Disallow: /SiteCollectionImages/ Disallow: /WorkflowTasks/ Disallow: /Search.aspx Disallow: /search.aspx Disallow: Search.aspx Disallow: search.aspx # adding new 2008 wild card syntax (see http://www.searchtools.com/robots/robots-txt-elements.html#wildcards) Disallow: */_layouts/ Disallow: */Documents/Forms/ Disallow: */Pages/Forms/ Disallow: */ReusableContent/ Disallow: */Reports%20List/ Disallow: */Search/ Disallow: */SiteCollectionDocuments/ Disallow: */SiteCollectionImages/ Disallow: */WorkflowTasks/ # page-level exclusion Disallow: *WebFldr.aspx$ Disallow: *Upload.aspx$ Disallow: *EditForm.aspx$ Disallow: *external.aspx$ Disallow: *search.aspx$ Disallow: *Search.aspx Disallow: *search.aspx