DownloadsDocumentationAPI ReferenceChangelogOpen A TicketContact

## **Using Sitemaps**

  • You can index only the sitemaps listed on the webpage by enabling sitemaps in WEB collection settings.

  • We support the sitemap.xml if it is in the robots.txt when the settings ”Follow sitemap" is enabled.

  • If the sitemap.xml has a list of other sitemaps.xml SearchBlox would index the same.

  • **Standard XML sitemaps are supported** by SearchBlox. Sitemaps with compressed XML files with tar or gzip file extensions are not supported currently in SearchBlox.

  • Only sitemaps would be indexed on disabling ignore sitemaps that is, the links available in `http://example.com/sitemap.xml` would be indexed.

  • By default, ignore sitemaps would be enabled in WEB collection settings.

  • Multiple sitemaps can be indexed on providing multiple URLs in the root path.

  • Allow/Disallow collection path rules can be considered while indexing sitemap URLs.

  • Spider depth and other WEB settings are** not applicable** for sitemaps.