Using Sitemaps

You can index only the sitemaps listed on the webpage by enabling sitemaps in HTTP collection settings.
We support the sitemap.xml if it is in the robots.txt when the settings ”Follow sitemap" is enabled.
If the sitemap.xml has a list of other sitemaps.xml SearchBlox would index the same.
Standard XML sitemaps are supported by SearchBlox. Sitemaps with compressed XML files with tar or gzip file extensions are not supported currently in SearchBlox.
Only sitemaps would be indexed on disabling ignore sitemaps that is, the links available in http://example.com/sitemap.xml would be indexed.
By default, ignore sitemaps would be enabled in HTTP collection settings.
Multiple sitemaps can be indexed on providing multiple URLs in the root path.
Allow/Disallow collection path rules can be considered while indexing sitemap URLs.
Spider depth and other HTTP settings are not applicable for sitemaps.