Using Remove Duplicates

Remove Duplicates

• When Remove Duplicates is enabled, pages with 100% same content that is, duplicates, will not be indexed.
• Here the content for checking duplicates includes title, keywords, description, other meta fields as well as the content of the page.
• If there is any difference in the whole content then the files/URLs will not be considered as duplicates.
• By default Remove Duplicates would be disabled in Collection Settings.
• The user needs to enable it to remove the duplicate content while indexing.