Using Remove Duplicates

Remove Duplicates

  • When Remove Duplicates is enabled, pages with 100% same content that is, duplicates, will not be indexed.
  • Here the content for checking duplicates includes title, keywords, description, other meta fields as well as the content of the page.
  • If there is any difference in the whole content then the files/URLs will not be considered as duplicates.
  • By default Remove Duplicates would be disabled in Collection Settings.
  • The user needs to enable it to remove the duplicate content while indexing.