Using Remove Duplicates

Remove Duplicates

1. The Remove Duplicates feature prevents pages with identical content from being indexed multiple times, improving search efficiency and result quality.
2. When enabled, the system will index only one instance of pages with 100% identical content. The Remove Duplicates feature is disabled by default in Collection Settings. Users must explicitly enable this feature to filter out duplicate content during indexing
3. The content comparison includes:
   • Page title
   • Keywords
   • Description
   • All meta fields
   • Main page content