Filesystem Collection

SearchBlox includes a built-in file system crawler to index content from file systems.

Creating a File Collection

To create a File System Collection in SearchBlox, follow these steps:

1. Log in to the Admin Console

   • Open SearchBlox and sign in.

2. Go to the Collections tab

   • Click "Create a New Collection" or the "+" button.

3. Choose the Collection Type

   • Select "File System Collection".

4. Name your Collection

   • Enter a unique name (example: "SalesDocs").

5. Set the RAG Options

   • Enable if you want to use Hybrid RAG search.
   • Disable if you don’t need it.

6. Set Access Permissions

   • Pick Private or Public access for the collection.
   • Turn Encryption ON or OFF based on your security needs.

7. Select the Content Language

   • Choose a language if it’s not English.

8. Click Save

   • Your File System Collection is now created.

Once the File Collection is created, you will be taken to the Path tab to set directory paths and apply file or folder filters for the Collection.

File Collection Path Settings

Directory Paths

The directory path is where the crawler starts searching for files. It automatically index through all folders inside it. In the Paths tab, just enter at least one folder path (like c:\salesdocs or /var/web/html/salesdoc) and save the settings.

Allow/Disallow Paths

Allow and Disallow filters help you control what gets indexed by letting you include only the folders you want and exclude any unwanted files or documents.

The path settings are listed in the table:

File Collection Settings

The Settings tab contains adjustable options for the file system crawler and indexer. SearchBlox already provides default settings when you create a new collection, and you can modify the available options as needed.

📘

Generate Title, Description and Topics using SearchAI PrivateLLM and Enable Hybrid Search:

• Choose and enable Generate Using LLM and Auto Relevance

• By clicking Compare Keyword Search with Hybrid will redirect to the Comparison Plugin

Settings	Description
Title	Generates concise and relevant titles for the indexed documents using LLM.
Description	Generates the description for indexed documents using LLM.
Topic	Generates relevant topics for indexed documents using LLM based on document's content.
Auto Relevance	Enable/Disable Hybrid Search for automatic relevance ranking

Indexing File Share Using File collection

• To index files from a file share, enter the share path (UNC path) of the folder containing the files you want to index.
• 

• If the file share is on another server in the same network and needs permission, run the SearchBlox server service with admin access and enter the required credentials.
• Using an admin account or any account that has access to the shared files will ensure the files are indexed successfully.

Schedule and Index

You can set how often and when a collection should start indexing from its root paths. SearchBlox supports the following schedule options:

• Once
• Hourly
• Daily
• Every 48 Hours
• Every 96 Hours
• Weekly
• Monthly

The following operation can be performed in file collection:

👍

Best Practices

• Always enter a correct directory path or valid UNC path in the File Collection settings.
• When mapping a directory to a URL, make sure the same directory path is entered in the “Map Directory Path” field.
• If you have multiple collections, schedule them so only 2–3 collections index or refresh at the same time.

Manage Documents Tab

• The Manage Documents tab allows you to perform these actions:

  1. Add or update documents
  2. Apply filters
  3. View document content
  4. View metadata
  5. Refresh documents
  6. Delete documents
  

• To add a document, click the "+" icon.

• Enter the document or URL, then click Add/Update.

• After adding it, the document URL will appear, and you can perform all the actions listed above.

• To delete a document, enter its file path and click Delete.

• To check the status of an indexed file, click View Metadata.

Models

Embedding

• Provider specifies the embedding provider used to generate vector representations of documents.
• Model defines the embedding model used to convert document content into vectors for semantic search.

Reranker

• Provider specifies the reranker provider used for improving search result relevance.
• Model defines the reranker model used to re-score and reorder search results based on relevance.

LLM

• Provider specifies the Large Language Model provider used for AI-powered features.

• Model defines the LLM used for tasks such as document enrichment, summaries, and SmartFAQs.

• These settings override global configurations and apply only to the current collection.