Amazon S3 Collection

Creating Amazon S3 Collection

You can create an Amazon S3 collection by following the steps given below.

  • After logging in to the Admin Console, select the Collections tab and click on Create a New Collection or "+" icon.
  • Choose Amazon S3 Collection as Collection Type
  • Enter a unique name for your collection (for example, AmazonS3).
  • Choose Private/Public Collection Access and Collection Encryption as per the requirements.
  • Choose the language of the content (if the language is other than English).
  • Click Save to create the collection.
  • Once the AmazonS3 collection is created you will be taken to the AmazonS3 tab

AmazonS3 Collection Settings

  • The Settings sub-tab holds settings for Amazon S3 and tunable parameters for the search.
  • Amazon S3 settings must be set explicitly in the Amazon S3 collections.
  • The mandatory fields for AmazonS3 collection are
    • Access key
    • Secret key
    • Bucket name
  • SearchBlox also comes pre-configured with few other AmazonS3 parameters like includes, excludes when a new collection is created.
  • The following table has the list of settings for AmazonS3 Collection

Field

Description

Access Key

Access key from Amazon S3 security credentials.
Mandatory field.

Secret Key

Security key from Amazon S3 security credentials.
Mandatory field.

Name

Optional name.

Bucket

Amazon S3 bucket to index.
Mandatory field.

Path Prefix

Path prefix to index in this bucket example: Work/.
This is optional. If specified, it should be an existing path with the trailing /.

Includes

File types to be included. example: .pdf, .jpg.

Excludes

File types to be excluded. example: *.zip.

Relevance - Remove Duplicates

Avoids the indexing of duplicate documents, i.e., documents which have the same exact content. The default is NO

Relevance - Stemming

Stemming considers the inflected words of the root form within the search page. For example, "running", "runs", and "ran" are all inflected forms of run. The default is YES.

Relevance - Spelling Suggestions

When enabled, a spelling index is created at the end of the indexing process.

Keyword-in-Context Display

The keyword-in-context returns search results with the description displayed from content areas where the search term occurs.

Enable Detailed Log Settings

When debug mode is enabled, indexing activity gets logged in detail within the index.log. Log details include: Indexing status of each URL along with timestamp, URL indexing status along with timestamp, status code and time taken for indexing. By default this is set to NO.

📘

Additional Note

  • Do not log transactions to S3 buckets since those log files will also be indexed, increasing bandwidth usage.
  • If logging is needed, then disallow the log files by excluding them (using extensions) in Collection Settings.

Schedule and Index

Sets the frequency and the start date/time for indexing a collection. Schedule Frequency supported in SearchBlox is as follows:

  • Once
  • Hourly
  • Daily
  • Every 48 Hours
  • Every 96 Hours
  • Weekly
  • Monthly

The following operations can be performed in AmazonS3 collection:

Activity

Description

Schedule

For each collection, indexing can be scheduled based on the above options.

Data Fields Tab

Using Data Fields tab we can create custom fields for search and we can see the Default Data Fields with non-encrypted collection. SearchBlox supports 4 types of Data Fields as listed below:

Keyword
Number
Date
Text

  • Once the Data fields are configured, collection must be cleared and re-indexed to take effect.

To know more about Data Fields please refer to Data Fields Tab

👍

Best Practices

  • It is mandatory to provide access key, secret key, bucket name and update rate in S3 collection settings.
  • It is possible to include or exclude file types using collection settings. Please use them to avoid indexing unnecessary file types.
  • If you have multiple collections, always schedule the activity to prevent more than 2-3 collections indexing at the same time.

Did this page help you?