Amazon S3 Collection

You can create an Amazon S3 collection by following the steps given below.

Creating Amazon S3 Collection

• After logging in to the Admin Console, click Add Collection button. The Add Collection screen will be displayed.
• Enter a unique name for your collection (for example, AmazonS3).
• Select Amazon S3 collection radio button.
• Click Add to create the collection.

Amazon S3 Collection Settings

• The Settings sub-tab holds settings for Amazon S3 and tunable parameters for the search.
• Amazon S3 settings must be set explicitly in the Amazon S3 collections.
• The mandatory fields for AmazonS3 collection are
  • Access key
  • Secret key
  • Bucket name
• SearchBlox also comes pre-configured with few other Amazon S3 parameters like includes, excludes when a new collection is created.
• The following table has the list of settings for Amazon S3 Collection

📘

Additional Note

• Do not log transactions to S3 buckets since those log files will also be indexed, increasing bandwidth usage.
• If logging is needed, then disallow the log files by excluding them (using extensions) in Collection Settings.

Indexing and Other Operations

The following operations can be performed in AmazonS3 collection:

• Indexer activity is controlled from the Index sub-tab in the collection. The current status of an indexer for a particular collection is indicated.
• Indexing operation starts the indexer for the Amazon S3 collection.
• On reindexing that is, clicking on index again after the initial index operation, all crawled documents will be reindexed. If documents have been deleted from S3 since the first index operation, they will be deleted from the index. New documents will also be indexed.
• Also, indexing is controlled from the Index sub-tab for a collection or through API. The current status of a collection is always indicated on the Collection Dashboard and the Index page.
• Index operation can also be initiated from the Collection Dashboard.
• Scheduling can be performed only from the Index sub-tab.

Schedule Frequency

Schedule Frequency supported in SearchBlox is as follows:

• Once
• Every Minute
• Hourly
• Daily
• Every 48 Hours
• Every 96 Hours
• Weekly
• Monthly

👍

Best Practices

• It is mandatory to provide access key, secret key, bucket name and update rate in S3 collection settings.
• It is possible to include or exclude file types using collection settings. Please use them to avoid indexing unnecessary file types.
• Do not schedule the same time for index and clear operations
• If you have multiple collections, always schedule the activity to prevent more than 2-3 collections indexing at the same time.