Flume can modify and drop events using Interceptors, which can be attached to any Flume source. The Solr UUIDInterceptor sets a universally unique 128-bit identifier (such as f692639d-483c-1b5f-cd61-183cb1726ae0) on each event.

Cloudera recommends assigning UUIDs to events as early as possible (for example, in the first Flume source of your data flow). This allows you to de-duplicate events that are duplicated as a result of replication or re-delivery in a Flume pipeline that is designed for high availability and high performance. If available, application-level UUIDs are preferable to auto-generated UUIDs because they enable subsequent updates and deletion of the document in Solr using that key. If application-level UUIDs are not present, you can use UUIDInterceptor to automatically assign UUIDs to document events.

The UUIDInterceptor supports the following configuration options (required options in bold):

For examples, see BlobHandler and BlobDeserializer.