Cloudera Enterprise 6.0.x | Other versions

Flume Solr BlobHandler Configuration Options

Flume accepts events by HTTP POST and GET operations using the HTTPSource Flume source.

HTTPSource transforms JSON input into events by default. You can also configure a BlobHandler for HTTPSource to return events containing the request parameters along with the binary large object (BLOB) uploaded with the request. Because the entire BLOB is buffered in RAM, this usage is not generally appropriate for very large objects.

The Flume Solr BlobHandler supports the following configuration options (required options in bold):

Property Name	Default	Description
`handler`		Must be set to the fully qualified class name (FQCN) `org.apache.flume.sink. solr.morphline.BlobHandler`.
`handler.maxBlobLength`	`100000000` (100 MB)	Specifies the maximum number of bytes to read and buffer per request.

This example shows a section for an HTTPSource named httpSrc with a BlobHandler for an agent named agent:

agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
agent.sources.httpSrc.port = 5140
agent.sources.httpSrc.handler = org.apache.flume.sink.solr.morphline.BlobHandler
agent.sources.httpSrc.handler.maxBlobLength = 2000000000
agent.sources.httpSrc.interceptors = uuidinterceptor
agent.sources.httpSrc.interceptors.uuidinterceptor.type = org.apache.flume.sink.solr.morphline.UUIDInterceptor$Builder
agent.sources.httpSrc.interceptors.uuidinterceptor.headerName = id
#agent.sources.httpSrc.interceptors.uuidinterceptor.preserveExisting = false
#agent.sources.httpSrc.interceptors.uuidinterceptor.prefix = flume01.example.com
agent.sources.httpSrc.channels = memoryChannel

Page generated July 25, 2018.