Flume agents can store data on an HDFS filesystem. For clusters configured to use Kerberos authentication, Flume requires a Kerberos principal and keytab to authenticate to the cluster, which then interacts with HDFS, MapReduce, and other cluster services on behalf of the Flume agent.

Enabling Flume to use Kerberos authentication on a cluster assumes that cluster has been configured to integrate with Kerberos.

The steps below have been tested with CDH 5 and MIT Kerberos 5 only. The discussion includes an example of configuring user flume for Kerberos authentication as an HDFS client. Configuring authentication between Flume agents is not covered.

Continue reading:

• Configuring Kerberos for Flume Thrift Source and Sink Using Cloudera Manager
• Writing to a Secure HBase Cluster
• Using Substitution Variables for Flume Kerberos Principal and Keytab