Cloudera Enterprise 6.0.x | Other versions

Kudu Security Overview

Kudu includes security features that allow Kudu clusters to be hardened against access from unauthorized users. Kudu uses strong authentication with Kerberos, while communication between Kudu clients and servers can now be encrypted with TLS. Kudu also allows you to use HTTPS encryption to connect to the web UI.

The rest of this topic describes the security capabilities of Apache Kudu and how to configure a secure Kudu cluster. Currently, there are a few known limitations in Kudu security that might impact your cluster. For the list, see Security Limitations.

Kudu Authentication with Kerberos

Kudu can be configured to enforce secure authentication among servers, and between clients and servers. Authentication prevents untrusted actors from gaining access to Kudu, and securely identifies connecting users or services for authorization checks. Authentication in Kudu is designed to interoperate with other secure Hadoop components by utilizing Kerberos.

Configure authentication on Kudu servers using the --rpc-authentication flag, which can be set to one of the following options:
  • required - Kudu will reject connections from clients and servers who lack authentication credentials.
  • optional - Kudu will attempt to use strong authentication, but will allow unauthenticated connections.
  • disabled - Kudu will only allow unauthenticated connections.
By default, the flag is set to optional. To secure your cluster, set --rpc-authentication to required.

Internal Private Key Infrastructure (PKI)

Kudu uses an internal PKI to issue X.509 certificates to servers in the cluster. Connections between peers who have both obtained certificates will use TLS for authentication. In such cases, neither peer needs to contact the Kerberos KDC.

X.509 certificates are only used for internal communication among Kudu servers, and between Kudu clients and servers. These certificates are never presented in a public facing protocol. By using internally-issued certificates, Kudu offers strong authentication which scales to huge clusters, and allows TLS encryption to be used without requiring you to manually deploy certificates on every node.

Authentication Tokens

After authenticating to a secure cluster, the Kudu client will automatically request an authentication token from the Kudu master. An authentication token encapsulates the identity of the authenticated user and carries the Kudu master's RSA signature so that its authenticity can be verified. This token will be used to authenticate subsequent connections. By default, authentication tokens are only valid for seven days, so that even if a token were compromised, it cannot be used indefinitely. For the most part, authentication tokens should be completely transparent to users. By using authentication tokens, Kudu is able to take advantage of strong authentication, without paying the scalability cost of communicating with a central authority for every connection.

When used with distributed compute frameworks such as Apache Spark, authentication tokens can simplify configuration and improve security. For example, the Kudu Spark connector will automatically retrieve an authentication token during the planning stage, and distribute the token to tasks. This allows Spark to work against a secure Kudu cluster where only the planner node has Kerberos credentials.

Client Authentication to Secure Kudu Clusters

Users running client Kudu applications must first run the kinit command to obtain a Kerberos ticket-granting ticket. For example:

$ kinit admin@EXAMPLE-REALM.COM

Once authenticated, you use the same client code to read from and write to Kudu servers with and without the Kerberos configuration.

Scalability

Kudu authentication is designed to scale to thousands of nodes, which means it must avoid unnecessary coordination with a central authentication authority (such as the Kerberos KDC) for each connection. Instead, Kudu servers and clients use Kerberos to establish initial trust with the Kudu master, and then use alternate credentials for subsequent connections. As described previously, the Kudu master issues internal X.509 certificates to tablet servers on startup, and temporary authentication tokens to clients on first contact.

Encryption

Kudu allows you to use TLS to encrypt all communications among servers, and between clients and servers. Configure TLS encryption on Kudu servers using the --rpc-encryption flag, which can be set to one of the following options:
  • required - Kudu will reject unencrypted connections.
  • optional - Kudu will attempt to use encryption, but will allow unencrypted connections.
  • disabled - Kudu will not use encryption.
By default, the flag is set to optional. To secure your cluster, set --rpc-encryption to required.
  Note: Kudu will automatically turn off encryption on local loopback connections, since traffic from these connections is never exposed externally. This allows locality-aware compute frameworks, such as Spark and Impala, to avoid encryption overhead, while still ensuring data confidentiality.

Coarse-grained Authorization

Kudu supports coarse-grained authorization checks for client requests based on the client's authenticated Kerberos principal (user or service). Access levels are granted based on whitelist-style Access Control Lists (ACLs), one for each level. Each ACL specifies a comma-separated list of users, or may be set to '*' to indicate that all authenticated users have access rights at the specified level.

The two levels of access which can be configured are:

  • Superuser - Principals authorized as a superuser can perform certain administrative functions such as using the kudu command line tool to diagnose and repair cluster issues.
  • User - Principals authorized as a user are able to access and modify all data in the Kudu cluster. This includes the ability to create, drop, and alter tables, as well as read, insert, update, and delete data. The default value for the User ACL is '*', which allows all users access to the cluster. However, if authentication is enabled, this will restrict access to only those users who are able to successfully authenticate using Kerberos. Unauthenticated users on the same network as the Kudu servers will be unable to access the cluster.
  Note: Internally, Kudu has a third access level for the daemons themselves called Service. This is used to ensure that users cannot connect to the cluster and pose as tablet servers.

Web UI Encryption

The Kudu web UI can be configured to use secure HTTPS encryption by providing each server with TLS certificates. Use the --webserver-certificate-file and --webserver-private-key-file properties to specify the certificate and private key to be used for communication.

Alternatively, you can choose to completely disable the web UI by setting --webserver-enabled flag to false on the Kudu servers.

Web UI Redaction

To prevent sensitive data from being included in the web UI, all row data is redacted. Table metadata, such as table names, column names, and partitioning information is not redacted. Alternatively, you can choose to completely disable the web UI by setting the --webserver-enabled flag to false on the Kudu servers.
  Note: Disabling the web UI will also disable REST endpoints such as /metrics. Monitoring systems rely on these endpoints to gather metrics data.

Log Redaction

To prevent sensitive data from being included in Kudu server logs, all row data will be redacted. You can turn off log redaction using the --redact flag.

Configuring a Secure Kudu Cluster using Cloudera Manager

  Warning: If you are upgrading from Kudu 1.2.0 / CDH 5.10.x, you must upgrade both Kudu and CDH parcels (or packages) at the same time. If you upgrade Kudu but do not upgrade CDH, new Kudu features such as Security will not be available. Note that even though you might be able to see the updated configuration options for Kudu security in Cloudera Manager, configuring them will have no effect.

Use the following set of instructions to secure a Kudu cluster using Cloudera Manager:

Enabling Kerberos Authentication and RPC Encryption

  Important: The following instructions assume you already have a secure Cloudera Manager cluster with Kerberos authentication enabled. If this is not the case, first secure your cluster using the steps described at Enabling Kerberos Authentication Using the Cloudera Manager Wizard.
To enable Kerberos authentication for Kudu:
  1. Go to the Kudu service.
  2. Click the Configuration tab.
  3. Select Category > Main.
  4. In the Search field, type Kerberos to show the relevant properties.
  5. Edit the following properties according to your cluster configuration:
    Field Usage Notes
    Kerberos Principal Set to the default principal, kudu. Currently, Kudu does not support configuring a custom service principal for Kudu processes.
    Enable Secure Authentication And Encryption Select this checkbox to enable authentication and RPC encryption between all Kudu clients and servers, as well as between individual servers. Only enable this property after you have configured Kerberos.
  6. Click Save Changes.
  7. You will see an error message that tells you the Kudu keytab is missing. To generate the keytab, go to the top navigation bar and click Administration > Security.
  8. Go to the Kerberos Credentials tab. On this page you will see a list of the existing Kerberos principals for services running on the cluster.
  9. Click Generate Missing Credentials. Once the Generate Missing Credentials command has finished running, you will see the Kudu principal added to the list.

Configuring Coarse-grained Authorization with ACLs

  1. Go to the Kudu service.
  2. Click the Configuration tab.
  3. Select Category > Security.
  4. In the Search field, type ACL to show the relevant properties.
  5. Edit the following properties according to your cluster configuration:
    Field Usage Notes
    Superuser Access Control List Add a comma-separated list of superusers who can access the cluster. By default, this property is left blank.

    '*' indicates that all authenticated users will be given superuser access.
    User Access Control List Add a comma-separated list of users who can access the cluster. By default, this property is set to '*'.

    The default value of '*' allows all users access to the cluster. However, if authentication is enabled, this will restrict access to only those users who are able to successfully authenticate using Kerberos. Unauthenticated users on the same network as the Kudu servers will be unable to access the cluster.

    Add the impala user to this list to allow Impala to query data in Kudu. You might choose to add any other relevant usernames if you want to give access to Spark Streaming jobs.
  6. Click Save Changes.

Configuring HTTPS Encryption for the Kudu Master and Tablet Server Web UIs

Use the following steps to enable HTTPS for encrypted connections to the Kudu master and tablet server web UIs.
  1. Go to the Kudu service.
  2. Click the Configuration tab.
  3. Select Category > Security.
  4. In the Search field, type TLS/SSL to show the relevant properties.
  5. Edit the following properties according to your cluster configuration:
    Field Usage Notes
    Master TLS/SSL Server Private Key File (PEM Format)

    Set to the path containing the Kudu master host's private key (PEM-format). This is used to enable TLS/SSL encryption (over HTTPS) for browser-based connections to the Kudu master web UI.
    Tablet Server TLS/SSL Server Private Key File (PEM Format)

    Set to the path containing the Kudu tablet server host's private key (PEM-format). This is used to enable TLS/SSL encryption (over HTTPS) for browser-based connections to Kudu tablet server web UIs.
    Master TLS/SSL Server Certificate File (PEM Format)

    Set to the path containing the signed certificate (PEM-format) for the Kudu master host's private key (set in Master TLS/SSL Server Private Key File). The certificate file can be created by concatenating all the appropriate root and intermediate certificates required to verify trust.
    Tablet Server TLS/SSL Server Certificate File (PEM Format)

    Set to the path containing the signed certificate (PEM-format) for the Kudu tablet server host's private key (set in Tablet Server TLS/SSL Server Private Key File). The certificate file can be created by concatenating all the appropriate root and intermediate certificates required to verify trust.
    Enable TLS/SSL for Master Server Enables HTTPS encryption on the Kudu master web UI.
    Enable TLS/SSL for Tablet Server Enables HTTPS encryption on the Kudu tablet server Web UIs.
  6. Click Save Changes.
Page generated July 25, 2018.