Setting Up Navigator Metadata Server
In addition to the basic configuration topics discussed below, see Navigator Metadata Server Tuning for memory sizing and other considerations.
Continue reading:
- Setting up Navigator Role Instances on Different Hosts
- Adding the Navigator Metadata Server Role
- Starting, Stopping, and Restarting a Navigator Metadata Server Role Instance
- Moving the Navigator Metadata Server to a New Host
- Setting the Navigator Metadata Server Java Heap Size
- Configuring the Navigator Metadata Server Storage Directory
- Configuring the Navigator Metadata Server Port
- Configuring the Navigator Metadata Server Log Directory
- Configuring the Navigator Metadata Server Session Timeout
- Disabling Anonymous Usage Collection
- Configuring Navigator Metadata Server for Kerberos and Custom Keytab Retrieval
- Navigator Metadata Server Configuration Settings Summary
Setting up Navigator Role Instances on Different Hosts
By default, the Cloudera Manager installation wizard assigns the Navigator Audit Server and Navigator Metadata Server roles to the same Cloudera Management Service host. This configuration can support small clusters but larger production clusters often need these roles distributed to different nodes for better performance. You can distribute these roles to different nodes at install time or you can move the Navigator Metadata Server later if necessary.
Adding the Navigator Metadata Server Role
- Configure the database where policies, roles, and audit report metadata is stored. For production
environments, Cloudera supports only external databases. The internal PostgreSQL database should be used for non-production clusters only, such as proof-of-concept.
- For external databases (production environments): The database should be up and running before you add the Navigator Metadata Server role to the cluster. You must have the database schema name (user name), password, and other details to provide when adding the role. When you have these ready, you can proceed.
- For internal databases (non-production environments): No additional preliminary information is needed. You can proceed.
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Instances tab.
- Click the Add Role Instances button. The Customize Role Assignments page displays and lists the nodes available to support additional roles, assuming the cluster has available resources. Reassign and customize role instances as needed.
- Select the Navigator Metadata Server role and assign it to appropriate host for your cluster.
- When finished, click Continue. The Database Setup page displays.
- Click Use Custom Database.
- In the Navigator Metadata Server section, enter the details of your database instance:
- Database host name
- Database type
- Database name
- Username
- Password
Note: Do not use the embedded database for production deployments. It is not a supported configuration. For non-production environments only you can select Use Embedded Database to have Cloudera Manager create and configure the database for you. Make a note of the generated passwords.Here is an example of a configured custom (external) database instance that uses MySQL for the Navigator Metadata Server database:
- Click Test Connection to verify the communication between the cluster and the external database. If the test fails, check the database settings and try again. If you selected embedded database, a message displays regarding database creation.
- Click Continue.
- The Cluster Setup Review Changes page displays.
- Click Finish.
Starting, Stopping, and Restarting a Navigator Metadata Server Role Instance
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Instances tab.
- Click the link for the Navigator Metadata Server from the Role Type list. The Actions for Selected button activates and displays (1) for the selected role.
- From the Actions for Selected (1) menu, select the Action you want to perform on
this role instance:
- Start
- Stop
- Restart
- Enter Maintenance Mode
- Exit Maintenance Mode
- Delete
- Click Cancel to abandon the process, or click Action to execute the action.
Moving the Navigator Metadata Server to a New Host
- Stop the Navigator Metadata Server. See Starting, Stopping, and Restarting a Navigator Metadata Server Role Instance for details.
- Delete it from the existing host.
- Add it to a new host. See Adding the Navigator Metadata Server Role for details.
- If the storage directory is not on NFS/SAN, move the data to the same path on the new host. See Configuring the Navigator Metadata Server Storage Directory.
- Start the Navigator Metadata Server.
Setting the Navigator Metadata Server Java Heap Size
((num_nav_elements + num_nav_relations) * 200 bytes) + 2 GBBriefly, the total of num_nav_elements and num_nav_relations is the number of Solr documents (per the Navigator Metadata Server log file) that comprise the index. See sizing guidelines for complete details. If you are just starting your Navigator deployment, set the Java heap size to at least 10 GB and as much as 30 GB if you have the memory available on the Navigator host.
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Configuration tab.
- Select .
- For Filter Category, click Resource Management to display the Java heap size property. The default value is 1.5 GiB
(specified in binary units).
- Use the drop-down selector to change the unit scale to B (Bytes), KiB, MiB, or GiB and set your preferred heap size.
- Click Save Changes. The setting takes effect only after restarting the role. Restart the Navigator Metadata Server role now or make other configuration changes and restart after you are finished with all changes. To restart the role:
- Click the Instances tab to display list the list of instances of each role type.
- Click the link for the Navigator Metadata Server to select it.
- From the Actions for Selected (1) menu, select Restart to restart the Navigator Metadata Server role instance using the new Java heap size.
Configuring the Navigator Metadata Server Storage Directory
Navigator Metadata Server uses a Solr instance (a search engine) to index extracted metadata for fast search and retrieval. The searchable Solr documents are stored in a directory on the host on which the Navigator Metadata Server runs. The default location for the directory is /var/lib/cloudera-scm-navigator.
To change the location of the storage directory:
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Instances tab.
- Click the link for the Navigator Metadata Server to select it.
- From the Actions for Selected (1) menu, select Stop.
- Open a terminal session and log in to the host configured to run Navigator Metadata Server.
- Copy the contents of the current storage directory to the new location. When copying is complete, return to the Cloudera Manager Admin Console.
- Click the Configuration tab.
- Select .
- Enter the directory path in the Navigator Metadata Server Storage Dir property. The property applies to the default role group, but you can apply to other role groups as needed. See Modifying Configuration Properties Using Cloudera Manager for details.
- Click Save Changes. This change does not take effect until the Navigator Metadata Server role is restarted.
- Click the Instances tab.
- Click the link for the Navigator Metadata Server from the list of instances.
- From the Actions for Selected menu, select Restart to restart the Navigator Metadata Server role instance.
Configuring the Navigator Metadata Server Port
The Cloudera Navigator console is accessed at port 7187 by default, on the host supporting the Navigator Metadata Server role. Follow the steps below to change setting.
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Configuration tab.
- In the Search box, type the word "ports" to display properties.
- Specify the port in the Navigator Metadata Server Port property. The property applies to the default role group, but you can apply to other role groups as needed. See Modifying Configuration Properties Using Cloudera Manager for details.
- Click Save Changes.
- Click the Instances tab.
- Restart the Navigator Metadata Server role.
Configuring the Navigator Metadata Server Log Directory
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Configuration tab.
- Under the Filter for Scope, click Navigator Metadata Server.
- Under Filter Category, click Logs.
- Set the Navigator Metadata Server Log Directory property. The property applies to the default role group, but you can apply to other role groups as needed. See Modifying Configuration Properties Using Cloudera Manager for details.
- Click Save Changes.
- Restart the Navigator Metadata Server role instance.
Configuring the Navigator Metadata Server Session Timeout
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Configuration tab.
- Under the Filter for Scope, click Navigator Metadata Server.
- For Filter Category, click Advanced. Scroll to find the Advanced Configuration Snippet (Safety Valve) entry field for cloudera-navigator.properties.
- Set the value for the nav.max_inactive_interval property to the number of seconds that can elapse before system times out:
nav.max_inactive_interval=num-seconds-until-timeout
This example reduces the timeout period from the default of 30 minutes to 15 minutes:
- Click Save Changes.
- Restart the Navigator Metadata Server role instance.
Disabling Anonymous Usage Collection
Cloudera Navigator by default sends anonymous usage information to Cloudera for analysis. The usage information helps Cloudera improve Cloudera Navigator functionality and pro-actively identify areas of the product to enhance. The information is completely anonymous and cannot be traced to any customer installation.
To disable anonymous usage information collection:
- Log in to the Cloudera Manager Admin Console.
- Select .
- Click the Configuration tab.
- Under the Filter for Scope, click Navigator Metadata Server.
- Click the Allow Usage Data Collection to clear the checkbox.
- Click Save Changes.
- Restart the Navigator Metadata Server role instance.
Cloudera Navigator stops emitting anonymous usage information.
This capability can be re-enabled at any time by clicking the Allow Usage Data Collection box that has been cleared.
Configuring Navigator Metadata Server for Kerberos and Custom Keytab Retrieval
If Navigator is running on a Kerberized cluster and the cluster is set up to use a custom keytab retrieval script, you need to manually put keytabs for HDFS and Hue principals into the Navigator home directory (typically /var/lib/cloudera-scm-navigator) for use by the Navigator web server.
If the keytabs are not available, you'll see Navigator Metadata Server log errors such as the following:
Error running extraction task. java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Login failure for hdfs/node.example.com@EXAMPLE.COM from keytab /var/lib/cloudera-scm-navigator/temp/1234/hadoop1234.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user
Navigator Metadata Server Configuration Settings Summary
Access to the Cloudera Manager Admin Console and the ability to change any of the settings in the table below requires the Cloudera Manager user role of Navigator Administrator or Full Administrator.
Property | Default | Usage |
---|---|---|
Java Heap Size of Cloudera Navigator Metadata Server in Bytes | 1536 MiB (1.5 GiB) | Increase Java heap size temporarily during an upgrade. Increase Java heap size for large Solr indexes. See Estimating Optimal Java Heap Size Using Solr Document Counts for details. |
Navigator Metadata Storage Dir | /var/lib/cloudera-scm-navigator | Change to another path if you like. Be sure to move existing content from the default directory to the new directory to maintain state. Offload content from this directory prior to upgrading for faster upgrades. |
Cloudera Navigator Server Port | 7187 | Change to another port if necessary. This is the port used to access Cloudera Navigator console. |
Navigator Metadata Server Log Directory | /var/log/cloudera-scm-navigator | Examine the log to find out how many documents the Solr index contains, and use that count to estimate Java heap size needed for normal operations and for upgrades. Use the log to troubleshoot issues. |
Server Session Timeout | 30 minutes (1800 seconds) | Timeout is specified in number of seconds that must elapse before the system generates a timeout event. |
Allow Usage Data Collection | enabled | Cloudera Navigator uses Google Analytics to anonymous usage data to Cloudera for product enhancement. Disable this setting if you do not want to send usage data. |
<< Navigator Metadata Server Management | ©2016 Cloudera, Inc. All rights reserved | Navigator Metadata Server Tuning >> |
Terms and Conditions Privacy Policy |