Hortonworks HDP / Ambari Install – Configure Open File Descriptors

Apache Ambari Logo

This is a very simple script I run in the target hosts that I am prepping for Apache Ambari Server or Agent and Hadoop deployment. The main thing it achieves is displaying the current settings for Linux Open File Descriptors, it then allows the user to specify whether they are below Ambari’s and/or Hadoop’s minimum system requirements, if that is the case the script will update the settings for you:

 

#!/bin/bash
#Script Name: ignacio_ofd.scr
#Author: Ignacio de la Torre
#Independent Contractor Profile: http://linkedin.com/in/idelatorre
#################
#Configure Maximum Open File Descriptors
echo ">>> Configure Maximum Open File Descriptors..."
echo "! ! ! Pay attention to the output below, if any of the two numbers displayed is less than 10,000, enter y at the prompt:"

ulimit -Sn
ulimit -Hn
echo "Enter y if the limits are below 10,000:"
read var_yesno
if [ "$var_yesno" = "y" ]
then
     echo "Updating /etc/security/limits.conf"
    #this updates the limits globally
    sudo chmod 666 /etc/security/limits.conf
    sudo echo "ubuntu    hard    nofile    10000" >> /etc/security/limits.conf
    sudo echo "ubuntu    soft    nofile    10000" >> /etc/security/limits.conf
    sudo echo "root    hard    nofile    100000" >> /etc/security/limits.conf
    sudo echo "root    soft    nofile    100000" >> /etc/security/limits.conf
    sudo chmod 644 /etc/security/limits.conf
else
    echo "ulimit not updated, not necessary"
fi

 

 

 

 

Hortonworks HDP / Ambari Install – Configure Network Time Protocol (NTP)

Apache Ambari aims at making Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive, easy-to-use Hadoop management web UI backed by its RESTful APIs.

This is a small script I developed to configure NTP on my hosts before deploying the Ambari server or agent and Hadoop:

 

 

 

 

 

#!/bin/bash
#Script Name: ignacio_config_ntp.scr
#Author: Ignacio de la Torre
#Independent Contractor Profile: https://linkedin.com/in/idelatorre
#################
#configure ntp to auto-start at boot time
#Install NTP
sudo yum install -y -q ntp

#Disable autostart
sudo systemctl disable ntpd
sudo timedatectl set-ntp no

#configure NTP
sudo ntpdate pool.ntp.org
sudo timedatectl set-timezone America/Los_Angeles

#re-enable NTP autostart
sudo systemctl enable ntpd
sudo timedatectl set-ntp on

How To: Download, install, configure and verify you have the latest version of Opatch

On this post we will discuss all the steps necessary to ensure you have the correct version of the OPatch patching utility for oracle software running on your system. If you find this post useful please or Share our site!

Environment Variables:

Set your ORACLE_INSTANCE path to a valid OBIEE 11g instance

Set your ORACLE_HOME path to the OBIEE 11g home (Oracle_BI1)

Validating Your Environment:

OPatch is installed by default, locate it go to the ORACLE_HOME path for your OBIEE installation:

\Oracle_BI1\OPatch

To verify your current session is correctly configured verify the installed version by running the following command:

opatch version

Minimum Required Versions:

OBIEE 11.1.1.5.0 -> OPatch version 11.1.0.8.3 or higher (do NOT use OPatch 12.x)

OBIEE 11.1.1.6.0 -> No OPatch packages available as of March 12, 2012

Downloading a Newer Version of OPatch:

To find and download the appropriate version of OPatch for yor system please go to ORacle Support and find the knoledge base article below:

Note 224346.1 – Opatch – Where Can I Find the Latest Version of Opatch?

Installing OPatch:

To install OPatch once you have downloaded the appropriate version follow the steps below:

  1. Rename your current OPatch directory (ORACLE_HOME\OPatch)
  2. Copy the zip file to your ORACLE_HOME
  3. Unzip the zip file
  4. Verify that the upgrade succeeded

    cd OPatch
    opatch version

File System Access:

OPatch will update the local Oracle Inventory so the user account running OPatch must have acces to the location of the OUI Inventory, to verify this you can run the following command:

opatch -lsinventory

Patch Directory Location (PATCH_TOP):

If you have a centralized location where you store code / releases it is a good idea to create a directory called PATCH_TOP to store patches as they are applied to each environment (DEV, QA, STAGE, PRD).

On this post we will discuss all the steps necessary to ensure you have the correct version of the OPatch patching utility for oracle software running on your system.

Configuring the windows host file to correctly use the loopback adapter for Oracle product installations

This procedure is required when installing Oracle products such at the Oracle Database or Oracle Business Intelligence Enterprise Edition on a host machine that uses DHCP.  Follow the steps to configure a loopback adapterand then:

  1. Open the host file under c:\windows\system32\drivers\etc using notepad
  2. Identify your computer’s host name by right clicking on My Computer, selecting properties and clicking on the computer name tab Identify your computers host name by right clicking on My Computer, selecting properties and clicking on the computer name tab
  3. On the first line of your host file add a link between the static IP address you assigned your loopback adapter and your computer’s host name, for example: # IP                 hostname.domain        hostname 10.10.10.35    virtualXP.acme.com virtualXP
  4. On the second line associate the same static IP with the localhost default alias, for example: 10.10.10.35 localhost.localdomain    localhost

Once you have completed these steps you can proceed with the next steps in your Oracle product installation. If you find this post useful please or Share our site! This procedure is required when installing Oracle server products such at the Oracle Database or Oracle Business Intelligence Enterprise Edition on a host machine that uses DHCP.  Follow the steps to configure a loopback adapter and then the steps outlined on this article…

Configuring a new loopback adapter in windows


  1. Start > Control Pannel> Add  Hardware
  2. Click next on the Welcome screen
    Click next on the Welcome screen
  3. When ask if you have connected your hardware, specify yes
    When ask if you have connected your hardware, specify yes
  4. In the next dialog select Add new hardware device
    In the next dialog select Add new hardware device
  5. On the following screen select Network Adapters
    On the following screen select Network Adapters
  6. Select Microsoft > Microsoft Loopback Adapter from the list
    Select Microsoft > Microsoft Loopback Adapter from the list
  7. Select next in the following screen to install your loopback adapter
    Select next in the following screen to install your loopback adapter
  8. Click finish to complete this section of the installation process
    Click finish to complete this section of the installation process
  9. Find your newly created adapter by going to Start > Control Panel > Network Connections
    Find your newly created adapter by going to Start > Control Panel > Network Connections
  10. Right click your new connection and select Properties
  11. Select the Internet Protocol on the second section of the screen and click properties
    Select the Internet Protocol on the second section of the screen and click properties
  12. In the IP properties dialog configure a static IP address following the example belowIP Address: 10.10.10.35 Subnet Mask: 255.255.255.0
    In the IP properties dialog configure a static IP address following the example belowIP Address: 10.10.10.35 Subnet Mask: 255.255.255.0

Once you have configured a static IP address you will have completed the configuration of your new loopback adapter.

For instructions on configuring your hosts file to correctly use your loopback adapter when installing Oracle products see this post.

If you find this post useful please or Share our site!

This short reference article will guide readers through the configuration process for a loopback adapter, this is a pre-requisite for most Oracle server software installs…