Tuesday, January 24, 2012

How to install KVM and libvirt on CentOS 6.2 with bridged networking

This is a how to install the KVM hypervisor and libvirt virtualization library on Linux CentOS 6.2. At the end of this guide you will have a CentOS box (name it host) with the following capabilities:
  • virtualization capacity: chance for multiple guests (Linux/Windows) running and sharing the host's hardware 
  • bridge network configuration: you'll be able to directly access to the guests as if they were physical machines on the same LAN and viceversa.
  • visual management: using virt-manager (I use Ubuntu 11.10 on this how to) you will be able to remotely admin the virtual machines on the CentOS host.

What's what?

KVM:

"KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V). It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module, kvm-intel.ko or kvm-amd.ko..."
"Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc."

libvirt:


  • "A toolkit to interact with the virtualization capabilities of recent versions of Linux... "
  • "A Free software available under the GNU Lesser General Public License."
  • "A long term stable C API"
  • "A set of bindings for common languages"

Intel-VT and AMD-V

Requirements

  • Hardware
    • Processor with support for Intel-VT or AMD-V technology
    • Plenty RAM memory depending on the number of guests
    • Network connectivity
  • Software
    • A previosly installed Linux CentOS 6.2 with a recent Linux kernel. On this how to:
    • $ uname -r
      2.6.32-220.2.1.el6.x86_64
      
      $ lsb_release -a
      LSB Version: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
      Distributor ID: CentOS
      Description: CentOS release 6.2 (Final)
      Release: 6.2
      Codename: Final
      
    • A Linux client with a Desktop environment installed
    • An available CentOS mirror/repository

Installation Steps

  1. Login into the CentOS as bozz user (a sudoer user) and check if your hardware support for  virtualization extensions, on my hardware it was:
  2. $ egrep '^flags.*(vmx|svm)' /proc/cpuinfo
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
    
    
  3. Install kvm and libvirt packages:
  4. $ sudo yum install kvm libvirt
  5. Update both packages to the latest version available on repositories/mirrors:
  6. $ sudo yum update libvirt kvm
    ...
    $ yum info libvirt
    Installed Packages
    Name        : libvirt
    Arch        : x86_64
    Version     : 0.9.4
    Release     : 23.el6_2.4
    ...
    $ yum info qemu-kvm
    Installed Packages
    Name        : qemu-kvm
    Arch        : x86_64
    Epoch       : 2
    Version     : 0.12.1.2
    Release     : 2.209.el6_2.1
    ...
    
  7. Restart the libvirtd daemon:
  8. $ sudo service libvirtd restart
  9. Verify if the kvm module is loaded, you should see amd or intel depending on the hardware:
  10. $ lsmod | grep kvm
    kvm_intel              50380  0
    kvm                   305113  1 kvm_intel
    
  11. Issue a virsh command to ensure local root connectivity first:
  12. $ sudo virsh sysinfo
    <sysinfo type="smbios">
    ...
    
  13. [OPTIONAL] To use KVM by a non-root user verify if kvm group was created on installation:
  14. $ cat /etc/group | grep kvm
    kvm:x:36:qemu
    
    then add the bozz user to kvm group, so it can gain access to hypervisor:
    $ sudo usermod -a -G kvm bozz
    $ logout
    login again as the bozz user and verify kvm's membership:
    $ id
    uid=500(bozz) gid=500(bozz) groups=500(bozz),10(wheel),36(kvm) context=...
    and verify if /dev/kvm is owned by group kvm:
    $ ls -l /dev/kvm
    crw-rw-rw-. 1 root kvm 10, 232 Jan 17 14:50 /dev/kvm
    
    on a system that runs udev, you will probably need to add the following line in your udev configuration so it will automatically give the right group to the newly created device:
    $ cat  /etc/udev/rules.d/80-kvm.rules 
    KERNEL=="kvm", GROUP="kvm", MODE="0666"
    
  15. To manage libvirt with a non-root account you should use PolicyKit. Define access control for a libvirt group:
  16. $ sudo groupadd libvirt
    $ sudo usermod -a -G libvirt bozz
    $ logout
    
    login again as bozz user and edit a new archive:
    $ sudo nano /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla
    with this content:
    [libvirt Management Access]
    # For allowing access to specific user only:
    #Identity=unix-user:bozz
    # For allowing access to a group (like this guide):
    Identity=unix-group:libvirt
    Action=org.libvirt.unix.manage
    ResultAny=yes
    ResultInactive=yes
    ResultActive=yes
    
    restart libvirtd daemon:
    $ sudo service libvirtd restart
    verify if bozz user can locally access to qemu:///system (NOTE: the use of qemu:///session is discouraged):
    $ virsh -c qemu:///system sysinfo
    <sysinfo type="smbios">
    ...
    verify if bozz user can remotely access to qemu+ssh://bozz@SERVER/system too. So on the Linux client issue:

    for Ubuntu client (like in this guide):
    $ sudo apt-get install libvirt-bin
    for CentOS client:
    $ sudo yum install libvirt
    then:
    $ virsh -c qemu+ssh://bozz@SERVER/system sysinfo
    <sysinfo type="smbios">
    ...
    change group ownership and permissions on the default images directory:
    $ sudo chown root:libvirt /var/lib/libvirt/images
    $ sudo chmod g+rw /var/lib/libvirt/images
  17. [OPTIONAL] When libvirtd (>= 0.9.3) is running as non-root it tries to read ~/.libvirt/libvirtd.conf (see here). A workaround is: 
  18. $ mkdir -p ~/.libvirt
    $ touch ~/.libvirt/libvirtd.conf 
    
    then issue a virsh command as bozz user:
    $ virsh list
     Id Name                 State
    ----------------------------------
    
    
  19. Configure Bridged Network by creating a new network script at /etc/sysconfig/network-scripts/ifcfg-br0:
  20. $ sudo nano /etc/sysconfig/network-scripts/ifcfg-br0
    
    and configuring the parameters according to your LAN settings (NOTE: options are case sensitive i.e. Bridge and bridge are two different options):
    DEVICE="br0"
    TYPE=Bridge
    DELAY=0
    ONBOOT="yes"
    BOOTPROTO=static
    IPADDR=192.168.11.12
    NETMASK=255.255.255.0
    NETWORK=192.168.11.0
    GATEWAY=192.168.11.1
    DNS1=192.168.11.2
    PEERDNS="yes"
    NM_CONTROLLED=no
    
    then edit the Ethernet network script /etc/sysconfig/network-scripts/ifcfg-eth0:
    $ sudo nano /etc/sysconfig/network-scripts/ifcfg-eth0
    with the following content (NOTE: the hardware address depends on your NIC, an arbitrary MAC address is used here):
    DEVICE="eth0"
    HWADDR="00:2C:C2:85:29:A3"
    ONBOOT="yes"
    BRIDGE=br0
    NM_CONTROLLED=no
    
      restart the networking service:
    $ sudo service network restart
    and verify the bridge config:
    $ brctl show
    bridge name bridge id  STP enabled interfaces
    br0  8000.002cc28529a3 no  eth0
    ...
    
    configure iptables to allow all traffic to be forwarded across the bridge:
    $ sudo iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
    $ sudo service iptables save
    $ sudo service iptables restart
    prevent bridged traffic from being processed by iptables rules, this improves the bridge’s performance. In /etc/sysctl.conf append the following lines:
    net.bridge.bridge-nf-call-ip6tables = 0
    net.bridge.bridge-nf-call-iptables = 0
    net.bridge.bridge-nf-call-arptables = 0
    reload the kernel parameters configured with sysctl:
    $ sudo sysctl -p /etc/sysctl.conf
    restart the libvirt daemon.
    $ sudo service libvirtd reload

Post-install Steps

  • Creating a virtual machine / guest by means of Virtual Machine Manager (virt-manager) from the Linux client machine. The guest OS will be an Ubuntu Server 11.10 Oneiric Ocelot:


  • for Ubuntu client (like in this guide):
    $ sudo apt-get install virt-manager
    for CentOS client:
    $ sudo yum install virt-manager
    then download oneiric-server-amd64.iso from the Ubuntu site on the Linux client: 
    $ wget http://releases.ubuntu.com/11.10/ubuntu-11.10-server-amd64.iso
    copy the downloaded ISO to SERVER:
    $ scp ubuntu-11.10-server-amd64.iso bozz@SERVER:/var/lib/libvirt/images/
    on the Linux client run virt-manager:
    $ virt-manager & 
    
    goto "File" / "Add Connection..." with this settings: 
    • Hypervisor: QEMU/KVM
    • Method: ssh
    • Username: bozz
    • Host: SERVER
    once connected click on the "Create a new virtual machine" button.
    1. Set the name to ubuntu-oneiric and choose "Local install media (ISO image or CDROM)":
    2. Browse and select the ISO located at /var/lib/libvirt/images/ubuntu-11.10-server-amd64.iso, OS type "Linux" and Version "Ubuntu 11.10 (Oneiric Ocelot)":
    3. Choose memory and CPU setting as you wish:
    4. Choose storage settings:
    5. Choose the previously created bridge network device "Host device eth0 (Bridge 'br0')", Virt type "kvm" and Architecture "x86_64":
    6. press "Finish" button and install the guest OS.

    References

Thursday, January 19, 2012

Installing Oracle Service Registry 11g on CentOS 6.2

This post is a how to install Oracle® Fusion Middleware Service Registry 11g (11.1.1)  on Linux CentOS 6.2.

Requirements

Hardware Requirements

  • CPU 1GHz+,  RAM 1 GB+: actual requirements depend on the on usage patterns of the target environment.
  • Free HDD Space ~500MB: enough if the selected database system is installed on another machine. The database server machine must have sufficient space for the selected database system.

Software Requirements

Installation Steps

  1. Create the OSR (Oracle Serice Registry) tablespaces and user:
    1. Create a SQL script archive createdb.sql with the following content (NOTE: assuming you already created the directory /opt/app/oracle/oradata/soagov/):
    2. -- createdb.sql --
      CREATE TABLESPACE uddinode
       LOGGING
       DATAFILE '/opt/app/oracle/oradata/soagov/uddinode.dbf' 
               SIZE 50M REUSE AUTOEXTEND ON NEXT 10M MAXSIZE UNLIMITED
       DEFAULT STORAGE (
        INITIAL 5M
        NEXT 5M
        MINEXTENTS 1
        MAXEXTENTS UNLIMITED
        PCTINCREASE 50 );
      
      CREATE USER uddiuser
       PROFILE "DEFAULT"
       IDENTIFIED BY "uddiuser"
       DEFAULT TABLESPACE uddinode
       TEMPORARY TABLESPACE "TEMP"
       ACCOUNT UNLOCK;
      GRANT "CONNECT" TO uddiuser;
      GRANT "RESOURCE" TO uddiuser;
      
      GRANT UNLIMITED TABLESPACE TO uddiuser;
      GRANT CREATE ANY SYNONYM TO uddiuser;
      GRANT DROP ANY SYNONYM TO uddiuser;
      
      
    3. Run sqlplus an connect to your SID, soagov in this case:
      1. $ sqlplus sys@soagov AS SYSDBA
        
        embed the SQL file and exit:
        SQL> @createdb.sql
        SQL> exit
        
    4. Test connectivity to the database:
    5. $ sqlplus uddiuser/uddiuser
      
  2. Start the OSR installation. By default, It will start in graphical mode. You can use the -mode option to start it in console or silent mode. Or my preferred one, SSH X11 forwarding:
  3. $ java -jar oracle-service-registry-11.1.1.jar -log=~/OSR_install.log
    
  4. Follow the wizard and choose "Connect to Schema" on the "Database Setup" panel. Provide the credentials written below.

Known issues

Tuesday, January 10, 2012

The Parking Permit Demo (4)

Here I let you the episode #6 of The Parking Permit Demo with Oracle BPM/SOA suite.

pp-06_Creating_WSDL_n_including_existing_schema
  • Creating an empty WSDL named verification_services.wsdl with target
    namespace http://ex.datys.cu/ParkingPermit
  • Including the parking_permit.xsd schema document into the <types>/<schema>
    section of the WSDL

The Parking Permit Demo (3)

I let you here the #5 video of the Parking Permit Demo with Oracle BPM/SOA suite.


pp-05 Defining Schema Type and Element for ParkingPermitApplication

  • Defining XSD complexType & element for ParkingPermitApplication

Creating a portable MySQL in CentOS 6 and Ubuntu 11.10 Linux from sources

This is a how to create a portable MySQL installation on GNU/Linux from the sources. It applies to:

  • MySQL 5.5.19
  • CentOS 6.x / Ubuntu 11.10 Oneiric Ocelot
At the end of this guide you will obtain a portable MySQL installation on a target directory with it's own databases, binaries, logs, pid files, etc.  Consider always the use of a permission preserving packaging (like TAR) for moving the installation between systems or removable storages.

Requirements


Steps

  1. Define some environment variables to make the installation smoothly:
  2. $ TARGET=$HOME/mysql
    $ BASEDIR=$TARGET/usr/local/mysql
    $ DATADIR=$TARGET/usr/local/mysql/data
    $ PORT=9797
    $ VERSION=5.5.19
    
  3. Install cmake, ncurses and bison:
    • On CentOS:
    • $ sudo yum install cmake ncurses-devel bison
      
    • On Ubuntu:
    • $ sudo apt-get install cmake libncurses5-dev bison 
      
  4. Unpack and make (NOTE: mysql-5.5.19.tar.gz is already downloaded in /tmp)
  5. $ pushd /tmp
    $ tar zxvf mysql-${VERSION}.tar.gz
    $ cd mysql-${VERSION}
    $ cmake .
    $ make 
    
  6. Install into target directory
  7. $ mkdir -p $TARGET
    $ make install DESTDIR="$TARGET"
    
  8. Create system databases
  9. $ pushd $BASEDIR
    $ scripts/mysql_install_db --user=$USER \
      --basedir=$BASEDIR \
      --datadir=$DATADIR \
      --ldata=$DATADIR
    $ mkdir -p $TARGET/var/run/mysql
    $ mkdir -p $TARGET/var/log/mysql
    $ popd
    $ popd
    

Post install steps

  • Running the portable MySQL (NOTE: bind-address is settled to 0.0.0.0, it means listening on all network interfaces, you can change it to 127.0.0.1 for local connections only or to an specific network interfaces address like 192.168.122.45)
  • $ $BASEDIR/bin/mysqld_safe --user=$USER \
      --basedir=$BASEDIR \
      --datadir=$DATADIR \
      --pid-file=$TARGET/var/run/mysql/mysql.pid \
      --skip-syslog \
      --log-error=$TARGET/var/log/mysql/mysql.err \
      --port=$PORT \
      --socket=$TARGET/var/run/mysqld/mysqld.sock \
      --ledir=$BASEDIR/bin \
      --mysqld=mysqld \
      --bind-address=0.0.0.0
    
  • Connecting locally (via socket) to the portable MySQL
  • $ $BASEDIR/bin/mysql -u root --socket=$TARGET/var/run/mysqld/mysqld.sock
    
  • Creating a sample database and granting all privileges to a user
  • $ $BASEDIR/bin/mysql -u root --socket=$TARGET/var/run/mysqld/mysqld.sock <<EOT
    create database alfresco;
    grant all privileges on alfresco.* to alfresco@'%' identified by 'alfresco';
    EOT
    
  • Connecting remotely to the created database (NOTE: verify firewall settings on the server before connecting remotely)
  • $ mysql -u alfresco -h SERVER --port=9797 -p alfresco
    
  • Changing the root password
  • $ $BASEDIR/bin/mysqladmin -u root password 'root' --socket=$TARGET/var/run/mysqld/mysqld.sock
    

Friday, January 6, 2012

How to install Oracle 11g Database Server on CentOS 6.2

UPDATED!:

The former post I wrote was a how to install an Oracle-ready CentOS 6.2 Linux Server box.Its time then to install the Oracle 11g Database server. This post is about the same thing, but using a CentOS 5, so I based this how to on it and improve some minor things, but the whole idea is pretty much the same.

Hardware Requirements

To check out your recommended hardware requirements, login to the CentOS 6.2 Linux Server box as the bozz user and verify the below list:

    How much memory:

$ grep -i memtotal /proc/meminfo


        results: should be at least 1GB

     How much swap:

$ grep -i swaptotal /proc/meminfo

        results:
       
        - should be 1.5 the size of memtotal if memtotal < 2GB

             - should be equal to memtotal if memtotal is between 2GB and 8GB

             - should be .75 the size of memtotal if memtotal > 8GB

    What processor type:

$ grep "model name" /proc/cpuinfo

        results: informational
   
    How much available RAM and swap:

$ free

        results: informational

    How much shared memory available:

$ df -k /dev/shm

        results: informational

        purpose: The shared memory should be sized to be at least the greater of MEMORY_MAX_TARGET and MEMORY_TARGET for each Oracle instance on the computer.

    How much disk space in /tmp:

$ df -k /tmp

        results: should have at least 400MB available

    How much disk space for Oracle binaries:

$ df -k

        results: Depending on installation type you should have between 1.5GB and 3.5GB


Software Requirements

To check out your recommended software requirements, login to the CentOS 6.2 Linux Server box as the bozz user and verify the below list:      

   What distribution and version of Linux is installed:

$ cat /proc/version
        results:  Linux version 2.6.32-220.2.1.el6.x86_64 (mockbuild@c6-x8664-build.centos.org) 

(gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) ) 

#1 SMP Fri Dec 23 02:21:33 CST 2011


    What version of the kernel is installed:

$ uname -r
        results: 2.6.32-220.2.1.el6.x86_64

    Ensure that the following packages are installed:

$ rpm -q compat-libstdc++-33.x86_64 binutils elfutils-libelf elfutils-libelf-devel 
$ rpm -q glibc glibc-common glibc-devel glibc-headers gcc gcc-c++ libaio-devel 
$ rpm -q libaio libgcc libstdc++ libstdc++ make sysstat unixODBC unixODBC-devel
    if some of this packages are missing then install them
  
$ sudo yum install compat-libstdc++-33.x86_64 binutils elfutils-libelf elfutils-libelf-devel 
$ sudo yum install glibc glibc-common glibc-devel glibc-headers gcc gcc-c++ libaio-devel 
$ sudo yum install libaio libgcc libstdc++ libstdc++ make sysstat unixODBC unixODBC-devel

     also install unzip package:
$ sudo yum install unzip 

Install Steps

  1. Login in as bozz user
  2. Create Oracle's user and groups
    1. Create groups oinstall & dba:
    2. $ sudo groupadd oinstall
      $ sudo groupadd dba
      
    3. Create the oracle user and set its password:
    4. $ sudo useradd -m -g oinstall -G dba -s /bin/bash oracle
      $ sudo passwd oracle
      
    5. Check if nobody user exists, this user is almost always created on a base CentOS installation:
    6. $ id nobody
      uid=99(nobody) gid=99(nobody) groups=99(nobody)
      
      if this user doesn't exists then use the following command to create it:
      $ sudo useradd nobody
      
  3. Configure some kernel parameters. The following kernel parameters must be validated and possibly changed. Use the commands given below to view a particular kernel setting and then change it if the setting is lower than the recommendation given here. The changed parameters are in bold in my case.
    1. Verify kernel parameters: 
    2. # semaphores: semmsl, semmns, semopm, semmni
      $ sysctl -a | grep kernel.sem
      
      results should be >= than: 
      
      kernel.sem = 250        32000   100     128
      
      $ sysctl -a | grep kernel.shm
      
      results should be >= than: 
      
      kernel.shmmax = 536870912
      
      kernel.shmmni = 4096
      
      kernel.shmall = 2097152
      
      $ sysctl -a | grep file-max
      
      results should be >= than: 
      
      fs.file-max = 6815744
      
      $ sysctl -a | grep ip_local_port_range
      
      results should be >= than: 
      
      net.ipv4.ip_local_port_range = 9000 65500
      
      $ sysctl -a | grep rmem_default
      
      results should be >= than: 
      
      net.core.rmem_default = 262144
      
      $ sysctl -a | grep rmem_max
      
      results should be >= than: 
      
      net.core.rmem_max = 4194304
      
      $ sysctl -a | grep wmem_default
      
      results should be >= than: 
      
      net.core.wmem_default = 262144
      
      $ sysctl -a | grep wmem_max
      
      results should be >= than: 
      
      net.core.wmem_max = 1048576
      
      $ sysctl -a | grep aio-max-nr
      
      results should be >= than: 
      
      fs.aio-max-nr = 1048576
      
    3. Make the changes permanent by editing the archive /etc/sysctl.conf file. This is given as an example only and does not reflect the results obtained above by the sysctl commands. Only add lines that are needed and do not change any values to be lower than the base installation might have assigned. In my case I only change the following settings on /etc/sysctl.conf:
    4. kernel.sem = 250 32000 100 128
      fs.file-max = 6815744
      net.ipv4.ip_local_port_range = 9000 65500
      net.core.rmem_default = 262144
      net.core.rmem_max = 4194304
      net.core.wmem_default = 262144
      net.core.wmem_max = 1048576
      fs.aio-max-nr = 1048576
      
    5. Apply the changes now:
    6. $ sudo sysctl -p
      
  4. Setting shell limits for the oracle user
    1. Edit the file /etc/security/limits.conf:
    2. $ sudo nano /etc/security/limits.conf
      
      and add the following:
      oracle soft nproc  2047
      oracle hard nproc  16384
      oracle soft nofile 1024
      oracle hard nofile 65536
      
    3. Edit /etc/pam.d/login:
    4. $ sudo nano /etc/pam.d/login
      and add the following depending on the architecture:
      session required /lib64/security/pam_limits.so
      session required pam_limits.so
      NOTE: be aware of the real location of pam_limits.so. It depens on the architecture and distro. On CentOS 6.2 64 bits /lib64/security/pam_limits.so
    5. Create a new profile archive at  /etc/profile.d/custom.sh:
    6. $ sudo nano /etc/profile.d/custom.sh
      and add the following content:
      #!/bin/bash
      
      if [ $USER = "oracle" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
          ulimit -p 16384
          ulimit -n 65536
        else
          ulimit -u 16384 -n 65536
        fi
      fi
      then add execution permissions to profile script: 
      $ sudo chmod +x /etc/profile.d/custom.sh
  5. Creating necessary directories to install the Oracle Software
    1. Check if you have enough space on disk and choose a directory to install:
    2. $ df -kh
    3. Create subdirectories and give permissions to the oracle user:
    4. $ sudo mkdir -p /opt/app/oracle/product/11.2.0
      $ sudo chown -R oracle:oinstall /opt/app
      $ sudo chmod -R 775 /opt/app
      
  6. Logout, login as oracle user (don't forget ssh -Y oracle@SERVER) and set the environment for the installation by adding the following to ~/.bash_profile: :
  7. $ nano ~/.bash_profile
    
    add the following line:
    umask 022
    
    export TMPDIR=$TMP
    export ORACLE_BASE=/opt/app/oracle
    export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
    export PATH=$ORACLE_HOME/bin:$PATH
    
    exit nano (Ctrl+X) and execute oracle's .bash_profile for testing purposes:
    $ source ~/.bash_profile
    logout from oracle user:
    $ exit
  8. Ensure oracle user is using an X Windows System. For example you can install and execute xterm:
  9. $ sudo yum install xterm
    
    logout and login remotely again as oracle user from a Linux client machine with Desktop environment:
    $ ssh -Y oracle@SERVER
    
    test if xterm works you should be able to see a X11 Windows on your client Desktop environment, if not then enable SSH X11 Forwarding (see last step)
    $ xterm
    
  10. Start the database installer (you should be able to see a X11 Windows with the Oracle database installation wizard on your client Desktop environment):
    $ cd /tmp/database 
    $ ./runInstaller
  11. NOTE: I assume you already had the database installer on the server. If not then copy the installer archives to the server via scp and uncompress them: 
    on the client run:
    $ scp linux.x64_11gR2_database_1of2.zip oracle@SERVER:/tmp
    $ scp linux.x64_11gR2_database_2of2.zip oracle@SERVER:/tmp
    
    on the server run:
    $ pushd /tmp
    $ unzip linux.x64_11gR2_database_1of2.zip
    $ unzip linux.x64_11gR2_database_2of2.zip
    $ popd 
    
  12. Optionally specify email address to be informed about security issues
  13. Choose "Install database software only", click "Next"
  14. Choose "Single instance database installation", click "Next"
  15. Add another language besides of English if you wish, click "Next"
  16. Choose 'Enterprise Edition' and on "Select options" choose the components you wish to install, click "Next"
  17. Set the following settings and click "Next":
    • verify Oracle Base: '/opt/app/oracle'
    • verify Oracle Home: '/opt/app/oracle/product/11.2.0/db_1'
  18. Verify path of inventory directory: '/opt/app/oraInventory'  and verify group name for install of 'oinstall', click "Next" 
  19. Prerequisite Checks:  the installer will verify the system environment and report on any issues it may find. If all goes well you will get the status of 'Succeeded' for every check. You have to check 'Ignore All' but ensure at least that the following checks pass
    1. Physical memory (also Available)
    2. Free space
    3. User and groups existence
    4. Group membership
    5. Hard and soft limits
    6. Architecture
    7. Kernel version
    8. OS Kernel parameters
    9. Swap size
    10. Run level.
    11. Packages
    12. Users with same ID
  20. Summary: This screen give a summary of the installation you are about to perform. Click and expand the products being installed to give you an idea of the what product and version is going to be installed.
    1. validate 'Global Settings' for directories
    2. validate 'Space Requirements'
    3. Click NEXT
  21. Install: This screen gives a status of what is being installed. If all system settings have been properly set and all packages been installed properly then this should proceed without error. 
    1. Note the time it takes for specific products. If you notice one hanging or taking a long amount of time (greater than 10 minutes) you could have a condition that requires a pre-installation patch. Don't stop the installer yet but instead start looking for a bug / patch for the Oracle 11g installation process and the actual product being installed.
    2. Execute configuration scripts, when the wizard ask it:
      • open up a different terminal window
      • login as the bozz user
      • run:
      • $ sudo /opt/app/oraInventory/orainstRoot.sh
        $ sudo /opt/app/oracle/product/11.2.0/db_1/root.sh
        accept a default of /usr/local/bin
      • click OK
  22. Cleanup: If you downloaded the software from the net you can now remove .zip file and the complete directory structure you unzipped it to.
  23. Final setup:  Add the following lines to oracle's $HOME/.bash_profile archive. This will allow you to execute various Oracle binaries to create databases, configure network settings, and generally manage the Oracle database environment:
  24. umask 022
    
    export TMPDIR=/tmp
    export ORACLE_BASE=/opt/app/oracle
    export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
    export PATH=$ORACLE_HOME/bin:$PATH
    NOTE: remember to first login as oracle user to edit the $HOME/.bash_profile archive. Finally ensure the execution permissions for this archive.

Post Install

References

Thursday, January 5, 2012

Step-by-step installing an Oracle-ready CentOS 6.2 Server

This is how to for installing an Oracle-ready Linux box with CentOS 6.2, It covers the steps involved on the preparation of an Oracle-ready server for the further installation of Oracle Database Server/Oracle Fusion Middleware Suite on a multi-role server deployment.

Requirements

  •  An USB stick with more than 4 GB of free storage
  • A 64 bits server machine with plenty RAM and HDD storage
  • Connectivity with CentOS Internet repositories or LAN mirrors
  • A Linux client machine with an installed desktop environment: CentOS, Debian, Ubuntu, etc. + Gnome, KDE, etc.

Steps

  1. Download a CentOS-6.2-x86_64-bin-DVD1.iso  (4.1GB) from the DVD mirrors or alternative download a minimal (322M), netinstall (227M), LiveCD (698M) or LiveDVD (1.6G) version. This how to uses CentOS-6.2-x86_64-bin-DVD1.iso  (4.1GB).
  2. Store the downloaded ISO into de USB key via this guide. I prefer the alternatives section with livecd-iso-to-disk script. 
  3. Connect the USB stick to one of the USB ports on the server machine and boot. Ensure that USB boot is enabled on the setup.
  4. On the CentOS installation wizard select Server and customize.
  5. Choose a wise partition schema depending on your hardware, for example 6 GB RAM + 2 x 500GB HDD:
  6. /            ext4    60GB   
    /boot    ext4    130M
    /home    xfs        ~403GB
    /opt    xfs        ~240GB      
    /var    xfs        ~210GB
    swap    swap    12GB 
  7.  On the package selection step ensure that no desktop environment is selected and ensure that ssh is installed. Select any additional package you want.
  8. Choose a complex root password.
  9. Once the OS installation is completed, reboot, login as root and ensure that ssh is working via:
  10. $ ssh localhost
  11. By default, the eth0 network interface is disabled. Configure networking settings:
    1. Login as root
    2. Edit the archive /etc/sysconfig/network-scripts/ifcfg-eth0  and configure the network interface to look like:
    3. DEVICE="eth0"
      HWADDR="00:1C:C0:95:59:55"
      NM_CONTROLLED="yes"
      ONBOOT="yes"
      NETWORK=192.168.192.0
      NETMASK=255.255.255.0
      IPADDR=192.168.192.201
      GATEWAY=192.168.192.1
      DNS1=192.168.192.2
      DNS2=192.168.192.2
      PEERDNS="yes"
      
    4. Restart networking daemon and test for connectivity with ping or other tool.
  12. Add a sudoer user -for example bozz- to avoid the use of root
    1. Login as root
    2. Create a new user and add it to wheel group:
    3. $ adduser -G wheel bozz
    4. Reset the password for bozz user (NOTE: choose a password you won't forget):
    5. $ passwd bozz
    6. Edit /etc/sudoers and uncomment the line starting with %wheel, it shall look like:
    7. %wheel  ALL=(ALL)    ALL
    8. Logout, login again with  bozz user and test sudoer's privileges:
    9. $ sudo ls -hal
    NOTE: From now, I only use the bozz user instead of root. You can also (optionally) disable the root account from remote access for security reasons.
  13. Login as bozz user
  14. Configure CentOS repositories via LAN mirrors:
    1. Ensure you can connect to the LAN mirror, example: http://mirrors.home.dev/centos
    2. $ pushd /tmp
      $ wget http://mirrors.home.dev/centos
      $ popd
      
    3. Create a new repo file at /etc/yum.repos.d/CentOS-Mirror.repo
    4. $ sudo nano /etc/yum.repos.d/CentOS-Mirror.repo
    5. Append the following entries pointing to the mirror:
    6. [base]
      name=CentOS-$releasever - Base
      baseurl=http://mirrors.home.dev/centos/$releasever/os/$basearch/
      gpgcheck=0
       
      #released updates 
      [updates]
      name=CentOS-$releasever - Updates
      baseurl=http://mirrors.home.dev/centos/$releasever/updates/$basearch/
      gpgcheck=0
      
      #additional packages that may be useful
      [extras]
      name=CentOS-$releasever - Extras
      baseurl=http://mirrors.home.dev/centos/$releasever/extras/$basearch/
      gpgcheck=0
      
    7. Comment or delete existing repository archives
      • comment every line of /etc/yum.repos.d/CentOS-Base.repo with #
      • or just:
      • delete the repository file
      • $ sudo rm /etc/yum.repos.d/CentOS-Base.repo
    8. Update the system using the mirror:
    9. $ sudo yum update
  15. Install Java JRE 7u1 64 bit on the server [Not required for Oracle Database, required for Oracle Fusion Middleware Suite]:
    1. Goto http://www.oracle.com/technetwork/java/javase/downloads/index.html and download jre-7u1-linux-x64.rpm
    2. Copy jre-7u1-linux-x64.rpm to the server via scp or other. Via scp from the Linux client:
    3. $ scp jre-7u1-linux-x64.rpm bozz@SERVER:~/
    4. Login into server and run installer:
    5. $ sudo rpm -Uvh jre-7u1-linux-x64.rpm
    6. Make use of Java JRE absolute version (/usr/java/jre1.7.0_01) in the alternatives configuration for java and javaws:
    7. $  sudo alternatives --install /usr/bin/java java /usr/java/jre1.7.0_01/bin/java 20000
      $  sudo alternatives --install /usr/bin/javaws javaws /usr/java/jre1.7.0_01/bin/javaws 20000
      
    8. Ensure the correct java version:
    9. $ java -version
      java version "1.7.0_01"
      Java(TM) SE Runtime Environment (build 1.7.0_01-b08)
      Java HotSpot(TM) 64-Bit Server VM (build 21.1-b02, mixed mode)
    10. Ensure that the alternatives is properly configured:
    11. $ alternatives --config java
      There is 1 program that provides 'java'.
      
          Selection    Command
        -----------------------------------------------
        *+ 1           /usr/java/jre1.7.0_01/bin/java
      
        Enter to keep the current selection[+], or type selection number: 
       
      $ alternatives --config javaws
      
        There is 1 program that provides 'javaws'.
      
          Selection    Command
        -----------------------------------------------
        *+ 1           /usr/java/jre1.7.0_01/bin/javaws
      
      
    12. Configure JRE_HOME system-wide environment variable. Edit /etc/environment:
    13. $ nano /etc/environment
    14. Append the following content:
    15. export JRE_HOME=/usr/java/jre1.7.0_01
      
    16. Logout and login again and test if JRE_HOME environment variable is correct:
    17. $ echo $JRE_HOME
      /usr/java/jre1.7.0_01
      
      $ ls -1 $JRE_HOME
      bin
      COPYRIGHT
      lib
      LICENSE
      man
      plugin
      README
      release
      THIRDPARTYLICENSEREADME.txt
      Welcome.html
      
      
  16. Prepare the CentOS server for installing Oracle products without sacrifice performance with a graphical environment. Oracle Fusion Middleware products use an X11 graphical environment on installation wizards. So the  SSH X11 Forwarding should be installed and enabled.
    1. On the server, edit /etc/ssh/sshd_config:
    2. $ sudo nano  /etc/ssh/sshd_config
    3. Set the following values:
    4. X11Forwarding yes
      X11DisplayOffset 10
      X11UseLocalhost yes
      
    5. On the server, install the packages  xorg-x11-xauth and libXtst:
    6. $ sudo yum install xorg-x11-xauth libXtst
    7. On the client, install the fontconfig packages xorg-x11-fonts-base and liberation-fonts:

    8. for a CentOS client:
      $ sudo yum install xorg-x11-fonts-base liberation-fonts
      for Ubuntu client:
      $ sudo apt-get install xfonts-base ttf-liberation
    9. On the client, try to login to the remote server using "-Y" option:
    10. $ ssh -Y bozz@SERVER
    11. Remotely logged, try to run your X program, such as the xterm or other:
    12. $ sudo yum install xterm
      $ xterm
      And you shall see the X program pop up in your local desktop. You can also add the following into your $HOME/.ssh/config :
      ForwardAgent yes
      ForwardX11 yes
      ForwardX11Trusted yes
      
       

References