Thursday, July 5, 2012

Building and packaging the latest Gearman server in CentOS 6.2

This is a how to package, install and test the latest version of Gearman server in RPM format using CentOS 6.2.

Motivation

The latest version available of Gearman in the Fedora repository is very outdated, CentOS is even more. So, if you plan to use the latest Gearman features, you have two choices 1) compile it using the tarball; 2) package it (hence compile it) in RPM format.
This guide uses the second approach, but not from scratch. Instead using the latest available SRPM German's package and performing some minor changes.

At this moment, the latest Gearman version is 0.33 and the latest Fedora-based SRPM is 0.23.

Hands on bash

  1. Append EPEL repository by creating the file /etc/yum.repos.d/epel.repo with the following content:
  2. [epel]
    name=Extra Packages for Enterprise Linux 6 - $basearch
    #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
    mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
    failovermethod=priority
    enabled=1
    gpgcheck=0
    
    [epel-debuginfo]
    name=Extra Packages for Enterprise Linux 6 - $basearch - Debug
    #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch/debug
    mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-6&arch=$basearch
    failovermethod=priority
    enabled=0
    gpgcheck=0
    
    [epel-source]
    name=Extra Packages for Enterprise Linux 6 - $basearch - Source
    #baseurl=http://download.fedoraproject.org/pub/epel/6/SRPMS
    mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-source-6&arch=$basearch
    failovermethod=priority
    enabled=0
    gpgcheck=0
    
  3. Install building dependencies:
  4. $ sudo yum groupinstall "Development Tools"
    $ sudo yum install libevent-devel libuuid-devel
    $ sudo yum install boost-devel
    $ sudo yum install libmemcached-devel memcached google-perftools-devel 
    
  5. Create a temporal cache and define a downloader command:
  6. $ mkdir -p /tmp/cache
    
    #/** 
    # * Downloads a file to the cache if doesn't exists
    # *
    # * @param $1 the file to download
    # * @param $2 the url where the file is located
    # */
    $ get() {
     [ -f /tmp/cache/$1 ] || wget -t inf -w 5 -c $2/$1 -O /tmp/cache/$1
    }
    
  7. Download latest Gearman tarball with sources and the latest SRPM package provided by Fedora
  8. $ lastver=0.33
    $ gm=gearmand-$lastver.tar.gz
    $ get $gm https://launchpadlibrarian.net/104788829/$gm 
    
    $ srcver=0.23
    $ srpm=gearmand-$srcver-1.fc16.src.rpm
    $ get $srpm http://www.muug.mb.ca/mirror/fedora/linux/releases/16/Everything/source/SRPMS/$srpm
    
    
  9. Create a clean RPM build environment and install the SRPM package on it:
  10. $ rm -rf ~/rpmbuild ~/.rpmmacros
    $ mkdir -p ~/rpmbuild/{BUILD,RPMS,S{OURCE,PEC,RPM}S}
    $ cat > ~/.rpmmacros <<< "%_topdir $HOME/rpmbuild"
    
    $ rpm -ivh /tmp/cache/$srpm
    
    
  11. Remove the unpacked old Gearman tarball and copy the new one to SOURCES:
  12. $ cd ~/rpmbuild
    $ cp /tmp/cache/gearmand-$lastver.tar.gz SOURCES/
    $ rm SOURCES/gearmand-$srcver.tar.gz
    
  13. Now the trick. The sed command below performs some changes on gearmand.spec file: 1) replaces the old version number with the new one; 2) Comments dependencies from systemd packages not available yet on CentOS; and 3) Adds various file/directory entries only available on the latest version of Gearman.
  14. $ sed -i \
      -e 's/\(Version:[[:space:]]\+\)'${srcver}'/\1'${lastver}'/' \
      -e 's/^BuildRequires:[[:space:]]\+systemd-units/#&/' \
      -e 's/^Requires(post):[[:space:]]\+systemd-sysv/#&/' \
      -e 's/^Requires(post):[[:space:]]\+systemd-units/#&/' \
      -e 's/^Requires(preun):[[:space:]]\+systemd-units/#&/' \
      -e 's/^Requires(postun):[[:space:]]\+systemd-units/#&/' \
      -e 's/install -m 0644 %{SOURCE3} %{buildroot}%{_unitdir}\/%{name}.service/#&/' \
      -e 's/^%changelog/%files\n%defattr(-,root,root,-)\n\/usr\/include\/libgearman-1.0\n\/etc\/rc.d\/init.d\/gearmand\n\/etc\/sysconfig\/gearmand\n\/usr\/bin\/gearadmin\n\/usr\/bin\/gearman\n\/usr\/sbin\/gearmand\n\/usr\/share\/man\n\n&/' \
      SPECS/gearmand.spec
    
  15. Build and install the Gearman RPM packages:
  16. $ rpmbuild -bb SPECS/gearmand.spec
    
    $ sudo rpm -ivh RPMS/x86_64/libgearman-$lastver-1.el6.x86_64.rpm RPMS/x86_64/gearmand-$lastver-1.el6.x86_64.rpm
    
    
  17. Register gearmand as a daemon on standard runleves:
  18. $ sudo chkconfig --add gearmand
    $ sudo chkconfig gearmand on
    
  19. Create an emtpy log archive with permissions for gearmand user amd group (created by the RPM installer), crate pid directory and start the daemon:
  20. $ sudo mkdir -p /usr/local/var
    $ sudo ln -s /var/log /usr/local/var/log 
    
    $ sudo touch /var/log/gearmand.log
    $ sudo chown gearmand:gearmand /var/log/gearmand.log
    
    $ sudo mkdir -p /var/run/gearmand/
    
    $ sudo /etc/init.d/gearmand start
    

Testing

  1. Use the examples provided by Gearman to test. Enter examples directory and run the reverse_worker example:
  2. $ cd BUILD/gearmand-$lastver/examples
    
    $ ./reverse_worker 
    
    
  3. Open a new terminal, enter examples directory and run the reverse_client:
  4. cd ~/rpmbuild/BUILD/gearmand-$lastver/examples
    
    $ ./reverse_client "Hello Gearman"
    

    in the reverse_worker terminal you should see:
    Recieved 12 bytes
    Job=H:centosarq62.localdomain:1  Reversed=HelloGearman
    
    

Now your Gearman with the latest features is ready for battle.

Enjoy it!

Wednesday, July 4, 2012

The NGINX's RAM-only Jail. Adding nginx to RAM-only diskless Linux box

Without any doubt, nginx is one of the best ever created HTTP servers, it solves the C10K problem with an elegant event-drive architecture. Many big sites around the WWW use it: Wikipedia, Hyves, etc.

This is a how to prepare your rambox to run nginx smoothly. It makes sense only after the preparation of the RAM-only PXE boot in my previous post. This post isn't about any kind of nginx optimization stuff beside of its compilation in a single binary rock. At the end of this guide, you will be running the nginx very much like in a jail.

Run these steps BEFORE the step "Change ownership to everything" at RAM-only PXE boot in my previous post.
  1. Download latest stable version nginx and upack it.
  2. $ pushd /tmp/wrk
    $ ngx=nginx-1.2.1.tar.gz
    $ cache $ngx http://nginx.org/download/$ngx
    $ tar -xvzf /tmp/cache/$ngx -C .
    $ mv $ngx nginx
    
  3. Add options depending on the features that you want to support. Beside of the default options I only add static compilation options:
  4. $ pushd nginx
    $ ./configure --with-ld-opt="-static -static-libgcc" \
      --with-cc-opt="-static -static-libgcc"
    
    make it (jN means N threads devoted to compilation and linking):
    $ make -j2
    
    ensure that it's not a dynamic executable:
    $ ldd objs/nginx
     not a dynamic executable
    $ popd
    
  5. Copy the nginx executable to rambox's /sbin :
    $ popd 
    $ pushd sbin
    $ chmod +w .
    $ cp ../../nginx/objs/nginx .
    $ chmod -w .
    $ popd
    
  6. Create /usr/local/nginx directories:
  7. $ mkdir -p -m 0755 usr/local/nginx/{conf,logs}
    
  8. Copy some needed libs:
  9. $ chmod +w lib
    $ cp /lib64/{ld-2.12.so,ld-linux-x86-64.so.2} lib/
    $ cp /lib64/{libc-2.12.so,libc.so.6} lib/
    $ cp /lib64/{libnsl-2.12.so,libnsl.so.1} lib/
    $ cp /lib64/{libnss_compat-2.12.so,libnss_compat.so.2} lib/
    $ chmod -w lib
    
  10. Copy mime conf file:
  11. $ cp ../nginx/conf/mime.types usr/local/nginx/conf/
    
  12. Create the nginx.conf file with some basic settings:
  13. $ dd of=usr/local/nginx/conf/nginx.conf << EOT
    # user and group to run nginx
    user www www;
    
    # numbers of dedicated CPUs
    worker_processes 1;
    
    # pid archive
    pid /var/run/nginx.pid;
    
    events {
        # max connections in WAIT 
        worker_connections  128;
      
        # accept & enqueue NEW connections, put them in WAIT
        multi_accept        on;
    }
    
    http {
            include       mime.types;
            default_type  application/octet-stream;
    
            server {
                    listen 80;
    
                    autoindex on;
     
                    location / {
                            root /var/www;
                            index  index.php index.html index.htm;
                    }
            }
    }
    EOT
    
  14. Resume the rambox creation process and once it gets started run:
  15. $ nginx
Enjoy it by browsing at http://HOSTADDRESS

Post install steps

Daemonize nginx by running it at /init script!

Tuesday, July 3, 2012

The Parking Permit Demo. README and Sources for Season 01

Here I let you the README and Sources for the Season 01 of The Parking Permit Demo with Oracle BPM/SOA suite.

 

 Enjoy it!