Thursday, July 5, 2012

Building and packaging the latest Gearman server in CentOS 6.2

This is a how to package, install and test the latest version of Gearman server in RPM format using CentOS 6.2.

Motivation

The latest version available of Gearman in the Fedora repository is very outdated, CentOS is even more. So, if you plan to use the latest Gearman features, you have two choices 1) compile it using the tarball; 2) package it (hence compile it) in RPM format.
This guide uses the second approach, but not from scratch. Instead using the latest available SRPM German's package and performing some minor changes.

At this moment, the latest Gearman version is 0.33 and the latest Fedora-based SRPM is 0.23.

Hands on bash

  1. Append EPEL repository by creating the file /etc/yum.repos.d/epel.repo with the following content:
  2. [epel]
    name=Extra Packages for Enterprise Linux 6 - $basearch
    #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
    mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch
    failovermethod=priority
    enabled=1
    gpgcheck=0
    
    [epel-debuginfo]
    name=Extra Packages for Enterprise Linux 6 - $basearch - Debug
    #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch/debug
    mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-6&arch=$basearch
    failovermethod=priority
    enabled=0
    gpgcheck=0
    
    [epel-source]
    name=Extra Packages for Enterprise Linux 6 - $basearch - Source
    #baseurl=http://download.fedoraproject.org/pub/epel/6/SRPMS
    mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-source-6&arch=$basearch
    failovermethod=priority
    enabled=0
    gpgcheck=0
    
  3. Install building dependencies:
  4. $ sudo yum groupinstall "Development Tools"
    $ sudo yum install libevent-devel libuuid-devel
    $ sudo yum install boost-devel
    $ sudo yum install libmemcached-devel memcached google-perftools-devel 
    
  5. Create a temporal cache and define a downloader command:
  6. $ mkdir -p /tmp/cache
    
    #/** 
    # * Downloads a file to the cache if doesn't exists
    # *
    # * @param $1 the file to download
    # * @param $2 the url where the file is located
    # */
    $ get() {
     [ -f /tmp/cache/$1 ] || wget -t inf -w 5 -c $2/$1 -O /tmp/cache/$1
    }
    
  7. Download latest Gearman tarball with sources and the latest SRPM package provided by Fedora
  8. $ lastver=0.33
    $ gm=gearmand-$lastver.tar.gz
    $ get $gm https://launchpadlibrarian.net/104788829/$gm 
    
    $ srcver=0.23
    $ srpm=gearmand-$srcver-1.fc16.src.rpm
    $ get $srpm http://www.muug.mb.ca/mirror/fedora/linux/releases/16/Everything/source/SRPMS/$srpm
    
    
  9. Create a clean RPM build environment and install the SRPM package on it:
  10. $ rm -rf ~/rpmbuild ~/.rpmmacros
    $ mkdir -p ~/rpmbuild/{BUILD,RPMS,S{OURCE,PEC,RPM}S}
    $ cat > ~/.rpmmacros <<< "%_topdir $HOME/rpmbuild"
    
    $ rpm -ivh /tmp/cache/$srpm
    
    
  11. Remove the unpacked old Gearman tarball and copy the new one to SOURCES:
  12. $ cd ~/rpmbuild
    $ cp /tmp/cache/gearmand-$lastver.tar.gz SOURCES/
    $ rm SOURCES/gearmand-$srcver.tar.gz
    
  13. Now the trick. The sed command below performs some changes on gearmand.spec file: 1) replaces the old version number with the new one; 2) Comments dependencies from systemd packages not available yet on CentOS; and 3) Adds various file/directory entries only available on the latest version of Gearman.
  14. $ sed -i \
      -e 's/\(Version:[[:space:]]\+\)'${srcver}'/\1'${lastver}'/' \
      -e 's/^BuildRequires:[[:space:]]\+systemd-units/#&/' \
      -e 's/^Requires(post):[[:space:]]\+systemd-sysv/#&/' \
      -e 's/^Requires(post):[[:space:]]\+systemd-units/#&/' \
      -e 's/^Requires(preun):[[:space:]]\+systemd-units/#&/' \
      -e 's/^Requires(postun):[[:space:]]\+systemd-units/#&/' \
      -e 's/install -m 0644 %{SOURCE3} %{buildroot}%{_unitdir}\/%{name}.service/#&/' \
      -e 's/^%changelog/%files\n%defattr(-,root,root,-)\n\/usr\/include\/libgearman-1.0\n\/etc\/rc.d\/init.d\/gearmand\n\/etc\/sysconfig\/gearmand\n\/usr\/bin\/gearadmin\n\/usr\/bin\/gearman\n\/usr\/sbin\/gearmand\n\/usr\/share\/man\n\n&/' \
      SPECS/gearmand.spec
    
  15. Build and install the Gearman RPM packages:
  16. $ rpmbuild -bb SPECS/gearmand.spec
    
    $ sudo rpm -ivh RPMS/x86_64/libgearman-$lastver-1.el6.x86_64.rpm RPMS/x86_64/gearmand-$lastver-1.el6.x86_64.rpm
    
    
  17. Register gearmand as a daemon on standard runleves:
  18. $ sudo chkconfig --add gearmand
    $ sudo chkconfig gearmand on
    
  19. Create an emtpy log archive with permissions for gearmand user amd group (created by the RPM installer), crate pid directory and start the daemon:
  20. $ sudo mkdir -p /usr/local/var
    $ sudo ln -s /var/log /usr/local/var/log 
    
    $ sudo touch /var/log/gearmand.log
    $ sudo chown gearmand:gearmand /var/log/gearmand.log
    
    $ sudo mkdir -p /var/run/gearmand/
    
    $ sudo /etc/init.d/gearmand start
    

Testing

  1. Use the examples provided by Gearman to test. Enter examples directory and run the reverse_worker example:
  2. $ cd BUILD/gearmand-$lastver/examples
    
    $ ./reverse_worker 
    
    
  3. Open a new terminal, enter examples directory and run the reverse_client:
  4. cd ~/rpmbuild/BUILD/gearmand-$lastver/examples
    
    $ ./reverse_client "Hello Gearman"
    

    in the reverse_worker terminal you should see:
    Recieved 12 bytes
    Job=H:centosarq62.localdomain:1  Reversed=HelloGearman
    
    

Now your Gearman with the latest features is ready for battle.

Enjoy it!

2 comments:

  1. I was searching for symfony web development and landed up onto your column and i ought say thank you for sharing such practical information.

    ReplyDelete
  2. Excellent guide, thank you very much!

    ReplyDelete