Ganglia - Basic installation and configuration guide

After playing around, with both Zabbix and Ganglia, I decided to go ahead with Ganglia. The installation is tricky but once it is installed, configuring and adding hosts is easy. Here I will discuss, installation and configuration of Ganglia. Both the client and server daemon will be installed on the same machine.


Ganglia is a monitoring system for high-performance computing systems. Ganglia comprises of two daemons. The daemon gmond, acts like a client and sends machine information( as xml fomat), to daemon gmetad which stores these statistics in a round robin database.


A PHP-based web frontend is used to dispay the graphs in near real time.
Before starting Ganglia a few dependencies are required to be installed.

# tcsh

# setenv INSTALL_DIR /opt/rrdtool-1.4.4

# setenv BUILD_DIR /opt/build/rrdbuild

# cd $BUILD_DIR  

# wget http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.4.4.tar.gz 

# gunzip -c rrdtool-1.4.4.tar.gz | tar xf – # cd rrdtool-1.4.4

# ./configure --prefix=$INSTALL_DIR && make && make install

This would have installed round robin database. It is required only on the head node. The head node (server) would be the one where you want to install gmetad daemon.
Another dependency required for Ganglia is libconfuse.

# cd /opt/build
# wget http://bzero.se/confuse/confuse-2.6.tar.gz
# /bin/tar -xzf confuse-2.6.tar.gz
# cd confuse-2.6
# ./configure --prefix=/opt/ganglia
# env CFLAGS="-O3 -fPIC" ./configure --prefix=/opt/ganglia
# make
# make install

Now we download Ganglia source package and begin installation.

# cd /opt/build/ 

# wget http://sourceforge.net/projects/ganglia/files/ganglia%20monitoring%20core/3.1.7/ganglia-3.1.7.tar.gz/download 

# tar -xzf ganglia-3.1.7.tar.gz 

# cd ganglia-3.1.7  

# setenv CFLAGS "-I/opt/ganglia/include/ -I/opt/rrdtool-1.4.4/include"  

# setenv LDFLAGS "-L/opt/ganglia/lib/ -L/opt/rrdtool-1.4.4/lib"  

# ./configure --prefix=/opt/ganglia --with-gmetad --sysconfdir=/etc/ganglia  

# make  

# make install

Make sure the permissions are appropriate on path /opt/ganglia/rrds. This is the path where I have my round robin database.

# cd /opt/ganglia/sbin/

# ./gmond --default-config > gmond.conf

# ./gmetad --default-config > gmetad.conf

Setup config files for gmetad and gmond.conf
# vi gmond.conf

Set up your udp_send_channel to look like this:

 udp_send_channel {
  host = masterhostname
  port = 8650
  ttl = 1
}
# vi gmetad.conf

Modify directives as:

data_source "MyCluster" localhost

rrd_rootdir "/opt/ganglia/rrds"

# cp gmond.conf /etc/ganglia/.

# cp gmetad.conf /etc/ganglia/.

# cd $BUILD_DIR
# cp gmond/gmond.init /etc/rc.d/init.d/gmond

# vi /etc/rc.d/init.d/gmond

Set this variable as
GMOND=/opt/ganglia/sbin/gmond


Optional (this would enable running the daemons as an automated service) :

# chkconfig --add gmond

# chkconfig --list gmond
gmond 0:off 1:off 2:on 3:on 4:on 5:on 6:off

# /etc/rc.d/init.d/gmond start
Starting GANGLIA gmond: [ OK ]
# cd /opt/build/ganglia-3.1.7/gmetad
# cp gmetad.init /etc/rc.d/init.d/gmetad
# vi /etc/rc.d/init.d/gmetad


GMETAD=/opt/ganglia/sbin/gmetad


# chkconfig --add gmetad

# chkconfig --list gmetad
gmetad 0:off 1:off 2:on 3:on 4:on 5:on 6:off


# /etc/rc.d/init.d/gmetad start
Starting GANGLIA gmetad: [ OK ]

Edit httpd.conf


Add the following lines if they are not a part of your config file.

LoadModule php5_module modules/libphp5.so

Edit DirectoryIndex like

DirectoryIndex index.html index.html.var index.php
Addtype application/x-httpd-php .php

AddType application/x-httpd-php-source .phps

Copy “web” folder to httpd document root location


Edit conf.php in web folder:
define("RRDTOOL", "/opt/rrdtool-1.4.4/bin/rrdtool");


RRDTOOL should point to rrdtool path

# /sbin/service httpd restart

You can point your browser to the web front end location and it should display you the graphs.

Extjs + PHP - How to embed Treepanel, Gridpanel and Formpanel in a page?

Here is a step by step procedure to get started with a basic extjs application. It covers handling data rendering in gridpanel as well.

1. Download ExtJS and unzip it to your web path.

2. Create an html file (say test_page.html) with these contents. This will be used to render the page.

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title> Main Page </title>
<link rel="stylesheet" type="text/css" href="ext/resources/css/ext-all.css" />
<script type="text/javascript" src="ext/adapter/ext/ext-base.js"></script>
<script type="text/javascript" src="ext/ext-all.js"> </script>
<script type="text/javascript" src="javascripts/test_ext_widget.js"> </script>
</head>
<body>
<div id="header">
</div>
</body>
</html>

3. Create php file (extjs.php) with the following contents. This file return a JSON object which will be rendered by extjs.

<?php
$start = @$_REQUEST["start"];
$limit = @$_REQUEST["limit"];

$start = $start ? $start : 0;
$limit = $limit ? $limit : 5;

$data = array(
array("first"=>'Jack', "last"=>'Shephard',"bpay"=>7.75,"cpay"=>8.75,"email"=>'jackx@clemsonx.edu',"uid"=>'jackx', "hiredate"=>'8/1 12:00am'),
array("first"=>'James', "last"=>'Ford',"bpay"=>7.75,"cpay"=>8.75, "email"=>'jamesx@clemson.edu',"uid"=>'jamesx', "hiredate"=>'9/1 12:00am'),
array("first"=>'Kate',"last"=>'Austen', "bpay"=>7.75,"cpay"=>8.75, "email"=>'katex@clemson.edu',"uid"=>'katex', "hiredate"=>'10/1 12:00am'),
array("first"=>'Juliet',"last"=>'Burke',"bpay"=>7.75,"cpay"=>8.75, "email"=>'julietx@clemson.edu',"uid"=>'julietx', "hiredate"=>'9/1 10:00am'),
array("first"=>'Sayid',"last"=>'Jarrah',"bpay"=>7.75,"cpay"=>8.75, "email"=>'sayidx@clemson.edu',"uid"=>'sayidx', "hiredate"=>'9/1 11:00am'),
);

$a = array();
for($i = $start; $i < 5; $i++) {
$a[] = $data[$i];
}
$o = array(
"success"=>true
,"totalCount"=>sizeof($data)
,"rows"=>$a
);

echo json_encode($o);
?>

4. Finally a js file (say test_ext_widget.js) which will contain the javascript to render the page. This would render a page with three panel objects (gridpanel, formpanel, treepanel).

Ext.ns('app_harpreet');

Ext.BLANK_IMAGE_URL = 'ext/resources/images/default/s.gif';

var item_list = [ {xtype : 'textfield', fieldLabel : 'First name'},
{xtype : 'textfield', fieldLabel : 'Last name'},
{xtype : 'textfield', fieldLabel : 'Email'},
{xtype : 'textfield', fieldLabel : 'Student ID'},
{xtype : 'textfield', fieldLabel : 'Current Pay'},
{xtype : 'textfield', fieldLabel : 'Base Pay'},
{xtype : 'datefield', fieldLabel : 'Hire Date'}
];

var final_cbox = new Ext.form.Checkbox( {
name : 'final_question',
id : 'final_question',
inputValue : 1,
fieldLabel : 'Suspended',
checked : true

});
item_list.push(final_cbox);

var btns = [];
btns.push( {text : 'Save', scope : this});
btns.push( {text : 'Cancel', scope : this});

// now the actual form
var uForm = new Ext.form.FormPanel( {
id:'u_form', region:'center', bodyStyle:'padding:30px', items:item_list, buttons:btns});

var window1 = new Ext.Window( {
id : 'user_edit_win',
title : 'Edit',
width : 900,
height : 650,
minWidth : 500,
minHeight : 600,
layout : 'border',
plain : true,
modal : true,
bodyStyle : 'padding:55px;',
items : [ uForm ]
});
// Custom grid pre-configured class
app_harpreet.Grid = Ext.extend(Ext.grid.GridPanel, {
//double click on user row opens a pop up window which can be used for updating record
_user_row_click : function() {window1.show();},

initComponent : function() {
var config = {
store : new Ext.data.JsonStore( {
id : 'members',
totalProperty : 'totalCount',
root : 'rows',
url : 'extjs.php',
fields : [ {name : 'first'},
{name : 'last'},
{name : 'email'},
{name : 'uid'},
{name : 'cpay', type : 'float'},
{name : 'bpay', type : 'float'},
{name : 'hiredate', type : 'date', dateFormat : 'n/j h:ia'}
]
}),
listeners : {
rowdblclick : this._user_row_click,
scope : this
},
columns : [ {
id : 'uid',
header : "UID",
width : 40,
sortable : true,
dataIndex : 'uid'
},{
id : 'first',
header : "First",
width : 40,
sortable : true,
dataIndex : 'first'
},{
id : 'last',
header : "Last",
width : 40,
sortable : true,
dataIndex : 'last'
}, {
id : 'base',
header : "Base Pay",
width : 40,
sortable : true,
renderer: Ext.util.Format.usMoney,
dataIndex : 'bpay'
}, {
id : 'pay',
header : "Current Pay",
width : 40,
sortable : true,
renderer: Ext.util.Format.usMoney,
dataIndex : 'cpay'
},{
id : 'email',
header : "Email",
width : 40,
sortable : true,
dataIndex : 'email'
}, {
header : "Hire Date",
width : 20,
sortable : true,
renderer : Ext.util.Format.dateRenderer('m/d/Y'),
dataIndex : 'hiredate'
},{
id : 'suspend',
header : "Suspend Until",
width : 40,
sortable : true,
dataIndex : 'suspend'
}],
viewConfig : {forceFit : true}
};

// apply config
Ext.apply(this, Ext.apply(this.initialConfig, config));

this.bbar = new Ext.PagingToolbar( {
store : this.store,
displayInfo : true,
pageSize : 10,
items : [ { xtype : 'textfield' } ]
});

// call parent
app_harpreet.Grid.superclass.initComponent.apply(this, arguments);
}

,
onRender : function() {
// call parent
app_harpreet.Grid.superclass.onRender.apply(this, arguments);
// load the store
this.store.load( {params : {start : 0,limit : 10}});
}});

Ext.reg('membergrid', app_harpreet.Grid);
var item_list1 = [ {xtype : 'textfield',fieldLabel : 'First name'},
{xtype : 'textfield',fieldLabel : 'Last name'},
{xtype : 'textfield', fieldLabel : 'Email'},
{xtype : 'textfield', fieldLabel : 'Student ID'} ];

var btns1 = [];
btns1.push( { text : 'Edit' });

Ext.onReady(function() {
var viewport = new Ext.Viewport( {
layout : 'border',
items : [ { //left tree panel
xtype : 'treepanel',
region : 'west',
collapsible : true,
title : 'Navigation',
id : 'tpNavigation',
root : new Ext.tree.AsyncTreeNode( {
expanded : true,
text : 'Home',
children : [ {text : 'User',leaf : true},
{text : 'Manage users',leaf : true},
{text : 'Clients',leaf : true}]
}),
rootVisible : false,
width : 200
// the west region uses a TreePanel with Accordion layout
}, { //bottom panel
region : 'south',
title : 'Search',
collapsible : true,
height : 200,
minHeight : 200,
items : [ {
xtype : 'form',
items : item_list1,
buttons : btns1
} ]
}, { // main panel to dispay records
region : 'center',
layout : 'fit',
title : 'Center',
xtype : 'membergrid'
} ]
});
});

Now you can point your browser to your html page to render the treepanel, formpanel and gridpanel. The page would make a call to extjs.php, to retrieve the data, as a Json object.
You can double click on the rows to open a pop up window, which can be used to update the data in the row.(row update is not accomplished in the above code)

Please let me know of any improvements/corrections.

Perl - How to create a tcp socket?

The following program can be used to create a tcp connection to a host on a specified port.
The program attempts connection at a periodic interval, which is passed as 'execution_interval'.

#!/usr/bin/perl
use IO::Socket;
use POSIX qw/strftime/;
if (@ARGV != 4)
{
  usage(); 
  exit;              
}
my $hostname = $ARGV[0];
my $port = $ARGV[1];
my $timeout = $ARGV[2];
my $interval = $ARGV[3];

my $logdir = "/tmp/";
my $datestamp = strftime('%m-%d-%y_%H:%M:%S',localtime);
$mday = (localtime(time))[3];

my $logfile = $logdir."$hostname\_$port\_tcp_socket\_$datestamp.log";
open LOG, ">>","$logfile" or die $!;
LOG->autoflush(1);
while(1) {
    my $sock = new IO::Socket::INET (PeerAddr => $hostname,
                                     PeerPort => $port,
                                     Proto => 'tcp',
                                     Timeout => $timeout);
    $cur_datestamp = strftime('%m-%d-%y_%H:%M:%S',localtime);
    $cur_mday = (localtime(time))[3];   
    if(!$sock) {
        print LOG "\n$cur_datestamp : Error - Could not create socket to $hostname on $port: $!";
        sleep $interval;
        next;       
    }   
    if($cur_mday != $mday) {
        close(LOG);
        $logfile = $logdir."$hostname\_$port\_tcp_socket\_$datestamp.log";
        open LOG, ">>","$logfile" or die $!;
        LOG->autoflush(1);
        $mday = $cur_mday;
    }   
   
    print LOG "\n$cur_datestamp : Socket created to $hostname on $port";   
    close($sock);
    sleep $interval;
}

close(LOG);

sub usage
{
  print "Invalid parameters \nUsage: ./tcp_socket.pl \n";
}


Please let me know, of any corrections/improvements.

Perl - send email without using sendmail

In a recent exercise, I was required to write a script which would attempt sending email through an smtp server instead of sendmail. At a periodic interval, the script would check if the smtp server is able to send emails or not. Basically, this script ran as a background process.

#!/usr/bin/perl

use Net::SMTP;
use POSIX qw/strftime/;
if (@ARGV != 5)
{
  usage(); 
  exit;              
}

my $smtpmailhost = $ARGV[0];
my $from_address = $ARGV[1];
my $to_address = $ARGV[2];
my $timeout = $ARGV[3];
my $interval = $ARGV[4];

my $logdir = "/tmp/email_logs/";

my $datestamp = strftime('%m-%d-%y_%H:%M:%S',localtime);
$mday = (localtime(time))[3];

my $logfile = $logdir."$smtpmailhost\_email_log\_$datestamp.log";
open LOG, ">>","$logfile" or die $!;
LOG->autoflush(1);
while(1) {
    $smtp = Net::SMTP->new($smtpmailhost,
                            Timeout => $timeout);
    $cur_datestamp = strftime('%m-%d-%y_%H:%M:%S',localtime);
    $cur_mday = (localtime(time))[3];
    if(!$smtp) {
        print LOG "\n$cur_datestamp : Error creating smtp object for $smtpmailhost: $!";
        sleep $interval;
        next;               
    }       
    if($cur_mday != $mday) {
        close(LOG);
        $logfile = $logdir."$smtpmailhost\_email_log\_$cur_datestamp.log";
        open LOG, ">>","$logfile" or die $!;
        LOG->autoflush(1);
        $mday = $cur_mday;
    }   
    $smtp->mail($from_address);
    $smtp->to($to_address);
    $smtp->data();
    $smtp->datasend("Subject: $smtpmailhost - Test email message\n");
    $smtp->datasend("\n");
    $smtp->datasend("Test email has been sent successfully at $cur_datestamp!");
    $smtp->dataend();
    $smtp->quit;
    print LOG "\n$cur_datestamp : Test Email sent via $smtpmailhost!";   
     sleep $interval;   
}

close(LOG);
sub usage
{
  print "Invalid parameters \nUsage: ./smtpemail_send.pl \n";
}



Please let me know, of any corrections/improvements.

Yii + MySql + Wamp

First, I installed WampServer, which is a package of Apache, Php and MySql coupled into one. In the httpd.conf file I pointed the “DocumentRoot” and “Directory” configurations to the directory, where I would have my application and Yii framework.

I copied the Yii framwork in "d:/workspace/app_demo/”

To create the stub application from yii, I executed these commands for my setup:

>cd D:\workspace\app_demo

>yii\framework\yiic webapp myapp

-------

This created an application “myapp” under “app_demo” folder.

The “/app_demo/myapp/protected/config/main.php” file contains the database configurations. By default the database was pointed to an sqlite database. I changed that, to point to Mysql database. So I had these settings for my mysql database.

/* 'db'=>array(

'connectionString' => 'sqlite:'.dirname(__FILE__).'/../data/testdrive.db',

),*/

'db'=>array(

'connectionString' => 'mysql:host=localhost;dbname=pages_db',

'emulatePrepare' => true,

'username' => 'root',

'password' => '',

'charset' => 'utf8',

),

I also enabled logging through this file to look like this:

'log'=>array(

'class'=>'CLogRouter',

'routes'=>array( array('class'=>'CFileLogRoute',

'levels'=>'error, warning',

),

array('class'=>'CWebLogRoute',

), ), ),

-------

My database had two tables (user and address). The sql is shown below:

CREATE TABLE IF NOT EXISTS `address` (

`addressid` int(11) NOT NULL AUTO_INCREMENT,

`firstline` varchar(45) DEFAULT NULL,

`secondline` varchar(45) DEFAULT NULL,

`city` varchar(45) DEFAULT NULL,

`state` varchar(45) DEFAULT NULL,

`country` varchar(45) DEFAULT NULL,

PRIMARY KEY (`addressid`)

) ENGINE=InnoDB DEFAULT CHARSET=latin1;

CREATE TABLE IF NOT EXISTS `user` (

`userid` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'This contains the userid of the user',

`fname` varchar(128) NOT NULL COMMENT 'This contains the first name of the user',

`lname` varchar(128) DEFAULT NULL COMMENT 'This contains the last name of the user',

`sex` varchar(1) NOT NULL COMMENT 'This contains the sex of the user',

`age` int(100) DEFAULT NULL COMMENT 'This contains the age of the user',

`creation_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'This contains the user account creation',

`user_addressid` int(11) DEFAULT NULL,

PRIMARY KEY (`userid`),

KEY `fk_user_address` (`user_addressid`)

) ENGINE=InnoDB DEFAULT CHARSET=latin1;

ALTER TABLE `user` ADD CONSTRAINT `fk_user_address` FOREIGN KEY

(`user_addressid`) REFERENCES `address` (`addressid`) ON DELETE NO ACTION ON

UPDATE NO ACTION;

-------

I had to create corresponding models for the tables. After the database configuration, I ran the following commands to create the models in my app.

cd app_demo\myapp

D:\workspace\app_demo\myapp>protected\yiic shell index.php

This opened the Yii Interactive Tool.

Yii Interactive Tool v1.1 (based on Yii v1.1.2)

Please type 'help' for help. Type 'exit' to quit.

>> model *

The following model classes (tables) match your criteria:

1. Address (address)

2. User (user)

Do you want to generate the above classes? [Yes|No] yes

generate models/Address.php

generate fixtures/address.php

generate unit/AddressTest.php

generate models/User.php

generate fixtures/user.php

generate unit/UserTest.php

The following model classes are successfully generated:

Address, User

If you have a 'db' database connection, you can test these models now with:

$model=User::model()->find();

print_r($model);

-------

After this, I edited the actionIndex function in SiteController.php to look like this:

public function actionIndex()

{

// renders the view file 'protected/views/site/index.php'

// using the default layout 'protected/views/layouts/main.php'

$this->render('index', array(

'user' => User::model()->findAll()

));

}

In the index.php, I added the following table to display the records from the database:

foreach($user AS $us):?> 

 echo $us->fname;?> 

echo $us->lname;?> 

echo $us->age;?> 

echo $us->user_address->city;?> 

 endforeach;?>

Then I pointed my browser to http://localhost/myapp/index.php and this generated a table with the results.

More on this later...

Spidvid: Let's create better videos!

The start-up scene is always sprawling with action. Individuals keep coming up with brilliant ideas to tackle a problem at hand. If you ever heard of Youtube, you can relate to the fact, that the online video sharing platform is filled with low quality videos. Enter the new service round the corner - Spidvid. It is a platform which brings together video creators and professionals, who would ultimately produce quality entertaining videos. It doesn't end here; they go further, distribute the videos and generate compensation for their contribution to the effort.

It is an inventive concept accented by a beautiful user interface. It is easy to use and very responsive. You can choose any role you want on the video creation life cycle. You may be a video creator, a professional who can act, write, edit or even an amateur who has video ideas which, you would like to be created by others. This way you get in touch with people with similar interests and learn from their experiences.

Spidvid will raise the bar for online videos. It brings together people from different corners of the planet, with a common goal of producing quality videos. Right now, Spidvid is offered as a beta service. As more and more users join the bandwagon, the talent pool will get richer, as you have more contributors.

Getting content onto the website is a task which the marketing team needs to pay attention to. I would like to have a section to see some clips of the videos created on Spidvid.

Overall, it is a good beginning to a new frontier in video collaboration. Spidvid has given users the tools which they need to connect and develop videos outside a single fortified studio. By making it open to public, the content will cross the geographical boundaries and be enjoyed by a wide range of viewers.

So, let's get started and create some videos!

Windows - the Seventh

“And the SEVENTH angel poured out his vial into the air; and there came a great voice out of the temple of heaven, from the throne saying, It is DONE.” (Revelation 16 : 17)

The number 'Seven' has a lot of significance. There are seven days in a week and it was on the seventh day that the world was completed. 'Seven' symbolizes completeness or perfection. Perfection is only divine. At an earthly level, I would attribute seven to refinement. In that respect, Microsoft has achieved elegance with W7. A few things I like about W7 are:
1. The annoying UAC has become much more placid. Introduced in Vista, as a security measure, it turned out to irritate the users with its frequent pop-ups.
2. The boot performance is significantly faster. Windows 7 should have achieved a 15-sec boot up by the time it is up for final release.
3. W7 lets the users connect to wireless networks instantly with fewer clicks which will prove to be of lot of help, for users without much technical know-how.
4. The footprint of the OS is much lighter compared to Vista. It installs faster and occupies lesser space on the hard disk. Some of additional software like Windows Mail have been made optional downloads instead of default installations.
5. The new OS uses lesser battery power and is known to save energy by 20%. That is a very good news for netbook makers as, Windows 7 will prove to be the OS for netbooks, too.
6. There are some cool mouse gestures included: Aero Peek and Aero Shake. Aero Peek makes all the open windows transparent and lets the users peek through their desktop revealing hidden gadgets and icons while, Aero Shake lets the users minimize unnecessary windows from the desktop by shaking the required window.
7. The OS supports backward compatibility. Legacy applications can run in a Windows XP mode, inside of Window 7.

And, something I don't like about W7:
1. Microsoft failed to come up with a suitable strategy for integrating the cloud. With computing being the talk of the town, and Microsoft itself working on Azure, the Software as a Service(Saas) integration seems to have been lost in translation. Microsoft could have amalgamated its Live Mesh with Windows 7 for data backup.
2. W7 still lives on to the concept of one partition OS. I would have liked this to change. The data like Documents, Music, Pictures and other user settings should reside on a different partition than the OS itself. This would be of great value in case of a system crash or an OS reinstall.

Overall, W7 is a nice compact mixture of niceties and removes some of the nuisances of Vista.