Node details:
Node1 – 10.10.YY.X1
Node2 - 10.10.YY.X2
Node3 - 10.10.YY.X3
The basic packages required for ganglia to run are:
[1] ganglia-gmond – required on all nodes from where metrics
needs to be collected
[2] ganglia-gmetad – required on node which will perform aggregation
(typically on one node)
[3] ganglia-web – required on the node running the Ganglia
web UI
Following is my setup: (Concentrating on Ganglia and HBase)
Node1 – 10.10.YY.X1 – Hbase regionserver, gmond
Node2 - 10.10.YY.X3 – Hbase regionserver, gmond
Node3 - 10.10.YY.X2 – Hbase regionserver, Hbase master,
gmond, gemetad, ganglia-web
For installing the packages in CentOS, use the following command:
yum install ganglia-gmond -y
yum install ganglia-gmetad -y
yum install ganglia-web -y
Edit ‘/etc/ganglia/gmond.conf’ on all nodes running the
gmond service (sample is shown below)
Part 1:
cluster {
name =
"ThreeNodeClusterAJames"
owner =
"unspecified"
latlong =
"unspecified"
url =
"unspecified"
}
Part 2:
udp_send_channel {
bind_hostname = yes
# Highly recommended, soon to be default.
# This option tells gmond to use a source address
# that resolves to the machine's hostname. Without
# this, the metrics may appear to come from any
# interface and the DNS names associated with
# those IPs will be used to create the RRDs.
#mcast_join =
239.2.11.71
host = 10.10.YY.XX
port = 8649
ttl = 1
}
NOTE:
·
We are using unicast protocol instead of
multicast protocol. Please comment ‘mcast_join = 239.2.11.71’ and add “host = <IP
of localhost running the gmond service>”
·
If there is any change in the port used, please
change the same in ‘port’ property. Here I am using the default port which is
8649.
Part 3:
udp_recv_channel {
#mcast_join =
239.2.11.71
port = 8649
#bind = 239.2.11.71
#bind = 10.10.72.154
#retry_bind = true
# Size of the UDP
buffer. If you are handling lots of metrics you really
# should bump it up
to e.g. 10MB or even higher.
# buffer = 10485760
}
Edit ‘/etc/ganglia/gmetad.conf’ in gmetad server and add the
following:
data_source "ThreeNodeClusterAJames" 10.10.YY.X1:8649
10.10.YY.X2:8649 10.10.YY.X3:8649
The cluster name ‘ThreeNodeClusterAJames’ should be same in
all the gmond servers and gmetad server.
Edit ‘/opt/mapr/conf/hadoop-metrics.properties’ file and add
Part 1:
# Configuration of the "cldb" context for ganglia
cldb.class=com.mapr.fs.cldb.counters.MapRGangliaContext31
cldb.period=10
cldb.servers=10.10.YY.X1:8649,10.10.YY.X2:8649,10.10.YY.X3:8649
cldb.spoof=1
Part 2:
# Configuration of the "fileserver" context for
ganglia
fileserver.class=com.mapr.fs.cldb.counters.MapRGangliaContext31
fileserver.period=37
fileserver.servers=10.10.YY.X1:8649,10.10.YY.X2:8649,10.10.YY.X3:8649
fileserver.spoof=1
Execute the following in any of the node and restart cldb
service on all cldb nodes:
maprcli config save -values
{"cldb.ganglia.cldb.metrics":"1"}
maprcli config save -values
{"cldb.ganglia.fileserver.metrics":"1"}
Execute the following in all nodes
sudo setenforce 0
Execute the following gmetad server:
chown -R apache:apache /usr/share/ganglia
chown -R ganglia:ganglia /var/lib/ganglia/rrd*
chcon -R -t httpd_sys_content_t *
Edit ‘/etc/httpd/conf.d/ganglia.conf’ file:
<Location /ganglia>
Order deny,allow
# Deny from all
Allow from 127.0.0.1
Allow from ::1
# Allow from
.example.com
</Location>
Restart the httpd, gmond and gmetad:
/etc/init.d/httpd restart
/etc/init.d/gmond restart
/etc/init.d/httpd restart
Edit ‘/opt/mapr/hbase/hbase-<version>/conf/hadoop-metrics2-hbase.properties’
file
hbase.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
hbase.sink.ganglia.servers=<ganglia-server>:8649
hbase.sink.ganglia.period=10
hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
hbase.period=10
hbase.servers=<server-running-hbase-regionserver-1>:8649,
<server-running-hbase-regionserver-2>:8649
# Configuration of the "jvm" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
jvm.period=10
jvm.servers=<server-running-hbase-regionserver-1>:8649,
<server-running-hbase-regionserver-2>:8649
...
# Configuration of the "rpc" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
# rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
rpc.period=10
rpc.servers=<server-running-hbase-regionserver-1>:8649,
<server-running-hbase-regionserver-2>:8649
...
# Configuration of the "rest" context for ganglia
# Pick one: Ganglia 3.0 (former) or Ganglia 3.1 (latter)
#
rest.class=org.apache.hadoop.metrics.ganglia.GangliaContext
rest.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
rest.period=10
rest.servers=<server-running-hbase-regionserver-1>:8649,
<server-running-hbase-regionserver-2>:8649
Restart Hbase master and regionserver in all nodes.
You should be able to see the hbase metrics in ganglia web UI.
Ganglia UI listing the metrics collected:
Ganglia UI showing HBase metrics:
