= Hadoop Cluster Status =

##{{{#!wiki warning
##'''Offline'''
##
##The cluster is currently not operational. See notes below.
##}}}

##{{{#!wiki note
##'''Monitoring'''
##
##Ganglia monitoring is currently having some issues.
##}}}

==== Cluster status ====

|| 2015-10-15 20:01:00 PDT || Cluster back online. ||
|| 2015-10-15 19:40:00 PDT || Cluster issues due to a DNS configuration failure. ||
|| 2015-07-06 11:20:00 PDT || Cluster is online, Yarn back on good behavior. ||
|| 2015-07-05 12:15:00 PDT || Jobs stuck in Yarn queue. ||
|| 2015-06-12 15:15:00 PDT || Cluster is online. ||
|| 2015-06-11 16:30:00 PDT || Cluster is offline for upgrade to CDH 5.4.2 and Spark. ||
|| 2015-05-11 15:40:00 PDT || Cluster is online. Problem fixed. ||
|| 2015-05-11 07:05:00 PDT || HDFS and local FS full. Jobs are not running. ||
|| 2015-05-05 15:40:00 PDT || Cluster is online. Problem fixed. ||
|| 2015-05-05 10:05:00 PDT || Cluster is online but jobs are not getting processed. Looking into the issue. ||
|| 2015-04-30 11:40:00 PDT || Cluster is online. Upgraded to CDH 5.4.0 and JDK 8u45. ||
|| 2015-04-29 16:30:00 PDT || Cluster is offline due to package upgrades (moving to CDH 5.4.0). ||
|| 2015-04-09 06:42:00 PDT || Cluster is online. ||
|| 2015-04-08 06:00:00 PDT || Cluster is offline due to NameNode issues. Investigating + trying to repair the problem. ||
|| 2014-12-05 22:32:00 PST || Cluster is online. ||
|| 2014-12-04 21:05:00 PST || Maintenance in progress. Installing the secondary namenode, Spark, etc. ||
|| 2014-12-04 06:28:00 PST || Ganglia back to normal. Cluster is online. ||
|| 2014-12-04 05:40:00 PST || Ganglia reporting is currently having issues. ||
|| 2014-12-04 05:18:00 PST || Cluster is back online. ||
|| 2014-12-03 19:15:00 PST || Cluster offline for system updates. Should be back online by --(22:00 PST)-- 2014-12-04 02:00 PST. ||

==== Cluster usage statistics ====

|| '''CPU load''' <<BR>> [%] || '''RAM used''' <<BR>> [bytes] ||
|| <<Ganglia(hadoop_cpu,ilHadoop,)>> || <<Ganglia(hadoop_mem,ilHadoop,)>> ||
|| '''NETWORK traffic''' <<Color2(IN,green,fontWeight=bold)>> <<Color2(OUT,blue,fontWeight=bold)>> <<BR>> [bytes/s] || '''DISK read''' <<BR>> [bytes/s]  || '''DISK write''' <<BR>> [bytes/s] ||
|| <<Ganglia(hadoop_network,ilHadoop,)>> || <<Ganglia(hadoop_disk,ilHadoop,,read,sda*sdb*sdc*sdd)>> || <<Ganglia(hadoop_disk,ilHadoop,,write,sda*sdb*sdc*sdd)>> ||

==== References ====

 * [[ilHadoopStats|Hadoop Cluster Statistics]]
 * [[http://ilHadoop1.stanford.edu:50070/dfshealth.html|NameNode information]]
 * [[http://ilHadoop1.stanford.edu:8088/cluster|Application tracker]]