Infolab wiki
Hadoop
Server |
OS |
Memory [GB] |
CPU Type |
Arch |
CPU Speed |
CPUs |
Cores |
Threads |
Local Storage [TB] |
Login |
Description |
ilhadoop |
CentOS 6.6 |
64 |
Opteron 6276 |
64 bit |
2.3 |
2 |
32 |
32 |
2 |
CS |
JobTracker, NameNode |
ilh01-ilh40 |
CentOS 6.9 |
64 |
Opteron 6320 |
64 bit |
2.8 |
1 |
8 |
8 |
8 |
N/A |
TaskTracker, DataNode |
Before updating notes:
Java version
adrijan@ilh02:~$ java -version java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
Download: http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html
File attached (registration needed to download older versions of java) jre-8u45-linux-x64.tar.gz
Ubuntu 16.04.3:
- update-alternatives --install /usr/bin/java java /opt/java/jre1.8.0_45/bin/java 1
- update-alternatives --set java /opt/java/jre1.8.0_45/bin/java
What to take from old machines:
- UID and GID of hdfs
- fstab
- hadoop configuration files
#START DATANODE sudo -H -u hdfs bash -c 'export JAVA_HOME=/opt/java; cd /opt/hadoop-2.6.0-cdh5.4.2 ; /opt/hadoop-2.6.0-cdh5.4.2/sbin/hadoop-daemon.sh --config /opt/hadoop-2.6.0-cdh5.4.2/etc/hadoop --script /opt/hadoop/sbin/hdfs start datanode' #START YARN sudo -H -u yarn bash -c 'export JAVA_HOME=/opt/java; cd /opt/hadoop-2.6.0-cdh5.4.2 ; /opt/hadoop/sbin/yarn-daemon.sh --config /opt/hadoop-2.6.0-cdh5.4.2/etc/hadoop start nodemanager' #STOP DATANODE sudo -H -u hdfs bash -c 'export JAVA_HOME=/opt/java; cd /opt/hadoop-2.6.0-cdh5.4.2 ; /opt/hadoop-2.6.0-cdh5.4.2/sbin/hadoop-daemon.sh --config /opt/hadoop-2.6.0-cdh5.4.2/etc/hadoop --script /opt/hadoop/sbin/hdfs stop datanode' #STOP YARN sudo -H -u yarn bash -c 'export JAVA_HOME=/opt/java; cd /opt/hadoop-2.6.0-cdh5.4.2 ; /opt/hadoop/sbin/yarn-daemon.sh --config /opt/hadoop-2.6.0-cdh5.4.2/etc/hadoop stop nodemanager'