Live On-Line Training:
Scalable Data Pipelines with Hadoop, Spark, and Kafka

User Tools

Site Tools

Trace:
linux_hadoop_minimal_installation_instructions_version_2

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
linux_hadoop_minimal_installation_instructions_version_2 [2024/01/25 13:43]
deadline [Installation Steps for Intel-x86_64 Based Hosts (with Videos)] update LHM links 		linux_hadoop_minimal_installation_instructions_version_2 [2024/01/30 14:07] (current)
deadline [Issues/Bugs]
Line 7: Line 7:
  
 === Intel x86_64 with VirtualBox=== === Intel x86_64 with VirtualBox===
-   * Version: 2.0-beta8\\ +   * Version: 2.0-8.1\\ 
-   * Release Date: August 9, 2022+   * Release Date: 25-Jan-2024
  
 === Apple M1, M2 with UTM === === Apple M1, M2 with UTM ===
-  * Version: 2.0-beta8-M1-R2 +  * Version: 2.0-M8.1 
-  * Release Date: February 20, 2023+  * Release Date: 25-Jan-2024
  
 === Issues with Either version of the LHM=== === Issues with Either version of the LHM===
Line 74: Line 74:
 **IMPORTANT** The LHM should be stopped when not in use. The running services should be stopped in a ''graceful manner'' (powered down) as frequent standby or a sudden power interruption can leave some services in a broken state. To stop the LHM, it is suggested the you login as the root user (see [[linux_hadoop_minimal_installation_instructions_version_2#stopping_the_vm|Stopping the VM]] ) and issue the ''poweroff'' command. This will ensure a safe and orderly shutdown of the machine. **IMPORTANT** The LHM should be stopped when not in use. The running services should be stopped in a ''graceful manner'' (powered down) as frequent standby or a sudden power interruption can leave some services in a broken state. To stop the LHM, it is suggested the you login as the root user (see [[linux_hadoop_minimal_installation_instructions_version_2#stopping_the_vm|Stopping the VM]] ) and issue the ''poweroff'' command. This will ensure a safe and orderly shutdown of the machine.
  
-====Installation Steps for Apple-M1 Based Hosts ====+====Installation Steps for Apple-M Based Hosts ====
  
-**20-Feb-2023 VERSION-UPDATED**+**25-Jan-2024 VERSION-UPDATED**
  
-There is a partial LHM for the Apple M1 based machines. The following (aarch64) packages are installed: CentOS Linux 7.6, Python3 3.6.8, R 3.6.0, Hadoop 3.3.0, Hive 3.1.2, Apache Spark 2.4.5, Derby 10.14.2.0, Zeppelin 0.8.2, Sqoop 1.4.7, Kafka 2.5.0, HBase 2.4.10, NiFi 1.17.0 (KafkaEsque is not installed)+There is a full LHM for the Apple M based machines. The following (aarch64) packages are installed: CentOS Linux 7.6, Python3 3.6.8, R 3.6.0, Hadoop 3.3.0, Hive 3.1.2, Apache Spark 2.4.5, Derby 10.14.2.0, Zeppelin 0.8.2, Sqoop 1.4.7, Kafka 2.5.0, HBase 2.4.10, NiFi 1.17.0 (KafkaEsque is not installed)
  
 To run the VM on an Apple-M1 based machine, Perform the following steps:  To run the VM on an Apple-M1 based machine, Perform the following steps: 
Line 85: Line 85:
  
  
-**Step 2:** Download the LHM for Apple-M1 into your ''Downloads'' folder. The current version is [[http://161.35.229.207/download/Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm.zip|US]] [[http://134.209.239.225/download/Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm.zip|Europe]] (11G) The [[https://www.clustermonkey.net/download/Hands-on_Hadoop_Spark/Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.MD5.txt| MD5 file ]] can be used to verify the integrity of the download. **NOTE:** Google Chrome may prevent http downloads, right click the link, choose “Save Link As” then click “Keep” next to the blue discard box at the bottom of the browser.+**Step 2:** Download the LHM for Apple-M1 into your ''Downloads'' folder. The current version is [[http://161.35.229.207/download/Linux-Hadoop-Minimal-V2.0-M8.1.utm.zip|US]] [[http://134.209.239.225/download/Linux-Hadoop-Minimal-V2.0-M8.1.utm.zip|Europe]] (9G) The [[https://www.clustermonkey.net/download/Hands-on_Hadoop_Spark/Linux-Hadoop-Minimal-V2.0-M8.1.utm.zip.MD5.txt| MD5 file ]] can be used to verify the integrity of the download. **NOTE:** Google Chrome may prevent http downloads, right click the link, choose “Save Link As” then click “Keep” next to the blue discard box at the bottom of the browser.
  
-**Step 3:** The downloaded LHM is a zip file. Click to extract the file. It should create a directory called ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm''+**Step 3:** The downloaded LHM is a zip file. Click to extract the file. It should create a directory called ''Linux-Hadoop-Minimal-V2.0-M8.1.utm''
  
-**Step 4:** Start UTM (it will be in the applications folder, suggest keeping it in your dock) Click the ''+'' to the left of the ''UTM'' at the top. A window called ''Start'' will open. Toward the bottom click ''Open'' under ''Existing''. Next, navigate to the ''Downloads'' folder and click on ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm'' The LHM should now be listed in the left hand column in the UTM window.+**Step 4:** Start UTM (it will be in the applications folder, suggest keeping it in your dock) Click the ''+'' to the left of the ''UTM'' at the top. A window called ''Start'' will open. Toward the bottom click ''Open'' under ''Existing''. Next, navigate to the ''Downloads'' folder and click on ''Linux-Hadoop-Minimal-V2.0-M8.1.utm'' The LHM should now be listed in the left hand column in the UTM window.
  
 **Step 5:** Start the LHM by clicking on the big arrow in the middle of the window. (If it is not highlighted, click on the ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2'' on the left) The LHM should start. A new "terminal" window will open and show the VM boot-up sequence. When finished, a prompt will be displayed in this window (''localhost login:'') Minimize this terminal window. We will log into the LHM using the Mac terminal program.  **Step 5:** Start the LHM by clicking on the big arrow in the middle of the window. (If it is not highlighted, click on the ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2'' on the left) The LHM should start. A new "terminal" window will open and show the VM boot-up sequence. When finished, a prompt will be displayed in this window (''localhost login:'') Minimize this terminal window. We will log into the LHM using the Mac terminal program. 
Line 393: Line 393:
 3. If the **time on the LHM falls out of sync** with the host due to hibernation, the following commands can be run to reset the the ntpd time daemon (run as root). **NOTE:** the host must have Internet access. 3. If the **time on the LHM falls out of sync** with the host due to hibernation, the following commands can be run to reset the the ntpd time daemon (run as root). **NOTE:** the host must have Internet access.
  
-    systemctl -l start ntpd+    systemctl -l stop ntpd
     ntpdate -u pool.ntp.org     ntpdate -u pool.ntp.org
     systemctl -l start ntpd     systemctl -l start ntpd
linux_hadoop_minimal_installation_instructions_version_2.1706190200.txt.gz · Last modified: 2024/01/25 13:43 by deadline

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
CC Attribution-Noncommercial-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki