Live On-Line Training:
Scalable Data Pipelines with Hadoop, Spark, and Kafka

User Tools

Site Tools

Trace:
linux_hadoop_minimal_installation_instructions_version_2

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Last revision Both sides next revision
linux_hadoop_minimal_installation_instructions_version_2 [2024/01/05 21:37]
deadline [Linux Hadoop Minimal VM Notes VERSION 2] 		linux_hadoop_minimal_installation_instructions_version_2 [2024/01/30 13:41]
deadline [Installation Steps for Apple-M Based Hosts]
Line 7: Line 7:
  
 === Intel x86_64 with VirtualBox=== === Intel x86_64 with VirtualBox===
-   * Version: 2.0-beta8\\ +   * Version: 2.0-8.1\\ 
-   * Release Date: August 9, 2022+   * Release Date: 25-Jan-2024
  
 === Apple M1, M2 with UTM === === Apple M1, M2 with UTM ===
-  * Version: 2.0-beta8-M1-R2 +  * Version: 2.0-M8.1 
-  * Release Date: February 20, 2023+  * Release Date: 25-Jan-2024
  
 === Issues with Either version of the LHM=== === Issues with Either version of the LHM===
Line 66: Line 66:
 **Step 3a:** On MacIntosh Systems with //Big Sur//, you may get the ''Kernel driver not installed (rc=-1908)'' error. The error is do to new security levels in MacOS. See this [[ https://www.howtogeek.com/658047/how-to-fix-virtualboxs-%E2%80%9Ckernel-driver-not-installed-rc-1908-error/|page]] for a fix.  **Step 3a:** On MacIntosh Systems with //Big Sur//, you may get the ''Kernel driver not installed (rc=-1908)'' error. The error is do to new security levels in MacOS. See this [[ https://www.howtogeek.com/658047/how-to-fix-virtualboxs-%E2%80%9Ckernel-driver-not-installed-rc-1908-error/|page]] for a fix. 
  
-**Step 4:** Download the current Linux-Hadoop-Minimal-V2.0-xxxxx.ova image [[http://161.35.229.207/download/Linux-Hadoop-Minimal-V2.0-beta8.ova|US]] [[http://134.209.239.225/download/Linux-Hadoop-Minimal-V2.0-beta8.ova|Europe]] and load it into VirtualBox. +**Step 4:** Download the current Linux-Hadoop-Minimal-V2.0-xxxxx.ova image [[http://161.35.229.207/download/Linux-Hadoop-Minimal-V2.0-8.1.ova|US]] [[http://134.209.239.225/download/Linux-Hadoop-Minimal-V2.0-8.1.ova|Europe]] and load it into VirtualBox. 
  
 **Step 5:** Start the VM. All the essential Hadoop service should be started automatically. **Step 5:** Start the VM. All the essential Hadoop service should be started automatically.
Line 74: Line 74:
 **IMPORTANT** The LHM should be stopped when not in use. The running services should be stopped in a ''graceful manner'' (powered down) as frequent standby or a sudden power interruption can leave some services in a broken state. To stop the LHM, it is suggested the you login as the root user (see [[linux_hadoop_minimal_installation_instructions_version_2#stopping_the_vm|Stopping the VM]] ) and issue the ''poweroff'' command. This will ensure a safe and orderly shutdown of the machine. **IMPORTANT** The LHM should be stopped when not in use. The running services should be stopped in a ''graceful manner'' (powered down) as frequent standby or a sudden power interruption can leave some services in a broken state. To stop the LHM, it is suggested the you login as the root user (see [[linux_hadoop_minimal_installation_instructions_version_2#stopping_the_vm|Stopping the VM]] ) and issue the ''poweroff'' command. This will ensure a safe and orderly shutdown of the machine.
  
-====Installation Steps for Apple-M1 Based Hosts ====+====Installation Steps for Apple-M Based Hosts ====
  
-**20-Feb-2023 VERSION-UPDATED**+**25-Jan-2024 VERSION-UPDATED**
  
-There is a partial LHM for the Apple M1 based machines. The following (aarch64) packages are installed: CentOS Linux 7.6, Python3 3.6.8, R 3.6.0, Hadoop 3.3.0, Hive 3.1.2, Apache Spark 2.4.5, Derby 10.14.2.0, Zeppelin 0.8.2, Sqoop 1.4.7, Kafka 2.5.0, HBase 2.4.10, NiFi 1.17.0 (KafkaEsque is not installed)+There is a full LHM for the Apple M based machines. The following (aarch64) packages are installed: CentOS Linux 7.6, Python3 3.6.8, R 3.6.0, Hadoop 3.3.0, Hive 3.1.2, Apache Spark 2.4.5, Derby 10.14.2.0, Zeppelin 0.8.2, Sqoop 1.4.7, Kafka 2.5.0, HBase 2.4.10, NiFi 1.17.0 (KafkaEsque is not installed)
  
 To run the VM on an Apple-M1 based machine, Perform the following steps:  To run the VM on an Apple-M1 based machine, Perform the following steps: 
Line 85: Line 85:
  
  
-**Step 2:** Download the LHM for Apple-M1 into your ''Downloads'' folder. The current version is [[http://161.35.229.207/download/Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm.zip|US]] [[http://134.209.239.225/download/Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm.zip|Europe]] (11G) The [[https://www.clustermonkey.net/download/Hands-on_Hadoop_Spark/Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.MD5.txt| MD5 file ]] can be used to verify the integrity of the download. **NOTE:** Google Chrome may prevent http downloads, right click the link, choose “Save Link As” then click “Keep” next to the blue discard box at the bottom of the browser.+**Step 2:** Download the LHM for Apple-M1 into your ''Downloads'' folder. The current version is [[http://161.35.229.207/download/Linux-Hadoop-Minimal-V2.0-M8.1.utm.zip|US]] [[http://134.209.239.225/download/Linux-Hadoop-Minimal-V2.0-M8.1.utm.zip|Europe]] (9G) The [[https://www.clustermonkey.net/download/Hands-on_Hadoop_Spark/Linux-Hadoop-Minimal-V2.0-M8.1.utm.zip.MD5.txt| MD5 file ]] can be used to verify the integrity of the download. **NOTE:** Google Chrome may prevent http downloads, right click the link, choose “Save Link As” then click “Keep” next to the blue discard box at the bottom of the browser.
  
-**Step 3:** The downloaded LHM is a zip file. Click to extract the file. It should create a directory called ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm''+**Step 3:** The downloaded LHM is a zip file. Click to extract the file. It should create a directory called ''Linux-Hadoop-Minimal-V2.0-M8.1.utm''
  
-**Step 4:** Start UTM (it will be in the applications folder, suggest keeping it in your dock) Click the ''+'' to the left of the ''UTM'' at the top. A window called ''Start'' will open. Toward the bottom click ''Open'' under ''Existing''. Next, navigate to the ''Downloads'' folder and click on ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2.utm'' The LHM should now be listed in the left hand column in the UTM window.+**Step 4:** Start UTM (it will be in the applications folder, suggest keeping it in your dock) Click the ''+'' to the left of the ''UTM'' at the top. A window called ''Start'' will open. Toward the bottom click ''Open'' under ''Existing''. Next, navigate to the ''Downloads'' folder and click on ''Linux-Hadoop-Minimal-V2.0-M8.1.utm'' The LHM should now be listed in the left hand column in the UTM window.
  
 **Step 5:** Start the LHM by clicking on the big arrow in the middle of the window. (If it is not highlighted, click on the ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2'' on the left) The LHM should start. A new "terminal" window will open and show the VM boot-up sequence. When finished, a prompt will be displayed in this window (''localhost login:'') Minimize this terminal window. We will log into the LHM using the Mac terminal program.  **Step 5:** Start the LHM by clicking on the big arrow in the middle of the window. (If it is not highlighted, click on the ''Linux-Hadoop-Minimal-V2.0-beta8-M1-R2'' on the left) The LHM should start. A new "terminal" window will open and show the VM boot-up sequence. When finished, a prompt will be displayed in this window (''localhost login:'') Minimize this terminal window. We will log into the LHM using the Mac terminal program. 
linux_hadoop_minimal_installation_instructions_version_2.txt · Last modified: 2024/01/30 14:07 by deadline

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
CC Attribution-Noncommercial-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki