Integrating LVM with Hadoop and providing Elasticity to DataNode Storage

Task Description :

Prerequisite :

  • Basic knowledge of Hadoop
  • Knowledge of Linux Partition and LVM

Hadoop :

Logical Volume Management :

Step 1: Start the NameNode Service

hadoop-daemon.sh start namenode

hadoop dfsadmin -report

Step 2: Add Hard Disk to the DataNode

fdisk -l

Step 3: Create Physical Volume from that hard disk.

pvcreate /dev/sdb

pvdisplay /dev/sdb

Step 4: Create the Volume Group

vgcreate dnvg /dev/sdb

vgdisplay dnvg

Step 5: Create Logical Volume of Size 30GiB

lvcreate — size 30G — name dnlv dnvg

lvdisplay dnvg/dnlv

Step 6: Format the Logical Volume

mkfs.ext4 /dev/dnvg/dnlv

Step 7: Mount the Logical Volume with the DataNode directory

mount /dev/dnvg/dnlv /dn

df -h

Step 8: Start Datanode service

hadoop-daemon.sh start datanode

hadoop dfsadmin -report

Step 9: Increate the Logical Volume Size

lvextend — size +10G /dev/dnvg/dnlv

Step 10: Format the extended Logical Volume

resize2fs /dev/dnvg/dnlv

df -h

Thank You !!

Hope you like it !!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aditya Raj

Aditya Raj

I'm passionate learner diving into the concepts of computing 💻