Arth Task

3 min readMay 19, 2021

Task Description:
Task 7.1: Elasticity Task
🔅Integrating LVM with Hadoop and
providing Elasticity to DataNode Storage
🔅Increase or Decrease the Size of Static
Partition in Linux.
🔅Automating LVM Partition using Python-Script.

Explanation:

We have to create HDFS Cluster and we have to provide LVM storage to our datanodes so that we can increase or decrease the Storage Size that is provided by datanode to namenode on the fly.

Solution:

This practical is performed on the top of RHEL 8 VM.

First of all, attach two hard-Disks to your V.M.

Now , go to the terminal of V.M. and run “fdisk -l” command , these two hard-Disks will appear.

Now, we have to convert these two Hard-Disks into PV(physical volume) and then we have to create a volume group, where we will add these two physical volumes that we created.

Now, we can create a logical volume from the volume group that we created.

Now, we have to format the logical volume.

Now, we have to create HDFS Cluster and then we have to mount this logical volume to the folder which datanode provide to namenode.

We have configure HDFS Cluster manually or we can use ansible-playbook to configure the HDFS Cluster.

Github Repo link: https://github.com/Nishant27Saini/Arth-Tasks.git, go to this link and get the ansible-playbook placed inside Task 11, to configure HDFS Cluster using ansible-playbook.

After configuring the HDFS cluster and mounting the logical volume to the folder that datanode will share to namenode, go to the WebUI of HDFS Cluster.

As datanode provided storage to namenode using Logical Volume, so we can increase or decrease the volume size that is provided by datanode to namenode.

Lets, increase the volume size that is provided by datanode to namenode, after increasing the size of logical volume, we have to format it also.