Sunday, 5 March 2017



Hadoop includes three main Components
  • HDFS
  • MapReduce
  • YARN
HDFS:
  • HDFS means Hadoop Distributed File System and it manages big data sets with high volume.
  • HDFS stores the data at distributed manner and it is the primary storage system
  • HDFS allows read and write the files but cannot updated the files in HDFS.
  • When we move file in HDFS that file are automatically split into small files and that small files are replicated of three different servers.
  • HDFS are implemented by Master Slave architecture.Master means namenode and Slave means datande.
NameNode:
  • NameNode is the heart and master oh Hadoop
  • It maintains the namespace system of hadoop
  • NameNode stores the metadata of data blocks that data are permanently stored on Local disk
  • It reduced disk space also.
  • There are two types of NameNode(active and standby)
Secondary Node:
  • Main role of secondary namenode is copy and merge the namespace.
  • Secondary namenode requires huge amount of memory to merge the files.
  • If namenode failure namespace images are stored in secondary namenode and it can be restart the namenode.
DataNode:
  • DataNode also known as slaves.
  • It can be used to actual storage.
  • Work of Datanode are based on NameNode instructions only
MapReduce:
1.Mapper Task:
  • It takes the one input and divide into small parts and distribute to another node.
  • To solve all small programs and send the results to master node.
2.Reducer Task:
  • It combines the all master node results and arranging the results at some formats.
YARN:
  • YARN means Yet Another Resource Negotiator.
  • YARN is the resource management responsible for managing resources in cluster and scheduling applications.
  • It also Known as MapReduce2
  • Responsible of YARN is managing and monitoring the work loads.
  • Yarn Having following two components
               1.Resource Manager(NameNode)
               2.Node Manager(DataNode)

7 comments:

  1. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    python online training
    python training in OMR
    python training in tambaram
    python training in annanagar

    ReplyDelete
  2. A universal message I suppose, not giving up is the formula for success I think. Some things take longer than others to accomplish, so people must understand that they should have their eyes on the goal, and that should keep them motivated to see it out til the end.

    Blueprism training in tambaram

    Blueprism training in annanagar

    ReplyDelete
  3. Thank you so much for a well written, easy to understand article on this. It can get really confusing when trying to explain it – but you did a great job. Thank you!


    angularjs Training in chennai
    angularjs-Training in pune

    angularjs-Training in chennai

    angularjs Training in chennai

    angularjs-Training in tambaram

    ReplyDelete