文档章节

what is Edge Note of MapR

我是彩笔
 我是彩笔
发布于 2014/11/07 09:41
字数 429
阅读 44
收藏 0

In my opinion, edge nodes in a Hadoop cluster are typically nodes that are responsible for running the client-side operations of a Hadoop cluster. Typically edge-nodes are kept separate from the nodes that contain Hadoop services such as HDFS, MapReduce, etc, mainly to keep computing resources separate. For smaller clusters only having a few nodes, it's common to see nodes playing a hybrid combination of roles for master services (JT, NN, etc.) , slave services (TT, DN, etc) and gateway services.

Note that running master and slave Hadoop services on the same node is not an ideal setup, and can cause scaling and resource issues depending on what's at use. This kind of configuration is typically seen on a small-scale dev environment.

With that said, here's some answers to your questions posted:

1) Does the edge node have to be part of the cluster The edge node does not have to be part of the cluster, however if it is outside of the cluster (meaning it doesn't have any specific Hadoop service roles running on it), it will need some basic pieces such as Hadoop binaries and current Hadoop cluster config files to submit jobs on the cluster.

2) What advantages do we have if it is inside the cluster? Depending on which distribution is in use, edge nodes run within the cluster allow for centralized management of all the Hadoop configuration entries on the cluster nodes which helps to reduce the amount of administration needed to update the config files. Usually this is a one-to-many approach, where config entries are updated in one location and are pushed out to all (many) nodes in the cluster.

However, when one of the nodes within the cluster is also used as an edge node, there are CPU and memory resources that are consumed by the client operations which detracts the available resources that could be utilized by the running Hadoop services in that node.

3) Does it store any blocks of data in hdfs? Unless the edge node is configured with a DataNode service, blocks of data will not be stored on that node.

4) Should the edge node be outside the cluster? As mentioned above, it can be dependent on the cluster environment and use-case; One of the supporting reasons to configure it outside of the cluster is to keep the client-running and Hadoop services separated.

Keeping an edge node separate allows that node to utilize the full computing resources available for Hadoop processing.

本文转载自:http://stackoverflow.com/questions/16685113/require-details-about-edge-node-in-cluster

共有 人打赏支持
我是彩笔
粉丝 7
博文 23
码字总数 1936
作品 0
浦东
私信 提问
MapR 初体验

文章转自:http://www.tbdata.org/archives/1833 一、MapR是什么? MapR是MapR Technologies, Inc的一个产品,号称下一代Hadoop,使Hadoop变为一个速度更快、可靠性更高、更易于管理、使用更...

红薯
2011/08/07
8.1K
1
一个 Hadoop 老兵的自白

Apache Hadoop 是一个免费软件,但实际上,除非是拥有庞大工程师团队的大公司,否则最好不要去创建仅供内部使用的Hadoop版本,因为如果要购买技术支持,那 Hadoop 就不是免费的了。Jim Scott...

oschina
2015/11/28
9.4K
28
Cloudera CDH 、Hortonworks DHP和MapR比较

目前啊,都知道,大数据集群管理方式分为手工方式(Apache hadoop)和工具方式(Ambari + hdp 和Cloudera Manger + CDH)。   手工部署呢,需配置太多参数,但是,好理解其原理,建议初学这...

hblt-j
2018/08/13
0
0
直击Hadoop Summit 2011:迎接海量数据挑战

海量数据正在不断生成,对于急需改变自己传统IT架构的企业而言,面对海量数据,如何分析并有效利用其价值,同时优化企业业务已成为现代企业转型过程中不可避免的问题。 作为海量数据处理的一...

疯狂的流浪
2011/07/01
3.9K
6
hadoop发行版本之间的区别

Hadoop是一个能够对大量数据进行分布式处理的软件框架。 Hadoop 以一种可靠、高效、可伸缩的方式进行数据处理。Hadoop的发行版除了有Apache hadoop外cloudera,hortonworks,mapR,华为,DKha...

左手的倒影
2018/09/11
0
0

没有更多内容

加载失败,请刷新页面

加载更多

记录replugin使用的一个坑

反复编译插件放入宿主中,一直出现如下错误: android.content.res.Resources$NotFoundException: Resource ID #0x7f050000 type #0x5 is not valid 回滚代码,重启AS还是出错。最终发现将宿...

Gemini-Lin
今天
1
0
Vert.x系列(二)--EventBusImpl源码分析

前言:Vert.x 实现了2种完成不同的eventBus: EventBusImpl(A local event bus implementation)和 它的子类 ClusteredEventBus(An event bus implementation that clusters with other Ve......

冷基
今天
1
0
Perl - 获取文件项目

参考:http://www.runoob.com/perl/perl-directories.html 下面返回JSON格式的文件列表 #!/usr/bin/perluse strict;use warnings;use utf8;use feature ':5.26';require Fi......

wffger
昨天
2
0
vue组件系列3、查询下载

直接源码,虽然样式样式不好看,逻辑也不是最优,但是可以留作纪念。毕竟以后类似的功能只需要优化就可以了,不用每次都重头开始。。。 <template> <div class="pre_upload"> <div ...

轻轻的往前走
昨天
2
0
java浅复制和深复制

之前写了数组的复制,所以这里继续总结一下浅复制和深复制。 浅拷贝:对基本数据类型进行值传递,对引用数据类型进行引用传递般的拷贝。 深拷贝:对基本数据类型进行值传递,对引用数据类型,...

woshixin
昨天
2
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部