暂无说说

storm集群的安装

storm jiajun 4个月前 (09-01) 82次浏览 0个评论 扫描二维码

环境

host

s100、s101、s102

对应 ip:

192.168.163.100 
192.168.163.101 
192.168.163.102

操作系统​:

centos7

步骤

1、关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

2、修改/etc/hosts 文件

sudo vi /etc/hosts

添加修改集群中主机和 IP 的映射关系

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.163.100 s100
192.168.163.101 s101
192.168.163.102 s102

分发到其它节点上

sudo scp /etc/hosts root@192.168.163.101:/etc/hosts
sudo scp /etc/hosts root@192.168.163.102:/etc/hosts

3、创建/soft 目录,用于安装软件(三个节点上都要创建)

sudo mkdir /soft
sudo chown hadoop:hadoop /soft

4、安装 jdk

chmod a+x jdk-6u45-linux-x64.bin
./jdk-6u45-linux-x64.bin
mv jdk1.6.0_45/ /soft
ln -s /soft/jdk1.6.0_45/ /soft/jdk

5、安装 Zookeeper

解压:

tar -zxvf zookeeper-3.4.5.tar.gz -C /soft
ln -s /soft/zookeeper-3.4.5/ /soft/zookeeper

配置 zookeeper-3.4.5/conf/zoo.cfg 文件

cd /soft/zookeeper/conf/
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg

修改如下内容:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/home/hadoop/zookeeper/data
# the port at which the clients will connect
clientPort=2181
server.1=s100:2888:3888
server.2=s101:2888:3888
server.3=s102:2888:3888

6、安装 python(centos 已经自带,略)

7、安装storm

解压:

tar -zxvf apache-storm-0.9.1-incubating.tar.gz -C /soft
ln -s /soft/apache-storm-0.9.1-incubating/ /soft/storm

修改 conf/storm.yaml 文件

cd /soft/storm/conf/
vi storm.yaml

修改添加下面内容:

########### These MUST be filled in for a storm configuration
 storm.zookeeper.servers:
     - "s100"
     - "s101"
     - "s102"
# storm.zookeeper.servers:
#     - "server1"
#     - "server2"
# 
 nimbus.host: "s100"
 storm.zookeeper.port: 2181
 storm.local.dir: "/home/hadoop/storm/data"
 supervisor.slots.ports:
      - 6700
      - 6701
      - 6702
      - 6703

注意:所有配置前面都要带空格,否则启动会报错,如:

(空格)storm.zookeeper.port: 2181

8、创建 myid

在修改 zookeeper 配置的时,设置 datadir 如下:

dataDir=/home/hadoop/zookeeper/data

因此需要分别在三个节点上创建/home/hadoop/zookeeper/data 目录

mkdir -p /home/hadoop/zookeeper/data

同时,由于指定了 server 信息如下:

server.1=s100:2888:3888
server.2=s101:2888:3888
server.3=s102:2888:3888

所以应分别在三个节点上的/home/hadoop/zookeeper/data 目录下创建 myid 文件

s100

cd /home/hadoop/zookeeper/data/
echo "1">myid

s101

cd /home/hadoop/zookeeper/data/
echo "2">myid

s102

cd /home/hadoop/zookeeper/data/
echo "3" >myid

9、修改/etc/profile

sudo vi /etc/profile

添加环境变量

export JAVA_HOME=/soft/jdk
export ZK_HOME=/soft/zookeeper
export STORM_HOME=/soft/storm
export PATH=$PATH:$JAVA_HOME/bin:$ZK_HOME/bin:$STORM_HOME/bin

刷新/etc/profile 使环境变量生效

source /etc/profile

查看 java 版本

[hadoop@s100 conf]$ java -version
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

10、分发到其它节点上

sudo scp /etc/profile root@s101:/etc/profile
sudo scp /etc/profile root@s102:/etc/profile
scp -rp /soft/* hadoop@s101:/soft/
scp -rp /soft/* hadoop@s102:/soft/

在 s101 跟 s102 上分别刷新/etc/profile

source /etc/profile

11、启动 zookeeper 集群

在三个节点上分别执行:

zkServer.sh start

查看 zookeeper 集群状态

zkServer.sh status

其中两个显示 follower,一个显示 leader

[hadoop@s100 conf]$ zkServer.sh status
JMX enabled by default
Using config: /soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower

[hadoop@s101 soft]$ zkServer.sh status 
JMX enabled by default
Using config: /soft/zookeeper/bin/../conf/zoo.cfg
Mode: leader

[hadoop@s102 soft]$ zkServer.sh status
JMX enabled by default
Using config: /soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower

10、启动 storm

在主节点上运行

storm nimbus & 
storm supervisor &

在工作节点上运行

storm supervisor &

在主节点上运行

storm ui &

在主节点上运行

storm logviewer &

查看进程:

[hadoop@s100 conf]$ jps
10454 supervisor
9735 nimbus
10951 Jps
9777 core
9829 logviewer
9358 QuorumPeerMain

[hadoop@s101 soft]$ jps
9295 supervisor
9733 Jps
9191 QuorumPeerMain

[hadoop@s102 soft]$ jps
9789 Jps
9300 supervisor
9189 QuorumPeerMain

webui:

http://192.168.163.100:8080/

至此,storm 集群安装完毕。

 

喜欢 (0)
发表我的评论
取消评论
表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址