暂无说说

hive安装

hadoop jiajun 8个月前 (02-23) 116次浏览 0个评论 扫描二维码

安装 mysql

执行以下命令安装 mysql

sudo apt-get install mysql-server

验证

mysql -uroot -proot

mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.7.25-0ubuntu0.16.04.2 (Ubuntu)

Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

安装hive

解压hive

tar -zxvf apache-hive-2.3.4-bin.tar.gz -C /soft/
cd /soft
ln -s apache-hive-2.3.4-bin/ hive

复制 mysql 驱动

cp mysql-connector-java-5.1.47.jar /soft/hive/lib/

配置环境变量

sudo nano /etc/profile
export HIVE_HOME=/soft/hive
export PATH=$PATH:$HIVE_HOME/bin
source /etc/profile

修改配置文件

1、修改 hive-site.xml

cd $HIVE_HOME/conf/
cp hive-default.xml.template  hive-site.xml
nano hive-site.xml

<property>
    <name>hive.metastore.local</name>
    <value>false</value>
</property>
<property>
	<name>javax.jdo.option.ConnectionURL</name>
	<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
</property>
<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
</property>

2、修改 hive-env.sh

cp hive-env.sh.template hive-env.sh
nano hive-env.sh
HADOOP_HOME=/soft/hadoop

3、修改 hadoop-env.sh

cd $HADOOP_HOME/etc/hadoop
nano hadoop-env.sh
export HADOOP_CLASSPATH=.:$CLASSPATH:$HADOOP_CLASSPATH:$HADOOP_HOME/bin

4、修改 hive-config.sh

cd $HIVE_HOME/bin
nano hive-config.sh
export JAVA_HOME=/soft/jdk
export HIVE_HOME=/soft/hive
export HADOOP_HOME=/soft/hadoop

5、初始化数据库,生成元数据

schematool -initSchema -dbType mysql

排错

1、Hive 启动时,报错 RelativepathinabsoluteURI:${system:java.io.tmpdir}

错误信息

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

 

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: 
Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
	at org.apache.hadoop.fs.Path.initialize(Path.java:205)
	at org.apache.hadoop.fs.Path.(Path.java:171)
	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:644)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:563)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:531)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
	at java.net.URI.checkPath(URI.java:1823)
	at java.net.URI.(URI.java:745)
	at org.apache.hadoop.fs.Path.initialize(Path.java:202)
	... 12 more

错误修改

修改 hive-site.xml 中所有包含${system:Java.io.tmpdir}字段的 value 路径(vim 下 / 表示搜索,后面跟你的关键词,比如搜索 hello,则为/hello, 再回车即可),你可以自己新建一个目录来替换它,例如/home/hive/iotmp.我这边把 hive-site.xml 中所有出现${system:Java.io.tmpdir}的地方都替换成了:

/tmp/hive

2、创建表时,报以下错误

SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))

原因:

这是由于没有初始化数据库,生成元数据。 需要执行以下指令(在 hive 的 bin 目录下执行)

./schematool -initSchema -dbType mysql

最后一个参数为 hive 存放元数据所使用的数据库,我用的是 mysql。如果使用 derby,将最后的参数改为 derby 就可以了。在使用 hive1.x 时不记得遇到过,在配置 2.1.0 时出现。

喜欢 (0)
发表我的评论
取消评论

表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址