Hadoop(二、Hive(数据仓库)搭建)（hadoop3 hive）

说明：此篇文章环境基于【Hadoop（一、环境搭建）】，请先阅读上一篇文章并搭建好环境。

1、在hadoop中core-site.xml中整合hive

cd /export/server/hadoop-3.2.0/etc/hadoop

vim core-site.xml

<!--整合hive-->

<property>

<name>hadoop.proxyuser.root.hosts</name>

<value>*</value>

</property>

<property>

<name>hadoop.proxyuser.root.groups</name>

<value>*</value>

</property>

2、安装mysql

2.1、卸载mariadb

执行命令：rpm -qa|grep mariadb

输出：mariadb-libs-5.5.68-1.el7.x86_64

执行删除命令：rpm -e mariadb-libs-5.5.68-1.el7.x86_64 --nodeps

2.2、安装mysql

在/export/software目录下新建目录：mkdir mysql

本地windows拷贝到虚拟机：scp -r mysql-5.7.22-1.el7.x86_64.rpm-bundle.tar root@192.168.137.129:/export/software/mysql

解压：tar xvf mysql-5.7.22-1.el7.x86_64.rpm-bundle.tar

安装依赖：yum -y install libaio

执行安装：rpm -ivh mysql-community-common-5.7.22-1.el7.x86_64.rpm mysql-community-libs-5.7.22-1.el7.x86_64.rpm mysql-community-client-5.7.22-1.el7.x86_64.rpm mysql-community-server-5.7.22-1.el7.x86_64.rpm

说明：执行安装中rpm -ivh 后面的几个rpm文件是解压mysql-5.7.22-1.el7.x86_64.rpm-bundle.tar后生成的，实际可根据mysql版本修改。

3、初始化mysql

进入mysql解压目录：cd /export/software/mysql

初始化：mysqld --initialize

更改组：chown mysql:mysql /var/lib/mysql -R

首次启动：systemctl start mysqld.service

查看自动生成的密码：cat /var/log/mysqld.log

登录mysql: mysql -u root -p (下面提示输入密码：julv:AeO#1MU)

修改密码：alter user user() identified by "hadoop";

授权：use mysql;

授权：GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop' WITH GRANT OPTION;

刷新：FLUSH PRIVILEGES;

设置mysql开机自启动：systemctl enable mysqld

4、hive安装

4.1、下载hive包

http://archive.apache.org/dist/hive/hive-3.1.2/

下载这个版本：apache-hive-3.1.2-bin.tar.gz

远程上传到虚拟机：scp -r apache-hive-3.1.2-bin.tar.gz root@192.168.137.129:/export/server

解压：tar xvzf apache-hive-3.1.2-bin.tar.gz

4.2、解决guava版本不一致

cd /export/server/apache-hive-3.1.2-bin/lib/

rm -rf guava-19.0.jar

cp /export/server/hadoop-3.2.0/share/hadoop/common/lib/guava-11.0.2.jar ./

说明：若/export/server/hadoop-3.2.0/share/hadoop/common/lib/和/export/server/apache-hive-3.1.2-bin/lib/下哪个guava的版本搞用哪一个。否则hive启动报错。

4.3、配置文件

进入此目录：cd /export/server/apache-hive-3.1.2-bin/conf

4.3.1配置hive.env.sh

先修改文件名称：mv hive-env.sh.template hive-env.sh

编辑：vim hive-env.sh，添加内容：

export HADOOP_HOME=/export/server/hadoop-3.2.0

export HIVE_CONF_DIR=/export/server/apache-hive-3.1.2-bin/conf

export HIVE_AUX_JARS_PATH=/export/server/apache-hive-3.1.2-bin/lib

4.3.2、配置hive-site.xml（没有就新增）

编辑：vim hive-site.xml，添加内容：

<configuration>

<!--存储元数据mysql配置-->

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://hp1:3306/hive3?createDatabaseIfNotExist=true&useSSL=false&useUnicode=true&characterEncoding=UTF-8</value>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

</property>

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>root</value>

</property>

<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value>hadoop</value>

</property>

<!--h2s绑定host-->

<property>

<name>hive.server2.thrift.bind.host</name>

<value>hp1</value>

</property>

<!--远程模式部署metastore地址-->

<property>

<name>hive.metastore.uris</name>

<value>thrift://hp1:9083</value>

</property>

<!--关闭元数据存储授权-->

<property>

<name>hive.metastore.event.db.notification.api.auth</name>

<value>false</value>

</property>

</configuration>

4.3.3、上传驱动

将驱动mysql-connector-java-5.1.32.jar上传到/export/server/apache-hive-3.1.2-bin/lib下

scp -r mysql-connector-java-5.1.32.jar root@192.168.137.129:/export/server/apache-hive-3.1.2-bin/lib

4.3.4初始化hive

cd /export/server/apache-hive-3.1.2-bin

bin/schematool -initSchema -dbType mysql -verbos

5、验证

执行完4.3.4之后会在hive3数据库中生成74张表