联合网站设计网站空间 支持什么程序
目录
Part.01 关于HDP
Part.02 核心组件原理
Part.03 资源规划
Part.04 基础环境配置
Part.05 Yum源配置
Part.06 安装OracleJDK
Part.07 安装MySQL
Part.08 部署Ambari集群
Part.09 安装OpenLDAP
Part.10 创建集群
Part.11 安装Kerberos
Part.12 安装HDFS
Part.13 安装Ranger
Part.14 安装YARN+MR
Part.15 安装HIVE
Part.16 安装HBase
Part.17 安装Spark2
Part.18 安装Flink
Part.19 安装Kafka
Part.20 安装Flume
四、基础环境配置
1.SSH免密登录
在hdp01、hdp02上生成公钥，配置免密登录到其他节点
ssh-keygen -t rsa -f ~/.ssh/id_rsa -C username_root
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root@192.168.111.201
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root@192.168.111.202
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root@192.168.111.203
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root@192.168.111.204
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 22 root@192.168.111.205
2.ansible安装
配置ansible和rhel-system-roles，创建配置文件
mkdir /root/ansible
cd /root/ansible
cp /etc/ansible/ansible.cfg /root/ansible/
修改配置文件，/root/ansible/ansible.cfg
[defaults]
inventory = /root/ansible/inventory
ask_pass = false
remote_user = root
配置inventory文件，/root/ansible/inventory
[hdp:children]
nn
dn
[nn]
192.168.111.201 hostname=hdp01
192.168.111.202 hostname=hdp02
[dn]
192.168.111.203 hostname=hdp03
192.168.111.204 hostname=hdp04
192.168.111.205 hostname=hdp05
3.修改hostname
创建playbook，/root/ansible/hostname.yml
---
- name: modify hostnamehosts: alltasks:- name: modify hostname permanentlyraw: "echo {{ hostname | quote }} > /etc/hostname"- name: modify hostname temporarilyshell: hostname {{ hostname | quote }}
执行并确认
ansible-playbook /root/ansible/hostname.yml
ansible all -m shell -a 'hostname'
4.修改hosts列表
在nn01上修改主机列表，/etc/hosts
192.168.111.201 hdp01.hdp.com hdp01
192.168.111.202 hdp02.hdp.com hdp02
192.168.111.203 hdp03.hdp.com hdp03
192.168.111.204 hdp04.hdp.com hdp04
192.168.111.205 hdp05.hdp.com hdp05
分发至其他节点
ansible all -m template -a 'src=/etc/hosts dest=/etc/hosts'
5.安装基础软件
安装vim等基础软件，/root/ansible/packages.yml
---
- hosts: alltasks:- name: install packagesyum:name:- pciutils- bash-completion- vim- chronystate: present
6.关闭firewall和SELinux
关闭firewall
ansible all -m service -a 'name=firewalld state=stopped enabled=no'
ansible all -m shell -a 'systemctl status firewalld | grep Active'
关闭SELinux
ansible all -m selinux -a 'policy=targeted state=disabled'
ansible all -m shell -a 'getenforce'
7.NTP时钟
以hdp01为时钟源，其余节点从nn01进行时钟同步
服务端（hdp01）
修改配置文件/etc/chrony.conf
# 不指定外部NTP源
# 允许本网段其节点作为客户端访问
allow 192.168.111.0/24
# 如果时间服务可不用，则使用本地时间作为标准时间授权，层数为10
local stratum 10
重启服务
systemctl restart chronyd
客户端（hdp02-hdp05）
安装ntp时钟，/root/ansible/timesync.yml
---
- hosts: 192.168.111.202,dnvars:timesync_ntp_servers:- hostname: 192.168.111.201iburst: yesroles:- rhel-system-roles.timesync
执行
ansible-playbook /root/ansible/timesync.yml
确认时钟同步情况
ansible 192.168.111.202,dn -m shell -a 'chronyc sources -v'
8.磁盘分区、文件系统及挂载目录
创建分区parted.yml文件
---
- hosts: alltasks:- name: parted devicesparted:device: "{{ item }}"number: 1label: gptstate: presentloop:- /dev/sdb- /dev/sdc- /dev/sddignore_errors: yes
确认结果
ansible all -m shell -a 'lsblk -f'
创建文件系统mkfs.yml文件
---
- hosts: alltasks:- name: mkdir of nnfile:path: /data01state: directorywhen: inventory_hostname in groups['nn']- name: mkdir of dnfile:path: "{{ item }}"state: directoryloop:- /data01- /data02- /data03when: inventory_hostname in groups['dn']- name: mkfsfilesystem:fstype: xfsdev: "{{ item }}"loop:- /dev/sdb1- /dev/sdc1- /dev/sdd1ignore_errors: yes- name: mount of nnmount:path: /data01src: /dev/sdb1fstype: xfsstate: mountedwhen: inventory_hostname in groups['nn']- name: mount of dnmount:path: "{{ item.p_dir }}"src: "{{ item.s_dir }}"fstype: xfsstate: mountedloop:- { p_dir: /data01, s_dir: /dev/sdb1 }- { p_dir: /data02, s_dir: /dev/sdc1 }- { p_dir: /data03, s_dir: /dev/sdd1 }when: inventory_hostname in groups['dn']