https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_requirements_supported_versions.html

centos7禁用ipv6的方法
https://blog.csdn.net/cjm712/article/details/87886614


Cluster hosts must have a working network name resolution system and correctly formatted /etc/hosts file. All cluster hosts must have properly configured forward and reverse host resolution through DNS. The /etc/hosts files must:
Contain consistent information about hostnames and IP addresses across all hosts
Not contain uppercase hostnames
Not contain duplicate IP addresses

Cluster hosts must not use aliases, either in /etc/hosts or in configuring DNS. A properly formatted /etc/hosts file should be similar to the following example:

127.0.0.1 localhost.localdomain localhost
192.168.1.1 cluster-01.example.com cluster-01
192.168.1.2 cluster-02.example.com cluster-02
192.168.1.3 cluster-03.example.com cluster-03 

For RHEL and CentOS, the /etc/sysconfig/network file on each host must contain the correct hostname.

https://docs.cloudera.com/documentation/enterprise/6/latest/topics/cm_ig_create_local_parcel_repo.html
wget -nc or --no-clobber  不再加在已经重名的文件

sudo mkdir -p /mnt/c/xpc/share/cloudera-repos
cd /mnt/c/xpc/share/cloudera-repos

Downloading and Publishing the Parcel Repository

sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.0/parcels/ -P /mnt/c/xpc/share/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.0/parcels/ -P /mnt/c/xpc/share/cloudera-repos
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/cdh6
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/gplextras6

sudo mkdir -p /mnt/c/xpc/share/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/parcels/1.7.2/ -P /mnt/c/xpc/share/cloudera-repos
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/accumulo-c5

sudo mkdir -p /mnt/c/xpc/share/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/spark2/parcels/2.3.0.cloudera3/ -P /mnt/c/xpc/share/cloudera-repos
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/spark2

sudo mkdir -p /mnt/c/xpc/share/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories http://archive.cloudera.com/sqoop-connectors/parcels/latest/ -P /mnt/c/xpc/share/cloudera-repos
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/sqoop-connectors


https://archive.cloudera.com/cm6/6.2.0/
https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_60_packaging.html
https://docs.cloudera.com/documentation/enterprise/6/latest/topics/cm_ig_create_local_package_repo.html?cdoc-os=redhat+7


sudo mkdir -p /mnt/c/xpc/share/cloudera-repos
cd /mnt/c/xpc/share/cloudera-repos

Cloudera Manager 6
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.0/redhat7/ -P /mnt/c/xpc/share/cloudera-repos
sudo wget https://archive.cloudera.com/cm6/6.3.0/allkeys.asc -P /mnt/c/xpc/share/cloudera-repos/cm6/6.3.0/
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/cm6

CDH 6
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.0/redhat7/ -P /mnt/c/xpc/share/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.0/redhat7/ -P /mnt/c/xpc/share/cloudera-repos
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/cdh6
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/gplextras6

Apache Accumulo for CDH
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/redhat/ -P /mnt/c/xpc/share/cloudera-repos
sudo chmod -R ugo+rx /mnt/c/xpc/share/cloudera-repos/accumulo-c5


Configuring Hosts to Use the Internal Repository
cat /etc/yum.repos.d/cloudera-repo.repo
[cloudera-repo]
name=cloudera-repo
#baseurl=http://<web_server>/cm/5
baseurl=http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/
enabled=1
gpgcheck=0 


远程 Parcel 存储库 URL
https://archive.cloudera.com/cdh6/{latest_supported}/parcels/
 
https://archive.cloudera.com/cdh5/parcels/5.16/
 
https://parcels.repos.intel.com/mkl/latest
 
https://archive.cloudera.com/accumulo-c5/parcels/latest/
 
https://archive.cloudera.com/accumulo6/6.1/parcels/
 
https://archive.cloudera.com/kafka/parcels/latest/
 
http://archive.cloudera.com/kudu/parcels/latest/
 
https://archive.cloudera.com/spark/parcels/latest/
 
https://archive.cloudera.com/sqoop-teradata-connector1/latest/
 
https://archive.cloudera.com/sqoop-connectors/parcels/latest/
 
https://archive.cloudera.com/sqoop-netezza-connector1/latest/

#filename: cdhdld.sh

REPOS_PATH=/mnt/c/xpc/share/cloudera-repos
mkdir -p $REPOS_PATH

wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.0/redhat7/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.0/repo-as-tarball/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.0/allkeys.asc -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.0/cloudera-manager-installer.bin -P ${REPOS_PATH}

wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.0/parcels/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.0/redhat7/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.0/docs/ -P ${REPOS_PATH}

wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.0/parcels/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.0/redhat7/ -P ${REPOS_PATH}

wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/redhat/7/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/parcels/1.7.2/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories http://archive.cloudera.com/sqoop-connectors/parcels/1.7.23/ -P ${REPOS_PATH}
wget --recursive --level=10 --no-parent --no-host-directories https://archive.cloudera.com/spark2/parcels/2.3.0.cloudera3/ -P ${REPOS_PATH}

#wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.0/cloudera-manager-installer.bin -P /mnt/c/xpc/share/cloudera-repos

https://docs.cloudera.com/documentation/enterprise/6/latest/topics/poc_run_installer.html
http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/cloudera-manager.repo

http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/cloudera-manager-installer.bin
chmod u+x cloudera-manager-installer.bin

rm -f /etc/yum.repos.d/cloudera*
cat << 'EOF' > /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name=Cloudera Manager 6.3.0
baseurl=http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/
gpgkey=http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/RPM-GPG-KEY-cloudera
gpgcheck=1
enabled=1
autorefresh=0
type=rpm-md
EOF
yum repoinfo cloudera-manager

sudo ./cloudera-manager-installer.bin --skip_repo_package=1

Installs the Cloudera Manager repository files.
Installs the Oracle JDK.
Installs the Cloudera Manager Server and embedded PostgreSQL packages.
Starts the embedded PostgreSQL database and Cloudera Manager Server.

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

http://node7:7180/cmf/login
admin / admin

node7,node8,node9

http://192.168.1.8/chfs/shared/share/cloudera-repos/cdh6/6.3.0/parcels

REPOS_PATH=/as4k/cloudera-repos
mkdir -p $REPOS_PATH
wget --recursive --level=10 --no-parent --no-host-directories http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/RPMS/x86_64/ -P ${REPOS_PATH}

cd /etc/yum.repos.d; gzip *
yum install oracle-j2sdk1.8
cd /etc/yum.repos.d; gunzip Centos-7.repo.gz  epel-7.repo.gz
yum install cloudera-manager-agent

在WSL上用python3构建离线yum仓库
cd /mnt/c/xpc/share/cloudera-repos
python3 -m http.server 8000
http://192.168.1.8:8000


rm -f /etc/yum.repos.d/cloudera*
cat << 'EOF' > /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name=Cloudera Manager 6.3.0
baseurl=http://192.168.1.8:8000/cm6/6.3.0/redhat7/yum/
gpgkey=http://192.168.1.8:8000/6.3.0/redhat7/yum/RPM-GPG-KEY-cloudera
gpgcheck=1
enabled=1
EOF
yum repoinfo cloudera-manager
#rpm --import http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/RPM-GPG-KEY-clouderaa
rpm --import http://192.168.1.8:8000/cm6/6.3.0/redhat7/yum/RPM-GPG-KEY-clouderaa

[root@node7 ~]# rpm --import http://192.168.1.8:8000/cm6/6.3.0/redhat7/yum/RPM-GPG-KEY-cloudera
[root@node7 ~]# rpm -K oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm 
oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm: rsa sha1 (md5) pgp md5 OK


http://192.168.1.8/chfs/shared/share/cloudera-repos/cm6/6.3.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0%2Bupdate181-1.x86_64.rpm
yum install oracle-j2sdk1.8
yum install ./oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm

On the Cloudera Manager Server host, type the following commands to install the Cloudera Manager packages.
yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

mkdir /as4k; cd /as4k/; 
wget http://192.168.1.8/chfs/shared/share/mysql/mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar    
tar xf mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar
yum install mysql-community-{server,client,common,libs}-*
(现在不要启动MySQL)

cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0

key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

systemctl enable mysqld
systemctl start mysqld
systemctl status mysqld

[root@node7 as4k]# journalctl -u mysqld | grep "temporary password"
Dec 27 13:19:03 node7 mysqld_pre_systemd[21485]: 2019-12-27T05:19:03.429047Z 1 [Note] A temporary password is generated for root@localhost: bapK9Ry%egj2


cat /etc/my.cnf
[mysqld]
plugin-load=validate_password.so
validate-password=OFF

echo; echo; cat /etc/my.cnf
systemctl restart mysqld

tmp_mysql_password=$(journalctl -u mysqld | grep "temporary password" | awk '{print $NF}')
echo $tmp_mysql_password | cat -A
mysql --connect-expired-password -uroot -p"${tmp_mysql_password}" -e "ALTER USER 'root'@'localhost' IDENTIFIED BY 'As4k.top'"
mysql -uroot -p'As4k.top'  -e  "GRANT ALL PRIVILEGES ON *.* TO root@'%' IDENTIFIED BY '123456' WITH GRANT OPTION"
mysql -uroot -p'As4k.top'  -e "select user,host from mysql.user"
mysql -uroot -p'As4k.top'  -e "show databases"

Install the JDBC driver on the Cloudera Manager Server host, as well as any other hosts running services that require database access.

Installing the MySQL JDBC Driver
https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
http://192.168.1.8/chfs/shared/share/mysql/mysql-connector-java-5.1.46.tar.gz

mkdir /as4k; cd /as4k; wget http://192.168.1.8/chfs/shared/share/mysql/mysql-connector-java-5.1.46.tar.gz; tar xf mysql-connector-java-5.1.46.tar.gz
mkdir -p /usr/share/java/; cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

CREATE DATABASE <database> DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON <database>.* TO '<user>'@'%' IDENTIFIED BY '<password>';

for i in scm amon rman hue sentry nav navms oozie; do
#echo $i
echo "CREATE DATABASE ${i} DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;"
echo "GRANT ALL ON ${i}.* TO '${i}'@'%' IDENTIFIED BY '123456';"
done

mysql -h192.168.1.121 -uroot -p123456
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456';
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456';


Example 1: Running the script when MySQL or MariaDB is co-located with the Cloudera Manager Server
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

systemctl start cloudera-scm-server

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

http://<server_host>:7180

cat << 'EOF' >  /etc/sysconfig/network
# Created by anaconda
NETWORKING=yes
HOSTNAME=node9
EOF
hwclock --systohc

一般你会遇到两个警告信息

Cloudera 建议将 /proc/sys/vm/swappiness 设置为最大值 10。当前设置为 30。使用 sysctl 命令在运行时更改该设置并编辑 /etc/sysctl.conf，以在重启后保存该设置。您可以继续进行安装，但 Cloudera Manager 可能会报告您的主机由于交换而运行状况不良。以下主机将受到影响

echo 'vm.swappiness=10' >> /etc/sysctl.conf
sysctl -p
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

以禁用此设置，然后将同一命令添加到 /etc/rc.local 等初始化脚本中，以便在系统重启时予以设置。以下主机将受到影响

http://www.cnblogs.com/jasondan/p/4011153.html
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cm_ig_uninstall_host.html
https://www.cnblogs.com/CaptainLin/p/7089766.html
https://www.cnblogs.com/ee900222/p/hadoop_3.html
https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_intro.html

open source Cloudera Distribution including Apache Hadoop (CDH)
https://segmentfault.com/a/1190000020188183?utm_source=tag-newest#articleHeader14

https://docs.cloudera.com/documentation/

https://archive.cloudera.com/cm6/6.3.0/cloudera-manager-installer.bin

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cm_6_version_download.html

Installing Cloudera Manager, CDH, and Managed Services
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/install_cm_cdh.html

Cloudera Enterprise 6 Requirements and Supported Versions
https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_requirements_supported_versions.html
https://docs.cloudera.com/documentation/enterprise/6/latest/topics/installation_reqts.html
http://192.168.1.8/chfs/shared/share/cloudera_ref_arch_metal.pdf

CDH离线搭建
https://segmentfault.com/a/1190000020188183?utm_source=tag-newest#articleHeader14