build hadoop, spark, hbase cluster-白红宇

build hadoop, spark, hbase cluster

阅读量：6237 次

发布时间：2019-06-22

本文共 2220 字，大约阅读时间需要 7 分钟。

1,something:

1,arc land

506 git branch

507 git status

508 git reset multicloud/qcloud/cluster_management/spark_cluster/dependent_file/spark-1.6.2-bin-hadoop2.6.tgz

509 git status

510 git add multicloud/qcloud/tools/ssh.py

511 git status

512 git log

513 git commit -a

514 git status

515 git log

516 git pull --rebase origin master

517 git log

518 arc diff

519 arc diff

520 history

git checkout -b test1  432  git status  433  git branch  434  ls  435  cd core/  436  ls  437  cd ..  438  cd multicloud/  439  ls  440  cd qcloud/  441  ls  442  cd ..  443  cd ..  444  ls  445  ls multicloud/qcloud/spark_cluster/README.md   446  vim multicloud/qcloud/spark_cluster/README.md   447  git branch  448  git status  449  git commit -a 'test1'  450  git commit -a  451  git log  452  clear  453  git pull --rebase origin master  454  arc diff  455  pwd  456  arc help  457  source ~/.bashrc   458  arc  459  arc help  460  ls ~/.bashrc   461  arc diff  462  arc diff  463  arc land

View Code

2,ssh aliyun

ssh -i aliyun.pem root@101.201.233.28

chmod 600 aliyun.pem

3,ssh免密码登录

(1)

ssh-keygen -t rsa

scp -i ~/aliyun.pem id_rsa.pub root@172.17.148.92:~/.ssh

(2)in 172.17.148.92

cat id_rsa.pub >> authorized_keys

chmod 600 authorized_keys

http://blog.csdn.net/m15851813962/article/details/53150967

4,docker

docker exec -it ssss /bin/bash

5, scp:

scp -i ~/work/keys/alicloud.pem * hadoopmaster:/root

6, firewall

systemctl stop firewalld

7,auto build command

spark:

python auto_build_spark.py -f /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/dev_conf/alicloud_dev_spark_cluster_conf.json -k /Users/chunyueli/work/keys/alicloud.pem -F /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/restart_qcloud_cluster.py -i 114.240.80.157 -a restart

hadoop

python auto_build_hadoop.py -f /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/dev_conf/alicloud_dev_hadoop_cluster_conf.json -k /Users/chunyueli/work/keys/alicloud.pem -F /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/restart_qcloud_cluster.py -i 114.240.85.26

8, docker command

转载于:https://www.cnblogs.com/yueyebigdata/p/6262955.html

你可能感兴趣的文章