使用Rancher在Kubernetes上部署EMQ X集群

2019/08/23 15:26
阅读数 53

本文描述如何通过Rancher2.0部署kubernetes集群,并将EMQ X部署到kubernetes集群上

实验环境:

公有云环境:AWS EC2

操作系统:ubuntu 16.04

Docker version:18.09.0

通过Rancher部署kubernetes集群

Rancher的安装以及部署kubernetes集群的步骤推荐直接按照快速入门执行。

创建Rancher Api密钥

EMQ X通过访问kube-apiserver来实现自动集群功能,在Rancher中,Rancher对kube-apiserver做了一层代理,在访问kube-apiserver的时候必须提供用于向Rancher进行身份验证的API密钥。参考用户手册创建并保存API Key。本实验中创建的Access Key为:token-dksbl,Secret Key为:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz,组合成的Token为:token-dksbl:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz

下载并配置kubectl

下载并安装kubectl

进入Rancher集群页面,点击Kubeconfig文件。

将kubeconfig文件保存到~/.kube/config

执行kubectl cluster-info验证配置是否成功

$ kubectl cluster-infoKubernetes master is running at https://13.125.244.172/k8s/clusters/c-vvgjq KubeDNS is running at https://13.125.244.172/k8s/clusters/c-vvgjq/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy ​ To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. 访问kube-apiserver

EMQ X通过访问kube-apiserver来实现自动集群,kube-apiserver的地址可以查看~/.ssh/config文件或者执行kubectl cluster-info获取,本实验中kube-apiserver的地址为:https://13.125.244.172/k8s/clusters/c-vvgjq。

直接访问kube-apiserver,可以看到会报错需要认证。

$ curl -k https://13.125.244.172/k8s/clusters/c-vvgjq{"type":"error","status":"401","message":"must authenticate"} 在头部加上Authorization认证则可以正常访问

$ curl -k -H 'Authorization: Bearer token-dksbl:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz' https://13.125.244.172/k8s/clusters/c-vvgjq 编辑emqx.yaml

在Kubernetes 上安装 EMQ X 系列文章之二 :EMQ X 自动集群一文中分享了EMQ X部署kubernetes集群的yaml文件如下,在Rancher上部署EMQ X集群的话需要稍加改动。

$cat emqx.yaml ​ apiVersion: v1 kind: Service metadata: name: emqx spec: ports:

  • port: 32333 nodePort: 32333 targetPort: emqx-dashboard protocol: TCP selector: app: emqx type: NodePort ​

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: emqx labels: app: emqx spec: replicas: 2 template: metadata: labels: app: emqx spec: containers: - name: emqx image: emqx/emqx:latest ports: - name: emqx-dashboard containerPort: 18083 env: - name: EMQX_CLUSTER__DISCOVERY value: k8s - name: EMQX_NAME value: emqx - name: EMQX_CLUSTER__K8S__APISERVER value: http://172.31.19.161:8080 - name: EMQX_CLUSTER__K8S__NAMESPACE value: default - name: EMQX_CLUSTER__K8S__SERVICE_NAME value: emqx - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE value: ip - name: EMQX_CLUSTER__K8S__APP_NAME value: emqx tty: true EMQ X可以读取/var/run/secrets/kubernetes.io/serviceaccount/token文件中的内容组合Authorization认证访问kube-apiserver,所以只需要把Rancher的API Token通过Secret挂载到容器中就可以了。

Secret解决了密码、token、密钥等敏感数据的配置问题,而不需要把这些敏感数据暴露到镜像或者Pod Spec中。Secret可以以Volume或者环境变量的方式使用。 Secret有三种类型: Service Account :用来访问Kubernetes API,由Kubernetes自动创建,并且会自动挂载到Pod的/run/secrets/kubernetes.io/serviceaccount目录中;

Opaque :base64编码格式的Secret,用来存储密码、密钥等;

kubernetes.io/dockerconfigjson :用来存储私有docker registry的认证信息。

首先对API Token做base64编码

$ echo -n token-dksbl:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz | base64 -w 0dG9rZW4tZGtzYmw6cHNoaGhmNWNwOGQ1djV4N2J6amRtODJxZnJ3Z3g3ZjJiemtzbnI3NDhqNDJ4bWJ2dmtsYmR6 在yaml文件中创建Secret

$vim emqx.yamlapiVersion: v1 kind: Secret metadata: name: emqx-secret type: Opaque data: token: dG9rZW4tcGI2MjU6eDZ2eGJ0Y2NmdG1waGpseHR3NGNjdGN2d2txdzk5aDJzYmhxNHFtaDh4c2ZnbXd6dzJ0d2Rw


...... 修改Deployment,将环境变量中的EMQX_CLUSTER__K8S__APISERVER改为Rancher的Kube-apiserver的地址,增加volumeMounts

$vim emqx.yaml...... apiVersion: extensions/v1beta1 kind: Deployment metadata: name: emqx labels: app: emqx spec: replicas: 2 template: metadata: labels: app: emqx spec: volumes: - name: emqx-secret secret: secretName: emqx-secret containers: - name: emqx image: emqx/emqx:latest ports: - name: emqx-dashboard containerPort: 18083 - name: emqx-http containerPort: 8083 - name: emqx-mqtt containerPort: 1883 env: - name: EMQX_CLUSTER__DISCOVERY value: k8s - name: EMQX_NAME value: emqx - name: EMQX_CLUSTER__K8S__APISERVER value: https://13.125.244.172/k8s/clusters/c-vvgjq - name: EMQX_CLUSTER__K8S__NAMESPACE value: default - name: EMQX_CLUSTER__K8S__SERVICE_NAME value: emqx - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE value: ip - name: EMQX_CLUSTER__K8S__APP_NAME value: emqx tty: true volumeMounts: - name: emqx-secret mountPath: "/var/run/secrets/kubernetes.io/serviceaccount" readOnly: true 部署EMQ X

查看修改后的emqx.yaml

$cat emqx.yamlapiVersion: v1 kind: Secret metadata: name: emqx-secret type: Opaque data: token: dG9rZW4tcGI2MjU6eDZ2eGJ0Y2NmdG1waGpseHR3NGNjdGN2d2txdzk5aDJzYmhxNHFtaDh4c2ZnbXd6dzJ0d2Rw ​

apiVersion: v1 kind: Service metadata: name: emqx spec: ports:

  • port: 32333 nodePort: 32333 targetPort: emqx-dashboard protocol: TCP selector: app: emqx type: NodePort ​

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: emqx labels: app: emqx spec: replicas: 2 template: metadata: labels: app: emqx spec: volumes: - name: emqx-secret secret: secretName: emqx-secret containers: - name: emqx image: emqx/emqx:latest ports: - name: emqx-dashboard containerPort: 18083 - name: emqx-http containerPort: 8083 - name: emqx-mqtt containerPort: 1883 env: - name: EMQX_CLUSTER__DISCOVERY value: k8s - name: EMQX_NAME value: emqx - name: EMQX_CLUSTER__K8S__APISERVER value: https://13.125.244.172/k8s/clusters/c-vvgjq - name: EMQX_CLUSTER__K8S__NAMESPACE value: default - name: EMQX_CLUSTER__K8S__SERVICE_NAME value: emqx - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE value: ip - name: EMQX_CLUSTER__K8S__APP_NAME value: emqx tty: true volumeMounts: - name: emqx-secret mountPath: "/var/run/secrets/kubernetes.io/serviceaccount" readOnly: true 部署EMQ X

$ kubectl create -f emqx.yamlsecret/emqx-secret created service/emqx created deployment.extensions/emqx created 查看状态

$ kubectl get podsNAME READY STATUS RESTARTS AGE emqx-67b5fcf4d-gwzfn 1/1 Running 0 36s emqx-67b5fcf4d-rb7m6 1/1 Running 0 36s 集群成功

$ kubectl exec emqx-67b5fcf4d-gwzfn /opt/emqx/bin/emqx_ctl cluster statusCluster status: [{running_nodes,['emqx@10.42.1.24','emqx@10.42.2.19']}] 使用Rancher Dashboard部署EMQ X(可选)

删除刚刚部署的EMQ X

$ kubectl delete -f emqx.yamlsecret "emqx-secret" deleted service "emqx" deleted deployment.extensions "emqx" deleted 进入Rancher集群工作负载页面,点击导入YAML

在导入页面将emqx.yaml文件的内容复制进去

点击导入,等待导入成功。

展开阅读全文
打赏
0
0 收藏
分享
加载中
更多评论
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部