在k8s集群中部署jenkins(使用glusterfs存储)

上一篇我们主要说了glusterfs的集群安装和部署，这篇我们是基于上一篇在集群中部署jenkins，存储就使用我们上篇创建的存储卷，如果有错过上篇内容的，建议先看一下，点这里看k8s存储卷选型和glusterfs集群安装，在上一篇最后我们创建了k8s-volume-jenkins-master名称的数据卷，接下来我们看如何使用。

首先创建endpoints.yaml文件：

--- apiVersion: v1 kind: Endpoints metadata: name: glusterfs-cluster namespace: ci subsets: - addresses: [{"ip": "192.168.1.1"}, {"ip": "192.168.1.2"}] ports: [{"port": 24007}]

1 2 3 4 5 6 7 8 9

--- apiVersion: v1 kind: Endpoints metadata:   name: glusterfs-cluster   namespace: ci subsets:   - addresses: [{"ip": "192.168.1.1"}, {"ip": "192.168.1.2"}] ports: [{"port": 24007}]

创建gs_service.yaml文件：

--- apiVersion: v1 kind: Service metadata: name: glusterfs-cluster namespace: ci spec: ports: - port: 24007 targetPort: 24007 protocol: TCP

1 2 3 4 5 6 7 8 9 10 11

--- apiVersion: v1 kind: Service metadata:   name: glusterfs-cluster   namespace: ci spec:   ports:     - port: 24007       targetPort: 24007       protocol: TCP

然后运行kubectl执行即可，yaml文件创建了一个namespace，名称叫ci, 指定glusterfs机器的ip和端口，有几台就写几台。

接下来我们创建jenkins-master-pv.yaml:

apiVersion: v1 kind: PersistentVolume metadata: name: pv-jenkins-master namespace: ci spec: capacity: storage: 1Gi accessModes: - ReadWriteMany glusterfs: endpoints: "glusterfs-cluster" path: "k8s-volume-jenkins-master" readOnly: false

1 2 3 4 5 6 7 8 9 10 11 12 13 14

apiVersion: v1 kind: PersistentVolume metadata:   name: pv-jenkins-master   namespace: ci spec:   capacity:     storage: 1Gi   accessModes:     - ReadWriteMany   glusterfs:     endpoints: "glusterfs-cluster"     path: "k8s-volume-jenkins-master" readOnly: false

创建jenkins-master-pvc.yaml:

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-jenkins-master namespace: ci spec: accessModes: - ReadWriteMany resources: requests: storage: 1Gi

1 2 3 4 5 6 7 8 9 10 11

apiVersion: v1 kind: PersistentVolumeClaim metadata:   name: pvc-jenkins-master   namespace: ci spec:   accessModes:     - ReadWriteMany   resources:     requests:       storage: 1Gi

下一步创建jenkins_depoly.yaml:

apiVersion: apps/v1 kind: Deployment metadata: namespace: ci name: jenkins spec: replicas: 1 selector: matchLabels: app: jenkins template: metadata: labels: app: jenkins spec: securityContext: fsGroup: 1000 runAsUser: 0 containers: - name: jenkins image: jenkins ports: - containerPort: 8000 volumeMounts: - name: home mountPath: "/var/jenkins_home" volumes: - name: home persistentVolumeClaim: claimName: pvc-jenkins-master

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

apiVersion: apps/v1 kind: Deployment metadata:   namespace: ci   name: jenkins spec:   replicas: 1   selector:     matchLabels:       app: jenkins   template:     metadata:       labels:         app: jenkins     spec:       securityContext:         fsGroup: 1000         runAsUser: 0       containers:         - name: jenkins           image: jenkins           ports:             - containerPort: 8000           volumeMounts:             - name: home               mountPath: "/var/jenkins_home"       volumes:         - name: home           persistentVolumeClaim:             claimName: pvc-jenkins-master

在deploy文件中指定了我们刚才创建的pvc，看claimName这行，对应pvc中的name。

最后创建:jenkins_service.yaml文件：

--- apiVersion: v1 kind: Service metadata: namespace: ci name: jenkins labels: name: jenkins spec: ports: - port: 8080 targetPort: 8080 name: jenkins protocol: TCP selector: app: jenkins type: NodePort

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

--- apiVersion: v1 kind: Service metadata:   namespace: ci   name: jenkins   labels:     name: jenkins spec:   ports:   - port: 8080     targetPort: 8080     name: jenkins     protocol: TCP   selector:     app: jenkins   type: NodePort

这里我们选择NodePort来暴露服务端口，方便测试验证。

到此部署就结束了，在K8s上部署应用就是写资源清单的过程，在实际部署过程中出现了Jenkins启动错误：

backoffrestarfailling

1	backoffrestarfailling

查日志：

kubectl logs jenkins-7c97f9d6d9-q6fk9 -n ci touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?

1 2 3

kubectl logs jenkins-7c97f9d6d9-q6fk9  -n ci touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?

这是因为权限问题导致的，因为jenkins 运行的用户 id是1000，所以加上权限后解决。

获取jenkins初始密码：

进入容器： kubectl exec -it -n ci jenkins-bff9cccdd-45jzx – bash more /var/jenkins_home/secrets/initialAdminPassword #查看初始密码

1 2 3 4

进入容器： kubectl exec -it -n ci jenkins-bff9cccdd-45jzx – bash   more /var/jenkins_home/secrets/initialAdminPassword  #查看初始密码

登录访问：