KubernetesにJenkinsをインストールする

Kubernetes上にJenkinsをインストールする方法を説明します。前回「VirtualBoxでKubernetes環境構築」の応用編になります。

今回の目的はKubernetes上にJenkinsをインストールし、以下の仕様を達成することです：

  • JenkinsのジョブでDockerが使えること
  • Jenkinsのデータを簡単にバックアップできること (VirtualBoxの仮想HDDを利用)

まず最初に、Jenkinsのデータを保存するVolumeのための仮想HDDを作成します。VirtualBoxで以下の仮想HDDを3個作成します：

  • Jenkins Home用 : 32 GiB
  • Jenkins Workspace用 : 128 GiB
  • Jenkins Docker認証書用 : 1 GiB

そしてこの仮想HDDをJenkinsを運用するKubernetesのWorker Nodeにマウントし、フォーマットしておきます。(/etc/fstabで再起動しても自動でマウントするようにしておくのも忘れずに)

  • /mnt/volumes/jenkins-home
  • /mnt/volumes/jenkins-workspace
  • /mnt/volumes/jenkins-docker-certs

では本格的にKubernetesの設定を行います。まずはPersistentVolumeを作ります。PersistentVolumeをマウントするWorker Nodeの名前は「k8s-node-0」という前提です。

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-home-pv
labels:
volume-name: jenkins-home
spec:
capacity:
storage: 32Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /mnt/volumes/jenkins-home
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-node-0
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-workspace-pv
labels:
volume-name: jenkins-workspace
spec:
capacity:
storage: 128Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /mnt/volumes/jenkins-workspace
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-node-0
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-docker-certs-pv
labels:
volume-name: jenkins-docker-certs
spec:
capacity:
storage: 1Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /mnt/volumes/jenkins-docker-certs
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-node-0
apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-home-pv labels: volume-name: jenkins-home spec: capacity: storage: 32Gi volumeMode: Filesystem accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: local-storage local: path: /mnt/volumes/jenkins-home nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - k8s-node-0 --- apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-workspace-pv labels: volume-name: jenkins-workspace spec: capacity: storage: 128Gi volumeMode: Filesystem accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: local-storage local: path: /mnt/volumes/jenkins-workspace nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - k8s-node-0 --- apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-docker-certs-pv labels: volume-name: jenkins-docker-certs spec: capacity: storage: 1Gi volumeMode: Filesystem accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: local-storage local: path: /mnt/volumes/jenkins-docker-certs nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - k8s-node-0
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-home-pv
  labels:
    volume-name: jenkins-home
spec:
  capacity:
    storage: 32Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /mnt/volumes/jenkins-home
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-node-0
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-workspace-pv
  labels:
    volume-name: jenkins-workspace
spec:
  capacity:
    storage: 128Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /mnt/volumes/jenkins-workspace
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-node-0
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-docker-certs-pv
  labels:
    volume-name: jenkins-docker-certs
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /mnt/volumes/jenkins-docker-certs
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-node-0

上記をjenkins-volume.yamlに保存し、Kubernetesに反映します。

kubectl apply -f jenkins-volume.yaml

これでJenkinsのデータを保存するためのPersistentVolumeが作られました。次にJenkins本体とDocker実行のためのコンテナが入ったPodを作ります。Jenkinsは再起動後も各種設定やデータを保持している必要があるのでStatefulSetで作ります。

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
apiVersion: v1
kind: Namespace
metadata:
name: jenkins
---
apiVersion: v1
kind: Service
metadata:
name: jenkins
namespace: jenkins
labels:
app: jenkins
spec:
type: NodePort
ports:
- nodePort: 30000
port: 8080
targetPort: 8080
protocol: TCP
name: http
selector:
app: jenkins
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: jenkins
namespace: jenkins
spec:
serviceName: "jenkins"
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
securityContext:
fsGroup: 1000
containers:
- name: jenkins-blueocean
image: jenkinsci/blueocean
ports:
- containerPort: 8080
name: http
- containerPort: 50000
name: inbound
env:
- name: DOCKER_HOST
value: "tcp://localhost:2376"
- name: DOCKER_CERT_PATH
value: "/certs/client"
- name: DOCKER_TLS_VERIFY
value: "1"
- name: JENKINS_OPTS
value: "--prefix=/jenkins"
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
- name: jenkins-workspace
mountPath: /var/jenkins_home/workspace
- name: jenkins-docker-certs
mountPath: /certs/client
readOnly: true
livenessProbe:
httpGet:
path: /jenkins/login
port: 8080
initialDelaySeconds: 300
periodSeconds: 60
timeoutSeconds: 30
- name: jenkins-docker
image: docker:dind
securityContext:
privileged: true
ports:
- containerPort: 2376
name: docker-daemon
env:
- name: DOCKER_TLS_CERTDIR
value: "/certs"
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
- name: jenkins-workspace
mountPath: /var/jenkins_home/workspace
- name: jenkins-docker-certs
mountPath: /certs/client
volumeClaimTemplates:
- metadata:
name: jenkins-home
spec:
selector:
matchLabels:
volume-name: jenkins-home
accessModes: [ "ReadWriteOnce" ]
storageClassName: "local-storage"
resources:
requests:
storage: 32Gi
- metadata:
name: jenkins-workspace
spec:
selector:
matchLabels:
volume-name: jenkins-workspace
accessModes: [ "ReadWriteOnce" ]
storageClassName: "local-storage"
resources:
requests:
storage: 128Gi
- metadata:
name: jenkins-docker-certs
spec:
selector:
matchLabels:
volume-name: jenkins-docker-certs
accessModes: [ "ReadWriteOnce" ]
storageClassName: "local-storage"
resources:
requests:
storage: 1Gi
apiVersion: v1 kind: Namespace metadata: name: jenkins --- apiVersion: v1 kind: Service metadata: name: jenkins namespace: jenkins labels: app: jenkins spec: type: NodePort ports: - nodePort: 30000 port: 8080 targetPort: 8080 protocol: TCP name: http selector: app: jenkins --- apiVersion: apps/v1 kind: StatefulSet metadata: name: jenkins namespace: jenkins spec: serviceName: "jenkins" replicas: 1 selector: matchLabels: app: jenkins template: metadata: labels: app: jenkins spec: securityContext: fsGroup: 1000 containers: - name: jenkins-blueocean image: jenkinsci/blueocean ports: - containerPort: 8080 name: http - containerPort: 50000 name: inbound env: - name: DOCKER_HOST value: "tcp://localhost:2376" - name: DOCKER_CERT_PATH value: "/certs/client" - name: DOCKER_TLS_VERIFY value: "1" - name: JENKINS_OPTS value: "--prefix=/jenkins" volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home - name: jenkins-workspace mountPath: /var/jenkins_home/workspace - name: jenkins-docker-certs mountPath: /certs/client readOnly: true livenessProbe: httpGet: path: /jenkins/login port: 8080 initialDelaySeconds: 300 periodSeconds: 60 timeoutSeconds: 30 - name: jenkins-docker image: docker:dind securityContext: privileged: true ports: - containerPort: 2376 name: docker-daemon env: - name: DOCKER_TLS_CERTDIR value: "/certs" volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home - name: jenkins-workspace mountPath: /var/jenkins_home/workspace - name: jenkins-docker-certs mountPath: /certs/client volumeClaimTemplates: - metadata: name: jenkins-home spec: selector: matchLabels: volume-name: jenkins-home accessModes: [ "ReadWriteOnce" ] storageClassName: "local-storage" resources: requests: storage: 32Gi - metadata: name: jenkins-workspace spec: selector: matchLabels: volume-name: jenkins-workspace accessModes: [ "ReadWriteOnce" ] storageClassName: "local-storage" resources: requests: storage: 128Gi - metadata: name: jenkins-docker-certs spec: selector: matchLabels: volume-name: jenkins-docker-certs accessModes: [ "ReadWriteOnce" ] storageClassName: "local-storage" resources: requests: storage: 1Gi
apiVersion: v1
kind: Namespace
metadata:
  name: jenkins
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: jenkins
  labels:
    app: jenkins
spec:
  type: NodePort
  ports:
  - nodePort: 30000
    port: 8080
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: jenkins
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  namespace: jenkins
spec:
  serviceName: "jenkins"
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      securityContext:
        fsGroup: 1000
      containers:
      - name: jenkins-blueocean
        image: jenkinsci/blueocean
        ports:
        - containerPort: 8080
          name: http
        - containerPort: 50000
          name: inbound
        env:
        - name: DOCKER_HOST
          value: "tcp://localhost:2376"
        - name: DOCKER_CERT_PATH
          value: "/certs/client"
        - name: DOCKER_TLS_VERIFY
          value: "1"
        - name: JENKINS_OPTS
          value: "--prefix=/jenkins"
        volumeMounts:
        - name: jenkins-home
          mountPath: /var/jenkins_home
        - name: jenkins-workspace
          mountPath: /var/jenkins_home/workspace
        - name: jenkins-docker-certs
          mountPath: /certs/client
          readOnly: true
        livenessProbe:
          httpGet:
            path: /jenkins/login
            port: 8080
          initialDelaySeconds: 300 
          periodSeconds: 60
          timeoutSeconds: 30
      - name: jenkins-docker
        image: docker:dind
        securityContext:
          privileged: true
        ports:
        - containerPort: 2376
          name: docker-daemon
        env:
        - name: DOCKER_TLS_CERTDIR
          value: "/certs"
        volumeMounts:
        - name: jenkins-home
          mountPath: /var/jenkins_home
        - name: jenkins-workspace
          mountPath: /var/jenkins_home/workspace
        - name: jenkins-docker-certs
          mountPath: /certs/client
  volumeClaimTemplates:
  - metadata:
      name: jenkins-home
    spec:
      selector:
        matchLabels:
          volume-name: jenkins-home
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-storage"
      resources:
        requests:
          storage: 32Gi
  - metadata:
      name: jenkins-workspace
    spec:
      selector:
        matchLabels:
          volume-name: jenkins-workspace
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-storage"
      resources:
        requests:
          storage: 128Gi
  - metadata:
      name: jenkins-docker-certs
    spec:
      selector:
        matchLabels:
          volume-name: jenkins-docker-certs
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-storage"
      resources:
        requests:
          storage: 1Gi

上記をjenkins.yamlで保存し、Kubernetesに反映します。

kubectl apply -f jenkins.yaml

特に問題が起きてない場合、しばらくするとJenkinsのPodが起動します。Jenkinsのコンテナから、Jenkinsの初期セットアップに必要なパスワードを取得します。

kubectl logs jenkins-0  jenkins-blueocean

JenkinsのURLにアクセスし、初期セットアップを行います。

  • http://[Worker Nodeの外部IP]:30000/jenkins

これでKubernetes上にJenkinsをセットアップできました。Reverse Proxyを立ててKubernetesのノードIPを隠蔽するとより使いやすくなるでしょう。

JenkinsのデータはVirtualBoxの仮想HDDになっているので、シンプルにHDDファイルをコピーすることでバックアップが取れます。

今回の手順はJenkinsをDockerにインストールする手順を参考にKubernetesで動くよう修正を加えたものです。

[おまけ]
今回使ったjenkins.yamlの簡単な解説です：

  • JenkinsのServiceを生成
    • NodePortタイプ。ノードIP:30000をJenkinsPodの8080ポートに転送
  • JenkinsのPodを生成
    • JenkinsがPersistentVolumeにデータを書き込めるようにfsGroupを1000に設定
    • Jenkins本体のコンテナ生成
      • アクセスURLに/jenkins/を付ける
      • 各種PersistentVolumeをマウントする
      • Docker In Dockerのホスト情報を環境変数として設定
      • LivenessProbeでもしJenkinsが落ちたら自動で再起動するように。初回起動の5分後から1分間隔でチェック。
      • LocalPersistentVolumeを使うようにVolumeClaim設定
    • Docker In Dockerコンテナ生成
      • 各種PersistentVolumeをマウントする
      • LocalPersistentVolumeを使うようにVolumeClaim設定

作成者: kkc0923

KANOTYPE管理者。

コメントする

メールアドレスが公開されることはありません。 ※ が付いている欄は必須項目です

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Loading...