Gitlab持续集成CI CD(Gitlab Runner篇)

1. GitLab Runner 简介

一般来说，构建任务都会占用很多的系统资源 (譬如编译代码)，而 GitLab CI 又是 GitLab 的一部分，如果由 GitLab CI 来运行构建任务的话，在执行构建任务的时候，GitLab 的性能会大幅下降。

GitLab CI 最大的作用是管理各个项目的构建状态，因此，运行构建任务这种浪费资源的事情就交给 GitLab Runner 来做！

因为 GitLab Runner 可以安装到不同的机器上，所以在构建任务运行期间并不会影响到 GitLab 的性能。

本文主要内容：基于Docker按照gitlab-runner，并集成maven/jdk/docker/docker-compose

2. 基于 Docker 安装 GitLab Runner

环境

  • Ubuntu: 18.04.3 LTS
  • Docker: 19.03.3, build a872fc2f86
  • Docker-compose: 1.24.0, build 0aa59064
  • Harbor: v1.9.0 （可用nexus或者registry替代）
  • Gilab中文社区版: twang2218/gitlab-ce-zh:11.1.4

2.1 准备工作

  • 创建工作目录

mkdir -p /usr/local/docker/runner

  • 创建构建目录

mkdir -p /usr/local/docker/runner/environment

  • 下载 jdk-8u152-linux-x64.tar.gz

  • 下载 apache-maven-3.6.1-bin.tar.gz

  • cp jdk-8u152-linux-x64.tar.gz /usr/local/docker/runner/environment

  • cp apache-maven-3.6.1-bin.tar.gz /usr/local/docker/runner/environment

    可选：若有nexus伺服

    配置maven的setting.xml文件至/usr/local/docker/runner/environment目录下

2.2 daemon.json

在 /usr/local/docker/runner/environment 目录下创建 daemon.json，用于配置镜像加速器和仓库地址

1
2
3
4
5
6
7
8
9
10
sudo tee /usr/local/docker/runner/environment/daemon.json <<-'EOF'
{
  "registry-mirrors": [
  "https://registry.docker-cn.com" #docker-cn加速器
  ],
  "insecure-registries": [
    "192.168.10.133:5000"   #harbor仓库地址
  ]
}
EOF

推荐配置阿里云镜像加速器，可参考Linux安装Dokcer CE 3.5)

2.3 Dockerfile

在/usr/local/docker/runner/environment 目录下创建 Dockerfile

vim /usr/local/docker/runner/environment/Dockerfile

内容如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
FROM gitlab/gitlab-runner
MAINTAINER EricShen <ahsbt@126.com>

# 修改软件源
RUN echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial main restricted universe multiverse' > /etc/apt/sources.list && \
    echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial-security main restricted universe multiverse' >> /etc/apt/sources.list && \
    echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main restricted universe multiverse' >> /etc/apt/sources.list && \
    echo 'deb http://mirrors.aliyun.com/ubuntu/ xenial-backports main restricted universe multiverse' >> /etc/apt/sources.list && \
    apt-get update -y && \
    apt-get clean

# 通过官方脚本安装 Docker
RUN curl -fsSL get.docker.com -o get-docker.sh && \
    sh get-docker.sh --mirror Aliyun
COPY daemon.json /etc/docker/daemon.json

# 安装 Docker Compose
WORKDIR /usr/local/bin
RUN wget https://raw.githubusercontent.com/topsale/resources/master/docker/docker-compose
RUN chmod +x docker-compose

# 安装 Java
RUN mkdir -p /usr/local/java
WORKDIR /usr/local/java
COPY jdk-8u152-linux-x64.tar.gz /usr/local/java
RUN tar -zxvf jdk-8u152-linux-x64.tar.gz && \
    rm -fr jdk-8u152-linux-x64.tar.gz

# 安装 Maven
RUN mkdir -p /usr/local/maven
WORKDIR /usr/local/maven
# 若未下载maven，此处可下载
# RUN wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.1/apache-maven-3.6.1-bin.tar.gz
COPY apache-maven-3.6.1-bin.tar.gz /usr/local/maven
RUN tar -zxvf apache-maven-3.6.1-bin.tar.gz && \
    rm -fr apache-maven-3.6.1-bin.tar.gz
# 可选：配置maven的setting.xml
# COPY settings.xml /usr/local/maven/apache-maven-3.6.1/conf/settings.xml

# 配置环境变量
ENV JAVA_HOME /usr/local/java/jdk1.8.0_152
ENV MAVEN_HOME /usr/local/maven/apache-maven-3.6.1
ENV PATH $PATH:$JAVA_HOME/bin:$MAVEN_HOME/bin

WORKDIR /

2.4 docker-compose.yml

在 /usr/local/docker/runner 目录下创建 docker-compose.yml

1
2
3
4
5
6
7
8
9
10
version: '3.1'
services:
  gitlab-runner:
    build: environment #指定构建路径
    restart: always
    container_name: gitlab-runner
    privileged: true
    volumes:
      - ./config:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock

2.5 构建镜像

确认所需文件

ls /usr/local/docker/runner 有

  • docker-compose.yml文件

ls /usr/local/docker/runner/environment有

  • apache-maven-3.6.1-bin.tar.gz

  • daemon.json

  • Dockerfile

  • jdk-8u152-linux-x64.tar.gz

  • setting.xml

构建镜像

1
2
cd /usr/local/docker/runner
docker-compose build
1
2
3
...省略...
Successfully build xxxxxxx
Successfully tagged runner_gitlab-runner # 构建成功
1
2
3
docker images #查看镜像
REPOSITORY                               TAG                 IMAGE ID            CREATED             SIZE
runner_gitlab-runner                     latest              74b8379cca15        41 hours ago        1.37GB

3. 注册 Runner

首先获取Gitlab上项目路径=>设置=>CI/CD=>Runner=>Setup a specific Runner manually下Gitlab地址和token

  • Gitlab中文社区版

Gitlab中文社区版

  • 英文原版Gitlab社区版

英文原版Gitlab社区版

然后到服务器中执行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# 执行
docker exec -it gitlab-runner gitlab-runner register 

# 输入 GitLab 地址
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
http://192.192.1.12/

# 输入 GitLab Token
Please enter the gitlab-ci token for this runner:
PgC_sRjg3qZgrU4WGvjY

# 输入 Runner 的描述
Please enter the gitlab-ci description for this runner:
# 可以为空

# 设置 Tag，可以用于指定在构建规定的 tag 时触发 ci
Please enter the gitlab-ci tags for this runner (comma separated):
deploy

Registering runner... succeeded                     runner=PgC_sRjg
# 选择 runner 执行器，这里我们选择的是 shell
Please enter the executor: virtualbox, docker+machine, parallels, shell, ssh, docker-ssh+machine, kubernetes, docker, docker-ssh:
shell

Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

4. 使用 Runner

进入你想要进行CI/CD的项目中（FYI: 通过Spring Initializr快速构架一个基于maven构建的Spring Boot项目）

  • 编写Dockerfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
FROM openjdk:8-jre  #基于openjdk构建镜像
MAINTAINER EricShen <ahsbt@126.com>

ENV APP_VERSION 1.0.0-SNAPSHOT
ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz

RUN mkdir /app

COPY test-$APP_VERSION.jar /app/app.jar
ENTRYPOINT ["dockerize", "-timeout", "5m", "-wait", "tcp://192.192.1.12:3306", "java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/app/app.jar"]

EXPOSE 8080
  • 编写docker-compose.yml
1
2
3
4
5
6
7
version: '3.1'
services:
  test_java:
    image: 192.192.1.12:9090/test/test:v1.0.0
    container_name: test_container
    ports:
      - 8080:8080
  • 编写.gitlab-ci.yml文件，添加到项目根路径下
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
stages:
  - build
  - push
  - run
  - clean

#构建
build:
  stage: build
  script:
    - /usr/local/maven/apache-maven-3.6.1/bin/mvn clean package # mvn 打jar包
    - cp target/test-1.0.0-SNAPSHOT.jar docker 
    - cd docker
    - docker build -t 192.192.1.12:9090/test/test:v1.0.0 . #构建镜像

#推送
push:
  stage: push
  script:
    - docker login 192.192.1.12:9090 -u admin -p Harbor12345 # 登陆Harbor仓库
    - docker push 192.192.1.12:9090/test/test:v1.0.0 # 推送镜像至Harbor

#运行
run:
  stage: run
  script:
    - cd docker
    - docker-compose down # 关闭容器
    - docker-compose up -d # 启动容器

#清理
clean:
  stage: clean
  script:
    - docker image prune -f # 清理虚悬镜像

img

5. 测试集成效果

所有操作完成后 push 代码到服务器，Gitlab会自动进行构建，在CI/CD的Pipeline中查看构建的结果：

image.png

  • 点击进入查看每条Pipeline的详情，由下图可见Pipeline有4个Stages，build=>push=>run=>clean

image.png

  • 此时可以看到构建完成，通过访问8080端口即可查看项目已经成功启动

6. 其它命令

  • 删除注册信息
    docker exec -it gitlab-runner gitlab-runner --name "名称"

  • 删除所有注册信息

    docker exec -it gitlab-runner gitlab-runner unregister --all-runners

  • 查看注册列表
    docker exec -it gitlab-runner gitlab-runner list