2023-09-15
发表 yhan219 学习17 分钟读完 (大约2606个字)0次访问

基于gitea+K3s实现DevOps/CI/CD

对于需要自建Git服务实现CI/CD及DevOps的，gitea无疑是个非常好的选择。而对于小型运维团队来说，使用K3s能减少大量的维护成本。

gitea和K3s都是非常轻量和易于安装的。

gitea文档：https://docs.gitea.com/zh-cn/

K3s文档：https://docs.k3s.io/zh/

安装gitea

参考文档：https://docs.gitea.com/zh-cn/installation/install-with-docker

1
2
3
cd /usr/local
mkdir gitea
vim docker-compose.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
version: "3"

networks:
  gitea:
    external: false

services:
  server:
    image: gitea/gitea:1.20.4
    container_name: gitea
    environment:
      - USER_UID=1000
      - USER_GID=1000
    restart: always
    networks:
      - gitea
    volumes:
      - ./gitea:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "222:22"
1
2
# 启动gitea
docker compose up -d

启用action

1
2
3
4
5
6
7
8
vim gitea/gitea/conf/app.ini
------
#末尾添加
[actions]
ENABLED=true
------
## 重启gitea
docker restart gitea

访问http://your-ip:3000 ,配置管理员账号密码和其他相应内容，点击安装即可。安装完成后如下：

image-20230915150958396

建议将服务暴漏为https域名，更能节省时间。以下以https://gitea.mydomain.com为例。

安装act runner

参考文档：https://docs.gitea.com/zh-cn/usage/actions/act-runner

获取token

到管理后台-actions-Runner中，点击创建runner，复制token，如下

image-20230915151107307

1
2
# 例如如下token
AaJkPUFeQs37hpGm6SHUIqHx3Tw8kmm0xX5gJv6f

注册runner

1
2
3
mkdir runner
cd runner
vim docker-compose.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
version: "3.8"
services:
  runner:
    image: gitea/act_runner:nightly
    container_name: gitea_runner
    restart: always
    environment:
      CONFIG_FILE: /config.yaml
      GITEA_INSTANCE_URL: "https://your-ip:3000"
      GITEA_RUNNER_REGISTRATION_TOKEN: "AaJkPUFeQs37hpGm6SHUIqHx3Tw8kmm0xX5gJv6f"
      GITEA_RUNNER_NAME: "my-runner"
      GITEA_RUNNER_LABELS: "${RUNNER_LABELS}"#需要特殊环境部署的，可以修改该标签，或在web控制台修改
    volumes:
      - ./config.yaml:/config.yaml
      - ./data:/data
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "8088:8088"

为加快构建速度，一般来说我们需要允许缓存：创建config.yaml，允许缓存，请注意修改your-ip为runner所在服务器的内网ip。

1
vim config.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
# Example configuration file, it's safe to copy this as the default config file without any modification.

# You don't have to copy this file to your instance,
# just run `./act_runner generate-config > config.yaml` to generate a config file.

log:
  # The level of logging, can be trace, debug, info, warn, error, fatal
  level: info

runner:
  # Where to store the registration result.
  file: .runner
  # Execute how many tasks concurrently at the same time.
  capacity: 1
  # Extra environment variables to run jobs.
  envs:
    A_TEST_ENV_NAME_1: a_test_env_value_1
    A_TEST_ENV_NAME_2: a_test_env_value_2
    RUNNER_TOOL_CACHE: /toolcache
  # Extra environment variables to run jobs from a file.
  # It will be ignored if it's empty or the file doesn't exist.
  env_file: .env
  # The timeout for a job to be finished.
  # Please note that the Gitea instance also has a timeout (3h by default) for the job.
  # So the job could be stopped by the Gitea instance if it's timeout is shorter than this.
  timeout: 3h
  # Whether skip verifying the TLS certificate of the Gitea instance.
  insecure: false
  # The timeout for fetching the job from the Gitea instance.
  fetch_timeout: 5s
  # The interval for fetching the job from the Gitea instance.
  fetch_interval: 2s
  # The labels of a runner are used to determine which jobs the runner can run, and how to run them.
  # Like: ["macos-arm64:host", "ubuntu-latest:docker://node:16-bullseye", "ubuntu-22.04:docker://node:16-bullseye"]
  # If it's empty when registering, it will ask for inputting labels.
  # If it's empty when execute `deamon`, will use labels in `.runner` file.
  labels: []

cache:
  # Enable cache server to use actions/cache.
  enabled: true
  # The directory to store the cache data.
  # If it's empty, the cache data will be stored in $HOME/.cache/actcache.
  dir: ""
  # The host of the cache server.
  # It's not for the address to listen, but the address to connect from job containers.
  # So 0.0.0.0 is a bad choice, leave it empty to detect automatically.
  host: "your-ip"
  # The port of the cache server.
  # 0 means to use a random available port.
  port: 8088
  # The external cache server URL. Valid only when enable is true.
  # If it's specified, act_runner will use this URL as the ACTIONS_CACHE_URL rather than start a server by itself.
  # The URL should generally end with "/".
  external_server: ""

container:
  # Specifies the network to which the container will connect.
  # Could be host, bridge or the name of a custom network.
  # If it's empty, act_runner will create a network automatically.
  network: ""
  # Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker).
  privileged: false
  # And other options to be used when the container is started (eg, --add-host=my.gitea.url:host-gateway).
  options:
  # The parent directory of a job's working directory.
  # If it's empty, /workspace will be used.
  workdir_parent:
  # Volumes (including bind mounts) can be mounted to containers. Glob syntax is supported, see https://github.com/gobwas/glob
  # You can specify multiple volumes. If the sequence is empty, no volumes can be mounted.
  # For example, if you only allow containers to mount the `data` volume and all the json files in `/src`, you should change the config to:
    # valid_volumes:
  #   - data
  #   - /src/*.json
  # If you want to allow any volume, please use the following configuration:
  # valid_volumes:
  #   - '**'
  valid_volumes: []
  # overrides the docker client host with the specified one.
  # If it's empty, act_runner will find an available docker host automatically.
  # If it's "-", act_runner will find an available docker host automatically, but the docker host won't be mounted to the job containers and service containers.
  # If it's not empty or "-", the specified docker host will be used. An error will be returned if it doesn't work.
  docker_host: ""
  # Pull docker image(s) even if already present
  force_pull: false

host:
  # The parent directory of a job's working directory.
  # If it's empty, $HOME/.cache/act/ will be used.
  workdir_parent:
1
docker compose up -d

重新刷新页面，可看到新的runner

runner

安装k3s

参考文档https://docs.k3s.io/zh/quick-start

在需要部署应用的服务器(例如192.168.0.100)上安装K3s,安装命令

1
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

如果暴漏为https域名，可以省略以下步骤

私有仓库镜像配置

如果是配置了https域名的可以忽略本步

docker允许访问gitea仓库

1
vim /etc/docker/daemon.json 
1
2
3
4
5
6
7
{
  "log-driver": "json-file",
  "log-opts": {"max-size": "100m", "max-file": "100"},
  "insecure-registries": [
   "your-ip:3000"
  ]
}

k3s允许访问gitea

1
vim /etc/rancher/k3s/registries.yaml
1
2
3
4
5
6
7
8
9
mirrors:
  gitea.mydomain.com:
    endpoint:
      - "http://your-ip:3000"
configs:
  "your-ip:3000":
    auth:
      username: your-gitea-name # this is the registry username
      password: your-gitea-password # this is the registry password
1
2
systemctl restart docker
systemctl restart k3s

持续集成

以下步骤将会以ruoyi-vue-plus为例。

启用action

点击右上角+-迁移外部仓库将ruoyi-vue-plus仓库迁移到本地gitea中。待迁移完成后，点击项目设置-仓库-启用actions,点击更新仓库设置按钮，开启项目action.

image-20230915152636599

image-20230915152859461

获取gitea部署token

在个人信息及配置-应用中生成一个token，注意权限选择repository和user都是read，其他不选。

请注意令牌只会显示一次，复制保管好。

image-20230915153622434

添加CI文件

以最新分支5.X为例，切换到5.X分支，在项目中添加.gitea/workflows/main.yml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
name: Ruoyi Vue Plus
run-name: CI/CD runner
on:
  push:
    # Sequence of patterns matched against refs/heads
    branches:
      - 5.X

jobs:
  ci-and-cd:
    runs-on: ubuntu-latest
    container:
      image: catthehacker/ubuntu:act-latest
    env:
      # cache bug https://gitea.com/gitea/act_runner/issues/70
      RUNNER_TOOL_CACHE: /toolcache
      REGISTRY: gitea.mydomain.com
      APP_NAME: ruoyi-admin
    steps:
      - name: checkout from git
        uses: actions/checkout@v3
      - name: Set up Java
        uses: graalvm/setup-graalvm@v1
        with:
          java-version: '17.0.7'
          distribution: 'graalvm'
          cache: 'maven'
      - name: Build with Maven
        run: mvn package -DskipTests -Pprod
      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ env.REGISTRY }}/devops/${{ env.APP_NAME }}
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_PASSWORD }}
      - name: Build and push
        uses: docker/build-push-action@master
        with:
          context: ./ruoyi-admin
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
      - name: k8s Check and Apply New Deployment
        uses: actions-hub/kubectl@master
        env:
          KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }}
        with:
          args: apply -f http://${{ secrets.DEVOPS_GITEA_TOKEN }}@${{ env.REGISTRY }}/devops/ruoyi-vue-plus/raw/branch/master/.deploy/deployment.yml
      - name: k8s Update Deployment
        uses: actions-hub/kubectl@master
        env:
          KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }}
        with:
          args: rollout restart deployment ${{ env.APP_NAME }}-${{ gitea.ref_name }}

设置action密钥

可以在个人中心，action中设置全局的，也可以在仓库中设置对应仓库的。

添加docker hub配置

在actions中，添加密钥，名称为DOCKERHUB_USERNAME和DOCKERHUB_PASSWORD，分别对应你的gitea账号和密码。添加DEVOPS_GITEA_TOKEN，即上一步得到的token。

添加kube-config配置

获取kube-config，默认放在/etc/rancher/k3s/k3s.yaml，复制内容，并修改ip为192.168.0.100，

1
2
3
4
5
6
cp /etc/rancher/k3s/k3s.yaml myk3s.yaml
vim myk3s.yaml
# 修改 server: https://127.0.0.1:6443 为  server: https://192.168.0.100:6443

# 获取KUBE_CONFIG
cat myk3s.yaml | base64

添加acitons密钥KUBE_CONFIG,值为上面输出的Base64

image-20230921105647611

创建k8s secret

1
2
kubectl create secret docker-registry dockerhub-id --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>

创建deployment

在.deploy目录下创建文件deployment.yml文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
---
## k8s默认是访问不了外部服务的，需要代理,以mysql和redis为例，代理myservice名称到192.168.0.100，配置文件中对应的ip修改为myservice
apiVersion: v1
kind: Endpoints
metadata:
  name: myservice
subsets:
  - addresses:
      - ip: 192.168.0.100
    ports:
      - port: 6379
        protocol: TCP
        name: redis
      - port: 3306
        protocol: TCP
        name: mysql
---
apiVersion: v1
kind: Service
metadata:
  name: myservice
spec:
  ports:
    - port: 6379
      targetPort: 6379
      protocol: TCP
      name: redis
    - port: 3306
      targetPort: 3306
      protocol: TCP
      name: mysql

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ruoyi-admin
spec:
  replicas: 2
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: ruoyi-admin
  template:
    metadata:
      labels:
        app: ruoyi-admin
    spec:
      containers:
        - image: gitea.mydomain.com/devops/ruoyi-admin:5.X
          name: ruoyi-admin
          imagePullPolicy: Always
          readinessProbe:
            httpGet:
              path: /actuator/health
              port: 8080
            initialDelaySeconds: 1
            successThreshold: 5
          ports:
            - containerPort: 8080
      imagePullSecrets:
        - name: dockerhub-id

---

apiVersion: v1
kind: Service
metadata:
  name: ruoyi-admin
spec:
  type: NodePort
  externalTrafficPolicy: Local
  ports:
    - name: http-8080
      protocol: TCP
      port: 8080
      nodePort: 30088
  selector:
    app: ruoyi-admin

推送并部署

提交代码，查看actions是否成功。运行成功后，访问http://192.168.0.100:30088 即可访问成功。

image-20230915160639074

Q&A

Q:actions中的Set up job慢，github访问超时

A:将github action仓库导入到本地gitea，修改ci文件为本地url，例如checkout，将 https://github.com/actions/checkout.git 同步到本地，修改ci为

1
2
3
steps:
  - name: checkout from git
    uses: https://gitea.mydomain.com/devops/checkout@v3

其他同理。

Q：我需要部署的服务器是私有的，需要VPN才能访问，怎么办？

A：如果该私有服务器能访问到gitea，只需要在该私有服务器上部署一个runner，修改该runner的GITEA_RUNNER_LABELS标签，例如GITEA_RUNNER_LABELS: test-runner然后修改ci文件，指定cd步骤为test-runner即可。

ci:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
name: Ruoyi Vue Plus
run-name: CI/CD runner
on:
  push:
    # Sequence of patterns matched against refs/heads
    branches:
      - 5.X

jobs:
  ci:
    runs-on: ubuntu-latest
    container:
      image: catthehacker/ubuntu:act-latest
    env:
      # cache bug https://gitea.com/gitea/act_runner/issues/70
      RUNNER_TOOL_CACHE: /toolcache
      REGISTRY: gitea.mydomain.com
      APP_NAME: ruoyi-admin
    steps:
      - name: checkout from git
        uses: actions/checkout@v3
      - 同上，将Deploy to k8s步骤移到后面
  cd:
    needs: ci
    runs-on: test-runner # 指定runner
    container:
      image: catthehacker/ubuntu:act-latest
    env:
      # cache bug https://gitea.com/gitea/act_runner/issues/70
      RUNNER_TOOL_CACHE: /toolcache
      REGISTRY: gitea.mydomain.com
      APP_NAME: ruoyi-admin
    steps:
      - name: k8s Check and Apply New Deployment
        uses: actions-hub/kubectl@master
        env:
          KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }}
        with:
          args: apply -f http://${{ secrets.DEVOPS_GITEA_TOKEN }}@${{ env.REGISTRY }}/devops/ruoyi-vue-plus/raw/branch/master/.deploy/deployment.yml
      - name: k8s Update Deployment
        uses: actions-hub/kubectl@master
        env:
          KUBE_CONFIG: ${{ secrets.KUBE_CONFIG }}
        with:
          args: rollout restart deployment ${{ env.APP_NAME }}-${{ gitea.ref_name }}


基于gitea+K3s实现DevOps/CI/CD

https://blog.yhan219.com/gitea-k3s-devops/

作者

yhan219

发布于

2023-09-15

更新于

2023-09-15

许可协议

 

喜欢这篇文章？打赏一下作者吧

关注我

分类

最新文章

归档

标签

DateGrip1
Idea1
IntelliJ IDEA2
Jetbrains2
Redis1
VPN1
acme.sh1
alfred1
alpine1
arthas1
bash3
centos8
devops2
docker5
docker compose2
gitea1
http1
https1
idea2
java3
linux2
mysql2
navicat1
nginx5
rabbitmq2
redis3
sdkman1
seaweedfs1
shell7
show hosts1
spring boot1
ssh3
systemctl2
systemd2
vpncmd1
vps1
windows1
workflow1
xxl-job1
zookeeper1
中间件1
云盘1
免费证书1
内网穿透1
分布式1
分布式锁1
命令1
命令行6
小文件服务器1
数据库3
日志1
时区1
时间1
服务器1
消息队列1
激活2
索引1
缓存1
运维1
锁1
问题排查1
限流1
×