Scripted deployment with ArgoCD UI (#5)

* initial import of deploy_jitsi.sh with argocd support
* automatic Let's Encrypt cert
* add INSTALL.md
pull/6/head
Wei He 4 years ago committed by GitHub
parent 77a22f90e3
commit 8db8cd7d9a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 83
      INSTALL.md
  2. 10
      argocd/cmd-params-cm.yaml
  3. 26
      argocd/ingressroute-server.yaml.sh
  4. 20
      bootstrap.sh
  5. 225
      deploy_jitsi.sh
  6. 6
      jitsi/Chart.lock
  7. 7
      jitsi/Chart.yaml
  8. BIN
      jitsi/charts/jitsi-meet-1.2.2.tgz
  9. 16
      jitsi/templates/ingressroute-web.yaml
  10. 12
      jitsi/templates/ingressroutetcp-prosody.yaml
  11. 5
      jitsi/values-jvb-off.yaml
  12. 41
      jitsi/values.yaml
  13. 36
      traefik-config.yaml.sh
  14. 12
      values.yml

@ -0,0 +1,83 @@
# Installation
The deployment script is largely based on the helm chart [jitsi-helm](https://github.com/jitsi-contrib/jitsi-helm/). The dependencies include [k3s](https://k3s.io/), [traefik](https://traefik.io/) and [argocd](https://argoproj.github.io/). It also uses [Let's Encrypt](https://letsencrypt.org/) for signing TLS certificates.
## Prerequisites
* A GNU/Linux host with Debian/Ubuntu installed and root privileges
* The host has a public IPv4 address <PUBLIC_IP>
* Allow these traffic through firewall: 80/TCP, 443/TCP, 5222/TCP (necessary only for external jvb), 30000/UDP (30001/UDP for test)
* An email inbox address <ACME_EMAIL> for receiving ACME notification mails
* A domain name <PROD_HOSTNAME> that has a DNS A record pointing to <PUBLIC_IP>
* (optional) Another domain name <TEST_HOSTNAME> (for test deployment purposes) that has a DNS A record pointing to <PUBLIC_IP>
* (optional) Yet another domain name <CD_HOSTNAME> (for accessing the ArgoCD web UI) that has a DNS A record pointing to <PUBLIC_IP>
## Install/Upgrade
The initial installation needs to be run from command line. But afterwards, ArgoCD web UI can be used instead to fulfill the subsequent (re)install/upgrade/uninstall needs. **All the shell commands need to be run with root user.** There are two installation modes, prod and test. By default, prod is installed. It can be switched to test by setting environment variable `TEST_INSTALL`.
### Install/Upgrade from command line
Run the following shell command by providing the 2 mandatory arguments: fully-qualified domain name for accessing jitsi web, and an email address for receiving Let's Enrypt's ACME mails.
```bash
curl -sL https://raw.githubusercontent.com/shanghailug/jitsi-deploy/main/deploy_jitsi.sh |
bash -s - <PROD_HOSTNAME> <ACME_EMAIL>
```
Alternatively, an additional environment variable `ARGOCD_FQDN` can be provided to enable ArgoCD web server's ingress, so that it can be accessed post installation, for future operations:
```bash
curl -sL https://raw.githubusercontent.com/shanghailug/jitsi-deploy/main/deploy_jitsi.sh |
ARGOCD_FQDN=<CD_HOSTNAME> bash -s - <PROD_HOSTNAME> <ACME_EMAIL>
```
Before committing to a prod installation, the whole setup can be tested by using a test hostname, only requesting certificates from staging instance of Let's Encrypt, and installing into `test` k8s namespace. This can be done by setting `TEST_INSTALL` and `STAGING_CERT` environment variable and giving test hostname as command argument, like this:
```bash
curl -sL https://raw.githubusercontent.com/shanghailug/jitsi-deploy/main/deploy_jitsi.sh |
TEST_INSTALL=1 STAGING_CERT=1 ARGOCD_FQDN=<CD_HOSTNAME> bash -s - <TEST_HOSTNAME> <ACME_EMAIL>
```
The installed applications can then be updated/upgraded by rerunning exactly the same command, when the git repo is updated or it's desirable to enable ArgoCD web after initial installation is done. The already installed components will usually be kept as-is if their versions match, or be upgraded otherwise. If k3s needs to be upgraded, however, it's probably a better idea to [tear down](#tear-down) the whole setup before-hand.
### Install/Upgrade from ArgoCD web UI
If the initial installation enabled ArgoCD web UI's ingress by providing the environment variable `ARGOCD_FQDN`, then the ArgoCD web server can be accessed via `https://${ARGOCD_FQDN}/`.
Please refer to [ArgoCD docs](https://argo-cd.readthedocs.io/en/stable/getting_started/#6-create-an-application-from-a-git-repository) for more details about how to create/update applications using helm charts from a git repo. The login's name is `admin` and the login's password can be retrieved after initial installation, by running the following command on the host:
```bash
kubectl -n argocd get secret/argocd-initial-admin-secret -o jsonpath='{ .data.password }' | base64 -d
```
## Settings
The following list of environment variables can be used to customize or alter the installation.
Environment Variable | Description | Default Value | Default behaviour
--- | --- | --- | ---
`ARGOCD_FQDN` | fully-qualified hostname for accessing ArgoCD web UI | "" | don't enable web ingress for ArgoCD server
`ARGOCD_VERSION` | argocd release to install | "v2.3.3" |
`DEPLOY_GIT_REPO` | the git repo url for retrieving artifacts | `https://github.com/shanghailug/jitsi-deploy.git` |
`DEPLOY_GIT_VERSION` | the revision of artifacts to checkout and use from the repo | "" | use the default branch when git repo is cloned locally
`EXCLUDE_JVB` | Exclude built-in jvb component (so that an external one can be registered for use) | "" | include jvb
`K3S_VERSION` | k3s release to install | "v1.23.6+k3s1" |
`TEST_INSTALL` | when set to non-empty, install an app called `jitsitest` into `test` k8s namespace | "" | install an app called `jitsi` into `prod` k8s namespace
## Uninstall
### Uninstall the applications
The applications can be uninstalled either from ArgoCD web UI or by running something like the following commands:
```bash
argocd app delete jitsi
argocd app delete jitsitest
```
### Tear down
```bash
/usr/local/bin/k3s-uninstall.sh
```

@ -0,0 +1,10 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cmd-params-cm
namespace: argocd
labels:
app.kubernetes.io/name: argocd-cmd-params-cm
app.kubernetes.io/part-of: argocd
data:
server.insecure: "true"

@ -0,0 +1,26 @@
cat <<EOF
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: argocd-server
namespace: argocd
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(\`${ARGOCD_FQDN}\`)
priority: 10
services:
- name: argocd-server
port: 80
- kind: Rule
match: Host(\`${ARGOCD_FQDN}\`) && Headers(\`Content-Type\`, \`application/grpc\`)
priority: 11
services:
- name: argocd-server
port: 80
scheme: h2c
tls:
certResolver: le-prod
EOF

@ -1,20 +0,0 @@
#!/bin/bash
# k3s and set current context as k3s , k3s use trafik as ingress controller by default.
(which k3s &> /dev/null && test -f /etc/rancher/k3s/k3s.yaml) || \
curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
kubectl get namespace jitsi &> /dev/null || \
sudo -E kubectl create namespace jitsi
sudo -E kubectl config set-context --current --namespace=jitsi
# helm
which helm &> /dev/null || \
curl -sfL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash -s -
# jitsi
sudo -E helm repo add jitsi https://jitsi-contrib.github.io/jitsi-helm/
sudo -E helm install shlug-jitsi jitsi/jitsi-meet -f values.yml -n jitsi

@ -0,0 +1,225 @@
#!/usr/bin/env bash
function err {
echo $1 1>&2
exit 1
}
# check usage
if [ $# -ne 2 ]; then
err "usage: $0 <fully-qualified-host-name> <acme_email_address>"
fi
# check sudo
if [ $EUID -ne 0 ]; then
err "sudo?"
fi
# host OS packages
apt update && apt -y install grep bind9-dnsutils iproute2 curl wget git
# parameters
export FQDN=$1
export ACME_EMAIL=$2
export PUBLIC_IP=$(nslookup ${FQDN} | grep -A1 Name: | grep Address: | cut -d' ' -f2)
if [ -z "${PUBLIC_IP}" ]; then
err "can't resolve hostname: ${FQDN}"
else
echo "resolved hostname '${FQDN}' to ip address ${PUBLIC_IP}"
fi
if ! (curl -s https://ipinfo.io/ip | grep -q ${PUBLIC_IP}); then
err "the host doesn't have such public ip: ${PUBLIC_IP}"
fi
if [ -n "${TEST_INSTALL}" ]; then
export HELM_NAME=jitsitest
export NAMESPACE=test
export JVB_PORT=30001
else
export HELM_NAME=jitsi
export NAMESPACE=prod
export JVB_PORT=30000
fi
# versions
K3S_VERSION=${K3S_VERSION:-"v1.23.6+k3s1"}
HELM_VERSION=${HELM_VERSION:-"v3.8.2"}
ARGOCD_VERSION=${ARGOCD_VERSION:-"v2.3.3"}
HELM_ARCHIVE="helm-${HELM_VERSION}-linux-amd64.tar.gz"
DEPLOY_GIT_REPO=${DEPLOY_GIT_REPO:-"https://github.com/shanghailug/jitsi-deploy.git"}
# workspace
WS_DIR=${HOME}/deploy/$(date +"%Y%m%d_%H%M%S")
SRC_DIR=${WS_DIR}/jitsi-deploy
mkdir -p ${WS_DIR}
function get_helm {
if ! which helm || ! ( helm version | grep -q ${HELM_VERSION} ); then
cd ${WS_DIR}/
wget https://get.helm.sh/${HELM_ARCHIVE}
tar -zxvf ${HELM_ARCHIVE}
mv $(find -type f -name helm) /usr/local/bin/
fi
}
function get_src {
cd ${WS_DIR}/
git clone ${DEPLOY_GIT_REPO}
cd $SRC_DIR/
if [ -n "${DEPLOY_GIT_VERSION}" ]; then
git checkout ${DEPLOY_GIT_VERSION}
fi
}
function do_k3s {
INSTALL_K3S=
# nuke
if [ -n "${NUKE_K3S}" ] && [ -f /usr/local/bin/k3s-uninstall.sh ]; then
/usr/local/bin/k3s-uninstall.sh
INSTALL_K3S=1
elif ! which k3s; then
INSTALL_K3S=1
fi
# install k3s
if [ -n "${INSTALL_K3S}" ]; then
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=${K3S_VERSION} INSTALL_K3S_EXEC="--tls-san ${PUBLIC_IP}" sh -
fi
echo -n "waiting for k3s server node to become ready ."
while ! (kubectl get node | grep -q -w Ready); do
echo -n "."
sleep 1
done
echo "ready."
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
kubectl get node -o wide
}
function get_argocd {
if ! which argocd || ! (argocd -n argocd version --client | grep ^argocd: | grep -q ${ARGOCD_VERSION}); then
cd ${WS_DIR}/
wget https://github.com/argoproj/argo-cd/releases/download/${ARGOCD_VERSION}/argocd-linux-amd64
chmod a+x argocd-linux-amd64
mv argocd-linux-amd64 /usr/local/bin/argocd
fi
}
function do_traefik {
cd ${SRC_DIR}/
./traefik-config.yaml.sh | kubectl apply -f -
echo -n "waiting for helm-install-traefik to become ready ."
while [ $(kubectl -n kube-system get job | grep helm-install-traefik | grep -c '1/1') -ne 2 ]; do
echo -n "."
sleep 1
done
echo "ready."
kubectl -n kube-system get job -o wide
}
function do_argocd {
cd ${SRC_DIR}/
kubectl create ns argocd --dry-run=client -o yaml | kubectl apply -f -
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
if [ -n "${ARGOCD_FQDN}" ]; then
export ARGOCD_FQDN
kubectl apply -f argocd/cmd-params-cm.yaml
kubectl -n argocd rollout restart deploy/argocd-server
argocd/ingressroute-server.yaml.sh | kubectl apply -f -
# ARGOCD_PASSWD=$(kubectl -n argocd get secret/argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d)
fi
echo -n "waiting for argocd to become ready ."
while [ $(kubectl -n argocd get pods | grep -c '1/1') -ne 7 ]; do
echo -n "."
sleep 1
done
echo "ready."
kubectl -n argocd get all
}
function do_chart {
cd ${SRC_DIR}/jitsi
if [ -n "${EXCLUDE_JVB}" ]; then
EXCLUDE_JVB_VALUES_FILE="-f values-jvb-off.yaml"
fi
if [ -n "${STAGING_CERT}" ]; then
CERT_RESOLVER="le-staging"
else
CERT_RESOLVER="le-prod"
fi
helm -n ${NAMESPACE} upgrade -i --create-namespace ${HELM_NAME} . \
-f values.yaml \
$EXCLUDE_JVB_VALUES_FILE \
--set certResolver=${CERT_RESOLVER} \
--set fqdn=${FQDN} \
--set jitsi-meet.publicURL=https://${FQDN} \
--set jitsi-meet.jvb.publicIP=${PUBLIC_IP} \
--set jitsi-meet.jvb.UDPPort=${JVB_PORT}
}
function do_app {
cd ${WS_DIR}/
if [ -n "${DEPLOY_GIT_VERSION}" ]; then
SET_GIT_REVISION="--revision ${DEPLOY_GIT_VERSION}"
fi
if [ -n "${EXCLUDE_JVB}" ]; then
EXCLUDE_JVB_VALUES_FILE="--values values-jvb-off.yaml"
fi
if [ -n "${STAGING_CERT}" ]; then
CERT_RESOLVER="le-staging"
else
CERT_RESOLVER="le-prod"
fi
argocd login --core
ORIG_NAMESPACE=$(kubectl config view --minify -o jsonpath='{..namespace}')
kubectl config set-context --current --namespace=argocd
kubectl create ns ${NAMESPACE} --dry-run=client -o yaml | kubectl apply -f -
argocd app create ${HELM_NAME} \
--upsert \
--repo ${DEPLOY_GIT_REPO} \
--path jitsi \
${SET_GIT_REVISION} \
--dest-server https://kubernetes.default.svc \
--dest-namespace ${NAMESPACE} \
--values values.yaml \
${EXCLUDE_JVB_VALUES_FILE} \
--helm-set certResolver=${CERT_RESOLVER} \
--helm-set fqdn=${FQDN} \
--helm-set jitsi-meet.publicURL=https://${FQDN} \
--helm-set jitsi-meet.jvb.publicIP=${PUBLIC_IP} \
--helm-set jitsi-meet.jvb.UDPPort=${JVB_PORT}
sleep 5 # there is a race if sync happens too quickly, so that it becomes a partial sync
argocd app sync ${HELM_NAME}
kubectl config set-context --current --namespace=${ORIG_NAMESPACE}
}
# installation starts from here
(
get_helm
get_src
do_k3s
get_argocd # 'argocd version' depends on k3s setup
do_traefik
do_argocd
do_app
# installation ends here
) 2>&1 | tee ${WS_DIR}/deploy.log

@ -0,0 +1,6 @@
dependencies:
- name: jitsi-meet
repository: https://jitsi-contrib.github.io/jitsi-helm
version: 1.2.2
digest: sha256:165664c1a23bc9760177e63740a861360eee007b432d9044ea449e77fba95d94
generated: "2022-05-02T17:15:02.132446+08:00"

@ -0,0 +1,7 @@
apiVersion: v2
name: jitsi-deploy
version: 0.1.0
dependencies:
- name: jitsi-meet
version: 1.2.2
repository: "https://jitsi-contrib.github.io/jitsi-helm"

Binary file not shown.

@ -0,0 +1,16 @@
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: jitsi-web
namespace: {{ .Release.Namespace }}
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`{{ .Values.fqdn }}`)
services:
- name: {{ .Release.Name }}-jitsi-meet-web
port: 80
tls:
certResolver: {{ .Values.certResolver }}

@ -0,0 +1,12 @@
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: {{ .Release.Name }}-ingressroutetcp-prosody
spec:
entryPoints:
- xmpp-{{ .Release.Namespace }}
routes:
- match: HostSNI(`*`)
services:
- name: {{ .Release.Name }}-prosody
port: 5222

@ -0,0 +1,5 @@
jitsi-meet:
jvb:
replicaCount: 0
service:
enabled: false

@ -0,0 +1,41 @@
certResolver: le-staging
fqdn: ""
jitsi-meet:
publicURL: ""
tz: Asia/Shanghai
web:
ingress:
enabled: false
jicofo:
livenessProbe:
failureThreshold: 30
periodSeconds: 10
readinessProbe:
failureThreshold: 30
periodSeconds: 10
jvb:
service:
# enabled: true
type: NodePort
# It may be required to change the default port to a value allowed by Kubernetes (30000-32768)
UDPPort: 30000
livenessProbe:
failureThreshold: 30
periodSeconds: 10
readinessProbe:
failureThreshold: 30
periodSeconds: 10
websockets:
enabled: true
# Use public IP of one of your node, or the public IP of a loadbalancer in front of the nodes
publicIP: ""

@ -0,0 +1,36 @@
cat <<EOF
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
additionalArguments:
- "--log.level=DEBUG"
- "--certificatesresolvers.le-prod.acme.email=${ACME_EMAIL}"
- "--certificatesresolvers.le-prod.acme.storage=/data/acme-prod.json"
- "--certificatesresolvers.le-prod.acme.tlschallenge=true"
- "--certificatesresolvers.le-prod.acme.caServer=https://acme-v02.api.letsencrypt.org/directory"
- "--certificatesresolvers.le-staging.acme.email=${ACME_EMAIL}"
- "--certificatesresolvers.le-staging.acme.storage=/data/acme-staging.json"
- "--certificatesresolvers.le-staging.acme.tlschallenge=true"
- "--certificatesresolvers.le-staging.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory"
# dashboard:
# enabled: true
ports:
# traefik:
# expose: true
web:
redirectTo: websecure
xmpp-prod:
port: 5222
expose: true
exposedPort: 5222
protocol: TCP
xmpp-test:
port: 5223
expose: true
exposedPort: 5223
protocol: TCP
EOF

@ -1,12 +0,0 @@
nameOverride: "shlug"
jvb:
service:
type: NodePort
# It may be required to change the default port to a value allowed by Kubernetes (30000-32768)
UDPPort: 30000
# Use public IP of one of your node, or the public IP of a loadbalancer in front of the nodes
publicIP: 20.205.103.185
publicURL: jisti.ycy.me
Loading…
Cancel
Save