Ruan Bekker's DevOps Blog
3 Node Multi-master k3s Kubernetes Cluster with MySQL

3 Node Multi-master k3s Kubernetes Cluster with MySQL

Ruan Bekker's photo
Ruan Bekker
·

3 min read

In this post we will setup a 3 node multi-master kubernetes cluster using k3s which is backed with mysql.

I will be installing mariadb on one of my nodes, but if you want to get mysql or mariadb running on docker, you can check out mysql on dockerhub.

Our Setup

Our IP Addresses for our nodes will be:

  • MySQL: 192.168.0.200
  • Rpi-01: 192.168.0.21
  • Rpi-02: 192.168.0.22
  • Rpi-03: 192.168.0.23

Database

On our database node, i'm installing mariadb server 10.3:

$ sudo apt install mariadb-server-10.3 -y

The initial setup of our database for kubernetes:

$ sudo mysql_secure_password

Then enable the plugin for socket authentication:

$ sudo mysql -u root
> use mysql;
> UPDATE user SET plugin='' WHERE User='root';
> FLUSH PRIVILEGES;
> exit;

Now you can auth with your password:

$ mysql -u root -p

Create the k3s database, the rancher user that will own the database and set a password:

> CREATE database k3s;
> CREATE user 'rancher'@'%' IDENTIFIED BY 'password';
> GRANT ALL on k3s.* TO 'rancher'@'%';
> FLUSH PRIVILEGES'

> use mysql;
> UPDATE user SET plugin='' WHERE User='rancher';
> FLUSH PRIVILEGES;

Edit sudo vim /etc/mysql/mariadb.conf.d/50-server.cnf and comment out bind-address so that mysql listens on all interfaces, then restart:

$ sudo /etc/init.d/mysql restart

our database details for k3s:

host: 192.168.0.200
user: rancher
pass: password
db: k3s

Setup Kubernetes Nodes

On our first node, install k3s and use the arguments as you prefer, but the important one is to point our database using --datastore-endpoint .

You will notice that I am also using --node-taint k3s-controlplane=true:NoExecute" on all 3 nodes, you can remove that, but I also want to show how you can remove the taints manually from node2 and node3 after the deployment.

$ curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig-mode 644 --token secret-01010101-xabcdefx --tls-san k3s.mydomain.com --node-taint k3s-controlplane=true:NoExecute" sh -s - server --datastore-endpoint="mysql://rancher:password@tcp(192.168.0.200:3306)/k3s"

Your kubeconfig will be available at: /etc/rancher/k3s/k3s.yaml

When we list our kubernetes nodes:

$ kubectl get nodes
NAME     STATUS   ROLES    AGE   VERSION
rpi-01   Ready    master   16s   v1.18.4+k3s1

Do the same for node 2 and node 3, once they have been provisioned, you should see the following:

$ kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
rpi-01   Ready    master   2m30s   v1.18.4+k3s1
rpi-02   Ready    master   41s     v1.18.4+k3s1
rpi-03   Ready    master   25s     v1.18.4+k3s1

Untaint Nodes

As mentioned, all 3 nodes run with NoExecute taints, and now we would like to remove them from node2 and node3 as a demonstration, if you would like to do that.

You can read more up about taints from the documentation

Describe the nodes to view the taints:

$ kubectl describe nodes | grep Taints
Taints:             k3s-controlplane=true:NoExecute
Taints:             k3s-controlplane=true:NoExecute
Taints:             k3s-controlplane=true:NoExecute

Remove taint from node2:

$ kubectl taint nodes rpi-02 k3s-controlplane=true:NoExecute-
node/rpi-02 untainted

And remove from node3:

$ kubectl taint nodes rpi-03 k3s-controlplane=true:NoExecute-
node/rpi-03 untainted

Now when we describe our nodes again:

$ kubectl describe nodes | grep Taints
Taints:             k3s-controlplane=true:NoExecute
Taints:             <none>
Taints:             <none>

Deploy a sample application

To test our cluster, let's deploy a sample web application, which is a basic application that returns it's hostname:

$ wget https://gist.githubusercontent.com/ruanbekker/d3efa0ff72a6d26698e6fb3a0839886d/raw/2bba233ec7a06a866dee9aa9daa106960f3a4369/deployment-green-hostname-rpi.yaml 

$ kubectl apply -f deployment-green-hostname-rpi.yaml
namespace/color created
deployment.apps/green created
service/green created
ingress.extensions/colors-green created

Once we deployed our application, have a look at the deployment and pods:

$ kubectl get deployment,pod -n color
NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/green   4/4     4            4           2m1s

NAME                         READY   STATUS    RESTARTS   AGE
pod/green-545d49b94b-cxt84   1/1     Running   0          118s
pod/green-545d49b94b-84fdm   1/1     Running   0          117s
pod/green-545d49b94b-gghnf   1/1     Running   0          118s
pod/green-545d49b94b-d5rvf   1/1     Running   0          117s

And our ingress:

$ kubectl get ingress -n color
NAME           CLASS    HOSTS                        ADDRESS         PORTS   AGE
colors-green   <none>   hostname-green.homedns.xyz   192.168.0.21    80      2m49s

Make a http request to our application:

$ curl -H "Host:hostname-green.homedns.xyz" http://192.168.0.21
Green running on: green-545d49b94b-84fdm

Thank You

Thanks for reading, hope you found it useful.

To view more of my content, have a visit on my website ruan.dev or say hi on Twitter: @ruanbekker

raspberrypiKubernetesk3s