Running A Simple Operator in Minikube

Introduction

You may have heard of Kubernetes operator pattern. Maybe you have worked on one of projects that has Custom Resource Definition (CRD). Custom resource definitions are cool things once you define your custom resource on Kubernetes by using YAML. The Kubernetes API server component will implement CRUD API for you automatically. Then, you can use powerful Kubernetes clients, such as kubectl to interact with your own resources managed by Kubernetes.

In this article, we introduce what is the operator. In addition, we provide the reasons that why, in some circumstance, we need the operator. Finally, we give one example to create a operator in Minikube.

What is the Operator

Operators are defined as software SREs.[1]

An Operator is like an automated Site Reliability Engineer for its application. 
It encodes in software the skills of an expert administrator. 

We all know Kubenetes has two planes: control plane and application plane. Control plane has i) Controller Manager, ii) API Server, iii) Scheduler and iv) etcd. Application plane are all microservices running on each node.

Controller manager implements the loops to repeatedly compare declared desired state and actual state for each application.

By using Kubenetes API client, such as kubectl, we could manage CRUD its well known resources, such as Deployment, Service or ConfigMap.

Can we use kubectl or similar client to manage our own resources by extended API? The answer is the Operator pattern.

Why Do We Need a CRD?

It really depends on your specific project and needs. The Kubernetes Docs answer this question like so:

Use a ConfigMap if any of the following apply:

• There is an existing, well-documented config file format, such as a mysql.cnf or named.conf.
• You want to put the entire config file into one key of a ConfigMap.
• The main use of the config file is for a program running in a Pod on your cluster to consume the file to configure itself.
• Consumers of the file prefer to consume file via a Pod or an environment variable in a Pod, rather than the Kubernetes API.
• You want to perform rolling updates via Deployment, etc., when the file is updated.

Use a Custom Resource Definition (CRD or Aggregated API) if most of the following apply:

• You want to use Kubernetes client libraries and CLIs to create and update a new resource.
• You want top-level support from kubectl (for example: kubectl get my-object object-name).
• You want to build new automation that watches for updates on the new object, and then CRUD other objects, or vice versa.
• You want to write automation that handles updates to the object.
• You want to use Kubernetes API conventions like .spec, .status, and .metadata.
• You want the object to be an abstraction over a collection of controlled resources or a summation of other resources.

Creating a CRD

Let’s go through the basic steps to create a CRD.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: redisclusters.redis.database.coreos.com
spec:
  group: redis.database.coreos.com
  names:
    kind: RedisCluster
    listKind: RedisClusterList
    plural: redisclusters
    shortNames:
    - redisclus
    - redis
    singular: rediscluster
  scope: Namespaced
  version: v1beta2
  versions:
  - name: v1beta2
    served: true
    storage: true

kubectl create -f redis-operator-crd.yaml

Defining an Operator Service Account

We also need to define declarations for a service account and the capabilities that account needs to run the redis Operator.

Define Service Account

apiVersion: v1
kind: ServiceAccount
metadata:
  name: redis-operator-sa

kubectl create -f redis-operator-sa.yaml

Define a Role

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: redis-operator-role
rules:
- apiGroups:
  - redis.database.coreos.com
  resources:
  - redisclusters
  - redisbackups
  - redisrestores
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  verbs:
  - '*'
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get

kubectl create -f redis-operator-role.yaml

Define a RoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: redis-operator-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: redis-operator-role
subjects:
- kind: ServiceAccount
  name: redis-operator-sa
  namespace: default

kubectl create -f redis-operator-rolebinding.yaml

Deploying the Operator

The Operator is a type of controller.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-operator
spec:
  selector:
    matchLabels:
      app: redis-operator
  replicas: 1
  template:
    metadata:
      labels:
        app: redis-operator
    spec:
      containers:
      - name: redis-operator
        image: <redis-operator:image_tag>
        command:
        - redis-operator
        - --create-crd=false
        env:
        - name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        imagePullPolicy: IfNotPresent
      serviceAccountName: redis-operator-sa

kubectl create -f redis-operator-deployment.yaml

Declaring an Redis Cluster Resource

Now that we have an Operator watching RedisCluster resources, we can declare a RedisCluster with our desired state.

apiVersion: redis.database.coreos.com/v1beta2
kind: RedisCluster
metadata:
  name: example-redis-cluster
spec:
  size: 2
  version: 3.1.10

kubectl create -f redis-cluster-cr.yaml

After create our redis custom resource, the operator will target the desired state to create the redis cluster.

Cleaning Up

Finally, we may want to clean up our play ground in minikube.

$ kubectl delete -f redis-operator-sa.yaml
$ kubectl delete -f redis-operator-role.yaml
$ kubectl delete -f redis-operator-rolebinding.yaml 
$ kubectl delete -f redis-operator-crd.yaml
$ kubectl delete -f redis-operator-deployment.yaml 
$ kubectl delete -f redis-cluster-cr.yaml

Reference

[1] Kubernetes Operators: Automation the Container Orchestration Platform. Jason Dobies & Joshua Wood.

• ← Previous Post
• Next Post →