Skip to content

.NET

Creating a simple Kubernetes operator in .NET (C#)

Note: I'd recommend building a Kubernetes operator using Go as it is the language of choice for Kubernetes. This article is more of a proof of concept.

Creating a Kubernetes operator can seem a bit overwhelming at first. To help, there’s a simple NuGet package called CanSupportMe.Operator which can be as simple as watching for a secret or config map or creating a custom resource definition (CRD) and watching that. You then get call backs for new items, modified items, reconciling items, deleting items and deleted items. (* CRDs only)

The call backs also expose a data object manager which lets you do things like create secrets and config maps, force the reconciliation of an object, clear all finalizers and check if a resource exists.

Example

The follow is a console application with the following packages installed:

dotnet add package Microsoft.Extensions.Configuration.Abstractions
dotnet add package Microsoft.Extensions.Hosting
dotnet add package Microsoft.Extensions.Hosting.Abstractions
dotnet add package CanSupportMe.Operator

If you’re not use a context with a token in your KUBECONFIG, use the commented out failover token lines to specify one (how to generate one is on the NuGet package page).

using CanSupportMe.Operator.Extensions;
using CanSupportMe.Operator.Models;
using CanSupportMe.Operator.Options;
using Microsoft.Extensions.Hosting;

// const string FAILOVER_TOKEN = "<YOUR_TOKEN_GOES_HERE>";

try
{  
  Console.WriteLine("Application starting up");

  IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices((context, services) =>
    {
      services.AddOperator(options =>
      {
        options.Group = "";
        options.Kind = "Secret";
        options.Version = "v1";
        options.Plural = "secrets";
        options.Scope = ResourceScope.Namespaced;
        options.LabelFilters.Add("app.kubernetes.io/managed-by", "DemoOperator");

        options.OnAdded = (kind, name, @namespace, item, dataObjectManager) =>
        {
          var typedItem = (KubernetesSecret)item;

          Console.WriteLine($"On {kind} Add: {name} to {@namespace} which is of type {typedItem.Type} with {typedItem.Data?.Count} item(s)");
        };

        options.FailoverToken = FAILOVER_TOKEN;
      });
    })
    .Build();

  host.Run();
}
catch (Exception ex)
{
  Console.WriteLine($"Application start failed because {ex.Message}");
}

To create a secret that will trigger the OnAdded call back, create a file called secret.yaml with the following contents:

apiVersion: v1
kind: Secret
metadata:
  name: test-secret-with-label
  namespace: default
  labels:
    app.kubernetes.io/managed-by: DemoOperator
stringData:
  notImportant: SomeValue
type: Opaque

Then apply it to your Kubernetes cluster using the following command:

kubectl apply -f secret.yaml

This should result in the following being output to the console:

On Secret Add: test-secret-with-label to default which is of type Opaque with 1 item(s)

.NET Container Running as Non-Root in Kubernetes

This is a quick guide on how to get a standard .NET container running as a non-root, non privileged user in Kubernetes. It is not a complete security guide but rather just enough if you require your pods to not run under root.

Update Dockerfile

The first step is to update the Dockerfile. Two changes are required here; one to change the port and one to specify the user to use.

Exported Port

At the start of the Dockerfile, replace any EXPORT statements with the following:

ENV ASPNETCORE_URLS http://+:8000
EXPOSE 8000

This will expose your application to the cluster on port 8000 rather than port 80.

User

Next, just before the ENTRYPOINT instruction, add the following line:

USER $APP_UID

Now build and push the container to your container registry of choice either manually or via a CI/CD pipeline.

Kubernetes Manifests

Deployment Manifest

Add the following snippet to the deployment manifest under the container entry that is to be locked down:

securityContext:
  allowPrivilegeEscalation: false
  runAsNonRoot: true
  runAsUser: 1654

Service Manifest

As the exported container port has now changed, update any service you may have defined so it looks similar to the following service manifest:

apiVersion: v1
kind: Service
metadata:
  name: your-service
  namespace: service-namespace
spec:
  selector:
     app: your-app
  ports:
    - name: http
      port: 80
      targetPort: 8000
  type: ClusterIP

The pod will still be accessible via its service on port 80 so things like ingress or gateway definitions or references from other apps do not need to be updated.

.NET 5/6, Docker and custom NuGet server

When using a custom NuGet server and you've added a nuget.config file to the solution, you'll need to add the following line to the default Dockerfile build by Visual Studio to allow the container to be built.

COPY ["nuget.config", "/src/"]

This should be placed before the RUN dotnet restore … line.

The filename is case sensitive within the container so using all lowercase is recommended for the file name. If you need to change the case, you may need to do two commits of the file (e.g. rename it NuGet.config –commit–> nuget1.config –commit–> nuget.config).

If running in a CI/CD pipeline and you have fixed custom NuGet servers, you can inject a nuget.config file into the CI pipeline however the file will still need referencing in the Dockerfile as above to be correctly used by the container build process.