In our previous article on the topic, we discussed the challenges of implementing Dynamic Admission Control (DAC) in your Kubernetes cluster using Validating Admission Webhooks (VAW). We introduced the new Validation Admission Policy (VAP) resource and demonstrated a few simple use cases to its simplicity.

Today we’re going to focus a couple of more advanced features of VAP:

1. Resource matching and filtering
   Parameters in policies

Lastly, we will shortly discuss considerations for using VAPs over VAWs and some tips for migration.

All examples in the previous article and this one, can be found in a GitHub repository created for this purpose.

Resource matching and filtering

In the use case demonstrated in the previous article, we saw how we can apply a policy for only one type of resource (Deployment), which makes sense for resource-specific validations.

But what if we wanted to apply a more generic policy in the cluster, such as enforcing a naming convention?

We have several options here. First, we can go with the explicit option and create a resource matching rule for each type of resource we would like to validate:

 

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "demo-policy.example.com"
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups:   ["apps"]
        apiVersions: ["v1"]
        operations:  ["CREATE", "UPDATE"]
        resources:   ["deployments", "statefulsets"]
      - apiGroups:   ["autoscaling"]
        apiVersions: ["v2"]
        operations:  ["CREATE", "UPDATE"]
        resources:   ["horizontalpodautoscalers"]
  validations:
    - expression: "!object.metadata.name.contains('demo') || object.metadata.namespace == 'demo'"

Sure, you can use the same rule for resources that are part of the same API group/version, but this list can grow very long very fast and will need to be maintained (especially when introducing new resource types into the cluster).

The alternative would be to match all resources and then selectively exclude those that we don’t want to validate. Resource filtering can be done on two levels:

1. inside the  ValidatingAdmissionPolicy resource — using  .spec.matchConditions
2. inside the  ValidatingAdmissionPolicyBinding resource — using  .spec.matchResources.excludeResources

The first approach is documented, whereas the second one not so much.  .spec.matchConditions uses Common Expression Langauge (CEL) to filter requests that the rules have already matched (i.e. past the binding and the policy’s match constraints).

Resource exclusions  (.spec.matchResources.excludeResources)on the other hand, are performed before the resource even reaches the policy and use the same syntax used to match resources to create a resource exclusion list.

Here’s an example of a policy that will skip  Lease resources from the API group  coordination.k8s.io and any resource from the API group  rbac.authorization.k8s.io using CEL match conditions:

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "demo-policy.example.com"
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups:   ["*"]
        apiVersions: ["*"]
        operations:  ["CREATE", "UPDATE"]
        resources:   ["*"]
  matchConditions:
    - name: 'exclude-leases' # Each match condition must have a unique name
      expression: '!(request.resource.group == "coordination.k8s.io" && request.resource.resource == "leases")' # Match non-lease resources.
    - name: 'rbac' # Skip RBAC requests.
      expression: 'request.resource.group != "rbac.authorization.k8s.io"'
  validations:
    - expression: "!object.metadata.name.contains('demo') || object.metadata.namespace == 'demo'"

And here’s a binding that will filter out similar requests using match exclusions:

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: "demo-binding-test.example.com"
spec:
  policyName: "demo-policy.example.com"
  validationActions: [Deny]
  matchResources:
    excludeResourceRules:
      # exclude all resources in the rbac.authorization.k8s.io API group
      - apiGroups:   ["rbac.authorization.k8s.io"]
        apiVersions: ["*"]
        operations:  ["*"]
        resources:   ["*"]
      # exclude resource type leases from the coordination.k8s.io API group
      - apiGroups:   ["coordination.k8s.io"]
        apiVersions: ["*"]
        operations:  ["*"]
        resources:   ["leases"]
    namespaceSelector:
      matchLabels:
        environment: demo

Since both achieve virtually the same, I would suggest going with  excludeResourceRules if possible, since the filtering happens earlier on and will avoid running CEL expressions for no reason (we discussed the cost budget of running CEL within the Kubernetes API server in the previous post). Additionally, the fact it uses the same syntax as matching resources, I believe that it’s a bit more clear and easier to understand.

However, since  matchConditions uses CEL expressions for filtering and has access to the same objects as the validation expressions ( object ,  oldObject ,  request , and  params ), it also means that it has the power to filter for things that are not available with  excludeResourceRules , such as users/groups, or any other attribute on the available objects:

 

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
 name: "demo-policy.example.com"
spec:
  failurePolicy: Fail
  # match _all_ resources (for CREATE and UPDATE operations)
  matchConstraints:
    resourceRules:
      - apiGroups:   ["*"]
        apiVersions: ["*"]
        operations:  ["CREATE", "UPDATE"]
        resources:   ["*"]
  matchConditions:
    - name: 'exclude-kubelet-requests'
      expression: '!("system:nodes" in request.userInfo.groups)' # Allow requests made by non-node users.
  validations:
    - expression: 'object.metadata.name.startsWith("demo-")'

In short, I would suggest a combined approach: use  excludeResourceRules when you’d like to filter out specific resources (by API group, version, operation, resource type) and use  matchConditions when you need more advanced filtering (requests from specific users, groups, etc).

Parameterizing policies

To make policies even more powerful, you can parameterize them. This allows you to reuse a policy with different parameter values without having to redefine it for each permutation. Essentially separating the configuration from the policy itself.

Using parameters requires the cluster operator to create a  CustomResourceDefinition (CRD) to define and hold the parameters. This means that using parameters requires some extra work — but you don’t have to use them at all, so it’s up to you to decide.

Here’s a simple example of CRD that can be easily modified or extended if necessary:

 

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: parameters.example.com
spec:
  group: example.com
  names:
    kind: Parameters
    plural: parameters
  versions:
  - name: v1
    schema:
      openAPIV3Schema:
        type: object
        properties:
          maxReplicas:
            type: integer
    served: true
    storage: true
  scope: Cluster

 

The above CRD will allow us to create resources of the type  Parameters with a single supported custom field in the schema,  maxReplicas of type integer:

apiVersion: example.com/v1
kind: Parameters
metadata:
name: demo-parameters
maxReplicas: 3

Once we have that, we configure our policy to take parameters ( .spec.paramKind ) by pointing it to the correct API version and kind (as defined in our CRD) and can use them in our validation expressions:

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "demo-policy.example.com"
spec:
  failurePolicy: Fail
  paramKind:
    apiVersion: example.com/v1
    kind: Parameters
  matchConstraints:
    resourceRules:
      - apiGroups:   ["apps"]
        apiVersions: ["v1"]
        operations:  ["CREATE", "UPDATE"]
        resources:   ["deployments"]
  validations:
    - expression: "object.spec.replicas <= params.maxReplicas"
      messageExpression: "'Deployments cannot have more than ' + string(params.maxReplicas) + ' replicas, this one has ' + string(object.spec.replicas)"

Note that the  messageExpression is also updated to use the parameter, allowing us to avoid hard-coding values anywhere in the policy.

Eventually, we’ll update the binding to reference the  Parameters resource we created by name:

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: "demo-binding-test.example.com"
spec:
  policyName: "demo-policy.example.com"
  validationActions: [Deny]
  paramRef:
    name: "demo-parameters"
  matchResources:
    namespaceSelector:
      matchLabels:
        environment: demo

Once we applied everything to our cluster, trying to create a deployment with more than the configured replicas will fail as expected:

 

$ k create deployment nginx — image=nginx — replicas=6
error: failed to create deployment: deployments.apps "nginx" is forbidden: ValidatingAdmissionPolicy 'demo-policy.example.com' with binding 'demo-binding-test.example.com' denied request: Deployments cannot have more than 3 replicas, this one has 6

This would allow us to create different  Parameters resources and bind them to different resources/namespaces so that we can reuse our parameterized policies.

VAW or VAP, which to choose

Since VAPs are so much simpler to implement than VAWs, it seems likely that it will eventually replace VAWs, or at least become the standard for starting up with in-cluster validation in Kubernetes.

CEL support in Kubernetes has opened many doors for potential quality of life improvements from a cluster operator perspective. A Mutating Admission Policy is another example for a feature that might become a reality thanks to CEL. It’s safe to assume that VAPs will continue to receive focus and development until reaching maturity and general availability.

That said, VAPs are still not a full-fledged replacement to VAWs. With VAWs you can still perform any kind of custom validation logic that you can put into code, whether it’s a simple request attribute validation, or even checking the state of an external system before admitting a request. Moreover, the fact that you have the write the validation server on your own, gives you the freedom to choose your own technology and language — you are not limited to using CEL for validation.

Also, considering that VAPs are still in Alpha (starting with v1.26), it’s not yet a valid alternative for anyone using managed Kubernetes offerings from the major clouds. EKS and AKS do not support alpha features, whereas GKE does but only for testing purposes (alpha clusters expire after 30 days). That means that even if you could, you probably shouldn’t fully adopt VAPs as your organization’s only mechanism of in-cluster validation just yet.

Tips for migrating from VAWs to VAPs

Are you reading this at a time where VAPs are already at beta or GA and considering using it instead of your current solution? Luckily, these 2 features are not mutually exclusive. This means you could run them side by side during the transitional period.

Since VAWs and VAPs match resources in a similar fashion, the process of creating VAPs should be fairly straightforward. Here are some tips to get you started:

• Create a  validatingAdmissionPolicy that has the same  .spec.matchConstraints.resourceRules as the .webhooks.rules of your  ValidatingWebhookConfiguration resource.
• Migrate your validation expressions from the current langauge (likely Rego) to CEL. This is by far the most time consuming task, but I would suggest trying to use ChatGPT for this purpose, policy rules are usually straightforward enough that it can do a pretty decent job of it.
• Create a  ValidatingAdmissionPolicyBinding and set  .spec.validationActions to Warn to return errors to the client, but not block any API requests.
• Decide how to bind a policy to resources using  .spec.matchResources. This can be a simple namespace label matcher, or using a more advanced match expression. Label your namespaces accordingly.
• Attempt to create valid and invalid resources in managed namespaces, evaluate any returned (or missing) warnings and refine your expressions throughout the process.
• Once you’re confident in your policy, change the binding  .spec.ValidationActions  to  Deny to put the policy into full effect
• Remove any duplicate policies from the legacy system

Additional Resources

Effortless In-Cluster Validation with Kubernetes: Introducing Validating Admission Policies
validating-admission-policy-playground GitHub repository containing full working examples for Validating Admission Policies
Validating Admission Policy in the official Kubernetes documentation
Common Expression Language in Kubernetes
Kubernetes Enhancement Proposal (KEP)-3488 CEL for Admission Control
The Path to Self Contained CRDs talk by Cici Huang