首页 > 解决方案 > Kubernetes CRDs - 参考现有的验证规范

问题描述

有没有办法访问现有的验证规范?例如,我希望能够在我的 CRD 上设置 NodeAffinity,并且只想设置 $ref: 。我在这里找到了整个 API:https ://github.com/kubernetes/kubernetes/blob/master/api/openapi-spec/swagger.json 或 kubectl proxy -> localhost:8001/openapi/v2(来自我的集群)

我可以手动复制粘贴 api 验证模式,但我想知道是否有一种方法可以使用 $ref 从我的 CRD 中自动引用现有的 OpenAPI 验证规范。我想像 $ref: localhost:8001/openapi/v2/definitions/io.k8s.api.core.v1.NodeAffinity

如果这是可能的,它是否也会解决内部的 $refs ?

作为参考,以下是 API 中的 nodeaffinity 定义:

"io.k8s.api.core.v1.NodeAffinity": {
  "description": "Node affinity is a group of node affinity scheduling rules.",
  "properties": {
    "preferredDuringSchedulingIgnoredDuringExecution": {
      "description": "The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \"weight\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.",
      "items": {
        "$ref": "#/definitions/io.k8s.api.core.v1.PreferredSchedulingTerm"
      },
      "type": "array"
    },
    "requiredDuringSchedulingIgnoredDuringExecution": {
      "$ref": "#/definitions/io.k8s.api.core.v1.NodeSelector",
      "description": "If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node."
    }
  },
  "type": "object"
},

(使用带有 Ansible 的 Operator-SDK,以防万一)

编辑:(添加一个完整的例子来进一步解释)

我有一个名为 Workshop 的 CRD,我需要对某些规格参数进行验证。

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: workshops.k8s.example.tk
spec:
  group: k8s.example.tk
  names:
    kind: Workshop
    listKind: WorkshopList
    plural: workshops
    singular: workshop
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      type: object
      properties:
        spec:
          type: object
          required:
          - workshopID
          properties:
            workshopID: #
              type: string
              description: Unique identifier for this particular virtual 
workshop
              example: d8e8fca2dc0f896fd7cb4cb0031ba249

现在我需要添加一个 nodeAffinity 规范字段,该字段将应用于此 CustomResourceDefinition 下的任何 pod。它的验证将与 Pod 中的 nodeAffinity 验证完全相同。

让我从以下网址提取已经在 OpenApi 中编写的验证规范:https ://github.com/kubernetes/kubernetes/blob/master/api/openapi-spec/swagger.json 并将其转换为 YAML,然后将其添加到我的规范中.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: workshops.k8s.example.tk
spec:
  group: k8s.example.tk
  names:
    kind: Workshop
    listKind: WorkshopList
    plural: workshops
    singular: workshop
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      type: object
      properties:
        spec:
          type: object
          required:
          - workshopID
          properties:
            workshopID: #
              type: string
              description: Unique identifier for this particular virtual 
workshop
              example: d8e8fca2dc0f896fd7cb4cb0031ba249
            affinity: #
          type: object
          properties:
            nodeAffinity: #
              description: Node affinity is a group of node affinity scheduling rules.
              type: object
              properties:
                preferredDuringSchedulingIgnoredDuringExecution:
                  description: The scheduler will prefer to schedule pods to nodes that satisfy
                    the affinity expressions specified by this field, but it may choose a node that
                    violates one or more of the expressions. The node that is most preferred is
                    the one with the greatest sum of weights, i.e. for each node that meets all
                    of the scheduling requirements (resource request, requiredDuringScheduling affinity
                    expressions, etc.), compute a sum by iterating through the elements of this
                    field and adding "weight" to the sum if the node matches the corresponding matchExpressions;
                    the node(s) with the highest sum are the most preferred.
                  type: array
                  items:
                    description: An empty preferred scheduling term matches all objects with implicit
                      weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no
                      objects (i.e. is also a no-op).
                    type: object
                    required:
                    - weight
                    - preference
                    properties:
                      preference:
                        description: A node selector term, associated with the corresponding weight.
                          A null or empty node selector term matches no objects. The requirements
                          of them are ANDed. The TopologySelectorTerm type implements a subset of
                          the NodeSelectorTerm.
                        type: object
                        properties:
                          matchExpressions:
                            description: A list of node selector requirements by node's labels.
                            type: array
                            items:
                              description: A node selector requirement is a selector that contains
                                values, a key, and an operator that relates the key and values.
                              type: object
                              required:
                              - key
                              - operator
                              properties:
                                key:
                                  description: The label key that the selector applies to.
                                  type: string
                                operator:
                                  description: Represents a key's relationship to a set of values.
                                    Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and
                                    Lt.
                                  type: string
                                values:
                                  description: An array of string values. If the operator is In
                                    or NotIn, the values array must be non-empty. If the operator
                                    is Exists or DoesNotExist, the values array must be empty. If
                                    the operator is Gt or Lt, the values array must have a single
                                    element, which will be interpreted as an integer. This array
                                    is replaced during a strategic merge patch.
                                  type: array
                                  items:
                                    type: string
                          matchFields:
                            description: A list of node selector requirements by node's fields.
                            type: array
                            items:
                              description: A node selector requirement is a selector that contains
                                values, a key, and an operator that relates the key and values.
                              type: object
                              required:
                              - key
                              - operator
                              properties:
                                key:
                                  description: The label key that the selector applies to.
                                  type: string
                                operator:
                                  description: Represents a key's relationship to a set of values.
                                    Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and
                                    Lt.
                                  type: string
                                values:
                                  description: An array of string values. If the operator is In
                                    or NotIn, the values array must be non-empty. If the operator
                                    is Exists or DoesNotExist, the values array must be empty. If
                                    the operator is Gt or Lt, the values array must have a single
                                    element, which will be interpreted as an integer. This array
                                    is replaced during a strategic merge patch.
                                  type: array
                                  items:
                                    type: string
                      weight:
                        description: Weight associated with matching the corresponding nodeSelectorTerm,
                          in the range 1-100.
                        type: integer
                        format: int32
                requiredDuringSchedulingIgnoredDuringExecution:
                  description: If the affinity requirements specified by this field are not met
                    at scheduling time, the pod will not be scheduled onto the node. If the affinity
                    requirements specified by this field cease to be met at some point during pod
                    execution (e.g. due to an update), the system may or may not try to eventually
                    evict the pod from its node. A node selector represents the union of the results
                    of one or more label queries over a set of nodes; that is, it represents the
                    OR of the selectors represented by the node selector terms.
                  type: object
                  required:
                  - nodeSelectorTerms
                  properties:
                    nodeSelectorTerms:
                      description: Required. A list of node selector terms. The terms are ORed.
                      type: array
                      items:
                        description: A null or empty node selector term matches no objects. The
                          requirements of them are ANDed. The TopologySelectorTerm type implements
                          a subset of the NodeSelectorTerm.
                        type: object
                        properties:
                          matchExpressions:
                            description: A list of node selector requirements by node's labels.
                            type: array
                            items:
                              description: A node selector requirement is a selector that contains
                                values, a key, and an operator that relates the key and values.
                              type: object
                              required:
                              - key
                              - operator
                              properties:
                                key:
                                  description: The label key that the selector applies to.
                                  type: string
                                operator:
                                  description: Represents a key's relationship to a set of values.
                                    Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and
                                    Lt.
                                  type: string
                                values:
                                  description: An array of string values. If the operator is In
                                    or NotIn, the values array must be non-empty. If the operator
                                    is Exists or DoesNotExist, the values array must be empty. If
                                    the operator is Gt or Lt, the values array must have a single
                                    element, which will be interpreted as an integer. This array
                                    is replaced during a strategic merge patch.
                                  type: array
                                  items:
                                    type: string
                          matchFields:
                            description: A list of node selector requirements by node's fields.
                            type: array
                            items:
                              description: A node selector requirement is a selector that contains
                                values, a key, and an operator that relates the key and values.
                              type: object
                              required:
                              - key
                              - operator
                              properties:
                                key:
                                  description: The label key that the selector applies to.
                                  type: string
                                operator:
                                  description: Represents a key's relationship to a set of values.
                                    Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and
                                    Lt.
                                  type: string
                                values:
                                  description: An array of string values. If the operator is In
                                    or NotIn, the values array must be non-empty. If the operator
                                    is Exists or DoesNotExist, the values array must be empty. If
                                    the operator is Gt or Lt, the values array must have a single
                                    element, which will be interpreted as an integer. This array
                                    is replaced during a strategic merge patch.
                                  type: array
                                  items:
                                    type: string

哇,为了验证一个字段(及其子字段),我的 CRD 定义增加了 100 多行,所有这些都只是为了重新实现 Kubernetes 原生 pod api 定义中已经存在的东西。手动复制粘贴并手动解析 Kubernetes 规范中的所有引用也需要大约 15 分钟。这不是很有意义吗:

A) 将此长 API 规范存储在外部文件中,并使用 $ref: externalfile.json 将其拉入以保持我的 CRD 小而干净。

或者更好

B) 使用 $ref 标签插入已经存在的实际 Kubernetes 原生验证规范,如下所示:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: workshops.k8s.example.tk
spec:
  group: k8s.example.tk
  names:
    kind: Workshop
    listKind: WorkshopList
    plural: workshops
    singular: workshop
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      type: object
      properties:
        spec:
          type: object
          required:
          - workshopID
          properties:
            workshopID: #
              type: string
              description: Unique identifier for this particular virtual 
workshop
              example: d8e8fca2dc0f896fd7cb4cb0031ba249
            affinity:
              type: object
              properties:
                nodeAffinity:
                  $ref: <kubernetes-api>/openapi/v2#/definitions/io.k8s.api.core.v1.NodeAffinity

回到 30 行左右的代码,并且验证规范与 Kubernetes 原生验证保持同步,因为它从 Kubernetes API 本身提取信息。据此,应该支持 $ref 这样做:https ://swagger.io/docs/specification/using-ref/#syntax

标签: kuberneteskubectlopenapikubernetes-apiserverkubernetes-custom-resources

解决方案


不幸的是,目前没有办法很好地做到这一点。我们通过编写一个可怕的 Bash 脚本从 Kubernetes 中提取定义并通过 Helm 模板将其包含到我们的 CRD 中来解决了这个问题。


推荐阅读