spec,libcni: add support for injecting runtimeConfig into plugin stdin data
Add a new CapabilityArgs member to the RuntimeConf struct which runtimes can use to pass arbitrary capability-based keys to the plugin. Elements of this member will be filtered against the plugin's advertised capabilities (from its config JSON) and then added to a new "runtimeConfig" top-level map added to the config JSON sent to the plugin on stdin. Also "runtime_config"->"runtimeConfig" in CONVENTIONS.md to make capitalization consistent with other CNI config keys like "cniVersion".
This commit is contained in:
parent
7d8c23dd59
commit
71c4a60741
@ -29,18 +29,18 @@ This method of passing information to a plugin is recommended when the following
|
|||||||
* The configuration has specific meaning to the plugin (i.e. it's not just general meta data)
|
* The configuration has specific meaning to the plugin (i.e. it's not just general meta data)
|
||||||
* the plugin is expected to act on the configuration or return an error if it can't
|
* the plugin is expected to act on the configuration or return an error if it can't
|
||||||
|
|
||||||
Dynamic information (i.e. data that a runtime fills out) should be placed in a `runtime_config` section.
|
Dynamic information (i.e. data that a runtime fills out) should be placed in a `runtimeConfig` section.
|
||||||
|
|
||||||
| Area | Purpose| Spec and Example | Runtime implementations | Plugin Implementations |
|
| Area | Purpose| Spec and Example | Runtime implementations | Plugin Implementations |
|
||||||
| ------ | ------ | ------ | ------ | ------ | ------ |
|
| ------ | ------ | ------ | ------ | ------ | ------ |
|
||||||
| port mappings | Pass mapping from ports on the host to ports in the container network namespace. | Operators can ask runtimes to pass port mapping information to plugins, by setting the following in the CNI config <pre>"capabilities": {port_mappings": true} </pre> Runtimes should fill in the actual port mappings when the config is passed to plugins. It should be placed in a new section of the config "runtime_config" e.g. <pre>"runtime_config": {<br /> "port_mappings" : [<br /> { "host_port": 8080, "container_port": 80, "protocol": "tcp" },<br /> { "host_port": 8000, "container_port": 8001, "protocol": "udp" }<br /> ]<br />}</pre> | none | none |
|
| port mappings | Pass mapping from ports on the host to ports in the container network namespace. | Operators can ask runtimes to pass port mapping information to plugins, by setting the following in the CNI config <pre>"capabilities": {"portMappings": true} </pre> Runtimes should fill in the actual port mappings when the config is passed to plugins. It should be placed in a new section of the config "runtimeConfig" e.g. <pre>"runtimeConfig": {<br /> "portMappings" : [<br /> { "hostPort": 8080, "containerPort": 80, "protocol": "tcp" },<br /> { "hostPort": 8000, "containerPort": 8001, "protocol": "udp" }<br /> ]<br />}</pre> | none | none |
|
||||||
|
|
||||||
For example, the configuration for a port mapping plugin might look like this to an operator (it should be included as part of a [network configuration list](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-lists).
|
For example, the configuration for a port mapping plugin might look like this to an operator (it should be included as part of a [network configuration list](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-lists).
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"name" : "ExamplePlugin",
|
"name" : "ExamplePlugin",
|
||||||
"type" : "port-mapper",
|
"type" : "port-mapper",
|
||||||
"capabilities": {"port_mappings": true}
|
"capabilities": {"portMappings": true}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -49,9 +49,9 @@ But the runtime would fill in the mappings so the plugin itself would receive so
|
|||||||
{
|
{
|
||||||
"name" : "ExamplePlugin",
|
"name" : "ExamplePlugin",
|
||||||
"type" : "port-mapper",
|
"type" : "port-mapper",
|
||||||
"runtime_config": {
|
"runtimeConfig": {
|
||||||
"port_mappings": [
|
"portMappings": [
|
||||||
{"host_port": 8080, "container_port": 80, "protocol": "tcp"}
|
{"hostPort": 8080, "containerPort": 80, "protocol": "tcp"}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
4
SPEC.md
4
SPEC.md
@ -253,8 +253,8 @@ The list is described in JSON form, and can be stored on disk or generated from
|
|||||||
- `name` (string): Network name. This should be unique across all containers on the host (or other administrative domain).
|
- `name` (string): Network name. This should be unique across all containers on the host (or other administrative domain).
|
||||||
- `plugins` (list): A list of standard CNI network configuration dictionaries (see above).
|
- `plugins` (list): A list of standard CNI network configuration dictionaries (see above).
|
||||||
|
|
||||||
When executing a plugin list, the runtime MUST replace the `name` and `cniVersion` fields in each individual network configuration in the list with the `name` and `cniVersion` field of the list itself.
|
When executing a plugin list, the runtime MUST replace the `name` and `cniVersion` fields in each individual network configuration in the list with the `name` and `cniVersion` field of the list itself. This ensures that the name and CNI version is the same for all plugin executions in the list, preventing versioning conflicts between plugins.
|
||||||
This ensures that the name and CNI version is the same for all plugin executions in the list, preventing versioning conflicts between plugins.
|
The runtime may also pass capability-based keys as a map in the top-level `runtimeConfig` key of the plugin's config JSON if a plugin advertises it supports a specific capability via the `capabilities` key of its network configuration. The key passed in `runtimeConfig` MUST match the name of the specific capability from the `capabilities` key of the plugins network configuration. See CONVENTIONS.md for more information on capabilities and how they are sent to plugins via the `runtimeConfig` key.
|
||||||
|
|
||||||
For the ADD action, the runtime MUST also add a `prevResult` field to the configuration JSON of any plugin after the first one, which MUST be the Result of the previous plugin (if any) in JSON format ([see below](#network-configuration-list-runtime-examples)).
|
For the ADD action, the runtime MUST also add a `prevResult` field to the configuration JSON of any plugin after the first one, which MUST be the Result of the previous plugin (if any) in JSON format ([see below](#network-configuration-list-runtime-examples)).
|
||||||
For the ADD action, plugins SHOULD echo the contents of the `prevResult` field to their stdout to allow subsequent plugins (and the runtime) to receive the result, unless they wish to modify or suppress a previous result.
|
For the ADD action, plugins SHOULD echo the contents of the `prevResult` field to their stdout to allow subsequent plugins (and the runtime) to receive the result, unless they wish to modify or suppress a previous result.
|
||||||
|
@ -28,6 +28,12 @@ type RuntimeConf struct {
|
|||||||
NetNS string
|
NetNS string
|
||||||
IfName string
|
IfName string
|
||||||
Args [][2]string
|
Args [][2]string
|
||||||
|
// A dictionary of capability-specific data passed by the runtime
|
||||||
|
// to plugins as top-level keys in the 'runtimeConfig' dictionary
|
||||||
|
// of the plugin's stdin data. libcni will ensure that only keys
|
||||||
|
// in this map which match the capabilities of the plugin are passed
|
||||||
|
// to the plugin
|
||||||
|
CapabilityArgs map[string]interface{}
|
||||||
}
|
}
|
||||||
|
|
||||||
type NetworkConfig struct {
|
type NetworkConfig struct {
|
||||||
@ -57,22 +63,54 @@ type CNIConfig struct {
|
|||||||
// CNIConfig implements the CNI interface
|
// CNIConfig implements the CNI interface
|
||||||
var _ CNI = &CNIConfig{}
|
var _ CNI = &CNIConfig{}
|
||||||
|
|
||||||
func buildOneConfig(list *NetworkConfigList, orig *NetworkConfig, prevResult types.Result) (*NetworkConfig, error) {
|
func buildOneConfig(list *NetworkConfigList, orig *NetworkConfig, prevResult types.Result, rt *RuntimeConf) (*NetworkConfig, error) {
|
||||||
var err error
|
var err error
|
||||||
|
|
||||||
// Ensure every config uses the same name and version
|
inject := map[string]interface{}{
|
||||||
orig, err = InjectConf(orig, "name", list.Name)
|
"name": list.Name,
|
||||||
if err != nil {
|
"cniVersion": list.CNIVersion,
|
||||||
return nil, err
|
|
||||||
}
|
}
|
||||||
orig, err = InjectConf(orig, "cniVersion", list.CNIVersion)
|
// Add previous plugin result
|
||||||
|
if prevResult != nil {
|
||||||
|
inject["prevResult"] = prevResult
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure every config uses the same name and version
|
||||||
|
orig, err = InjectConf(orig, inject)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Add previous plugin result
|
return injectRuntimeConfig(orig, rt)
|
||||||
if prevResult != nil {
|
}
|
||||||
orig, err = InjectConf(orig, "prevResult", prevResult)
|
|
||||||
|
// This function takes a libcni RuntimeConf structure and injects values into
|
||||||
|
// a "runtimeConfig" dictionary in the CNI network configuration JSON that
|
||||||
|
// will be passed to the plugin on stdin.
|
||||||
|
//
|
||||||
|
// Only "capabilities arguments" passed by the runtime are currently injected.
|
||||||
|
// These capabilities arguments are filtered through the plugin's advertised
|
||||||
|
// capabilities from its config JSON, and any keys in the CapabilityArgs
|
||||||
|
// matching plugin capabilities are added to the "runtimeConfig" dictionary
|
||||||
|
// sent to the plugin via JSON on stdin. For exmaple, if the plugin's
|
||||||
|
// capabilities include "portMappings", and the CapabilityArgs map includes a
|
||||||
|
// "portMappings" key, that key and its value are added to the "runtimeConfig"
|
||||||
|
// dictionary to be passed to the plugin's stdin.
|
||||||
|
func injectRuntimeConfig(orig *NetworkConfig, rt *RuntimeConf) (*NetworkConfig, error) {
|
||||||
|
var err error
|
||||||
|
|
||||||
|
rc := make(map[string]interface{})
|
||||||
|
for capability, supported := range orig.Network.Capabilities {
|
||||||
|
if !supported {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if data, ok := rt.CapabilityArgs[capability]; ok {
|
||||||
|
rc[capability] = data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(rc) > 0 {
|
||||||
|
orig, err = InjectConf(orig, map[string]interface{}{"runtimeConfig": rc})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@ -90,7 +128,7 @@ func (c *CNIConfig) AddNetworkList(list *NetworkConfigList, rt *RuntimeConf) (ty
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
newConf, err := buildOneConfig(list, net, prevResult)
|
newConf, err := buildOneConfig(list, net, prevResult, rt)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@ -114,7 +152,7 @@ func (c *CNIConfig) DelNetworkList(list *NetworkConfigList, rt *RuntimeConf) err
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
newConf, err := buildOneConfig(list, net, nil)
|
newConf, err := buildOneConfig(list, net, nil, rt)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@ -134,6 +172,11 @@ func (c *CNIConfig) AddNetwork(net *NetworkConfig, rt *RuntimeConf) (types.Resul
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
net, err = injectRuntimeConfig(net, rt)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
return invoke.ExecPluginWithResult(pluginPath, net.Bytes, c.args("ADD", rt))
|
return invoke.ExecPluginWithResult(pluginPath, net.Bytes, c.args("ADD", rt))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -144,6 +187,11 @@ func (c *CNIConfig) DelNetwork(net *NetworkConfig, rt *RuntimeConf) error {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
net, err = injectRuntimeConfig(net, rt)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
return invoke.ExecPluginWithoutResult(pluginPath, net.Bytes, c.args("DEL", rt))
|
return invoke.ExecPluginWithoutResult(pluginPath, net.Bytes, c.args("DEL", rt))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -19,6 +19,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"io/ioutil"
|
"io/ioutil"
|
||||||
"net"
|
"net"
|
||||||
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
|
||||||
"github.com/containernetworking/cni/libcni"
|
"github.com/containernetworking/cni/libcni"
|
||||||
@ -35,19 +36,25 @@ type pluginInfo struct {
|
|||||||
debugFilePath string
|
debugFilePath string
|
||||||
debug *noop_debug.Debug
|
debug *noop_debug.Debug
|
||||||
config string
|
config string
|
||||||
|
stdinData []byte
|
||||||
}
|
}
|
||||||
|
|
||||||
func addNameToConfig(name, config string) ([]byte, error) {
|
type portMapping struct {
|
||||||
obj := make(map[string]interface{})
|
HostPort int `json:"hostPort"`
|
||||||
err := json.Unmarshal([]byte(config), &obj)
|
ContainerPort int `json:"containerPort"`
|
||||||
if err != nil {
|
Protocol string `json:"protocol"`
|
||||||
return nil, fmt.Errorf("unmarshal existing network bytes: %s", err)
|
}
|
||||||
|
|
||||||
|
func stringInList(s string, list []string) bool {
|
||||||
|
for _, item := range list {
|
||||||
|
if s == item {
|
||||||
|
return true
|
||||||
|
}
|
||||||
}
|
}
|
||||||
obj["name"] = name
|
return false
|
||||||
return json.Marshal(obj)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func newPluginInfo(configKey, configValue, prevResult string, injectDebugFilePath bool, result string) pluginInfo {
|
func newPluginInfo(configValue, prevResult string, injectDebugFilePath bool, result string, runtimeConfig map[string]interface{}, capabilities []string) pluginInfo {
|
||||||
debugFile, err := ioutil.TempFile("", "cni_debug")
|
debugFile, err := ioutil.TempFile("", "cni_debug")
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
Expect(debugFile.Close()).To(Succeed())
|
Expect(debugFile.Close()).To(Succeed())
|
||||||
@ -58,23 +65,155 @@ func newPluginInfo(configKey, configValue, prevResult string, injectDebugFilePat
|
|||||||
}
|
}
|
||||||
Expect(debug.WriteDebug(debugFilePath)).To(Succeed())
|
Expect(debug.WriteDebug(debugFilePath)).To(Succeed())
|
||||||
|
|
||||||
config := fmt.Sprintf(`{"type": "noop", "%s": "%s", "cniVersion": "0.3.0"`, configKey, configValue)
|
// config is what would be in the plugin's on-disk configuration
|
||||||
|
// without runtime injected keys
|
||||||
|
config := fmt.Sprintf(`{"type": "noop", "some-key": "%s"`, configValue)
|
||||||
if prevResult != "" {
|
if prevResult != "" {
|
||||||
config += fmt.Sprintf(`, "prevResult": %s`, prevResult)
|
config += fmt.Sprintf(`, "prevResult": %s`, prevResult)
|
||||||
}
|
}
|
||||||
if injectDebugFilePath {
|
if injectDebugFilePath {
|
||||||
config += fmt.Sprintf(`, "debugFile": "%s"`, debugFilePath)
|
config += fmt.Sprintf(`, "debugFile": "%s"`, debugFilePath)
|
||||||
}
|
}
|
||||||
|
if len(capabilities) > 0 {
|
||||||
|
config += `, "capabilities": {`
|
||||||
|
for i, c := range capabilities {
|
||||||
|
if i > 0 {
|
||||||
|
config += ", "
|
||||||
|
}
|
||||||
|
config += fmt.Sprintf(`"%s": true`, c)
|
||||||
|
}
|
||||||
|
config += "}"
|
||||||
|
}
|
||||||
config += "}"
|
config += "}"
|
||||||
|
|
||||||
|
// stdinData is what the runtime should pass to the plugin's stdin,
|
||||||
|
// including injected keys like 'name', 'cniVersion', and 'runtimeConfig'
|
||||||
|
newConfig := make(map[string]interface{})
|
||||||
|
err = json.Unmarshal([]byte(config), &newConfig)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
newConfig["name"] = "some-list"
|
||||||
|
newConfig["cniVersion"] = "0.3.0"
|
||||||
|
|
||||||
|
// Only include standard runtime config and capability args that this plugin advertises
|
||||||
|
newRuntimeConfig := make(map[string]interface{})
|
||||||
|
for key, value := range runtimeConfig {
|
||||||
|
if stringInList(key, capabilities) {
|
||||||
|
newRuntimeConfig[key] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(newRuntimeConfig) > 0 {
|
||||||
|
newConfig["runtimeConfig"] = newRuntimeConfig
|
||||||
|
}
|
||||||
|
|
||||||
|
stdinData, err := json.Marshal(newConfig)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
return pluginInfo{
|
return pluginInfo{
|
||||||
debugFilePath: debugFilePath,
|
debugFilePath: debugFilePath,
|
||||||
debug: debug,
|
debug: debug,
|
||||||
config: config,
|
config: config,
|
||||||
|
stdinData: stdinData,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
var _ = Describe("Invoking plugins", func() {
|
var _ = Describe("Invoking plugins", func() {
|
||||||
|
Describe("Capabilities", func() {
|
||||||
|
var (
|
||||||
|
debugFilePath string
|
||||||
|
debug *noop_debug.Debug
|
||||||
|
pluginConfig []byte
|
||||||
|
cniConfig libcni.CNIConfig
|
||||||
|
runtimeConfig *libcni.RuntimeConf
|
||||||
|
netConfig *libcni.NetworkConfig
|
||||||
|
)
|
||||||
|
|
||||||
|
BeforeEach(func() {
|
||||||
|
debugFile, err := ioutil.TempFile("", "cni_debug")
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
Expect(debugFile.Close()).To(Succeed())
|
||||||
|
debugFilePath = debugFile.Name()
|
||||||
|
|
||||||
|
debug = &noop_debug.Debug{}
|
||||||
|
Expect(debug.WriteDebug(debugFilePath)).To(Succeed())
|
||||||
|
|
||||||
|
pluginConfig = []byte(`{ "type": "noop", "cniVersion": "0.3.0", "capabilities": { "portMappings": true, "somethingElse": true, "noCapability": false } }`)
|
||||||
|
netConfig, err = libcni.ConfFromBytes(pluginConfig)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
|
cniConfig = libcni.CNIConfig{Path: []string{filepath.Dir(pluginPaths["noop"])}}
|
||||||
|
|
||||||
|
runtimeConfig = &libcni.RuntimeConf{
|
||||||
|
ContainerID: "some-container-id",
|
||||||
|
NetNS: "/some/netns/path",
|
||||||
|
IfName: "some-eth0",
|
||||||
|
Args: [][2]string{{"DEBUG", debugFilePath}},
|
||||||
|
CapabilityArgs: map[string]interface{}{
|
||||||
|
"portMappings": []portMapping{
|
||||||
|
{HostPort: 8080, ContainerPort: 80, Protocol: "tcp"},
|
||||||
|
},
|
||||||
|
"somethingElse": []string{"foobar", "baz"},
|
||||||
|
"noCapability": true,
|
||||||
|
"notAdded": []bool{true, false},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
AfterEach(func() {
|
||||||
|
Expect(os.RemoveAll(debugFilePath)).To(Succeed())
|
||||||
|
})
|
||||||
|
|
||||||
|
It("adds correct runtime config for capabilities to stdin", func() {
|
||||||
|
_, err := cniConfig.AddNetwork(netConfig, runtimeConfig)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
|
debug, err = noop_debug.ReadDebug(debugFilePath)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
Expect(debug.Command).To(Equal("ADD"))
|
||||||
|
|
||||||
|
conf := make(map[string]interface{})
|
||||||
|
err = json.Unmarshal(debug.CmdArgs.StdinData, &conf)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
|
// We expect runtimeConfig keys only for portMappings and somethingElse
|
||||||
|
rawRc := conf["runtimeConfig"]
|
||||||
|
rc, ok := rawRc.(map[string]interface{})
|
||||||
|
Expect(ok).To(Equal(true))
|
||||||
|
expectedKeys := []string{"portMappings", "somethingElse"}
|
||||||
|
Expect(len(rc)).To(Equal(len(expectedKeys)))
|
||||||
|
for _, key := range expectedKeys {
|
||||||
|
_, ok := rc[key]
|
||||||
|
Expect(ok).To(Equal(true))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
It("adds no runtimeConfig when the plugin advertises no used capabilities", func() {
|
||||||
|
// Replace CapabilityArgs with ones we know the plugin
|
||||||
|
// doesn't support
|
||||||
|
runtimeConfig.CapabilityArgs = map[string]interface{}{
|
||||||
|
"portMappings22": []portMapping{
|
||||||
|
{HostPort: 8080, ContainerPort: 80, Protocol: "tcp"},
|
||||||
|
},
|
||||||
|
"somethingElse22": []string{"foobar", "baz"},
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err := cniConfig.AddNetwork(netConfig, runtimeConfig)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
|
debug, err = noop_debug.ReadDebug(debugFilePath)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
Expect(debug.Command).To(Equal("ADD"))
|
||||||
|
|
||||||
|
conf := make(map[string]interface{})
|
||||||
|
err = json.Unmarshal(debug.CmdArgs.StdinData, &conf)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
|
// No intersection of plugin capabilities and CapabilityArgs,
|
||||||
|
// so plugin should not receive a "runtimeConfig" key
|
||||||
|
_, ok := conf["runtimeConfig"]
|
||||||
|
Expect(ok).Should(BeFalse())
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
Describe("Invoking a single plugin", func() {
|
Describe("Invoking a single plugin", func() {
|
||||||
var (
|
var (
|
||||||
debugFilePath string
|
debugFilePath string
|
||||||
@ -99,12 +238,19 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
}
|
}
|
||||||
Expect(debug.WriteDebug(debugFilePath)).To(Succeed())
|
Expect(debug.WriteDebug(debugFilePath)).To(Succeed())
|
||||||
|
|
||||||
|
portMappings := []portMapping{
|
||||||
|
{HostPort: 8080, ContainerPort: 80, Protocol: "tcp"},
|
||||||
|
}
|
||||||
|
|
||||||
cniBinPath = filepath.Dir(pluginPaths["noop"])
|
cniBinPath = filepath.Dir(pluginPaths["noop"])
|
||||||
pluginConfig = `{ "type": "noop", "some-key": "some-value", "cniVersion": "0.3.0" }`
|
pluginConfig = `{ "type": "noop", "some-key": "some-value", "cniVersion": "0.3.0", "capabilities": { "portMappings": true } }`
|
||||||
cniConfig = libcni.CNIConfig{Path: []string{cniBinPath}}
|
cniConfig = libcni.CNIConfig{Path: []string{cniBinPath}}
|
||||||
netConfig = &libcni.NetworkConfig{
|
netConfig = &libcni.NetworkConfig{
|
||||||
Network: &types.NetConf{
|
Network: &types.NetConf{
|
||||||
Type: "noop",
|
Type: "noop",
|
||||||
|
Capabilities: map[string]bool{
|
||||||
|
"portMappings": true,
|
||||||
|
},
|
||||||
},
|
},
|
||||||
Bytes: []byte(pluginConfig),
|
Bytes: []byte(pluginConfig),
|
||||||
}
|
}
|
||||||
@ -112,19 +258,36 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
ContainerID: "some-container-id",
|
ContainerID: "some-container-id",
|
||||||
NetNS: "/some/netns/path",
|
NetNS: "/some/netns/path",
|
||||||
IfName: "some-eth0",
|
IfName: "some-eth0",
|
||||||
Args: [][2]string{[2]string{"DEBUG", debugFilePath}},
|
Args: [][2]string{{"DEBUG", debugFilePath}},
|
||||||
|
CapabilityArgs: map[string]interface{}{
|
||||||
|
"portMappings": portMappings,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// inject runtime args into the expected plugin config
|
||||||
|
conf := make(map[string]interface{})
|
||||||
|
err = json.Unmarshal([]byte(pluginConfig), &conf)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
conf["runtimeConfig"] = map[string]interface{}{
|
||||||
|
"portMappings": portMappings,
|
||||||
|
}
|
||||||
|
newBytes, err := json.Marshal(conf)
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
expectedCmdArgs = skel.CmdArgs{
|
expectedCmdArgs = skel.CmdArgs{
|
||||||
ContainerID: "some-container-id",
|
ContainerID: "some-container-id",
|
||||||
Netns: "/some/netns/path",
|
Netns: "/some/netns/path",
|
||||||
IfName: "some-eth0",
|
IfName: "some-eth0",
|
||||||
Args: "DEBUG=" + debugFilePath,
|
Args: "DEBUG=" + debugFilePath,
|
||||||
Path: cniBinPath,
|
Path: cniBinPath,
|
||||||
StdinData: []byte(pluginConfig),
|
StdinData: newBytes,
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
AfterEach(func() {
|
||||||
|
Expect(os.RemoveAll(debugFilePath)).To(Succeed())
|
||||||
|
})
|
||||||
|
|
||||||
Describe("AddNetwork", func() {
|
Describe("AddNetwork", func() {
|
||||||
It("executes the plugin with command ADD", func() {
|
It("executes the plugin with command ADD", func() {
|
||||||
r, err := cniConfig.AddNetwork(netConfig, runtimeConfig)
|
r, err := cniConfig.AddNetwork(netConfig, runtimeConfig)
|
||||||
@ -149,6 +312,7 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
Expect(debug.Command).To(Equal("ADD"))
|
Expect(debug.Command).To(Equal("ADD"))
|
||||||
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
||||||
|
Expect(string(debug.CmdArgs.StdinData)).To(ContainSubstring("\"portMappings\":"))
|
||||||
})
|
})
|
||||||
|
|
||||||
Context("when finding the plugin fails", func() {
|
Context("when finding the plugin fails", func() {
|
||||||
@ -184,6 +348,7 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
Expect(debug.Command).To(Equal("DEL"))
|
Expect(debug.Command).To(Equal("DEL"))
|
||||||
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
||||||
|
Expect(string(debug.CmdArgs.StdinData)).To(ContainSubstring("\"portMappings\":"))
|
||||||
})
|
})
|
||||||
|
|
||||||
Context("when finding the plugin fails", func() {
|
Context("when finding the plugin fails", func() {
|
||||||
@ -241,10 +406,49 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
)
|
)
|
||||||
|
|
||||||
BeforeEach(func() {
|
BeforeEach(func() {
|
||||||
|
var err error
|
||||||
|
|
||||||
|
capabilityArgs := map[string]interface{}{
|
||||||
|
"portMappings": []portMapping{
|
||||||
|
{HostPort: 8080, ContainerPort: 80, Protocol: "tcp"},
|
||||||
|
},
|
||||||
|
"otherCapability": 33,
|
||||||
|
}
|
||||||
|
|
||||||
|
cniBinPath = filepath.Dir(pluginPaths["noop"])
|
||||||
|
cniConfig = libcni.CNIConfig{Path: []string{cniBinPath}}
|
||||||
|
runtimeConfig = &libcni.RuntimeConf{
|
||||||
|
ContainerID: "some-container-id",
|
||||||
|
NetNS: "/some/netns/path",
|
||||||
|
IfName: "some-eth0",
|
||||||
|
Args: [][2]string{{"FOO", "BAR"}},
|
||||||
|
CapabilityArgs: capabilityArgs,
|
||||||
|
}
|
||||||
|
|
||||||
|
expectedCmdArgs = skel.CmdArgs{
|
||||||
|
ContainerID: runtimeConfig.ContainerID,
|
||||||
|
Netns: runtimeConfig.NetNS,
|
||||||
|
IfName: runtimeConfig.IfName,
|
||||||
|
Args: "FOO=BAR",
|
||||||
|
Path: cniBinPath,
|
||||||
|
}
|
||||||
|
|
||||||
|
rc := map[string]interface{}{
|
||||||
|
"containerId": runtimeConfig.ContainerID,
|
||||||
|
"netNs": runtimeConfig.NetNS,
|
||||||
|
"ifName": runtimeConfig.IfName,
|
||||||
|
"args": map[string]string{
|
||||||
|
"FOO": "BAR",
|
||||||
|
},
|
||||||
|
"portMappings": capabilityArgs["portMappings"],
|
||||||
|
"otherCapability": capabilityArgs["otherCapability"],
|
||||||
|
}
|
||||||
|
|
||||||
|
ipResult := `{"dns":{},"ips":[{"version": "4", "address": "10.1.2.3/24"}]}`
|
||||||
plugins = make([]pluginInfo, 3, 3)
|
plugins = make([]pluginInfo, 3, 3)
|
||||||
plugins[0] = newPluginInfo("some-key", "some-value", "", true, `{"dns":{},"ips":[{"version": "4", "address": "10.1.2.3/24"}]}`)
|
plugins[0] = newPluginInfo("some-value", "", true, ipResult, rc, []string{"portMappings", "otherCapability"})
|
||||||
plugins[1] = newPluginInfo("some-key", "some-other-value", `{"dns":{},"ips":[{"version": "4", "address": "10.1.2.3/24"}]}`, true, "PASSTHROUGH")
|
plugins[1] = newPluginInfo("some-other-value", ipResult, true, "PASSTHROUGH", rc, []string{"otherCapability"})
|
||||||
plugins[2] = newPluginInfo("some-key", "yet-another-value", `{"dns":{},"ips":[{"version": "4", "address": "10.1.2.3/24"}]}`, true, "INJECT-DNS")
|
plugins[2] = newPluginInfo("yet-another-value", ipResult, true, "INJECT-DNS", rc, []string{})
|
||||||
|
|
||||||
configList := []byte(fmt.Sprintf(`{
|
configList := []byte(fmt.Sprintf(`{
|
||||||
"name": "some-list",
|
"name": "some-list",
|
||||||
@ -256,25 +460,13 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
]
|
]
|
||||||
}`, plugins[0].config, plugins[1].config, plugins[2].config))
|
}`, plugins[0].config, plugins[1].config, plugins[2].config))
|
||||||
|
|
||||||
var err error
|
|
||||||
netConfigList, err = libcni.ConfListFromBytes(configList)
|
netConfigList, err = libcni.ConfListFromBytes(configList)
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
})
|
||||||
|
|
||||||
cniBinPath = filepath.Dir(pluginPaths["noop"])
|
AfterEach(func() {
|
||||||
cniConfig = libcni.CNIConfig{Path: []string{cniBinPath}}
|
for _, p := range plugins {
|
||||||
runtimeConfig = &libcni.RuntimeConf{
|
Expect(os.RemoveAll(p.debugFilePath)).To(Succeed())
|
||||||
ContainerID: "some-container-id",
|
|
||||||
NetNS: "/some/netns/path",
|
|
||||||
IfName: "some-eth0",
|
|
||||||
Args: [][2]string{{"FOO", "BAR"}},
|
|
||||||
}
|
|
||||||
|
|
||||||
expectedCmdArgs = skel.CmdArgs{
|
|
||||||
ContainerID: "some-container-id",
|
|
||||||
Netns: "/some/netns/path",
|
|
||||||
IfName: "some-eth0",
|
|
||||||
Args: "FOO=BAR",
|
|
||||||
Path: cniBinPath,
|
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
@ -307,13 +499,10 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
debug, err := noop_debug.ReadDebug(plugins[i].debugFilePath)
|
debug, err := noop_debug.ReadDebug(plugins[i].debugFilePath)
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
Expect(debug.Command).To(Equal("ADD"))
|
Expect(debug.Command).To(Equal("ADD"))
|
||||||
newConfig, err := addNameToConfig("some-list", plugins[i].config)
|
|
||||||
Expect(err).NotTo(HaveOccurred())
|
|
||||||
|
|
||||||
// Must explicitly match JSON due to dict element ordering
|
// Must explicitly match JSON due to dict element ordering
|
||||||
debugJSON := debug.CmdArgs.StdinData
|
Expect(debug.CmdArgs.StdinData).To(MatchJSON(plugins[i].stdinData))
|
||||||
debug.CmdArgs.StdinData = nil
|
debug.CmdArgs.StdinData = nil
|
||||||
Expect(debugJSON).To(MatchJSON(newConfig))
|
|
||||||
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@ -351,13 +540,10 @@ var _ = Describe("Invoking plugins", func() {
|
|||||||
debug, err := noop_debug.ReadDebug(plugins[i].debugFilePath)
|
debug, err := noop_debug.ReadDebug(plugins[i].debugFilePath)
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
Expect(debug.Command).To(Equal("DEL"))
|
Expect(debug.Command).To(Equal("DEL"))
|
||||||
newConfig, err := addNameToConfig("some-list", plugins[i].config)
|
|
||||||
Expect(err).NotTo(HaveOccurred())
|
|
||||||
|
|
||||||
// Must explicitly match JSON due to dict element ordering
|
// Must explicitly match JSON due to dict element ordering
|
||||||
debugJSON := debug.CmdArgs.StdinData
|
Expect(debug.CmdArgs.StdinData).To(MatchJSON(plugins[i].stdinData))
|
||||||
debug.CmdArgs.StdinData = nil
|
debug.CmdArgs.StdinData = nil
|
||||||
Expect(debugJSON).To(MatchJSON(newConfig))
|
|
||||||
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
Expect(debug.CmdArgs).To(Equal(expectedCmdArgs))
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
@ -201,22 +201,24 @@ func LoadConfList(dir, name string) (*NetworkConfigList, error) {
|
|||||||
return ConfListFromConf(singleConf)
|
return ConfListFromConf(singleConf)
|
||||||
}
|
}
|
||||||
|
|
||||||
func InjectConf(original *NetworkConfig, key string, newValue interface{}) (*NetworkConfig, error) {
|
func InjectConf(original *NetworkConfig, newValues map[string]interface{}) (*NetworkConfig, error) {
|
||||||
config := make(map[string]interface{})
|
config := make(map[string]interface{})
|
||||||
err := json.Unmarshal(original.Bytes, &config)
|
err := json.Unmarshal(original.Bytes, &config)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("unmarshal existing network bytes: %s", err)
|
return nil, fmt.Errorf("unmarshal existing network bytes: %s", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if key == "" {
|
for key, value := range newValues {
|
||||||
return nil, fmt.Errorf("key value can not be empty")
|
if key == "" {
|
||||||
}
|
return nil, fmt.Errorf("keys cannot be empty")
|
||||||
|
}
|
||||||
|
|
||||||
if newValue == nil {
|
if value == nil {
|
||||||
return nil, fmt.Errorf("newValue must be specified")
|
return nil, fmt.Errorf("key '%s' value must not be nil", key)
|
||||||
}
|
}
|
||||||
|
|
||||||
config[key] = newValue
|
config[key] = value
|
||||||
|
}
|
||||||
|
|
||||||
newBytes, err := json.Marshal(config)
|
newBytes, err := json.Marshal(config)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
@ -115,6 +115,33 @@ var _ = Describe("Loading configuration from disk", func() {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
Describe("Capabilities", func() {
|
||||||
|
var configDir string
|
||||||
|
|
||||||
|
BeforeEach(func() {
|
||||||
|
var err error
|
||||||
|
configDir, err = ioutil.TempDir("", "plugin-conf")
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
|
pluginConfig := []byte(`{ "name": "some-plugin", "type": "noop", "cniVersion": "0.3.0", "capabilities": { "portMappings": true, "somethingElse": true, "noCapability": false } }`)
|
||||||
|
Expect(ioutil.WriteFile(filepath.Join(configDir, "50-whatever.conf"), pluginConfig, 0600)).To(Succeed())
|
||||||
|
})
|
||||||
|
|
||||||
|
AfterEach(func() {
|
||||||
|
Expect(os.RemoveAll(configDir)).To(Succeed())
|
||||||
|
})
|
||||||
|
|
||||||
|
It("reads plugin capabilities from network config", func() {
|
||||||
|
netConfig, err := libcni.LoadConf(configDir, "some-plugin")
|
||||||
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
Expect(netConfig.Network.Capabilities).To(Equal(map[string]bool{
|
||||||
|
"portMappings": true,
|
||||||
|
"somethingElse": true,
|
||||||
|
"noCapability": false,
|
||||||
|
}))
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
Describe("ConfFromFile", func() {
|
Describe("ConfFromFile", func() {
|
||||||
Context("when the file cannot be opened", func() {
|
Context("when the file cannot be opened", func() {
|
||||||
It("returns a useful error", func() {
|
It("returns a useful error", func() {
|
||||||
@ -286,18 +313,18 @@ var _ = Describe("Loading configuration from disk", func() {
|
|||||||
conf := &libcni.NetworkConfig{Network: &types.NetConf{Name: "some-plugin"},
|
conf := &libcni.NetworkConfig{Network: &types.NetConf{Name: "some-plugin"},
|
||||||
Bytes: []byte(`{ cc cc cc}`)}
|
Bytes: []byte(`{ cc cc cc}`)}
|
||||||
|
|
||||||
_, err := libcni.InjectConf(conf, "", nil)
|
_, err := libcni.InjectConf(conf, map[string]interface{}{"": nil})
|
||||||
Expect(err).To(MatchError(HavePrefix(`unmarshal existing network bytes`)))
|
Expect(err).To(MatchError(HavePrefix(`unmarshal existing network bytes`)))
|
||||||
})
|
})
|
||||||
|
|
||||||
It("returns key error", func() {
|
It("returns key error", func() {
|
||||||
_, err := libcni.InjectConf(testNetConfig, "", nil)
|
_, err := libcni.InjectConf(testNetConfig, map[string]interface{}{"": nil})
|
||||||
Expect(err).To(MatchError(HavePrefix(`key value can not be empty`)))
|
Expect(err).To(MatchError(HavePrefix(`keys cannot be empty`)))
|
||||||
})
|
})
|
||||||
|
|
||||||
It("returns newValue error", func() {
|
It("returns newValue error", func() {
|
||||||
_, err := libcni.InjectConf(testNetConfig, "test", nil)
|
_, err := libcni.InjectConf(testNetConfig, map[string]interface{}{"test": nil})
|
||||||
Expect(err).To(MatchError(HavePrefix(`newValue must be specified`)))
|
Expect(err).To(MatchError(HavePrefix(`key 'test' value must not be nil`)))
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
@ -305,7 +332,7 @@ var _ = Describe("Loading configuration from disk", func() {
|
|||||||
It("adds the new key & value to the config", func() {
|
It("adds the new key & value to the config", func() {
|
||||||
newPluginConfig := []byte(`{"name":"some-plugin","test":"test"}`)
|
newPluginConfig := []byte(`{"name":"some-plugin","test":"test"}`)
|
||||||
|
|
||||||
resultConfig, err := libcni.InjectConf(testNetConfig, "test", "test")
|
resultConfig, err := libcni.InjectConf(testNetConfig, map[string]interface{}{"test": "test"})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
||||||
Network: &types.NetConf{Name: "some-plugin"},
|
Network: &types.NetConf{Name: "some-plugin"},
|
||||||
@ -316,10 +343,10 @@ var _ = Describe("Loading configuration from disk", func() {
|
|||||||
It("adds the new value for exiting key", func() {
|
It("adds the new value for exiting key", func() {
|
||||||
newPluginConfig := []byte(`{"name":"some-plugin","test":"changedValue"}`)
|
newPluginConfig := []byte(`{"name":"some-plugin","test":"changedValue"}`)
|
||||||
|
|
||||||
resultConfig, err := libcni.InjectConf(testNetConfig, "test", "test")
|
resultConfig, err := libcni.InjectConf(testNetConfig, map[string]interface{}{"test": "test"})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
resultConfig, err = libcni.InjectConf(resultConfig, "test", "changedValue")
|
resultConfig, err = libcni.InjectConf(resultConfig, map[string]interface{}{"test": "changedValue"})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
||||||
@ -331,10 +358,10 @@ var _ = Describe("Loading configuration from disk", func() {
|
|||||||
It("adds existing key & value", func() {
|
It("adds existing key & value", func() {
|
||||||
newPluginConfig := []byte(`{"name":"some-plugin","test":"test"}`)
|
newPluginConfig := []byte(`{"name":"some-plugin","test":"test"}`)
|
||||||
|
|
||||||
resultConfig, err := libcni.InjectConf(testNetConfig, "test", "test")
|
resultConfig, err := libcni.InjectConf(testNetConfig, map[string]interface{}{"test": "test"})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
resultConfig, err = libcni.InjectConf(resultConfig, "test", "test")
|
resultConfig, err = libcni.InjectConf(resultConfig, map[string]interface{}{"test": "test"})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
||||||
@ -350,11 +377,11 @@ var _ = Describe("Loading configuration from disk", func() {
|
|||||||
newDNS := &types.DNS{Nameservers: servers, Domain: "local"}
|
newDNS := &types.DNS{Nameservers: servers, Domain: "local"}
|
||||||
|
|
||||||
// inject DNS
|
// inject DNS
|
||||||
resultConfig, err := libcni.InjectConf(testNetConfig, "dns", newDNS)
|
resultConfig, err := libcni.InjectConf(testNetConfig, map[string]interface{}{"dns": newDNS})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
// inject type
|
// inject type
|
||||||
resultConfig, err = libcni.InjectConf(resultConfig, "type", "bridge")
|
resultConfig, err = libcni.InjectConf(resultConfig, map[string]interface{}{"type": "bridge"})
|
||||||
Expect(err).NotTo(HaveOccurred())
|
Expect(err).NotTo(HaveOccurred())
|
||||||
|
|
||||||
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
Expect(resultConfig).To(Equal(&libcni.NetworkConfig{
|
||||||
|
@ -60,9 +60,10 @@ func (n *IPNet) UnmarshalJSON(data []byte) error {
|
|||||||
type NetConf struct {
|
type NetConf struct {
|
||||||
CNIVersion string `json:"cniVersion,omitempty"`
|
CNIVersion string `json:"cniVersion,omitempty"`
|
||||||
|
|
||||||
Name string `json:"name,omitempty"`
|
Name string `json:"name,omitempty"`
|
||||||
Type string `json:"type,omitempty"`
|
Type string `json:"type,omitempty"`
|
||||||
IPAM struct {
|
Capabilities map[string]bool `json:"capabilities,omitempty"`
|
||||||
|
IPAM struct {
|
||||||
Type string `json:"type,omitempty"`
|
Type string `json:"type,omitempty"`
|
||||||
} `json:"ipam,omitempty"`
|
} `json:"ipam,omitempty"`
|
||||||
DNS DNS `json:"dns"`
|
DNS DNS `json:"dns"`
|
||||||
|
Loading…
x
Reference in New Issue
Block a user