From 58f6bc9c8be42d145c8161327558ffde0976b58a Mon Sep 17 00:00:00 2001 From: Eugene Yakubovich Date: Mon, 1 Jun 2015 16:58:43 -0700 Subject: [PATCH 01/34] add flannel plugin This introduces a notion of a "meta" plugin. A meta plugin is one that delegates the actual work of setting up the interface to the main plugin. The meta plugin is used to select and dynamically configure the main plugin. The sequence of events, is as follows: Given netconf like: { "name": "mynet", "type": "flannel", "delegate": { "type": "bridge" } } flannel fills in values like "mtu", "ipam.subnet" and delegates to "bridge" main plugin. "bridge" plugin will operate as usual, calling into ipam module for IP assignment. Delegate dictionary should not contain "name" field as it will be filled in by the flannel plugin. --- flannel.md | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100644 flannel.md diff --git a/flannel.md b/flannel.md new file mode 100644 index 00000000..dcb6a663 --- /dev/null +++ b/flannel.md @@ -0,0 +1,86 @@ +# flannel plugin + +## Overview +This plugin is designed to work in conjunction with [flannel](https://github.com/coreos/flannel), a network fabric for containers. +When flannel daemon is started, it outputs a `/run/flannel/subnet.env` file that looks like this: +``` +FLANNEL_SUBNET=10.1.17.0/24 +FLANNEL_MTU=1472 +FLANNEL_IPMASQ=true +``` + +This information reflects the attributes of flannel network on the host. +The flannel CNI plugin uses this information to configure another CNI plugin, such as bridge plugin. + +## Operation +Given the following network configuration file and the contents of `/run/flannel/subnet.env` above, +``` +{ + "name": "mynet", + "type": "flannel" +} +``` +the flannel plugin will generate another network configuration file: +``` +{ + "name": "mynet", + "type": "bridge", + "mtu": 1472, + "ipMasq": false, + "isGateway": true, + "ipam": { + "type": "host-local", + "subnet": "10.1.17.0/24" + } +} +``` + +It will then invoke the bridge plugin, passing it the generated configuration. + +As can be seen from above, the flannel plugin, by default, will delegate to the bridge plugin. +If additional configuration values need to be passed to the bridge plugin, it can be done so via the `delegate` field: +``` +{ + "name": "mynet", + "type": "flannel", + "delegate": { + "bridge": "mynet0", + "mtu": 1400 + } +} +``` + +This supplies a configuration parameter to the bridge plugin -- the created bridge will now be named `mynet0`. +Notice that `mtu` has also been specified and this value will not be overwritten by flannel plugin. + +Additionally, the `delegate` field can be used to select a different kind of plugin altogether. +To use `ipvlan` instead of `bridge`, the following configuratoin can be specified: + +``` +{ + "name": "mynet", + "type": "flannel", + "delegate": { + "type": "ipvlan", + "master": "eth0" + } +} +``` + +## Network configuration reference + +* `name` (string, required): the name of the network +* `type` (string, required): "flannel" +* `subnetFile` (string, optional): full path to the subnet file written out by flanneld. Defaults to /run/flannel/subnet.env +* `delegate` (dictionary, optional): specifies configuration options for the delegated plugin. + +flannel plugin will always set the following fields in the delegated plugin configuration: + +* `name`: value of its "name" field. +* `ipam`: "host-local" type will be used with "subnet" set to `$FLANNEL_SUBNET`. + +flannel plugin will set the following fields in the delegated plugin configuration if they are not present: +* `ipMasq`: the inverse of `$FLANNEL_IPMASQ` +* `mtu`: `$FLANNEL_MTU` + +Additionally, for the bridge plugin, `isGateway` will be set to `true`, if not present. From 507805f8c49f8e052e0fbfc0552a7457e8af6295 Mon Sep 17 00:00:00 2001 From: Eugene Yakubovich Date: Fri, 10 Jul 2015 12:48:36 -0700 Subject: [PATCH 02/34] Document the plugins Adds a .md file to the Documentation/ folder for each plugin. Fixes #16 --- bridge.md | 37 +++++++++++++++++++++++++++++++++++++ dhcp.md | 35 +++++++++++++++++++++++++++++++++++ host-local.md | 36 ++++++++++++++++++++++++++++++++++++ ipvlan.md | 40 ++++++++++++++++++++++++++++++++++++++++ macvlan.md | 34 ++++++++++++++++++++++++++++++++++ 5 files changed, 182 insertions(+) create mode 100644 bridge.md create mode 100644 dhcp.md create mode 100644 host-local.md create mode 100644 ipvlan.md create mode 100644 macvlan.md diff --git a/bridge.md b/bridge.md new file mode 100644 index 00000000..bda5ac5c --- /dev/null +++ b/bridge.md @@ -0,0 +1,37 @@ +# bridge plugin + +## Overview + +With bridge plugin, all containers (on the same host) are plugged into a bridge (virtual switch) that resides in the host network namespace. +The containers receive one end of the veth pair with the other end connected to the bridge. +An IP address is only assigned to one end of the veth pair -- one residing in the container. +The bridge itself can also be assigned an IP address, turning it into a gateway for the containers. +Alternatively, the bridge can function purely in L2 mode and would need to be bridged to the host network interface (if other than container-to-container communication on the same host is desired). + +The network configuration specifies the name of the bridge to be used. +If the bridge is missing, the plugin will create one on first use and, if gateway mode is used, assign it an IP that was returned by IPAM plugin via the gateway field. + +## Example configuration +``` +{ + "name": "mynet", + "type": "bridge", + "bridge": "mynet0", + "isGateway": true, + "ipMasq": true, + "ipam": { + "type": "host-local", + "subnet": "10.10.0.0/16" + } +} +``` + +## Network configuration reference + +* `name` (string, required): the name of the network. +* `type` (string, required): "bridge". +* `bridge` (string, optional): name of the bridge to use/create. Defaults to "cni0". +* `isGateway` (boolean, optional): assign an IP address to the bridge. Defaults to false. +* `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. +* `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel. +* `ipam` (dictionary, required): IPAM configuration to be used for this network. diff --git a/dhcp.md b/dhcp.md new file mode 100644 index 00000000..298e96e0 --- /dev/null +++ b/dhcp.md @@ -0,0 +1,35 @@ +# dhcp plugin + +## Overview + +With dhcp plugin the containers can get an IP allocated by a DHCP server already running on your network. +This can be especially useful with plugin types such as [macvlan](https://github.com/appc/cni/blob/master/Documentation/macvlan.md). +Because a DHCP lease must be periodically renewed for the duration of container lifetime, a separate daemon is required to be running. +The same plugin binary can also be run in the daemon mode. + +## Operation +To use the dhcp IPAM plugin, first launch the dhcp daemon: + +``` +# Make sure the unix socket has been removed +$ rm -f /run/cni/dhcp.sock +$ ./dhcp daemon +``` + +Alternatively, you can use systemd socket activation protocol. +Be sure that the .socket file uses /run/cni/dhcp.sock as the socket path. + +With the daemon running, containers using the dhcp plugin can be launched. + +## Example configuration + +``` +{ + "ipam": { + "type": "dhcp", + } +} + +## Network configuration reference + +* `type` (string, required): "dhcp" diff --git a/host-local.md b/host-local.md new file mode 100644 index 00000000..ff6ee775 --- /dev/null +++ b/host-local.md @@ -0,0 +1,36 @@ +# host-local plugin + +## Overview + +host-local IPAM plugin allocates IPv4 addresses out of a specified address range. +It stores the state locally on the host filesystem, therefore ensuring uniqueness of IP addresses on a single host. + +## Example configuration +``` +{ + "ipam": { + "type": "host-local", + "subnet": "10.10.0.0/16", + "rangeStart": "10.10.1.20", + "rangeEnd": "10.10.3.50", + "gateway": "10.10.0.254", + "routes": [ + { "dst": "0.0.0.0/0" }, + { "dst": "192.168.0.0/16", "gw": "10.10.5.1" } + ] + } +} +``` + +## Network configuration reference + +* `type` (string, required): "host-local". +* `subnet` (string, required): CIDR block to allocate out of. +* `rangeStart` (string, optional): IP inside of "subnet" from which to start allocating addresses. Defaults to ".2" IP inside of the "subnet" block. +* `rangeEnd` (string, optional): IP inside of "subnet" with which to end allocating addresses. Defaults to ".254" IP inside of the "subnet" block. +* `gateway` (string, optional): IP inside of "subnet" to designate as the gateway. Defaults to ".1" IP inside of the "subnet" block. +* `routes` (string, optional): list of routes to add to the container namespace. Each route is a dictionary with "dst" and optional "gw" fields. If "gw" is omitted, value of "gateway" will be used. + +## Files + +Allocated IP addresses are stored as files in /var/lib/cni/networks/$NETWORK_NAME. diff --git a/ipvlan.md b/ipvlan.md new file mode 100644 index 00000000..f86a0159 --- /dev/null +++ b/ipvlan.md @@ -0,0 +1,40 @@ +# ipvlan plugin + +## Overview + +ipvlan is a new [addition](https://lwn.net/Articles/620087/) to the Linux kernel. +Like its cousin macvlan, it virtualizes the host interface. +However unlike macvlan which generates a new MAC address for each interface, ipvlan devices all share the same MAC. +The kernel driver inspects the IP address of each packet when making a decision about which virtual interface should process the packet. + +Because all ipvlan interfaces share the MAC address with the host interface, DHCP can only be used in conjunction with ClientID (currently not supported by DHCP plugin). + +## Example configuration + +``` +{ + "name": "mynet", + "type": "ipvlan", + "master": "eth0", + "ipam": { + "type": "host-local", + "subnet": "10.1.2.0/24", + } +} +``` + +## Network configuration reference + +* `name` (string, required): the name of the network. +* `type` (string, required): "ipvlan". +* `master` (string, required): name of the host interface to enslave. +* `mode` (string, optional): one of "l2", "l3". Defaults to "l2". +* `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel. +* `ipam` (dictionary, required): IPAM configuration to be used for this network. + +## Notes + +* `ipvlan` does not allow virtual interfaces to communicate with the master interface. +Therefore the container will not be able to reach the host via `ipvlan` interface. +Be sure to also have container join a network that provides connectivity to the host (e.g. `ptp`). +* A single master interface can not be enslaved by both `macvlan` and `ipvlan`. diff --git a/macvlan.md b/macvlan.md new file mode 100644 index 00000000..9e9f981d --- /dev/null +++ b/macvlan.md @@ -0,0 +1,34 @@ +# macvlan plugin + +## Overview + +[macvlan](http://backreference.org/2014/03/20/some-notes-on-macvlanmacvtap/) functions like a switch that is already connected to the host interface. +A host interface gets "enslaved" with the virtual interfaces sharing the physical device but having distinct MAC addresses. +Since each macvlan interface has its own MAC address, it makes it easy to use with exising DHCP servers already present on the network. + +## Example configuration + +``` +{ + "name": "mynet", + "type": "macvlan", + "master": "eth0", + "ipam": { + "type": "dhcp" + } +} +``` + +## Network configuration reference + +* `name` (string, required): the name of the network +* `type` (string, required): "macvlan" +* `master` (string, required): name of the host interface to enslave +* `mode` (string, optional): one of "bridge", "private", "vepa", "passthrough". Defaults to "bridge". +* `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel. +* `ipam` (dictionary, required): IPAM configuration to be used for this network. + +## Notes + +* If are testing on a laptop, please remember that most wireless cards do not support being enslaved by macvlan. +* A single master interface can not be enslaved by both `macvlan` and `ipvlan`. From f741bc324b1a6c4448b6e1ab403279a205eedd89 Mon Sep 17 00:00:00 2001 From: Eugene Yakubovich Date: Fri, 10 Jul 2015 12:18:25 -0700 Subject: [PATCH 03/34] Rename veth plugin to ptp and add docs. ptp (point-to-point) better describes the nature of the plugin as veth pairs are also used by the bridge plugin. --- ptp.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 ptp.md diff --git a/ptp.md b/ptp.md new file mode 100644 index 00000000..d4dde924 --- /dev/null +++ b/ptp.md @@ -0,0 +1,28 @@ +# ptp plugin + +## Overview +The ptp plugin creates a point-to-point link between a container and the host by using a veth device. +One end of the veth pair is placed inside a container and the other end resides on the host. +Both ends receive an IP address out of a /31 range. +The IP of the host end becomes the gateway address inside the container. + +Because ptp plugin requires a pair of IP addresses for each container, it should be used in conjuction with host-local-ptp IPAM plugin. + +## Example network configuration +``` +{ + "name": "mynet", + "type": "ptp", + "ipam": { + "type": "host-local-ptp", + "subnet": "10.1.1.0/24" + } +} + +## Network configuration reference + +* `name` (string, required): the name of the network +* `type` (string, required): "ptp" +* `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. +* `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to value chosen by the kernel. +* `ipam` (dictionary, required): IPAM configuration to be used for this network. From a41e1a699ecc18bc3bb315682e811c4afb3925d1 Mon Sep 17 00:00:00 2001 From: Alex Chan Date: Thu, 20 Aug 2015 16:41:25 +0100 Subject: [PATCH 04/34] Fix a few spelling mistakes in the docs --- flannel.md | 2 +- macvlan.md | 2 +- ptp.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/flannel.md b/flannel.md index dcb6a663..e223ddb3 100644 --- a/flannel.md +++ b/flannel.md @@ -54,7 +54,7 @@ This supplies a configuration parameter to the bridge plugin -- the created brid Notice that `mtu` has also been specified and this value will not be overwritten by flannel plugin. Additionally, the `delegate` field can be used to select a different kind of plugin altogether. -To use `ipvlan` instead of `bridge`, the following configuratoin can be specified: +To use `ipvlan` instead of `bridge`, the following configuration can be specified: ``` { diff --git a/macvlan.md b/macvlan.md index 9e9f981d..32448c61 100644 --- a/macvlan.md +++ b/macvlan.md @@ -4,7 +4,7 @@ [macvlan](http://backreference.org/2014/03/20/some-notes-on-macvlanmacvtap/) functions like a switch that is already connected to the host interface. A host interface gets "enslaved" with the virtual interfaces sharing the physical device but having distinct MAC addresses. -Since each macvlan interface has its own MAC address, it makes it easy to use with exising DHCP servers already present on the network. +Since each macvlan interface has its own MAC address, it makes it easy to use with existing DHCP servers already present on the network. ## Example configuration diff --git a/ptp.md b/ptp.md index d4dde924..a3270bcf 100644 --- a/ptp.md +++ b/ptp.md @@ -6,7 +6,7 @@ One end of the veth pair is placed inside a container and the other end resides Both ends receive an IP address out of a /31 range. The IP of the host end becomes the gateway address inside the container. -Because ptp plugin requires a pair of IP addresses for each container, it should be used in conjuction with host-local-ptp IPAM plugin. +Because ptp plugin requires a pair of IP addresses for each container, it should be used in conjunction with host-local-ptp IPAM plugin. ## Example network configuration ``` From a75401a13bca7316c2162b95c6551eb2917d2614 Mon Sep 17 00:00:00 2001 From: Stefan Junker Date: Sun, 16 Aug 2015 02:30:04 +0200 Subject: [PATCH 05/34] host-local: allow ip request via CNI_ARGS A specific IP can now be requested via the environment variable CNI_ARGS, e.g. `CNI_ARGS=ip=1.2.3.4`. The plugin will try to reserve the specified IP. If this is not successful the execution will fail. --- host-local.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/host-local.md b/host-local.md index ff6ee775..35aba355 100644 --- a/host-local.md +++ b/host-local.md @@ -31,6 +31,11 @@ It stores the state locally on the host filesystem, therefore ensuring uniquenes * `gateway` (string, optional): IP inside of "subnet" to designate as the gateway. Defaults to ".1" IP inside of the "subnet" block. * `routes` (string, optional): list of routes to add to the container namespace. Each route is a dictionary with "dst" and optional "gw" fields. If "gw" is omitted, value of "gateway" will be used. +## Supported arguments +The following [CNI_ARGS](https://github.com/appc/cni/blob/master/SPEC.md#parameters) are supported: + +* `ip`: request a specific IP address from the subnet. If it's not available, the plugin will exit with an error + ## Files Allocated IP addresses are stored as files in /var/lib/cni/networks/$NETWORK_NAME. From 4c57a90da6a15e7425310f482f39350c8ec85256 Mon Sep 17 00:00:00 2001 From: Stefan Junker Date: Wed, 23 Sep 2015 22:11:25 +0200 Subject: [PATCH 06/34] update ptp docs to reflect changes in plugin --- ptp.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/ptp.md b/ptp.md index a3270bcf..e7f2ffe7 100644 --- a/ptp.md +++ b/ptp.md @@ -3,10 +3,8 @@ ## Overview The ptp plugin creates a point-to-point link between a container and the host by using a veth device. One end of the veth pair is placed inside a container and the other end resides on the host. -Both ends receive an IP address out of a /31 range. -The IP of the host end becomes the gateway address inside the container. - -Because ptp plugin requires a pair of IP addresses for each container, it should be used in conjunction with host-local-ptp IPAM plugin. +The host-local IPAM plugin can be used to allocate an IP address to the container. +The traffic of the container interface will be routed through the interface of the host. ## Example network configuration ``` @@ -14,7 +12,7 @@ Because ptp plugin requires a pair of IP addresses for each container, it should "name": "mynet", "type": "ptp", "ipam": { - "type": "host-local-ptp", + "type": "host-local", "subnet": "10.1.1.0/24" } } From dd719a43cea9491cf98aece2e38c68d7a3607aba Mon Sep 17 00:00:00 2001 From: Eugene Yakubovich Date: Wed, 7 Oct 2015 15:11:54 -0700 Subject: [PATCH 07/34] flannel: doc fix and check for missing subnet.env fields - Document that flannel outputs FLANNEL_NETWORK b/c we use it - Check and error out if expecited fields are missing in subnetEnv file Fixes #72 --- flannel.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/flannel.md b/flannel.md index e223ddb3..2399ac48 100644 --- a/flannel.md +++ b/flannel.md @@ -4,7 +4,8 @@ This plugin is designed to work in conjunction with [flannel](https://github.com/coreos/flannel), a network fabric for containers. When flannel daemon is started, it outputs a `/run/flannel/subnet.env` file that looks like this: ``` -FLANNEL_SUBNET=10.1.17.0/24 +FLANNEL_NETWORK=10.1.0.0/16 +FLANNEL_SUBNET=10.1.17.1/24 FLANNEL_MTU=1472 FLANNEL_IPMASQ=true ``` From cf3c027960939f85311d90aa227050e01084a1c7 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Tue, 26 Jan 2016 18:54:56 +0100 Subject: [PATCH 08/34] *: add "dns" field to the configuration appc/cni#76 added a "dns" field in the result JSON. But before this patch, the plugins had no way of knowing which name server to return. There could be two ways of knowing which name server to return: 1. add it as an extra argument ("CNI_ARGS") 2. add it in the network configuration as a convenience (received via stdin) I chose the second way because it is easier. In the case of rkt, it means the user could just add the DNS name servers in /etc/rkt/net.d/mynetwork.conf. --- ptp.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/ptp.md b/ptp.md index e7f2ffe7..acde8515 100644 --- a/ptp.md +++ b/ptp.md @@ -14,7 +14,11 @@ The traffic of the container interface will be routed through the interface of t "ipam": { "type": "host-local", "subnet": "10.1.1.0/24" - } + }, + "dns": [ + "8.8.8.8", + "8.8.4.4" + ] } ## Network configuration reference @@ -24,3 +28,4 @@ The traffic of the container interface will be routed through the interface of t * `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. * `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to value chosen by the kernel. * `ipam` (dictionary, required): IPAM configuration to be used for this network. +* `dns` (string array, optional): name servers to return as is in the [Result](/SPEC.md#result). Defaults to empty list. From c7db40fb4c7bc09f7b4d809ea61f2d116e2f1f16 Mon Sep 17 00:00:00 2001 From: Stefan Junker Date: Wed, 27 Jan 2016 12:10:53 +0100 Subject: [PATCH 09/34] docs/ptp: update example and DNS description --- ptp.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/ptp.md b/ptp.md index acde8515..a7867109 100644 --- a/ptp.md +++ b/ptp.md @@ -15,10 +15,9 @@ The traffic of the container interface will be routed through the interface of t "type": "host-local", "subnet": "10.1.1.0/24" }, - "dns": [ - "8.8.8.8", - "8.8.4.4" - ] + "dns": { + "nameservers": [ "10.1.1.1", "8.8.8.8" ] + } } ## Network configuration reference @@ -28,4 +27,4 @@ The traffic of the container interface will be routed through the interface of t * `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. * `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to value chosen by the kernel. * `ipam` (dictionary, required): IPAM configuration to be used for this network. -* `dns` (string array, optional): name servers to return as is in the [Result](/SPEC.md#result). Defaults to empty list. +* `dns` (dictionary, optional): DNS information to return as described in the [Result](/SPEC.md#result). From 0b6ad716e8ac5b3eca041cb612bbc629e6f0c132 Mon Sep 17 00:00:00 2001 From: Alban Crequy Date: Mon, 8 Feb 2016 23:28:15 +0100 Subject: [PATCH 10/34] new plugin: tuning Allow users to tune net network parameters such as somaxconn. With this patch, users can add a new network configuration: > { > "name": "mytuning", > "type": "tuning", > "sysctl": { > "net.core.somaxconn": "500" > } > } The value /proc/sys/net/core/somaxconn will be set to 500 in the network namespace but will remain unchanged on the host. Only sysctl parameters that belong to the network subsystem can be modified. Related to: https://github.com/coreos/rkt/pull/2140 --- tuning.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 tuning.md diff --git a/tuning.md b/tuning.md new file mode 100644 index 00000000..bc786d88 --- /dev/null +++ b/tuning.md @@ -0,0 +1,36 @@ +# tuning plugin + +## Overview + +This plugin can change some system controls (sysctls) in the network namespace. +It does not create any network interfaces and therefore does not bring connectivity by itself. +It is only useful when used in addition to other plugins. + +## Operation +The following network configuration file +``` +{ + "name": "mytuning", + "type": "tuning", + "sysctl": { + "net.core.somaxconn": "500" + } +} +``` +will set /proc/sys/net/core/somaxconn to 500. +Other sysctls can be modified as long as they belong to the network namespace (`/proc/sys/net/*`). + +A successful result would simply be: +``` +{ + "cniVersion": "0.1.0" +} +``` + +## Network sysctls documentation + +Some network sysctls are documented in the Linux sources: + +- [Documentation/sysctl/net.txt](https://www.kernel.org/doc/Documentation/sysctl/net.txt) +- [Documentation/networking/ip-sysctl.txt](https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt) +- [Documentation/networking/](https://www.kernel.org/doc/Documentation/networking/) From 680d1f6ec29f98430421e24d4fb7e412f3810d2e Mon Sep 17 00:00:00 2001 From: Stefan Junker Date: Wed, 4 May 2016 16:06:05 +0200 Subject: [PATCH 11/34] *: appc/cni -> containernetworking/cni The project has been moved so internally we simply rename everything. Consumers are recommended to update their vendored version of cni. --- dhcp.md | 2 +- host-local.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dhcp.md b/dhcp.md index 298e96e0..7451c2c8 100644 --- a/dhcp.md +++ b/dhcp.md @@ -3,7 +3,7 @@ ## Overview With dhcp plugin the containers can get an IP allocated by a DHCP server already running on your network. -This can be especially useful with plugin types such as [macvlan](https://github.com/appc/cni/blob/master/Documentation/macvlan.md). +This can be especially useful with plugin types such as [macvlan](https://github.com/containernetworking/cni/blob/master/Documentation/macvlan.md). Because a DHCP lease must be periodically renewed for the duration of container lifetime, a separate daemon is required to be running. The same plugin binary can also be run in the daemon mode. diff --git a/host-local.md b/host-local.md index 35aba355..37446157 100644 --- a/host-local.md +++ b/host-local.md @@ -32,7 +32,7 @@ It stores the state locally on the host filesystem, therefore ensuring uniquenes * `routes` (string, optional): list of routes to add to the container namespace. Each route is a dictionary with "dst" and optional "gw" fields. If "gw" is omitted, value of "gateway" will be used. ## Supported arguments -The following [CNI_ARGS](https://github.com/appc/cni/blob/master/SPEC.md#parameters) are supported: +The following [CNI_ARGS](https://github.com/containernetworking/cni/blob/master/SPEC.md#parameters) are supported: * `ip`: request a specific IP address from the subnet. If it's not available, the plugin will exit with an error From b34210aa1c4c5a41988fc7f3fe65d31c146f86d9 Mon Sep 17 00:00:00 2001 From: Stefan Junker Date: Wed, 4 May 2016 16:06:05 +0200 Subject: [PATCH 12/34] *: appc/cni -> containernetworking/cni The project has been moved so internally we simply rename everything. Consumers are recommended to update their vendored version of cni. --- dhcp.md | 2 +- host-local.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dhcp.md b/dhcp.md index 298e96e0..7451c2c8 100644 --- a/dhcp.md +++ b/dhcp.md @@ -3,7 +3,7 @@ ## Overview With dhcp plugin the containers can get an IP allocated by a DHCP server already running on your network. -This can be especially useful with plugin types such as [macvlan](https://github.com/appc/cni/blob/master/Documentation/macvlan.md). +This can be especially useful with plugin types such as [macvlan](https://github.com/containernetworking/cni/blob/master/Documentation/macvlan.md). Because a DHCP lease must be periodically renewed for the duration of container lifetime, a separate daemon is required to be running. The same plugin binary can also be run in the daemon mode. diff --git a/host-local.md b/host-local.md index 35aba355..37446157 100644 --- a/host-local.md +++ b/host-local.md @@ -32,7 +32,7 @@ It stores the state locally on the host filesystem, therefore ensuring uniquenes * `routes` (string, optional): list of routes to add to the container namespace. Each route is a dictionary with "dst" and optional "gw" fields. If "gw" is omitted, value of "gateway" will be used. ## Supported arguments -The following [CNI_ARGS](https://github.com/appc/cni/blob/master/SPEC.md#parameters) are supported: +The following [CNI_ARGS](https://github.com/containernetworking/cni/blob/master/SPEC.md#parameters) are supported: * `ip`: request a specific IP address from the subnet. If it's not available, the plugin will exit with an error From 5de15979c6c870fbe2aa46f860760495f2425e6f Mon Sep 17 00:00:00 2001 From: Minhan Xia Date: Tue, 5 Apr 2016 10:50:28 -0700 Subject: [PATCH 13/34] plugins/bridge: add support to set hairpin mode --- bridge.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/bridge.md b/bridge.md index bda5ac5c..f9c7f2c4 100644 --- a/bridge.md +++ b/bridge.md @@ -19,6 +19,7 @@ If the bridge is missing, the plugin will create one on first use and, if gatewa "bridge": "mynet0", "isGateway": true, "ipMasq": true, + "hairpinMode": true, "ipam": { "type": "host-local", "subnet": "10.10.0.0/16" @@ -34,4 +35,5 @@ If the bridge is missing, the plugin will create one on first use and, if gatewa * `isGateway` (boolean, optional): assign an IP address to the bridge. Defaults to false. * `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. * `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel. +* `hairpinMode` (boolean, optional): set hairpin mode for interfaces on the bridge. Defaults to false. * `ipam` (dictionary, required): IPAM configuration to be used for this network. From 8814631f8026acc03cb76a3263177399dfcd13bc Mon Sep 17 00:00:00 2001 From: Stefan Junker Date: Fri, 4 Sep 2015 22:23:43 +0200 Subject: [PATCH 14/34] bridge: add isDefaultGateway field When isDefaultGateway is true it automatically sets isGateway to true. The default route will be added via the (bridge's) gateway IP. If a default gateway has been configured via IPAM in the same configuration file, the plugin will error out. --- bridge.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/bridge.md b/bridge.md index f9c7f2c4..dcf52b70 100644 --- a/bridge.md +++ b/bridge.md @@ -17,7 +17,7 @@ If the bridge is missing, the plugin will create one on first use and, if gatewa "name": "mynet", "type": "bridge", "bridge": "mynet0", - "isGateway": true, + "isDefaultGateway": true, "ipMasq": true, "hairpinMode": true, "ipam": { @@ -33,6 +33,7 @@ If the bridge is missing, the plugin will create one on first use and, if gatewa * `type` (string, required): "bridge". * `bridge` (string, optional): name of the bridge to use/create. Defaults to "cni0". * `isGateway` (boolean, optional): assign an IP address to the bridge. Defaults to false. +* `isDefaultGateway` (boolean, optional): Sets isGateway to true and makes the assigned IP the default route. Defaults to false. * `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. * `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel. * `hairpinMode` (boolean, optional): set hairpin mode for interfaces on the bridge. Defaults to false. From 04fbde4ce8a948abf5f5e8a0ccb610a965be80d9 Mon Sep 17 00:00:00 2001 From: Gabe Rosenhouse Date: Thu, 21 Jul 2016 13:03:58 -0700 Subject: [PATCH 15/34] docs: fix tuning doc, correct the result json it does not report the version, it never did --- tuning.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/tuning.md b/tuning.md index bc786d88..7ca16d0e 100644 --- a/tuning.md +++ b/tuning.md @@ -22,9 +22,7 @@ Other sysctls can be modified as long as they belong to the network namespace (` A successful result would simply be: ``` -{ - "cniVersion": "0.1.0" -} +{ } ``` ## Network sysctls documentation From 08529639f43bf7fcfa59b7de078f8d46ba979c88 Mon Sep 17 00:00:00 2001 From: Lukasz Zajaczkowski Date: Mon, 1 Aug 2016 12:53:46 +0200 Subject: [PATCH 16/34] documentation: add description for forceAddress parameter --- bridge.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/bridge.md b/bridge.md index dcf52b70..3abe99fd 100644 --- a/bridge.md +++ b/bridge.md @@ -18,6 +18,7 @@ If the bridge is missing, the plugin will create one on first use and, if gatewa "type": "bridge", "bridge": "mynet0", "isDefaultGateway": true, + "forceAddress": false, "ipMasq": true, "hairpinMode": true, "ipam": { @@ -34,6 +35,7 @@ If the bridge is missing, the plugin will create one on first use and, if gatewa * `bridge` (string, optional): name of the bridge to use/create. Defaults to "cni0". * `isGateway` (boolean, optional): assign an IP address to the bridge. Defaults to false. * `isDefaultGateway` (boolean, optional): Sets isGateway to true and makes the assigned IP the default route. Defaults to false. +* `forceAddress` (boolean, optional): Indicates if a new IP address should be set if the previous value has been changed. Defaults to false. * `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. * `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel. * `hairpinMode` (boolean, optional): set hairpin mode for interfaces on the bridge. Defaults to false. From 0ee270c769bb3420f73773bc4f18f5fbd8deef2a Mon Sep 17 00:00:00 2001 From: Dan Peterson Date: Sun, 9 Oct 2016 21:22:27 -0300 Subject: [PATCH 17/34] docs/ptp: fix markdown formatting --- ptp.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ptp.md b/ptp.md index a7867109..0ba1348a 100644 --- a/ptp.md +++ b/ptp.md @@ -7,6 +7,7 @@ The host-local IPAM plugin can be used to allocate an IP address to the containe The traffic of the container interface will be routed through the interface of the host. ## Example network configuration + ``` { "name": "mynet", @@ -19,6 +20,7 @@ The traffic of the container interface will be routed through the interface of t "nameservers": [ "10.1.1.1", "8.8.8.8" ] } } +``` ## Network configuration reference From 17b2a6d5e2de0cdf9da7c14b159d5562c9df8e84 Mon Sep 17 00:00:00 2001 From: Kuralamudhan Ramakrishnan Date: Sun, 23 Oct 2016 23:18:44 +0100 Subject: [PATCH 18/34] Update ipvlan.md invalid json format --- ipvlan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ipvlan.md b/ipvlan.md index f86a0159..988555ef 100644 --- a/ipvlan.md +++ b/ipvlan.md @@ -18,7 +18,7 @@ Because all ipvlan interfaces share the MAC address with the host interface, DHC "master": "eth0", "ipam": { "type": "host-local", - "subnet": "10.1.2.0/24", + "subnet": "10.1.2.0/24" } } ``` From ffbf7084062030dd53fa8c08b92e61790a8185cd Mon Sep 17 00:00:00 2001 From: "Mark St.Godard" Date: Wed, 9 Nov 2016 09:34:35 -0600 Subject: [PATCH 19/34] flannel: update flannel documentation add optional 'stateDir' to flannel docs --- flannel.md | 1 + 1 file changed, 1 insertion(+) diff --git a/flannel.md b/flannel.md index 2399ac48..c0ab3fde 100644 --- a/flannel.md +++ b/flannel.md @@ -73,6 +73,7 @@ To use `ipvlan` instead of `bridge`, the following configuration can be specifie * `name` (string, required): the name of the network * `type` (string, required): "flannel" * `subnetFile` (string, optional): full path to the subnet file written out by flanneld. Defaults to /run/flannel/subnet.env +* `stateDir` (string, optional): path to directory where plugin will store generated network configuration files. Defaults to `/var/lib/cni/flannel` * `delegate` (dictionary, optional): specifies configuration options for the delegated plugin. flannel plugin will always set the following fields in the delegated plugin configuration: From 475581873db89067c76e8173a6afefb4724c8c2d Mon Sep 17 00:00:00 2001 From: "Mark St.Godard" Date: Thu, 17 Nov 2016 15:54:29 -0600 Subject: [PATCH 20/34] flannel: rename stateDir to dataDir Rename StateDir to DataDir for flannel CNI plugin --- flannel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/flannel.md b/flannel.md index c0ab3fde..0efb6905 100644 --- a/flannel.md +++ b/flannel.md @@ -73,7 +73,7 @@ To use `ipvlan` instead of `bridge`, the following configuration can be specifie * `name` (string, required): the name of the network * `type` (string, required): "flannel" * `subnetFile` (string, optional): full path to the subnet file written out by flanneld. Defaults to /run/flannel/subnet.env -* `stateDir` (string, optional): path to directory where plugin will store generated network configuration files. Defaults to `/var/lib/cni/flannel` +* `dataDir` (string, optional): path to directory where plugin will store generated network configuration files. Defaults to `/var/lib/cni/flannel` * `delegate` (dictionary, optional): specifies configuration options for the delegated plugin. flannel plugin will always set the following fields in the delegated plugin configuration: From cd16252299831ffb84ddeb922a6bdd74400a062f Mon Sep 17 00:00:00 2001 From: Casey Callendrello Date: Mon, 21 Nov 2016 19:05:41 +0100 Subject: [PATCH 21/34] ipam/host-local: add ResolvConf argument for DNS configuration This adds the option `resolvConf` to the host-local IPAM configuration. If specified, the plugin will try to parse the file as a resolv.conf(5) type file and return it in the DNS response. --- host-local.md | 1 + 1 file changed, 1 insertion(+) diff --git a/host-local.md b/host-local.md index 37446157..00c1e84c 100644 --- a/host-local.md +++ b/host-local.md @@ -30,6 +30,7 @@ It stores the state locally on the host filesystem, therefore ensuring uniquenes * `rangeEnd` (string, optional): IP inside of "subnet" with which to end allocating addresses. Defaults to ".254" IP inside of the "subnet" block. * `gateway` (string, optional): IP inside of "subnet" to designate as the gateway. Defaults to ".1" IP inside of the "subnet" block. * `routes` (string, optional): list of routes to add to the container namespace. Each route is a dictionary with "dst" and optional "gw" fields. If "gw" is omitted, value of "gateway" will be used. +* `resolvConf` (string, optional): Path to a `resolv.conf` on the host to parse and return as the DNS configuration ## Supported arguments The following [CNI_ARGS](https://github.com/containernetworking/cni/blob/master/SPEC.md#parameters) are supported: From 98eb0d93a1abddb3e40325423f5a5f8dae8827a5 Mon Sep 17 00:00:00 2001 From: Gabe Rosenhouse Date: Wed, 15 Feb 2017 15:18:25 -0800 Subject: [PATCH 22/34] docs: consolidate host-local documentation - remove the README.md in the plugins/ipam/host-local (no other plugin has this) - merge that info into the main Documentation/host-local.md --- host-local.md | 50 +++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 45 insertions(+), 5 deletions(-) diff --git a/host-local.md b/host-local.md index 00c1e84c..073e369a 100644 --- a/host-local.md +++ b/host-local.md @@ -1,12 +1,17 @@ -# host-local plugin +# host-local IP address management plugin + +host-local IPAM allocates IPv4 and IPv6 addresses out of a specified address range. Optionally, +it can include a DNS configuration from a `resolv.conf` file on the host. ## Overview host-local IPAM plugin allocates IPv4 addresses out of a specified address range. It stores the state locally on the host filesystem, therefore ensuring uniqueness of IP addresses on a single host. -## Example configuration -``` +## Example configurations + +IPv4: +```json { "ipam": { "type": "host-local", @@ -17,11 +22,44 @@ It stores the state locally on the host filesystem, therefore ensuring uniquenes "routes": [ { "dst": "0.0.0.0/0" }, { "dst": "192.168.0.0/16", "gw": "10.10.5.1" } - ] + ], + "dataDir": "/var/my-orchestrator/container-ipam-state" } } ``` +IPv6: +```json +{ + "ipam": { + "type": "host-local", + "subnet": "3ffe:ffff:0:01ff::/64", + "rangeStart": "3ffe:ffff:0:01ff::0010", + "rangeEnd": "3ffe:ffff:0:01ff::0020", + "routes": [ + { "dst": "3ffe:ffff:0:01ff::1/64" } + ], + "resolvConf": "/etc/resolv.conf" + } +} +``` + +We can test it out on the command-line: + +```bash +$ export CNI_COMMAND=ADD +$ export CNI_CONTAINERID=f81d4fae-7dec-11d0-a765-00a0c91e6bf6 +$ echo '{ "name": "default", "ipam": { "type": "host-local", "subnet": "203.0.113.0/24" } }' | ./host-local +``` + +```json +{ + "ip4": { + "ip": "203.0.113.1/24" + } +} +``` + ## Network configuration reference * `type` (string, required): "host-local". @@ -31,6 +69,8 @@ It stores the state locally on the host filesystem, therefore ensuring uniquenes * `gateway` (string, optional): IP inside of "subnet" to designate as the gateway. Defaults to ".1" IP inside of the "subnet" block. * `routes` (string, optional): list of routes to add to the container namespace. Each route is a dictionary with "dst" and optional "gw" fields. If "gw" is omitted, value of "gateway" will be used. * `resolvConf` (string, optional): Path to a `resolv.conf` on the host to parse and return as the DNS configuration +* `dataDir` (string, optional): Path to a directory to use for maintaining state, e.g. which IPs have been allocated to which containers + ## Supported arguments The following [CNI_ARGS](https://github.com/containernetworking/cni/blob/master/SPEC.md#parameters) are supported: @@ -39,4 +79,4 @@ The following [CNI_ARGS](https://github.com/containernetworking/cni/blob/master/ ## Files -Allocated IP addresses are stored as files in /var/lib/cni/networks/$NETWORK_NAME. +Allocated IP addresses are stored as files in `/var/lib/cni/networks/$NETWORK_NAME`. The prefix can be customized with the `dataDir` option listed above. From f658c0b4b38b740d02891d3e1641947692a12115 Mon Sep 17 00:00:00 2001 From: Dan Williams Date: Sun, 26 Feb 2017 13:26:14 -0800 Subject: [PATCH 23/34] docs: CNI versioning for 0.3.0 upgrade --- spec-upgrades.md | 101 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 spec-upgrades.md diff --git a/spec-upgrades.md b/spec-upgrades.md new file mode 100644 index 00000000..b9ac77ec --- /dev/null +++ b/spec-upgrades.md @@ -0,0 +1,101 @@ +# CNI Plugin Versions, Compatibility, and Writing Plugins + +## Specification Changes between CNI Specification Versions 0.2.0 and +0.3.0 + +Plugins which conform to the CNI 0.3.0 specification may elect to +return more expressive IPAM JSON that is not compatible with earlier +specification versions. This requires updates to plugins that wish to +conform to the 0.3.0 specification. It **does not** require updates to +plugins that wish to only implement the 0.2.0 and earlier +specifications. + +## Multi-version Compatible Plugins + +Plugins can also choose to support multiple CNI versions. The plugin is +expected to return IPAM output conforming to the CNI specification +version passed in the plugin's configuration on stdin in the +`cniVersion` key. If that key is not present, then CNI specification +version 0.2.0 should be assumed and 0.2.0 conforming JSON returned. +Plugins should return all CNI specification versions they are +compatible with when given the `VERSION` command, so runtimes can +determine each plugins capabilities. + +For example, if the plugin advertises only CNI specification version +0.2.0 support and receives a `cniVersion` key of "0.3.0" the plugin +must return an error. If the plugin advertises CNI specification +support for 0.2.0 and 0.3.0, and receives `cniVersion` key of "0.2.0", +the plugin must return IPAM JSON conforming to the CNI specification +version 0.2.0. + +## libcni API and Go Types Changes in 0.3.0 + +With the 0.5.0 CNI reference plugins and libcni release a number of +important changes were made that affect both runtimes and plugin +authors. These changes make it easier to enhance the libcni and plugin +interfaces going forward. The largest changes are to the `types` +package, which now contains multiple versions of libcni types. The +`current` package contains the latest types, while types compatible +with previous libcni releases will be placed into versioned packages +like `types020` (representing the libcni types that conform to the CNI +specification version 0.2.0). + +The `Result` type is now an interface instead of a plain struct. The +interface has new functions to convert the Result structure between +different versions of the libcni types. Plugins and consumers of libcni +should generally be written to use the Go types of the latest libcni +release, and convert between the latest libcni types and the CNI +specification type they are passed in the `cniVersion` key of plugin +configuration. + +### Plugins + +For example, say your plugins supports both CNI specification version +0.2.0 and 0.3.0. Your plugin receives configuration with a `cniVersion` +key of "0.2.0". This means your plugin should return an IPAM structure +conforming to the CNI specification version 0.2.0. The easiest way to +code your plugin is to always use the most current libcni Result type +(from the `current` package) and then immediately before exiting, +convert that result to the requested CNI specification version result +(eg, `types020`) and print it to stdout. + +``` +import "github.com/containernetworking/cni/pkg/types" +import "github.com/containernetworking/cni/pkg/types/current" + +result := ¤t.Result{} +<<< populate result here >>> +return types.PrintResult(result, << CNI version from stdin net +config>>) +``` + +or if your plugin internally runs IPAM and needs to process the result: + +``` +import "github.com/containernetworking/cni/pkg/types" +import "github.com/containernetworking/cni/pkg/types/current" + +ipamResult, err := ipam.ExecAdd(n.IPAM.Type, args.StdinData) +result, err := current.NewResultFromResult(ipamResult) +<<< manipulate result here >>> +return types.PrintResult(result, << CNI version from stdin net +config>>) +``` + +### Runtimes + +Since libcni functions like AddNetwork() now return a Result interface +rather than a plain structure, only a single additional step is +required to convert that object to a structure you can manipulate +internally. Runtimes should code to the most current Go types provided +by the libcni they vendor into their sources, and can convert from +whatever IPAM result version the plugin provides to the most current +version like so: + +``` +resultInterface, err := cninet.AddNetwork(netconf, rt) +<<< resultInterface could wrap an object of any CNI specification +version >>> +realResult := current.NewResultFromResult(resultInterface) +<<< realResult is a struct, not an interface >>> +``` From 5b6d9556955eb19aa57a8f0fa05d2556e37c31b0 Mon Sep 17 00:00:00 2001 From: Gabe Rosenhouse Date: Sun, 26 Feb 2017 16:53:52 -0800 Subject: [PATCH 24/34] docs: Edits to v0.3.0 upgrade guidance --- spec-upgrades.md | 281 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 204 insertions(+), 77 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index b9ac77ec..7cfd2ed5 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -1,101 +1,228 @@ -# CNI Plugin Versions, Compatibility, and Writing Plugins +# How to upgrade to CNI Specification v0.3.0 -## Specification Changes between CNI Specification Versions 0.2.0 and -0.3.0 +Version 0.3.0 of the [CNI Specification](../SPEC.md) provides rich information +about container network configuration, including details of network interfaces +and support for multiple IP addresses. -Plugins which conform to the CNI 0.3.0 specification may elect to -return more expressive IPAM JSON that is not compatible with earlier -specification versions. This requires updates to plugins that wish to -conform to the 0.3.0 specification. It **does not** require updates to -plugins that wish to only implement the 0.2.0 and earlier -specifications. +To support this new data, the specification changed in a couple significant +ways that will impact CNI users, plugin authors, and runtime authors. -## Multi-version Compatible Plugins +This document provides guidance for how to upgrade: -Plugins can also choose to support multiple CNI versions. The plugin is -expected to return IPAM output conforming to the CNI specification -version passed in the plugin's configuration on stdin in the -`cniVersion` key. If that key is not present, then CNI specification -version 0.2.0 should be assumed and 0.2.0 conforming JSON returned. -Plugins should return all CNI specification versions they are -compatible with when given the `VERSION` command, so runtimes can -determine each plugins capabilities. +- [For CNI Users](#for-cni-users) +- [For Plugin Authors](#for-plugin-authors) +- [For Runtime Authors](#for-runtime-authors) -For example, if the plugin advertises only CNI specification version -0.2.0 support and receives a `cniVersion` key of "0.3.0" the plugin -must return an error. If the plugin advertises CNI specification -support for 0.2.0 and 0.3.0, and receives `cniVersion` key of "0.2.0", -the plugin must return IPAM JSON conforming to the CNI specification -version 0.2.0. +**Note**: the CNI Spec is versioned independently from the GitHub releases +for this repo. For example, Release v0.4.0 supports Spec version v0.2.0, +and Release v0.5.0 supports Spec v0.3.0. -## libcni API and Go Types Changes in 0.3.0 +---- -With the 0.5.0 CNI reference plugins and libcni release a number of -important changes were made that affect both runtimes and plugin -authors. These changes make it easier to enhance the libcni and plugin -interfaces going forward. The largest changes are to the `types` -package, which now contains multiple versions of libcni types. The -`current` package contains the latest types, while types compatible -with previous libcni releases will be placed into versioned packages -like `types020` (representing the libcni types that conform to the CNI -specification version 0.2.0). +## For CNI Users +If you maintain CNI configuration files for a container runtime that uses CNI, +ensure that the configuration files specify a `cniVersion` field and that the +version there is supported by your container runtime and CNI plugins. The CNI +spec includes example configuration files for [single plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations) +and for [lists of chained plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations). -The `Result` type is now an interface instead of a plain struct. The -interface has new functions to convert the Result structure between -different versions of the libcni types. Plugins and consumers of libcni -should generally be written to use the Go types of the latest libcni -release, and convert between the latest libcni types and the CNI -specification type they are passed in the `cniVersion` key of plugin -configuration. +Consult the documentation for your runtime and plugins to determine what +CNI spec versions they support. -### Plugins +## For Plugin Authors +This section provides guidance for upgrading plugins to CNI Spec Version 0.3.0. -For example, say your plugins supports both CNI specification version -0.2.0 and 0.3.0. Your plugin receives configuration with a `cniVersion` -key of "0.2.0". This means your plugin should return an IPAM structure -conforming to the CNI specification version 0.2.0. The easiest way to -code your plugin is to always use the most current libcni Result type -(from the `current` package) and then immediately before exiting, -convert that result to the requested CNI specification version result -(eg, `types020`) and print it to stdout. +### General guidance for all plugins (language agnostic) +To provide the smoothest upgrade path, **existing plugins should support +multiple versions of the CNI spec**. In particular, plugins with existing +installed bases should add support for CNI spec version 0.3.0 while maintaining +compatibility with older versions. -``` -import "github.com/containernetworking/cni/pkg/types" -import "github.com/containernetworking/cni/pkg/types/current" +To do this, two changes are required. First, a plugin should advertise which +CNI spec versions it supports. It does this by responding to the `VERSION` +command with the following JSON data: -result := ¤t.Result{} -<<< populate result here >>> -return types.PrintResult(result, << CNI version from stdin net -config>>) +```json +{ + "cniVersion": "0.3.0", + "supportedVersions": [ "0.1.0", "0.2.0", "0.3.0" ] +} ``` -or if your plugin internally runs IPAM and needs to process the result: +Second, for the `ADD` command, a plugin must respect the `cniVersion` field +provided in the [network configuration JSON](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration). +That field is a request for the plugin to return results of a particular format: +- If the `cniVersion` field is not present, then spec v0.2.0 should be assumed + and v0.2.0 format JSON returned. + +- If the plugin doesn't support the version, the plugin must error. + +- Otherwise, the plugin must return a [CNI Result](https://github.com/containernetworking/cni/blob/master/SPEC.md#result) + in the format requested. + +Result formats for older CNI spec versions are available in the +[git history for SPEC.md](https://github.com/containernetworking/cni/commits/master/SPEC.md). + +For example, suppose a plugin, via its `VERSION` response, advertises CNI specification +support for v0.2.0 and v0.3.0. When it receives `cniVersion` key of `0.2.0`, +the plugin must return result JSON conforming to CNI spec version 0.2.0. + +### Specific guidance for plugins written in Go +Plugins written in Go may leverage the Go language packages in this repository +to ease the process of upgrading and supporting multiple versions. CNI +[Library and Plugins Release v0.5.0](https://github.com/containernetworking/cni/releases) +includes important changes to the Golang APIs. Plugins using these APIs will +require some changes now, but should more-easily handle spec changes and +new features going forward. + +For plugin authors, the biggest change is that `types.Result` is now an +interface implemented by concrete struct types in the `types/current` and +`types/020` subpackages. + +Internally, plugins should use the `types/current` structs, and convert +to or from specific versions when required. In most plugins, the only +conversion required will at completion time, when it prints the result as +JSON to stdout. At that point, the result should be printed in the format +requested by the `cniVersion` field present in the network configuration. +Use the library function `types.PrintResult()` for that. + +Additionally, the plugin should advertise which CNI Spec versions it supports +via the 3rd argument to `skel.PluginMain()`. + +Here is some example code + +```go +import ( + "github.com/containernetworking/cni/pkg/skel" + "github.com/containernetworking/cni/pkg/types" + "github.com/containernetworking/cni/pkg/types/current" + "github.com/containernetworking/cni/pkg/version" +) + +func cmdAdd(args *skel.CmdArgs) error { + // determine spec version to use + var netConf struct { + types.NetConf + // other plugin-specific configuration goes here + } + err := json.Unmarshal(args.StdinData, &netConf) + cniVersion := netConf.CNIVersion + + // plugin does its work... + // set up interfaces + // assign addresses, etc + + // construct the result + result := ¤t.Result{ + Interfaces: []*current.Interface{ ... }, + IPs: []*current.IPs{ ... }, + ... + } + + // print result to stdout, in the format defined by the requested cniVersion + return types.PrintResult(result, cniVersion) +} + +func main() { + skel.PluginMain(cmdAdd, cmdDel, version.PluginSupports("0.1.0", "0.2.0", "0.3.0")) +} ``` -import "github.com/containernetworking/cni/pkg/types" -import "github.com/containernetworking/cni/pkg/types/current" -ipamResult, err := ipam.ExecAdd(n.IPAM.Type, args.StdinData) +Alternately, to use the result from a delegated IPAM plugin, the `result` +value might be formed like this: + +```go +ipamResult, err := ipam.ExecAdd(netConf.IPAM.Type, args.StdinData) result, err := current.NewResultFromResult(ipamResult) -<<< manipulate result here >>> -return types.PrintResult(result, << CNI version from stdin net -config>>) ``` -### Runtimes +Other examples of spec v0.3.0-compatible plugins are the +[main plugins in this repo](https://github.com/containernetworking/cni/tree/master/plugins/main) -Since libcni functions like AddNetwork() now return a Result interface -rather than a plain structure, only a single additional step is -required to convert that object to a structure you can manipulate -internally. Runtimes should code to the most current Go types provided -by the libcni they vendor into their sources, and can convert from -whatever IPAM result version the plugin provides to the most current -version like so: +## For Runtime Authors + +This section provides guidance for upgrading container runtimes to support +CNI Spec Version 0.3.0. + +### General guidance for all runtimes + +#### Support multiple CNI spec versions +To provide the smoothest upgrade path and support the broadest range of CNI +plugins, **container runtimes should support multiple versions of the CNI spec**. +In particular, runtimes with existing installed bases should add support for CNI +spec version 0.3.0 while maintaining compatibility with older versions. + +To support multiple versions of the CNI spec, runtimes should be able to +call both new and legacy plugins, and handle the results from either. + +When calling a plugin, the runtime must request that the plugin respond in a +particular format by specifying the `cniVersion` field in the +[Network Configuration](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration) +JSON block. The plugin will then respond with +a [Result](https://github.com/containernetworking/cni/blob/master/SPEC.md#result) +in the format defined by that CNI spec version, and the runtime must parse +and handle this result. + +#### Handle errors due to version incompatibility +Plugins may respond with error indicating that they don't support the requested +CNI version (see [Well-known Error Codes](https://github.com/containernetworking/cni/blob/master/SPEC.md#well-known-error-codes)), +e.g. +```json +{ + "cniVersion": "0.2.0", + "code": 1, + "msg": "CNI version not supported" +} ``` -resultInterface, err := cninet.AddNetwork(netconf, rt) -<<< resultInterface could wrap an object of any CNI specification -version >>> -realResult := current.NewResultFromResult(resultInterface) -<<< realResult is a struct, not an interface >>> +In that case, the runtime may retry with a lower CNI spec version, or take +some other action. + +#### (optional) Discover plugin version support +Runtimes may discover which CNI spec versions are supported by a plugin, by +calling the plugin with the `VERSION` command. The `VERSION` command was +added in CNI spec v0.2.0, so older plugins may not respect it. In the absence +of a successful response to `VERSION`, assume that the plugin only supports +CNI spec v0.1.0. + +#### Handle missing data in v0.3.0 results +The Result for the `ADD` command in CNI spec version 0.3.0 includes a new field +`interfaces`. An IP address in the `ip` field may describe which interface +it is assigned to, by placing a numeric index in the `interface` subfield. + +However, some plugins which are v0.3.0 compatible may nonetheless omit the +`interfaces` field and/or set the `interface` index value to `-1`. Runtimes +should gracefully handle this situation, unless they have good reason to rely +on the existence of the interface data. In that case, provide the user an +error message that helps diagnose the issue. + +### Specific guidance for container runtimes written in Go +Container runtimes written in Go may leverage the Go language packages in this +repository to ease the process of upgrading and supporting multiple versions. +CNI [Library and Plugins Release v0.5.0](https://github.com/containernetworking/cni/releases) +includes important changes to the Golang APIs. Runtimes using these APIs will +require some changes now, but should more-easily handle spec changes and +new features going forward. + +For runtimes, the biggest changes to the Go libraries are in the `types` package. +The top-level `types.Result` is now an opaque interface instead of a struct, and +APIs exposed by other packages, such as the high-level `libcni` package, have +been updated to use this interface. Concrete types are available as subpackages. +The `types/current` subpackage contains the latest (spec v0.3.0) types. + +A container runtime should use `current.NewResultFromResult()` to convert the +opaque `types.Result` to a concrete `current.Result` struct. It may then +work with the fields exposed by that struct: + +```go +// runtime invokes the plugin to get the opaque types.Result +// this may conform to any CNI spec version +resultInterface, err := libcni.AddNetwork(netConf, runtimeConf) + +// upconvert result to the current 0.3.0 spec +result, err := current.NewResultFromResult(resultInterface) + +// use the result fields .... +for _, ip := range result.IPs { ... } ``` From bdd70561ad7ac01ab7ef109287f6f15a4e3f84f8 Mon Sep 17 00:00:00 2001 From: Casey Callendrello Date: Mon, 27 Feb 2017 14:27:56 +0100 Subject: [PATCH 25/34] docs: minor improvements to 0.3.0 upgrade guidance --- spec-upgrades.md | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index 7cfd2ed5..b24411e4 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -206,10 +206,22 @@ require some changes now, but should more-easily handle spec changes and new features going forward. For runtimes, the biggest changes to the Go libraries are in the `types` package. -The top-level `types.Result` is now an opaque interface instead of a struct, and -APIs exposed by other packages, such as the high-level `libcni` package, have -been updated to use this interface. Concrete types are available as subpackages. -The `types/current` subpackage contains the latest (spec v0.3.0) types. +It has been refactored to make working with versioned results simpler. The top-level +`types.Result` is now an opaque interface instead of a struct, and APIs exposed by +other packages, such as the high-level `libcni` package, have been updated to use +this interface. Concrete types are now per-version subpackages. The `types/current` +subpackage contains the latest (spec v0.3.0) types. + +The versioned types understand how to convert between most versions. However, converting +to a higher version will mean some fields are missing. + +| From | 0.1 | 0.2 | 0.3 | +|--------|-----|-----|-----| +| To 0.1 | ✔ | ✔ | | +| To 0.2 | ✴ | ✔ | ✔ | +| To 0.3 | | ✴ | ✔ | + +*(pairs marked with ✴ will have some missing data)* A container runtime should use `current.NewResultFromResult()` to convert the opaque `types.Result` to a concrete `current.Result` struct. It may then From 1061ef1eb9808d65bf05742e3c7a319d25b3f741 Mon Sep 17 00:00:00 2001 From: Casey Callendrello Date: Mon, 27 Feb 2017 14:57:31 +0100 Subject: [PATCH 26/34] docs: add small upgrade instructions --- spec-upgrades.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index b24411e4..9c301aa1 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -22,12 +22,17 @@ and Release v0.5.0 supports Spec v0.3.0. ## For CNI Users If you maintain CNI configuration files for a container runtime that uses CNI, ensure that the configuration files specify a `cniVersion` field and that the -version there is supported by your container runtime and CNI plugins. The CNI -spec includes example configuration files for [single plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations) +version there is supported by your container runtime and CNI plugins. +Configuration files without the version field should be labeled version 0.2.0 +by default. The CNI spec includes example configuration files for +[single plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations) and for [lists of chained plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations). Consult the documentation for your runtime and plugins to determine what -CNI spec versions they support. +CNI spec versions they support. Test any plugin upgrades before deploying to +production. You may find [cnitool](https://github.com/containernetworking/cni/tree/master/cnitool) +useful. Specifically, your configuration version should be the lowest common +version supported by your plugins. ## For Plugin Authors This section provides guidance for upgrading plugins to CNI Spec Version 0.3.0. From 2aa863dab7a4ff4ba327adfb708b8c1a666984d3 Mon Sep 17 00:00:00 2001 From: Gabe Rosenhouse Date: Mon, 27 Feb 2017 07:11:04 -0800 Subject: [PATCH 27/34] docs: minor improvements to spec-upgrades --- spec-upgrades.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index 9c301aa1..d2b70473 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -86,11 +86,11 @@ interface implemented by concrete struct types in the `types/current` and `types/020` subpackages. Internally, plugins should use the `types/current` structs, and convert -to or from specific versions when required. In most plugins, the only -conversion required will at completion time, when it prints the result as -JSON to stdout. At that point, the result should be printed in the format -requested by the `cniVersion` field present in the network configuration. -Use the library function `types.PrintResult()` for that. +to or from specific versions when required. A typical plugin will only need +to do a single conversion. That is when it is about to complete and needs to +print the result JSON in the correct format to stdout. The library +function `types.PrintResult()` simplifies this by converting and printing in +a single call. Additionally, the plugin should advertise which CNI Spec versions it supports via the 3rd argument to `skel.PluginMain()`. @@ -151,7 +151,7 @@ Other examples of spec v0.3.0-compatible plugins are the This section provides guidance for upgrading container runtimes to support CNI Spec Version 0.3.0. -### General guidance for all runtimes +### General guidance for all runtimes (language agnostic) #### Support multiple CNI spec versions To provide the smoothest upgrade path and support the broadest range of CNI From 5124ac475930cfe92107251b50e2b3810507e5e5 Mon Sep 17 00:00:00 2001 From: Gabe Rosenhouse Date: Mon, 27 Feb 2017 07:53:09 -0800 Subject: [PATCH 28/34] docs: fill-out and correct version conversion table --- spec-upgrades.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index d2b70473..1543f193 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -217,16 +217,20 @@ other packages, such as the high-level `libcni` package, have been updated to us this interface. Concrete types are now per-version subpackages. The `types/current` subpackage contains the latest (spec v0.3.0) types. -The versioned types understand how to convert between most versions. However, converting -to a higher version will mean some fields are missing. +When up-converting older result types to spec v0.3.0, fields new in +spec v0.3.0 (like `interfaces`) may be empty. Conversely, when +down-converting v0.3.0 results to an older version, any data in those fields +will be lost. | From | 0.1 | 0.2 | 0.3 | |--------|-----|-----|-----| -| To 0.1 | ✔ | ✔ | | -| To 0.2 | ✴ | ✔ | ✔ | -| To 0.3 | | ✴ | ✔ | +| To 0.1 | ✔ | ✔ | x | +| To 0.2 | ✔ | ✔ | x | +| To 0.3 | ✴ | ✴ | ✔ | -*(pairs marked with ✴ will have some missing data)* +✔ : lossless conversion +✴ : higher-version output may have empty fields +x : lower-version output is missing some data A container runtime should use `current.NewResultFromResult()` to convert the opaque `types.Result` to a concrete `current.Result` struct. It may then From f15d521b2ab93dcff3dc3eb3f23e4ed94335197b Mon Sep 17 00:00:00 2001 From: Gabe Rosenhouse Date: Mon, 27 Feb 2017 07:57:00 -0800 Subject: [PATCH 29/34] docs: table formatting is hard --- spec-upgrades.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index 1543f193..4ebcb518 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -228,9 +228,13 @@ will be lost. | To 0.2 | ✔ | ✔ | x | | To 0.3 | ✴ | ✴ | ✔ | -✔ : lossless conversion -✴ : higher-version output may have empty fields -x : lower-version output is missing some data + +Key: +> ✔ : lossless conversion
+> ✴ : higher-version output may have empty fields
+> x : lower-version output is missing some data
+ + A container runtime should use `current.NewResultFromResult()` to convert the opaque `types.Result` to a concrete `current.Result` struct. It may then From c806f3d62b59736971c4a7d85eb23d34d8e8bcff Mon Sep 17 00:00:00 2001 From: Casey Callendrello Date: Tue, 28 Feb 2017 19:44:25 +0100 Subject: [PATCH 30/34] Minor rewording about default config version --- spec-upgrades.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/spec-upgrades.md b/spec-upgrades.md index 4ebcb518..f6447709 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -23,8 +23,8 @@ and Release v0.5.0 supports Spec v0.3.0. If you maintain CNI configuration files for a container runtime that uses CNI, ensure that the configuration files specify a `cniVersion` field and that the version there is supported by your container runtime and CNI plugins. -Configuration files without the version field should be labeled version 0.2.0 -by default. The CNI spec includes example configuration files for +Configuration files without a version field should be given version 0.2.0. +The CNI spec includes example configuration files for [single plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations) and for [lists of chained plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations). @@ -55,11 +55,11 @@ command with the following JSON data: ``` Second, for the `ADD` command, a plugin must respect the `cniVersion` field -provided in the [network configuration JSON](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration). +provided in the [network configuration JSON](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration). That field is a request for the plugin to return results of a particular format: - If the `cniVersion` field is not present, then spec v0.2.0 should be assumed - and v0.2.0 format JSON returned. + and v0.2.0 format result JSON returned. - If the plugin doesn't support the version, the plugin must error. From 8447ebc83a11c92316b92a758e293c2bb375e8f2 Mon Sep 17 00:00:00 2001 From: Dan Williams Date: Wed, 22 Mar 2017 14:21:53 -0500 Subject: [PATCH 31/34] spec/plugins: fix 'ip'->'ips' in the spec, bump to 0.3.1 --- spec-upgrades.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/spec-upgrades.md b/spec-upgrades.md index f6447709..c9efc67a 100644 --- a/spec-upgrades.md +++ b/spec-upgrades.md @@ -1,3 +1,9 @@ +# How to upgrade to CNI Specification v0.3.1 + +The 0.3.0 specification contained a small error. The Result structure's `ip` field should have been renamed to `ips` to be consistent with the IPAM result structure definition; this rename was missed when updating the Result to accommodate multiple IP addresses and interfaces. All first-party CNI plugins (bridge, host-local, etc) were updated to use `ips` (and thus be inconsistent with the 0.3.0 specification) and most other plugins have not been updated to the 0.3.0 specification yet, so few (if any) users should be impacted by this change. + +The 0.3.1 specification corrects the Result structure to use the `ips` field name as originally intended. This is the only change between 0.3.0 and 0.3.1. + # How to upgrade to CNI Specification v0.3.0 Version 0.3.0 of the [CNI Specification](../SPEC.md) provides rich information From 7d3fe6923b9beda96ff71683d9159ed7d67f0954 Mon Sep 17 00:00:00 2001 From: Avinash Sridharan Date: Thu, 30 Mar 2017 22:28:36 +0000 Subject: [PATCH 32/34] Documentation: Added documentation for `cnitool`. Added documentation to simplify the usage of `cnitool` and the settings expected to be used with `cnitool`. --- cnitool.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 cnitool.md diff --git a/cnitool.md b/cnitool.md new file mode 100644 index 00000000..a4bc7d35 --- /dev/null +++ b/cnitool.md @@ -0,0 +1,16 @@ +# Overview +The `cnitool` is a utility that can be used to test a CNI plugin +without the need for a container runtime. The `cnitool` takes a +`network name` and a `network namespace` and a command to `ADD` or +`DEL`,.i.e, attach or detach containers from a network. The `cnitool` +relies on the following environment variables to operate properly: +* `NETCONFPATH`: This environment variable needs to be set to a + directory. It defaults to `/etc/cni/net.d`. The `cnitool` searches + for CNI configuration files in this directory with the extension + `*.conf` or `*.json`. It loads all the CNI configuration files in + this directory and if it finds a CNI configuration with the `network + name` given to the cnitool it returns the corresponding CNI + configuration, else it returns `nil`. + +* `CNI_PATH`: For a given CNI configuration `cnitool` will search for + the corresponding CNI plugin in this path. From 9f52fc0374e3a8c2a75bd9a6c2dbd0813be57a3a Mon Sep 17 00:00:00 2001 From: Casey Callendrello Date: Mon, 15 May 2017 17:16:27 +0200 Subject: [PATCH 33/34] Move documentation folder to individual READMEs --- Documentation/cnitool.md | 16 -- Documentation/spec-upgrades.md | 259 ------------------ .../dhcp.md => plugins/ipam/dhcp/README.md | 2 +- .../ipam/host-local/README.md | 0 .../main/bridge/README.md | 0 .../main/ipvlan/README.md | 0 .../main/macvlan/README.md | 0 .../ptp.md => plugins/main/ptp/README.md | 2 +- .../meta/flannel/README.md | 0 .../meta/tuning/README.md | 0 10 files changed, 2 insertions(+), 277 deletions(-) delete mode 100644 Documentation/cnitool.md delete mode 100644 Documentation/spec-upgrades.md rename Documentation/dhcp.md => plugins/ipam/dhcp/README.md (91%) rename Documentation/host-local.md => plugins/ipam/host-local/README.md (100%) rename Documentation/bridge.md => plugins/main/bridge/README.md (100%) rename Documentation/ipvlan.md => plugins/main/ipvlan/README.md (100%) rename Documentation/macvlan.md => plugins/main/macvlan/README.md (100%) rename Documentation/ptp.md => plugins/main/ptp/README.md (92%) rename Documentation/flannel.md => plugins/meta/flannel/README.md (100%) rename Documentation/tuning.md => plugins/meta/tuning/README.md (100%) diff --git a/Documentation/cnitool.md b/Documentation/cnitool.md deleted file mode 100644 index a4bc7d35..00000000 --- a/Documentation/cnitool.md +++ /dev/null @@ -1,16 +0,0 @@ -# Overview -The `cnitool` is a utility that can be used to test a CNI plugin -without the need for a container runtime. The `cnitool` takes a -`network name` and a `network namespace` and a command to `ADD` or -`DEL`,.i.e, attach or detach containers from a network. The `cnitool` -relies on the following environment variables to operate properly: -* `NETCONFPATH`: This environment variable needs to be set to a - directory. It defaults to `/etc/cni/net.d`. The `cnitool` searches - for CNI configuration files in this directory with the extension - `*.conf` or `*.json`. It loads all the CNI configuration files in - this directory and if it finds a CNI configuration with the `network - name` given to the cnitool it returns the corresponding CNI - configuration, else it returns `nil`. - -* `CNI_PATH`: For a given CNI configuration `cnitool` will search for - the corresponding CNI plugin in this path. diff --git a/Documentation/spec-upgrades.md b/Documentation/spec-upgrades.md deleted file mode 100644 index c9efc67a..00000000 --- a/Documentation/spec-upgrades.md +++ /dev/null @@ -1,259 +0,0 @@ -# How to upgrade to CNI Specification v0.3.1 - -The 0.3.0 specification contained a small error. The Result structure's `ip` field should have been renamed to `ips` to be consistent with the IPAM result structure definition; this rename was missed when updating the Result to accommodate multiple IP addresses and interfaces. All first-party CNI plugins (bridge, host-local, etc) were updated to use `ips` (and thus be inconsistent with the 0.3.0 specification) and most other plugins have not been updated to the 0.3.0 specification yet, so few (if any) users should be impacted by this change. - -The 0.3.1 specification corrects the Result structure to use the `ips` field name as originally intended. This is the only change between 0.3.0 and 0.3.1. - -# How to upgrade to CNI Specification v0.3.0 - -Version 0.3.0 of the [CNI Specification](../SPEC.md) provides rich information -about container network configuration, including details of network interfaces -and support for multiple IP addresses. - -To support this new data, the specification changed in a couple significant -ways that will impact CNI users, plugin authors, and runtime authors. - -This document provides guidance for how to upgrade: - -- [For CNI Users](#for-cni-users) -- [For Plugin Authors](#for-plugin-authors) -- [For Runtime Authors](#for-runtime-authors) - -**Note**: the CNI Spec is versioned independently from the GitHub releases -for this repo. For example, Release v0.4.0 supports Spec version v0.2.0, -and Release v0.5.0 supports Spec v0.3.0. - ----- - -## For CNI Users -If you maintain CNI configuration files for a container runtime that uses CNI, -ensure that the configuration files specify a `cniVersion` field and that the -version there is supported by your container runtime and CNI plugins. -Configuration files without a version field should be given version 0.2.0. -The CNI spec includes example configuration files for -[single plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations) -and for [lists of chained plugins](https://github.com/containernetworking/cni/blob/master/SPEC.md#example-configurations). - -Consult the documentation for your runtime and plugins to determine what -CNI spec versions they support. Test any plugin upgrades before deploying to -production. You may find [cnitool](https://github.com/containernetworking/cni/tree/master/cnitool) -useful. Specifically, your configuration version should be the lowest common -version supported by your plugins. - -## For Plugin Authors -This section provides guidance for upgrading plugins to CNI Spec Version 0.3.0. - -### General guidance for all plugins (language agnostic) -To provide the smoothest upgrade path, **existing plugins should support -multiple versions of the CNI spec**. In particular, plugins with existing -installed bases should add support for CNI spec version 0.3.0 while maintaining -compatibility with older versions. - -To do this, two changes are required. First, a plugin should advertise which -CNI spec versions it supports. It does this by responding to the `VERSION` -command with the following JSON data: - -```json -{ - "cniVersion": "0.3.0", - "supportedVersions": [ "0.1.0", "0.2.0", "0.3.0" ] -} -``` - -Second, for the `ADD` command, a plugin must respect the `cniVersion` field -provided in the [network configuration JSON](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration). -That field is a request for the plugin to return results of a particular format: - -- If the `cniVersion` field is not present, then spec v0.2.0 should be assumed - and v0.2.0 format result JSON returned. - -- If the plugin doesn't support the version, the plugin must error. - -- Otherwise, the plugin must return a [CNI Result](https://github.com/containernetworking/cni/blob/master/SPEC.md#result) - in the format requested. - -Result formats for older CNI spec versions are available in the -[git history for SPEC.md](https://github.com/containernetworking/cni/commits/master/SPEC.md). - -For example, suppose a plugin, via its `VERSION` response, advertises CNI specification -support for v0.2.0 and v0.3.0. When it receives `cniVersion` key of `0.2.0`, -the plugin must return result JSON conforming to CNI spec version 0.2.0. - -### Specific guidance for plugins written in Go -Plugins written in Go may leverage the Go language packages in this repository -to ease the process of upgrading and supporting multiple versions. CNI -[Library and Plugins Release v0.5.0](https://github.com/containernetworking/cni/releases) -includes important changes to the Golang APIs. Plugins using these APIs will -require some changes now, but should more-easily handle spec changes and -new features going forward. - -For plugin authors, the biggest change is that `types.Result` is now an -interface implemented by concrete struct types in the `types/current` and -`types/020` subpackages. - -Internally, plugins should use the `types/current` structs, and convert -to or from specific versions when required. A typical plugin will only need -to do a single conversion. That is when it is about to complete and needs to -print the result JSON in the correct format to stdout. The library -function `types.PrintResult()` simplifies this by converting and printing in -a single call. - -Additionally, the plugin should advertise which CNI Spec versions it supports -via the 3rd argument to `skel.PluginMain()`. - -Here is some example code - -```go -import ( - "github.com/containernetworking/cni/pkg/skel" - "github.com/containernetworking/cni/pkg/types" - "github.com/containernetworking/cni/pkg/types/current" - "github.com/containernetworking/cni/pkg/version" -) - -func cmdAdd(args *skel.CmdArgs) error { - // determine spec version to use - var netConf struct { - types.NetConf - // other plugin-specific configuration goes here - } - err := json.Unmarshal(args.StdinData, &netConf) - cniVersion := netConf.CNIVersion - - // plugin does its work... - // set up interfaces - // assign addresses, etc - - // construct the result - result := ¤t.Result{ - Interfaces: []*current.Interface{ ... }, - IPs: []*current.IPs{ ... }, - ... - } - - // print result to stdout, in the format defined by the requested cniVersion - return types.PrintResult(result, cniVersion) -} - -func main() { - skel.PluginMain(cmdAdd, cmdDel, version.PluginSupports("0.1.0", "0.2.0", "0.3.0")) -} -``` - -Alternately, to use the result from a delegated IPAM plugin, the `result` -value might be formed like this: - -```go -ipamResult, err := ipam.ExecAdd(netConf.IPAM.Type, args.StdinData) -result, err := current.NewResultFromResult(ipamResult) -``` - -Other examples of spec v0.3.0-compatible plugins are the -[main plugins in this repo](https://github.com/containernetworking/cni/tree/master/plugins/main) - - -## For Runtime Authors - -This section provides guidance for upgrading container runtimes to support -CNI Spec Version 0.3.0. - -### General guidance for all runtimes (language agnostic) - -#### Support multiple CNI spec versions -To provide the smoothest upgrade path and support the broadest range of CNI -plugins, **container runtimes should support multiple versions of the CNI spec**. -In particular, runtimes with existing installed bases should add support for CNI -spec version 0.3.0 while maintaining compatibility with older versions. - -To support multiple versions of the CNI spec, runtimes should be able to -call both new and legacy plugins, and handle the results from either. - -When calling a plugin, the runtime must request that the plugin respond in a -particular format by specifying the `cniVersion` field in the -[Network Configuration](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration) -JSON block. The plugin will then respond with -a [Result](https://github.com/containernetworking/cni/blob/master/SPEC.md#result) -in the format defined by that CNI spec version, and the runtime must parse -and handle this result. - -#### Handle errors due to version incompatibility -Plugins may respond with error indicating that they don't support the requested -CNI version (see [Well-known Error Codes](https://github.com/containernetworking/cni/blob/master/SPEC.md#well-known-error-codes)), -e.g. -```json -{ - "cniVersion": "0.2.0", - "code": 1, - "msg": "CNI version not supported" -} -``` -In that case, the runtime may retry with a lower CNI spec version, or take -some other action. - -#### (optional) Discover plugin version support -Runtimes may discover which CNI spec versions are supported by a plugin, by -calling the plugin with the `VERSION` command. The `VERSION` command was -added in CNI spec v0.2.0, so older plugins may not respect it. In the absence -of a successful response to `VERSION`, assume that the plugin only supports -CNI spec v0.1.0. - -#### Handle missing data in v0.3.0 results -The Result for the `ADD` command in CNI spec version 0.3.0 includes a new field -`interfaces`. An IP address in the `ip` field may describe which interface -it is assigned to, by placing a numeric index in the `interface` subfield. - -However, some plugins which are v0.3.0 compatible may nonetheless omit the -`interfaces` field and/or set the `interface` index value to `-1`. Runtimes -should gracefully handle this situation, unless they have good reason to rely -on the existence of the interface data. In that case, provide the user an -error message that helps diagnose the issue. - -### Specific guidance for container runtimes written in Go -Container runtimes written in Go may leverage the Go language packages in this -repository to ease the process of upgrading and supporting multiple versions. -CNI [Library and Plugins Release v0.5.0](https://github.com/containernetworking/cni/releases) -includes important changes to the Golang APIs. Runtimes using these APIs will -require some changes now, but should more-easily handle spec changes and -new features going forward. - -For runtimes, the biggest changes to the Go libraries are in the `types` package. -It has been refactored to make working with versioned results simpler. The top-level -`types.Result` is now an opaque interface instead of a struct, and APIs exposed by -other packages, such as the high-level `libcni` package, have been updated to use -this interface. Concrete types are now per-version subpackages. The `types/current` -subpackage contains the latest (spec v0.3.0) types. - -When up-converting older result types to spec v0.3.0, fields new in -spec v0.3.0 (like `interfaces`) may be empty. Conversely, when -down-converting v0.3.0 results to an older version, any data in those fields -will be lost. - -| From | 0.1 | 0.2 | 0.3 | -|--------|-----|-----|-----| -| To 0.1 | ✔ | ✔ | x | -| To 0.2 | ✔ | ✔ | x | -| To 0.3 | ✴ | ✴ | ✔ | - - -Key: -> ✔ : lossless conversion
-> ✴ : higher-version output may have empty fields
-> x : lower-version output is missing some data
- - - -A container runtime should use `current.NewResultFromResult()` to convert the -opaque `types.Result` to a concrete `current.Result` struct. It may then -work with the fields exposed by that struct: - -```go -// runtime invokes the plugin to get the opaque types.Result -// this may conform to any CNI spec version -resultInterface, err := libcni.AddNetwork(netConf, runtimeConf) - -// upconvert result to the current 0.3.0 spec -result, err := current.NewResultFromResult(resultInterface) - -// use the result fields .... -for _, ip := range result.IPs { ... } -``` diff --git a/Documentation/dhcp.md b/plugins/ipam/dhcp/README.md similarity index 91% rename from Documentation/dhcp.md rename to plugins/ipam/dhcp/README.md index 7451c2c8..885eb09f 100644 --- a/Documentation/dhcp.md +++ b/plugins/ipam/dhcp/README.md @@ -3,7 +3,7 @@ ## Overview With dhcp plugin the containers can get an IP allocated by a DHCP server already running on your network. -This can be especially useful with plugin types such as [macvlan](https://github.com/containernetworking/cni/blob/master/Documentation/macvlan.md). +This can be especially useful with plugin types such as [macvlan](../../main/macvlan/README.md). Because a DHCP lease must be periodically renewed for the duration of container lifetime, a separate daemon is required to be running. The same plugin binary can also be run in the daemon mode. diff --git a/Documentation/host-local.md b/plugins/ipam/host-local/README.md similarity index 100% rename from Documentation/host-local.md rename to plugins/ipam/host-local/README.md diff --git a/Documentation/bridge.md b/plugins/main/bridge/README.md similarity index 100% rename from Documentation/bridge.md rename to plugins/main/bridge/README.md diff --git a/Documentation/ipvlan.md b/plugins/main/ipvlan/README.md similarity index 100% rename from Documentation/ipvlan.md rename to plugins/main/ipvlan/README.md diff --git a/Documentation/macvlan.md b/plugins/main/macvlan/README.md similarity index 100% rename from Documentation/macvlan.md rename to plugins/main/macvlan/README.md diff --git a/Documentation/ptp.md b/plugins/main/ptp/README.md similarity index 92% rename from Documentation/ptp.md rename to plugins/main/ptp/README.md index 0ba1348a..1467fb84 100644 --- a/Documentation/ptp.md +++ b/plugins/main/ptp/README.md @@ -29,4 +29,4 @@ The traffic of the container interface will be routed through the interface of t * `ipMasq` (boolean, optional): set up IP Masquerade on the host for traffic originating from this network and destined outside of it. Defaults to false. * `mtu` (integer, optional): explicitly set MTU to the specified value. Defaults to value chosen by the kernel. * `ipam` (dictionary, required): IPAM configuration to be used for this network. -* `dns` (dictionary, optional): DNS information to return as described in the [Result](/SPEC.md#result). +* `dns` (dictionary, optional): DNS information to return as described in the [Result](https://github.com/containernetworking/cni/blob/master/SPEC.md#result). diff --git a/Documentation/flannel.md b/plugins/meta/flannel/README.md similarity index 100% rename from Documentation/flannel.md rename to plugins/meta/flannel/README.md diff --git a/Documentation/tuning.md b/plugins/meta/tuning/README.md similarity index 100% rename from Documentation/tuning.md rename to plugins/meta/tuning/README.md From c9e3fc4890767a619624e804029a8f5975ff28a3 Mon Sep 17 00:00:00 2001 From: Casey Callendrello Date: Mon, 15 May 2017 19:07:19 +0200 Subject: [PATCH 34/34] Add references to all plugins to the README --- README.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 69c46457..5aa1dd55 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,23 @@ # plugins -Some additional CNI network plugins, matinained by the containernetworking team +Some CNI network plugins, maintained by the containernetworking team. For more information, see the individual READMEs. ## Plugins supplied: +### Main: interface-creating +* `bridge`: Creates a bridge, adds the host and the container to it. +* `ipvlan`: Adds an [ipvlan](https://www.kernel.org/doc/Documentation/networking/ipvlan.txt) interface in the container +* `loopback`: Creates a loopback interface +* `macvlan`: Creates a new MAC address, forwards all traffic to that to the container +* `ptp`: Creates a veth pair. +* `vlan`: Allocates a vlan device. + +### IPAM: IP address allocation +* `dhcp`: Runs a daemon on the host to make DHCP requests on behalf of the container +* `host-local`: maintains a local database of allocated IPs + +### Meta: other plugins +* `flannel`: generates an interface corresponding to a flannel config file +* `tuning`: Tweaks sysctl parameters of an existing interface + ### Sample - The sample plugin provides a base for building a sample plugin with tests. +The sample plugin provides an example for building your own plugin.