Create new Source Based Routing plugin
This creates a new plugin (sbr) which sets up source based routing, for use as a chained plugin for multi-network environments.
This commit is contained in:
parent
726759b29b
commit
29928cff4d
@ -26,6 +26,7 @@ Read [CONTRIBUTING](CONTRIBUTING.md) for build and test instructions.
|
||||
* `tuning`: Tweaks sysctl parameters of an existing interface
|
||||
* `portmap`: An iptables-based portmapping plugin. Maps ports from the host's address space to the container.
|
||||
* `bandwidth`: Allows bandwidth-limiting through use of traffic control tbf (ingress/egress).
|
||||
* `sbr`: A plugin that configures source based routing for an interface (from which it is chained).
|
||||
|
||||
### Sample
|
||||
The sample plugin provides an example for building your own plugin.
|
||||
|
127
plugins/meta/sbr/README.md
Normal file
127
plugins/meta/sbr/README.md
Normal file
@ -0,0 +1,127 @@
|
||||
# Source based routing plugin
|
||||
|
||||
## Introduction
|
||||
|
||||
This plugin performs Source Based Routing (SBR). The most common and standard way to
|
||||
perform routing is to base it purely on the destination. However, in some
|
||||
applications which are using network separation for traffic management and
|
||||
security, there is no way to tell *a priori* which interface should be used,
|
||||
but the application is capable of making the decision.
|
||||
|
||||
As an example, a Telco application might have two networks, a management
|
||||
network and a SIP (telephony) network for traffic, with rules which state that:
|
||||
|
||||
- SIP traffic (only) must be routed over the SIP network;
|
||||
|
||||
- all other traffic (but no SIP traffic) must be routed over the management
|
||||
network.
|
||||
|
||||
There is no way of configuring this based on destination IP, since there is no
|
||||
way of telling whether a destination IP on the internet is (say) an address
|
||||
used for downloading updated software packages or a remote SIP endpoint.
|
||||
|
||||
Hence Source Based Routing is used.
|
||||
|
||||
- The application explicitly listens on the correct interface for incoming
|
||||
traffic.
|
||||
|
||||
- When the application wishes to send to an address via the SIP network, it
|
||||
explicitly binds to the IP of the device on that network.
|
||||
|
||||
- Routes for the SIP interface are configured in a separate routing table, and
|
||||
a rule is configured to use that table based on the source IP address.
|
||||
|
||||
Note that in most cases there is a management device (the first one) and that
|
||||
the SBR plugin is called once for each device after the first, leaving the
|
||||
default routing table applied to the management device. However, this not
|
||||
mandatory, and source based routing may be configured on any or all of the
|
||||
devices as appropriate.
|
||||
|
||||
## Usage
|
||||
|
||||
This plugin runs as a chained plugin, and requires the following information
|
||||
passed in from the previous plugin (which has just set up the network device):
|
||||
|
||||
- What is the network interface in question?
|
||||
|
||||
- What is the IP address (or addresses) of that network interface?
|
||||
|
||||
- What is the default gateway for that interface (if any)?
|
||||
|
||||
It then reads all routes to the network interface in use.
|
||||
|
||||
Here is an example of what the plugin would do. (The `ip` based commands are
|
||||
implemented in go, but easier to describe via the command line.) Suppose that
|
||||
it reads that:
|
||||
|
||||
- The interface is `net1`.
|
||||
|
||||
- The IP address on that interface is `192.168.1.209`.
|
||||
|
||||
- The default gateway on that interface is `192.168.1.1`.
|
||||
|
||||
- There is one route configured on that network interface, which is
|
||||
`192.168.1.0/24`.
|
||||
|
||||
Then the actions it takes are the following.
|
||||
|
||||
- It creates a new routing table, and sets a rule to use it for the IP address in question.
|
||||
|
||||
ip rule add from 192.168.1.209/32 table 100
|
||||
|
||||
- It adds a route to the default gateway for the relevant table.
|
||||
|
||||
ip route add default via 192.168.1.1 dev net1 table 100
|
||||
|
||||
- It moves every existing route on the device to the new table.
|
||||
|
||||
ip route del 192.168.1.0/24 dev net1 src 192.168.1.209
|
||||
ip route add 192.168.1.0/24 dev net1 src 192.168.1.209 table 100
|
||||
|
||||
On deletion it:
|
||||
|
||||
- deletes the rule (`ip rule del from 192.168.1.209/32 table 100`), which it
|
||||
finds by deleting rules relating to IPs on the device which is about to be
|
||||
deleted.
|
||||
|
||||
- does nothing with routes (since the kernel automatically removes routes when
|
||||
the device with which they are associated is deleted).
|
||||
|
||||
## Future enhancements and known limitations
|
||||
|
||||
The following are possible future enhancements.
|
||||
|
||||
- The table number is currently selected by starting at 100, then incrementing
|
||||
the value until an unused table number is found. It might be nice to have an
|
||||
option to pass the table number as an input.
|
||||
|
||||
- There is no log severity, and there is no logging to file (pending changes to
|
||||
CNI logging generally).
|
||||
|
||||
- This plugin sets up Source Based Routing, as described above. In future,
|
||||
there may be a need for a VRF plugin (that uses
|
||||
[VRF routing](https://www.kernel.org/doc/Documentation/networking/vrf.txt)
|
||||
instead of source based routing). If and when this happens, it is likely that
|
||||
the logic would be virtually identical for the plugin, and so the same plugin
|
||||
might offer either SBR or VRF routing depending on configuration.
|
||||
|
||||
## Configuration
|
||||
|
||||
This plugin must be used as a chained plugin. There are no specific configuration parameters.
|
||||
|
||||
A sample configuration for this plugin acting as a chained plugin after flannel
|
||||
is the following.
|
||||
|
||||
~~~json
|
||||
{
|
||||
"name": "flannel-sbr",
|
||||
"cniVersion": "0.3.0",
|
||||
"plugins":
|
||||
[
|
||||
{ "type": "flannel" },
|
||||
{
|
||||
"type": "sbr",
|
||||
}
|
||||
]
|
||||
}
|
||||
~~~
|
378
plugins/meta/sbr/main.go
Normal file
378
plugins/meta/sbr/main.go
Normal file
@ -0,0 +1,378 @@
|
||||
// Copyright 2017 CNI authors
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// This is the Source Based Routing plugin that sets up source based routing.
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"net"
|
||||
|
||||
"github.com/alexflint/go-filemutex"
|
||||
"github.com/containernetworking/cni/pkg/skel"
|
||||
"github.com/containernetworking/cni/pkg/types"
|
||||
"github.com/containernetworking/cni/pkg/types/current"
|
||||
"github.com/containernetworking/cni/pkg/version"
|
||||
"github.com/containernetworking/plugins/pkg/ns"
|
||||
|
||||
"github.com/vishvananda/netlink"
|
||||
)
|
||||
|
||||
const firstTableID = 100
|
||||
|
||||
// PluginConf is the configuration document passed in.
|
||||
type PluginConf struct {
|
||||
types.NetConf
|
||||
|
||||
// This is the previous result, when called in the context of a chained
|
||||
// plugin. Because this plugin supports multiple versions, we'll have to
|
||||
// parse this in two passes. If your plugin is not chained, this can be
|
||||
// removed (though you may wish to error if a non-chainable plugin is
|
||||
// chained).
|
||||
RawPrevResult *map[string]interface{} `json:"prevResult"`
|
||||
PrevResult *current.Result `json:"-"`
|
||||
|
||||
// Add plugin-specific flags here
|
||||
}
|
||||
|
||||
// Wrapper that does a lock before and unlock after operations to serialise
|
||||
// this plugin.
|
||||
func withLockAndNetNS(nspath string, toRun func(_ ns.NetNS) error) error {
|
||||
// We lock on the network namespace to ensure that no other instance
|
||||
// clashes with this one.
|
||||
log.Printf("Network namespace to use and lock: %s", nspath)
|
||||
lock, err := filemutex.New(nspath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = lock.Lock()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = ns.WithNetNSPath(nspath, toRun)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Cleaner to unlock even though about to exit
|
||||
err = lock.Unlock()
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// parseConfig parses the supplied configuration (and prevResult) from stdin.
|
||||
func parseConfig(stdin []byte) (*PluginConf, error) {
|
||||
conf := PluginConf{}
|
||||
|
||||
if err := json.Unmarshal(stdin, &conf); err != nil {
|
||||
return nil, fmt.Errorf("failed to parse network configuration: %v", err)
|
||||
}
|
||||
|
||||
// Parse previous result.
|
||||
if conf.RawPrevResult != nil {
|
||||
resultBytes, err := json.Marshal(conf.RawPrevResult)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not serialize prevResult: %v", err)
|
||||
}
|
||||
res, err := version.NewResult(conf.CNIVersion, resultBytes)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not parse prevResult: %v", err)
|
||||
}
|
||||
conf.RawPrevResult = nil
|
||||
conf.PrevResult, err = current.NewResultFromResult(res)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not convert result to current version: %v", err)
|
||||
}
|
||||
}
|
||||
// End previous result parsing
|
||||
|
||||
return &conf, nil
|
||||
}
|
||||
|
||||
// getIPCfgs finds the IPs on the supplied interface, returning as IPConfig structures
|
||||
func getIPCfgs(iface string, prevResult *current.Result) ([]*current.IPConfig, error) {
|
||||
|
||||
if len(prevResult.IPs) == 0 {
|
||||
// No IP addresses; that makes no sense. Pack it in.
|
||||
return nil, fmt.Errorf("No IP addresses supplied on interface: %s", iface)
|
||||
}
|
||||
|
||||
// We do a single interface name, stored in args.IfName
|
||||
log.Printf("Checking for relevant interface: %s", iface)
|
||||
|
||||
// ips contains the IPConfig structures that were passed, filtered somewhat
|
||||
ipCfgs := make([]*current.IPConfig, 0, len(prevResult.IPs))
|
||||
|
||||
for _, ipCfg := range prevResult.IPs {
|
||||
// IPs have an interface that is an index into the interfaces array.
|
||||
// We assume a match if this index is missing.
|
||||
if ipCfg.Interface == nil {
|
||||
log.Printf("No interface for IP address %s", ipCfg.Address.IP)
|
||||
ipCfgs = append(ipCfgs, ipCfg)
|
||||
continue
|
||||
}
|
||||
|
||||
// Skip all IPs we know belong to an interface with the wrong name.
|
||||
intIdx := *ipCfg.Interface
|
||||
if intIdx >= 0 && intIdx < len(prevResult.Interfaces) && prevResult.Interfaces[intIdx].Name != iface {
|
||||
log.Printf("Incorrect interface for IP address %s", ipCfg.Address.IP)
|
||||
continue
|
||||
}
|
||||
|
||||
log.Printf("Found IP address %s", ipCfg.Address.IP.String())
|
||||
ipCfgs = append(ipCfgs, ipCfg)
|
||||
}
|
||||
|
||||
return ipCfgs, nil
|
||||
}
|
||||
|
||||
// cmdAdd is called for ADD requests
|
||||
func cmdAdd(args *skel.CmdArgs) error {
|
||||
conf, err := parseConfig(args.StdinData)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
log.Printf("Configure SBR for new interface %s - previous result: %v",
|
||||
args.IfName, conf.PrevResult)
|
||||
|
||||
if conf.PrevResult == nil {
|
||||
return fmt.Errorf("This plugin must be called as chained plugin")
|
||||
}
|
||||
|
||||
// Get the list of relevant IPs.
|
||||
ipCfgs, err := getIPCfgs(args.IfName, conf.PrevResult)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Do the actual work.
|
||||
err = withLockAndNetNS(args.Netns, func(_ ns.NetNS) error {
|
||||
return doRoutes(ipCfgs, conf.PrevResult.Routes, args.IfName)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Pass through the result for the next plugin
|
||||
return types.PrintResult(conf.PrevResult, conf.CNIVersion)
|
||||
}
|
||||
|
||||
// doRoutes does all the work to set up routes and rules during an add.
|
||||
func doRoutes(ipCfgs []*current.IPConfig, origRoutes []*types.Route, iface string) error {
|
||||
// Get a list of rules and routes ready.
|
||||
rules, err := netlink.RuleList(netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to list all rules: %v", err)
|
||||
}
|
||||
|
||||
routes, err := netlink.RouteList(nil, netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to list all routes: %v", err)
|
||||
}
|
||||
|
||||
// Pick a table ID to use. We pick the first table ID from firstTableID
|
||||
// on that has no existing rules mapping to it and no existing routes in
|
||||
// it.
|
||||
table := firstTableID
|
||||
for {
|
||||
foundExisting := false
|
||||
for _, rule := range rules {
|
||||
if rule.Table == table {
|
||||
foundExisting = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
for _, route := range routes {
|
||||
if route.Table == table {
|
||||
foundExisting = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if foundExisting {
|
||||
table++
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
log.Printf("First unreferenced table: %d", table)
|
||||
|
||||
link, err := netlink.LinkByName(iface)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Cannot find network interface %s: %v", iface, err)
|
||||
}
|
||||
|
||||
linkIndex := link.Attrs().Index
|
||||
|
||||
// Loop through setting up source based rules and default routes.
|
||||
for _, ipCfg := range ipCfgs {
|
||||
log.Printf("Set rule for source %s", ipCfg.String())
|
||||
rule := netlink.NewRule()
|
||||
rule.Table = table
|
||||
|
||||
// Source must be restricted to a single IP, not a full subnet
|
||||
var src net.IPNet
|
||||
src.IP = ipCfg.Address.IP
|
||||
if ipCfg.Version == "4" {
|
||||
src.Mask = net.CIDRMask(32, 32)
|
||||
} else {
|
||||
src.Mask = net.CIDRMask(64, 64)
|
||||
}
|
||||
|
||||
log.Printf("Source to use %s", src.String())
|
||||
rule.Src = &src
|
||||
|
||||
if err = netlink.RuleAdd(rule); err != nil {
|
||||
return fmt.Errorf("Failed to add rule: %v", err)
|
||||
}
|
||||
|
||||
// Add a default route, since this may have been removed by previous
|
||||
// plugin.
|
||||
if ipCfg.Gateway != nil {
|
||||
log.Printf("Adding default route to gateway %s", ipCfg.Gateway.String())
|
||||
|
||||
var dest net.IPNet
|
||||
if ipCfg.Version == "4" {
|
||||
dest.IP = net.IPv4zero
|
||||
dest.Mask = net.CIDRMask(0, 32)
|
||||
} else {
|
||||
dest.IP = net.IPv6zero
|
||||
dest.Mask = net.CIDRMask(0, 64)
|
||||
}
|
||||
|
||||
route := netlink.Route{
|
||||
Dst: &dest,
|
||||
Gw: ipCfg.Gateway,
|
||||
Table: table,
|
||||
LinkIndex: linkIndex}
|
||||
|
||||
err = netlink.RouteAdd(&route)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to add default route to %s: %v",
|
||||
ipCfg.Gateway.String(),
|
||||
err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Move all routes into the correct table. We are taking a shortcut; all
|
||||
// the routes have been added to the interface anyway but in the wrong
|
||||
// table, so instead of removing them we just move them to the table we
|
||||
// want them in.
|
||||
routes, err = netlink.RouteList(link, netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Unable to list routes: %v", err)
|
||||
}
|
||||
|
||||
for _, route := range routes {
|
||||
log.Printf("Moving route %s from table %d to %d",
|
||||
route.String(), route.Table, table)
|
||||
|
||||
err := netlink.RouteDel(&route)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to delete route: %v", err)
|
||||
}
|
||||
|
||||
route.Table = table
|
||||
|
||||
// We use route replace in case the route already exists, which
|
||||
// is possible for the default gateway we added above.
|
||||
err = netlink.RouteReplace(&route)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to readd route: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// cmdDel is called for DELETE requests
|
||||
func cmdDel(args *skel.CmdArgs) error {
|
||||
// We care a bit about config because it sets log level.
|
||||
_, err := parseConfig(args.StdinData)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
log.Printf("Cleaning up SBR for %s", args.IfName)
|
||||
err = withLockAndNetNS(args.Netns, func(_ ns.NetNS) error {
|
||||
return tidyRules(args.IfName)
|
||||
})
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// Tidy up the rules for the deleted interface
|
||||
func tidyRules(iface string) error {
|
||||
|
||||
// We keep on going on rule deletion error, but return the last failure.
|
||||
var errReturn error
|
||||
|
||||
rules, err := netlink.RuleList(netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
log.Printf("Failed to list all rules to tidy: %v", err)
|
||||
return fmt.Errorf("Failed to list all rules to tidy: %v", err)
|
||||
}
|
||||
|
||||
link, err := netlink.LinkByName(iface)
|
||||
if err != nil {
|
||||
log.Printf("Failed to get link %s: %v", iface, err)
|
||||
return fmt.Errorf("Failed to get link %s: %v", iface, err)
|
||||
}
|
||||
|
||||
addrs, err := netlink.AddrList(link, netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
log.Printf("Failed to list all addrs: %v", err)
|
||||
return fmt.Errorf("Failed to list all addrs: %v", err)
|
||||
}
|
||||
|
||||
RULE_LOOP:
|
||||
for _, rule := range rules {
|
||||
log.Printf("Check rule: %v", rule)
|
||||
if rule.Src == nil {
|
||||
continue
|
||||
}
|
||||
|
||||
for _, addr := range addrs {
|
||||
if rule.Src.IP.Equal(addr.IP) {
|
||||
log.Printf("Delete rule %v", rule)
|
||||
err := netlink.RuleDel(&rule)
|
||||
if err != nil {
|
||||
errReturn = fmt.Errorf("Failed to delete rule %v", err)
|
||||
log.Printf("... Failed! %v", err)
|
||||
}
|
||||
continue RULE_LOOP
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
return errReturn
|
||||
}
|
||||
|
||||
func main() {
|
||||
skel.PluginMain(cmdAdd, cmdGet, cmdDel, version.All, "TODO")
|
||||
}
|
||||
|
||||
func cmdGet(args *skel.CmdArgs) error {
|
||||
return fmt.Errorf("not implemented")
|
||||
}
|
430
plugins/meta/sbr/sbr_linux_test.go
Normal file
430
plugins/meta/sbr/sbr_linux_test.go
Normal file
@ -0,0 +1,430 @@
|
||||
// Copyright 2017 CNI authors
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"net"
|
||||
|
||||
"github.com/containernetworking/cni/pkg/skel"
|
||||
"github.com/containernetworking/plugins/pkg/ns"
|
||||
"github.com/containernetworking/plugins/pkg/testutils"
|
||||
|
||||
"github.com/vishvananda/netlink"
|
||||
"golang.org/x/sys/unix"
|
||||
|
||||
. "github.com/onsi/ginkgo"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
// Structures specifying state at start.
|
||||
type Device struct {
|
||||
Name string
|
||||
Addrs []net.IPNet
|
||||
Routes []netlink.Route
|
||||
}
|
||||
|
||||
type Rule struct {
|
||||
Src string
|
||||
Table int
|
||||
}
|
||||
|
||||
type netStatus struct {
|
||||
Devices []Device
|
||||
Rules []netlink.Rule
|
||||
}
|
||||
|
||||
// Create a link, shove some IP addresses on it, and push it into the network
|
||||
// namespace.
|
||||
func setup(targetNs ns.NetNS, status netStatus) error {
|
||||
// Get the status right
|
||||
err := targetNs.Do(func(_ ns.NetNS) error {
|
||||
for _, dev := range status.Devices {
|
||||
log.Printf("Adding dev %s\n", dev.Name)
|
||||
link := &netlink.Dummy{LinkAttrs: netlink.LinkAttrs{Name: dev.Name}}
|
||||
err := netlink.LinkAdd(link)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
err = netlink.LinkSetUp(link)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, addr := range dev.Addrs {
|
||||
log.Printf("Adding address %v to device %s\n", addr, dev.Name)
|
||||
err = netlink.AddrAdd(link, &netlink.Addr{IPNet: &addr})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
for _, route := range dev.Routes {
|
||||
log.Printf("Adding route %v to device %s\n", route, dev.Name)
|
||||
route.LinkIndex = link.Attrs().Index
|
||||
err = netlink.RouteAdd(&route)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
// Readback the routes and rules.
|
||||
func readback(targetNs ns.NetNS, devNames []string) (netStatus, error) {
|
||||
// Get the status right.
|
||||
var retVal netStatus
|
||||
|
||||
err := targetNs.Do(func(_ ns.NetNS) error {
|
||||
retVal.Devices = make([]Device, 2)
|
||||
|
||||
for i, name := range devNames {
|
||||
log.Printf("Checking device %s", name)
|
||||
retVal.Devices[i].Name = name
|
||||
|
||||
link, err := netlink.LinkByName(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Need to read all tables, so cannot use RouteList
|
||||
routeFilter := &netlink.Route{
|
||||
LinkIndex: link.Attrs().Index,
|
||||
Table: unix.RT_TABLE_UNSPEC,
|
||||
}
|
||||
|
||||
routes, err := netlink.RouteListFiltered(netlink.FAMILY_ALL,
|
||||
routeFilter,
|
||||
netlink.RT_FILTER_OIF|netlink.RT_FILTER_TABLE)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, route := range routes {
|
||||
log.Printf("Got %s route %v", name, route)
|
||||
}
|
||||
|
||||
retVal.Devices[i].Routes = routes
|
||||
}
|
||||
|
||||
rules, err := netlink.RuleList(netlink.FAMILY_ALL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
retVal.Rules = make([]netlink.Rule, 0, len(rules))
|
||||
|
||||
for _, rule := range rules {
|
||||
// Rules over 250 are the kernel defaults, that we ignore.
|
||||
if rule.Table < 250 {
|
||||
log.Printf("Got interesting rule %v", rule)
|
||||
retVal.Rules = append(retVal.Rules, rule)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
return retVal, err
|
||||
}
|
||||
|
||||
func equalRoutes(expected, actual []netlink.Route) bool {
|
||||
// Compare two sets of routes, comparing only destination, gateway and
|
||||
// table. Return true if equal.
|
||||
match := true
|
||||
|
||||
// Map used to make comparisons easy
|
||||
expMap := make(map[string]bool)
|
||||
|
||||
for _, route := range expected {
|
||||
expMap[route.String()] = true
|
||||
}
|
||||
|
||||
for _, route := range actual {
|
||||
routeString := route.String()
|
||||
if expMap[routeString] {
|
||||
log.Printf("Route %s expected and found", routeString)
|
||||
delete(expMap, routeString)
|
||||
} else {
|
||||
log.Printf("Route %s found, not expected", routeString)
|
||||
match = false
|
||||
}
|
||||
}
|
||||
|
||||
for expRoute := range expMap {
|
||||
log.Printf("Route %s expected, not found", expRoute)
|
||||
match = false
|
||||
}
|
||||
|
||||
return match
|
||||
}
|
||||
|
||||
func createDefaultStatus() netStatus {
|
||||
// Useful default status.
|
||||
devs := make([]Device, 2)
|
||||
rules := make([]netlink.Rule, 0)
|
||||
|
||||
devs[0] = Device{Name: "eth0"}
|
||||
devs[0].Addrs = make([]net.IPNet, 1)
|
||||
devs[0].Addrs[0] = net.IPNet{
|
||||
IP: net.IPv4(10, 0, 0, 2),
|
||||
Mask: net.IPv4Mask(255, 255, 255, 0),
|
||||
}
|
||||
devs[0].Routes = make([]netlink.Route, 2)
|
||||
devs[0].Routes[0] = netlink.Route{
|
||||
Dst: &net.IPNet{
|
||||
IP: net.IPv4(10, 2, 0, 0),
|
||||
Mask: net.IPv4Mask(255, 255, 255, 0),
|
||||
},
|
||||
Gw: net.IPv4(10, 0, 0, 5),
|
||||
}
|
||||
devs[0].Routes[1] = netlink.Route{
|
||||
Gw: net.IPv4(10, 0, 0, 1),
|
||||
}
|
||||
|
||||
devs[1] = Device{Name: "net1"}
|
||||
devs[1].Addrs = make([]net.IPNet, 1)
|
||||
devs[1].Addrs[0] = net.IPNet{
|
||||
IP: net.IPv4(192, 168, 1, 209),
|
||||
Mask: net.IPv4Mask(255, 255, 255, 0),
|
||||
}
|
||||
devs[1].Routes = make([]netlink.Route, 1)
|
||||
devs[1].Routes[0] = netlink.Route{
|
||||
Dst: &net.IPNet{
|
||||
IP: net.IPv4(192, 168, 2, 0),
|
||||
Mask: net.IPv4Mask(255, 255, 255, 0),
|
||||
},
|
||||
Gw: net.IPv4(192, 168, 1, 2),
|
||||
}
|
||||
|
||||
return netStatus{
|
||||
Devices: devs,
|
||||
Rules: rules}
|
||||
}
|
||||
|
||||
var _ = Describe("sbr test", func() {
|
||||
var targetNs ns.NetNS
|
||||
|
||||
BeforeEach(func() {
|
||||
var err error
|
||||
targetNs, err = testutils.NewNS()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
})
|
||||
|
||||
AfterEach(func() {
|
||||
targetNs.Close()
|
||||
})
|
||||
|
||||
It("Works with a 0.3.0 config", func() {
|
||||
ifname := "net1"
|
||||
conf := `{
|
||||
"cniVersion": "0.3.0",
|
||||
"name": "cni-plugin-sbr-test",
|
||||
"type": "sbr",
|
||||
"prevResult": {
|
||||
"interfaces": [
|
||||
{
|
||||
"name": "%s",
|
||||
"sandbox": "%s"
|
||||
}
|
||||
],
|
||||
"ips": [
|
||||
{
|
||||
"version": "4",
|
||||
"address": "192.168.1.209/24",
|
||||
"gateway": "192.168.1.1",
|
||||
"interface": 0
|
||||
}
|
||||
],
|
||||
"routes": []
|
||||
}
|
||||
}`
|
||||
conf = fmt.Sprintf(conf, ifname, targetNs.Path())
|
||||
args := &skel.CmdArgs{
|
||||
ContainerID: "dummy",
|
||||
Netns: targetNs.Path(),
|
||||
IfName: ifname,
|
||||
StdinData: []byte(conf),
|
||||
}
|
||||
|
||||
err := setup(targetNs, createDefaultStatus())
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
oldStatus, err := readback(targetNs, []string{"net1", "eth0"})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
_, _, err = testutils.CmdAddWithArgs(args, func() error { return cmdAdd(args) })
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
newStatus, err := readback(targetNs, []string{"net1", "eth0"})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
// Check results. We expect all the routes on net1 to have moved to
|
||||
// table 100 except for local routes (table 255); a new default gateway
|
||||
// route to have been created; and a single rule to exist.
|
||||
expNet1 := oldStatus.Devices[0]
|
||||
expEth0 := oldStatus.Devices[1]
|
||||
for i := range expNet1.Routes {
|
||||
if expNet1.Routes[i].Table != 255 {
|
||||
expNet1.Routes[i].Table = 100
|
||||
}
|
||||
}
|
||||
expNet1.Routes = append(expNet1.Routes,
|
||||
netlink.Route{
|
||||
Gw: net.IPv4(192, 168, 1, 1),
|
||||
Table: 100,
|
||||
LinkIndex: expNet1.Routes[0].LinkIndex})
|
||||
|
||||
Expect(len(newStatus.Rules)).To(Equal(1))
|
||||
Expect(newStatus.Rules[0].Table).To(Equal(100))
|
||||
Expect(newStatus.Rules[0].Src.String()).To(Equal("192.168.1.209/32"))
|
||||
devNet1 := newStatus.Devices[0]
|
||||
devEth0 := newStatus.Devices[1]
|
||||
Expect(equalRoutes(expNet1.Routes, devNet1.Routes)).To(BeTrue())
|
||||
Expect(equalRoutes(expEth0.Routes, devEth0.Routes)).To(BeTrue())
|
||||
|
||||
conf = `{
|
||||
"cniVersion": "0.3.0",
|
||||
"name": "cni-plugin-sbr-test",
|
||||
"type": "sbr"
|
||||
}`
|
||||
|
||||
// And now check that we can back it all out.
|
||||
args = &skel.CmdArgs{
|
||||
ContainerID: "dummy",
|
||||
Netns: targetNs.Path(),
|
||||
IfName: ifname,
|
||||
StdinData: []byte(conf),
|
||||
}
|
||||
err = testutils.CmdDelWithArgs(args, func() error { return cmdDel(args) })
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
retVal, err := readback(targetNs, []string{"net1", "eth0"})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
// Check results. We expect the rule to have been removed.
|
||||
Expect(len(retVal.Rules)).To(Equal(0))
|
||||
})
|
||||
|
||||
It("Works with a default route already set", func() {
|
||||
ifname := "net1"
|
||||
conf := `{
|
||||
"cniVersion": "0.3.0",
|
||||
"name": "cni-plugin-sbr-test",
|
||||
"type": "sbr",
|
||||
"prevResult": {
|
||||
"interfaces": [
|
||||
{
|
||||
"name": "%s",
|
||||
"sandbox": "%s"
|
||||
}
|
||||
],
|
||||
"ips": [
|
||||
{
|
||||
"version": "4",
|
||||
"address": "192.168.1.209/24",
|
||||
"gateway": "192.168.1.1",
|
||||
"interface": 0
|
||||
}
|
||||
],
|
||||
"routes": []
|
||||
}
|
||||
}`
|
||||
conf = fmt.Sprintf(conf, ifname, targetNs.Path())
|
||||
args := &skel.CmdArgs{
|
||||
ContainerID: "dummy",
|
||||
Netns: targetNs.Path(),
|
||||
IfName: ifname,
|
||||
StdinData: []byte(conf),
|
||||
}
|
||||
|
||||
preStatus := createDefaultStatus()
|
||||
// Remove final (default) route from eth0, then add another default
|
||||
// route to net1
|
||||
preStatus.Devices[0].Routes = preStatus.Devices[0].Routes[:0]
|
||||
routes := preStatus.Devices[1].Routes
|
||||
preStatus.Devices[1].Routes = append(preStatus.Devices[1].Routes,
|
||||
netlink.Route{
|
||||
Gw: net.IPv4(192, 168, 1, 1),
|
||||
LinkIndex: routes[0].LinkIndex})
|
||||
|
||||
err := setup(targetNs, preStatus)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
oldStatus, err := readback(targetNs, []string{"net1", "eth0"})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
_, _, err = testutils.CmdAddWithArgs(args, func() error { return cmdAdd(args) })
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
newStatus, err := readback(targetNs, []string{"net1", "eth0"})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
// Check results. We expect all the routes on net1 to have moved to
|
||||
// table 100 except for local routes (table 255); a new default gateway
|
||||
// route to have been created; and a single rule to exist.
|
||||
expNet1 := oldStatus.Devices[0]
|
||||
expEth0 := oldStatus.Devices[1]
|
||||
for i := range expNet1.Routes {
|
||||
if expNet1.Routes[i].Table != 255 {
|
||||
expNet1.Routes[i].Table = 100
|
||||
}
|
||||
}
|
||||
|
||||
Expect(len(newStatus.Rules)).To(Equal(1))
|
||||
Expect(newStatus.Rules[0].Table).To(Equal(100))
|
||||
Expect(newStatus.Rules[0].Src.String()).To(Equal("192.168.1.209/32"))
|
||||
devNet1 := newStatus.Devices[0]
|
||||
devEth0 := newStatus.Devices[1]
|
||||
Expect(equalRoutes(expEth0.Routes, devEth0.Routes)).To(BeTrue())
|
||||
Expect(equalRoutes(expNet1.Routes, devNet1.Routes)).To(BeTrue())
|
||||
})
|
||||
|
||||
It("works with a 0.2.0 config", func() {
|
||||
conf := `{
|
||||
"cniVersion": "0.2.0",
|
||||
"name": "cni-plugin-sbr-test",
|
||||
"type": "sbr",
|
||||
"anotherAwesomeArg": "foo",
|
||||
"prevResult": {
|
||||
"ip4": {
|
||||
"ip": "192.168.1.209/24",
|
||||
"gateway": "192.168.1.1",
|
||||
"routes": []
|
||||
}
|
||||
}
|
||||
}`
|
||||
|
||||
args := &skel.CmdArgs{
|
||||
ContainerID: "dummy",
|
||||
Netns: targetNs.Path(),
|
||||
IfName: "net1",
|
||||
StdinData: []byte(conf),
|
||||
}
|
||||
err := setup(targetNs, createDefaultStatus())
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
|
||||
_, _, err = testutils.CmdAddWithArgs(args, func() error { return cmdAdd(args) })
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
})
|
||||
|
||||
})
|
15
plugins/meta/sbr/sbr_suite_test.go
Normal file
15
plugins/meta/sbr/sbr_suite_test.go
Normal file
@ -0,0 +1,15 @@
|
||||
// The boilerplate needed for Ginkgo
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
. "github.com/onsi/ginkgo"
|
||||
. "github.com/onsi/gomega"
|
||||
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSample(t *testing.T) {
|
||||
RegisterFailHandler(Fail)
|
||||
RunSpecs(t, "plugins/sbr")
|
||||
}
|
Loading…
x
Reference in New Issue
Block a user