Skip to content

docs+fixes(cni): finish the CNI plugin-chain split — docs, containerlab, dead code #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
privateip wants to merge 1 commit into from
Open

docs+fixes(cni): finish the CNI plugin-chain split — docs, containerlab, dead code #307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 8 additions & 34 deletions cmd/galactic-cni/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@
package main

import (
"encoding/json"
"errors"
"fmt"
"io"
"log"
"os"
"strings"
Expand Down Expand Up @@ -108,27 +106,14 @@ func newRootCommand() *cobra.Command {
return nil
}

// Read stdin once so we can inspect the CNI config before the
// library runs its netns validation. We pipe the buffered bytes
// back as os.Stdin so the CNI library can still read them.
stdinData, _ := io.ReadAll(os.Stdin)
r, w, _ := os.Pipe()
go func() {
_, _ = w.Write(stdinData)
_ = w.Close()
}()
oldStdin := os.Stdin
os.Stdin = r

// Tap mode never enters a network namespace — all operations are
// host-side. Set the override so the CNI library skips its same-
// netns rejection check, which would otherwise reject kraftlet
// workloads that pass the host netns.
if isTapMode(stdinData) {
_ = os.Setenv("CNI_NETNS_OVERRIDE", "true")
}

defer func() { os.Stdin = oldStdin }()
// galactic-cni is veth-only: it always moves an interface into
// the container's own netns, so it always needs the CNI
// library's normal same-netns rejection check — unlike
// galactic-tap-cni (which unconditionally sets
// CNI_NETNS_OVERRIDE, since tap workloads never enter a netns
// at all), there is no stdin-peeking tap-mode detection here
// anymore. Interface kind is which binary you invoke now, not a
// config field this process branches on.
cni.RunPlugin()
return nil
},
Expand All @@ -142,17 +127,6 @@ func newRootCommand() *cobra.Command {
return cmd
}

// isTapMode returns true when the CNI config requests tap interface type.
// Only a minimal JSON parse is needed — full validation happens later in
// parseConf inside cmdAdd.
func isTapMode(stdinData []byte) bool {
var cfg struct {
InterfaceType string `json:"interface_type"`
}
_ = json.Unmarshal(stdinData, &cfg)
return cfg.InterfaceType == "tap"
}

func main() {
if err := newRootCommand().Execute(); err != nil {
log.Fatalf("error: %v", err)
Expand Down
16 changes: 4 additions & 12 deletions cmd/galactic-tap-cni/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ package main

import (
"fmt"
"io"
"log"
"os"
"strings"
Expand Down Expand Up @@ -71,17 +70,10 @@ func newRootCommand() *cobra.Command {
// Tap mode never enters a network namespace — all operations
// are host-side. Set the override so the CNI library skips its
// same-netns rejection check, which would otherwise reject
// kraftlet workloads that pass the host netns.
stdinData, _ := io.ReadAll(os.Stdin)
r, w, _ := os.Pipe()
go func() {
_, _ = w.Write(stdinData)
_ = w.Close()
}()
oldStdin := os.Stdin
os.Stdin = r
defer func() { os.Stdin = oldStdin }()

// kraftlet workloads that pass the host netns. Unconditional
// here (unlike galactic-cni, which has no override logic at
// all): every invocation of this binary is tap mode, so there
// is no config content to peek at first.
_ = os.Setenv("CNI_NETNS_OVERRIDE", "true")

cnitap.RunPlugin()
Expand Down
14 changes: 8 additions & 6 deletions deploy/containerlab/docs/tenants.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ pod-to-pod connectivity. Every one of them follows the same mechanism: Multus
attaches each `netshoot` pod to its VPC's `private` NetworkAttachmentDefinition
(via the `v1.multus-cni.io/default-network` annotation, which makes the VPC
interface the pod's `eth0` rather than an additional `net1` — there is no
`k8s.v1.cni.cncf.io/networks` annotation in play here), which invokes
`galactic-cni` to create a VRF, veth pair, SRv6 encapsulation route, and a
`BGPAdvertisement` CRD. The `galactic-router` controller then advertises each
pod's EVPN route to the route reflector, distributing reachability across
sites.
`k8s.v1.cni.cncf.io/networks` annotation in play here), which invokes the
galactic CNI plugin chain: `galactic-cni` creates a VRF and veth pair, then
`galactic-bgp` registers the attachment against the eBPF uSID datapath and
writes a `BGPAdvertisement` CRD (see [docs/cni-cmd-sequence.md](../../../docs/cni-cmd-sequence.md)
for the full per-binary ADD sequence). The `galactic-router` controller then
advertises each pod's EVPN route to the route reflector, distributing
reachability across sites.

They differ only in scope and addressing:

Expand All @@ -37,7 +39,7 @@ fabric. The low hextet of a pod's USID is `(Function << 12) | Argument`
`0xE` (`FunctionEndDT46`) for every plain L3 VRF attachment, and `Argument` is
a 12-bit value `galactic-router` allocates per-node as the lowest unused slot
in `[0x001, 0xFFF]` among that node's existing `BGPVRFInstance` CRDs
(`allocateArgument`, `internal/cni/bgp.go`) — **not** a decode of the NAD's
(`allocateArgument`, `internal/cnibgp/bgp.go`) — **not** a decode of the NAD's
`vpc`/`vpcattachment` values. Concretely, expect hextets in the
`0xe001`–`0xefff` range; the exact value depends on allocation order (`ns50`
is provisioned first in `task deploy`, then `ns10`, `ns20`, `ns30`, `ns40` in
Expand Down
27 changes: 20 additions & 7 deletions deploy/containerlab/resources/tenants/ns10/dfw/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,27 @@ metadata:
name: private
namespace: ns10
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:10:ff01::/48",
"address_families": ["ipv6"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:10:ff01::/48",
"address_families": ["ipv6"]
}
},
{
"type": "galactic-bgp",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system"
}
]
}
27 changes: 20 additions & 7 deletions deploy/containerlab/resources/tenants/ns10/iad/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,27 @@ metadata:
name: private
namespace: ns10
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:10:ff03::/48",
"address_families": ["ipv6"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:10:ff03::/48",
"address_families": ["ipv6"]
}
},
{
"type": "galactic-bgp",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system"
}
]
}
27 changes: 20 additions & 7 deletions deploy/containerlab/resources/tenants/ns10/sjc/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,27 @@ metadata:
name: private
namespace: ns10
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:10:ff02::/48",
"address_families": ["ipv6"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:10:ff02::/48",
"address_families": ["ipv6"]
}
},
{
"type": "galactic-bgp",
"vpc": "10",
"vpcattachment": "10",
"namespace": "galactic-system"
}
]
}
29 changes: 21 additions & 8 deletions deploy/containerlab/resources/tenants/ns20/dfw/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,28 @@ metadata:
name: private
namespace: ns20
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:20:ff01::/48",
"ipv4_subnet": "172.21.1.0/24",
"address_families": ["ipv6", "ipv4"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:20:ff01::/48",
"ipv4_subnet": "172.21.1.0/24",
"address_families": ["ipv6", "ipv4"]
}
},
{
"type": "galactic-bgp",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system"
}
]
}
29 changes: 21 additions & 8 deletions deploy/containerlab/resources/tenants/ns20/iad/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,28 @@ metadata:
name: private
namespace: ns20
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:20:ff03::/48",
"ipv4_subnet": "172.21.10.0/24",
"address_families": ["ipv6", "ipv4"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:20:ff03::/48",
"ipv4_subnet": "172.21.10.0/24",
"address_families": ["ipv6", "ipv4"]
}
},
{
"type": "galactic-bgp",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system"
}
]
}
29 changes: 21 additions & 8 deletions deploy/containerlab/resources/tenants/ns20/sjc/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,28 @@ metadata:
name: private
namespace: ns20
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:20:ff02::/48",
"ipv4_subnet": "172.21.20.0/24",
"address_families": ["ipv6", "ipv4"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:20:ff02::/48",
"ipv4_subnet": "172.21.20.0/24",
"address_families": ["ipv6", "ipv4"]
}
},
{
"type": "galactic-bgp",
"vpc": "20",
"vpcattachment": "20",
"namespace": "galactic-system"
}
]
}
27 changes: 20 additions & 7 deletions deploy/containerlab/resources/tenants/ns30/dfw/nad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,27 @@ metadata:
name: private
namespace: ns30
spec:
config: |
config: |-
{
"cniVersion": "1.0.0",
"name": "private",
"type": "galactic-cni",
"vpc": "30",
"vpcattachment": "30",
"namespace": "galactic-system",
"ipv6_subnet": "fd20:30:ff01::/48",
"address_families": ["ipv6"]
"plugins": [
{
"type": "galactic-cni",
"vpc": "30",
"vpcattachment": "30",
"namespace": "galactic-system",
"ipam": {
"type": "galactic-ipam",
"ipv6_subnet": "fd20:30:ff01::/48",
"address_families": ["ipv6"]
}
},
{
"type": "galactic-bgp",
"vpc": "30",
"vpcattachment": "30",
"namespace": "galactic-system"
}
]
}
Loading