Aravind Prabhakar

Systems Engineer | Networking | Security | PreSales | Cloud | Devops | AIOps

View on GitHub Linkedin Blogs Tags
20 July 2023

Service chaining JCNR and cSRX to offer security services

By Aravind

Containerized SRX and Integration with Juniper Cloud Native Router

Topology

topology

Prerequisites

Steps following assumes that the above has been completed and you have an all in one kubernetes cluster. Either on a bare metal or on a VM

Network attachment definition

Save the below into a file nad-trust.yaml

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-trust
spec:
  config: '{
    "cniVersion":"0.4.0",
    "name": "net-trust",
    "type": "jcnr",
    "args": {
      "vrfName": "trust",
      "vrfTarget": "11:11"
    },
    "kubeConfig":"/etc/kubernetes/kubelet.conf"
  }'

Save the below into a file nad-untrust.yaml

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-untrust
spec:
  config: '{
    "cniVersion":"0.4.0",
    "name": "net-untrust",
    "type": "jcnr",
    "args": {
      "vrfName": "untrust",
      "vrfTarget": "10:10"
    },
    "kubeConfig":"/etc/kubernetes/kubelet.conf"
  }'

Apply the NAD

kubectl apply -f nad-trust.yaml
kubectl apply -f nad-untrust.yaml

cSRX Manifest file

Below creates cSRX with kernel mode. Save the below to csrx.yaml

---
apiVersion: v1
kind: Pod
metadata:
  name: csrx
  annotations:
    k8s.v1.cni.cncf.io/networks: |
      [
        {
          "name": "net-trust",
          "interface":"eth1",
          "cni-args": {
            "mac":"aa:bb:cc:dd:01:12",
            "interfaceType":"veth",
            "rd": "11:11",
            "dataplane":"dpdk",
            "ipConfig":{
              "ipv4":{
                "address":"1.20.1.2/30",
                "gateway":"1.20.1.1"
              }
            }
          }
        },
        {
          "name": "net-untrust",
          "interface":"eth2",
          "cni-args": {
            "mac":"aa:bb:cc:dd:01:21",
            "interfaceType": "veth",
            "rd": "10:10",
            "dataplane":"dpdk",
            "ipConfig":{
              "ipv4":{
                "address":"1.21.1.2/30",
                "gateway":"1.21.1.1"
              }
            }
          }
        }
      ]
spec:
  containers:
  - name: csrx1
    securityContext:
       privileged: true
    image: csrx:23.2R1.13
    imagePullPolicy: IfNotPresent
    env:
    - name: CSRX_SIZE
      value: "large"
    - name: CSRX_HUGEPAGES
      value: "no"
    - name: CSRX_PACKET_DRIVER
      value: "interrupt"
    - name: CSRX_FORWARD_MODE
      value: "routing"
      #- name: CSRX_AUTO_ASSIGN_IP
      #value: "yes"
    - name: CSRX_MGMT_PORT_REORDER
      value: "no"
    - name: CSRX_TCP_CKSUM_CALC
      value: "yes"
    - name: CSRX_LICENSE_FILE
      value: "/var/jail/.csrx_license"
    - name: CSRX_JUNOS_CONFIG
      value: "/var/jail/csrx_config"
    - name: CSRX_CTRL_CPU
      value: "0x01"
    - name: CSRX_DATA_CPU
      value: "0x05"
    volumeMounts:
    - name: disk
      mountPath: "/dev"
  volumes:
  - name: disk
    hostPath:
      path: /dev
      type: Director

Apply cSRX manifest

kubectl apply -f csrx.yaml

Validate the interfaces

root@csrx:/# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.244.0.81  netmask 255.255.255.0  broadcast 10.244.0.255
        ether 16:f5:a9:7f:0d:3a  txqueuelen 0  (Ethernet)
        RX packets 9  bytes 830 (830.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15  bytes 1358 (1.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether aa:bb:cc:dd:01:12  txqueuelen 0  (Ethernet)
        RX packets 171650  bytes 258235769 (258.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 83924  bytes 5563556 (5.5 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether aa:bb:cc:dd:01:21  txqueuelen 0  (Ethernet)
        RX packets 83961  bytes 5566390 (5.5 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 170651  bytes 258108443 (258.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
pfe_tun: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

# tap0 will be bound to eth1 
tap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 1.20.1.1  netmask 255.255.255.252  broadcast 1.20.1.3
        inet6 fe80::a8bb:ccff:fedd:112  prefixlen 64  scopeid 0x20<link>
        ether aa:bb:cc:dd:01:12  txqueuelen 1000  (Ethernet)
        RX packets 9  bytes 378 (378.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 24  bytes 1280 (1.2 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

# tap1 will be bound to eth2 
tap1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 1.21.1.1  netmask 255.255.255.252  broadcast 1.21.1.3
        inet6 fe80::a8bb:ccff:fedd:121  prefixlen 64  scopeid 0x20<link>
        ether aa:bb:cc:dd:01:21  txqueuelen 1000  (Ethernet)
        RX packets 11  bytes 462 (462.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 28  bytes 1456 (1.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Configure cSRX

root@csrx# show | display set
set version 20230616.051920_builder.r1345402
set system root-authentication encrypted-password *disabled*
set system services ssh root-login allow
set interfaces ge-0/0/0 unit 0 family inet address 1.20.1.1/30
set interfaces ge-0/0/1 unit 0 family inet address 1.21.1.1/30
set routing-options static route 181.1.1.1/32 next-hop 1.21.1.2/32
set routing-options static route 180.1.1.1/32 next-hop 1.20.1.2/32
set security policies default-policy permit-all
set security zones security-zone untrust interfaces ge-0/0/1.0 host-inbound-traffic system-services all
set security zones security-zone untrust interfaces ge-0/0/1.0 host-inbound-traffic protocols all
set security zones security-zone trust interfaces ge-0/0/0.0 host-inbound-traffic protocols all

Configure JCNR with correct routes

set routing-instances trust routing-options static route 181.1.1.1/32 next-hop 1.20.1.1
set routing-instances untrust routing-options static route 180.1.1.1/32 next-hop 1.21.1.1

Pass traffic E2E

root@ubuntu:~# ping 181.1.1.1
PING 181.1.1.1 (181.1.1.1) 56(84) bytes of data.
64 bytes from 181.1.1.1: icmp_seq=1 ttl=61 time=8.53 ms
64 bytes from 181.1.1.1: icmp_seq=2 ttl=61 time=1.77 ms
64 bytes from 181.1.1.1: icmp_seq=3 ttl=61 time=1.60 ms
64 bytes from 181.1.1.1: icmp_seq=4 ttl=61 time=1.19 ms


root@csrx> show security flow session
Session ID: 1071, Policy name: default-policy-logical-system-00/2, Timeout: 4, Session State: Valid
  In: 180.1.1.1/52 --> 181.1.1.1/1;icmp, Conn Tag: 0x0, If: ge-0/0/0.0, Pkts: 1, Bytes: 84,
  Out: 181.1.1.1/1 --> 180.1.1.1/52;icmp, Conn Tag: 0x0, If: ge-0/0/1.0, Pkts: 1, Bytes: 84,

Session ID: 1072, Policy name: default-policy-logical-system-00/2, Timeout: 4, Session State: Valid
  In: 180.1.1.1/52 --> 181.1.1.1/2;icmp, Conn Tag: 0x0, If: ge-0/0/0.0, Pkts: 1, Bytes: 84,
  Out: 181.1.1.1/2 --> 180.1.1.1/52;icmp, Conn Tag: 0x0, If: ge-0/0/1.0, Pkts: 1, Bytes: 84,
Total sessions: 2

Debugging

[ kubernetes  ] tags: kubernetes