MACSEC
By Aravind
[[TOC]]
What is MACSEC ?
- Layer2 protocol which encrypts using gcm-aes encryption algorithm (128/256) and offers secure links in a LAN env.
- secures a P2P link preventing man in the middle attacks, passive wire tapping
- secures all traffic including ARP and DHCP which IPSEC does not
- operates at layer 2 as compared to IPSEC which operates at L3
Due to the above protects all higher layer traffic (L2 and above) exiting the interface which is enabled with macsec.
What does it offer ?
- data integrity
- Encryption
Few acryonyms
- MKA (macsec key aggrement)
- CA (Connectivity association)
- macsec supports single CA per interface (IFD). If logical is used (vlan tags) the vlan tags are not encrypted.
- creates inbound and outbound connections
- CAK (connectivity association key)
- can be static or dynamic
- static can be pre-shared-keys with/without automatic rollovers
- In dynamic peer node generates the security keys as part of the authentication process. The CAK and CKN are dynamically generated to create a secure connection
- hex value as input
- stored encrypted.
- CKN (connectivity association key name)
- hex value (even numbered)
- stored in plain text
Best practices
- full length CKN is 32octets preferred so that it can interop well.
- can compare CKN as username and CAK as password hence keep them different
- use hitless key rollovers
- ensure both systems are the same time (although not required but preferred)
- Ensure same transmit interval is programmed on both ends of the link. 6000ms (6s) is recommended. default is 2seconds.
- harden shared secrets (PSKs) using a master password
set system master-password plain-text-password <password> - use extended packet numbering (XPN) for high speed links.
Topology
Configuration using static preshared key
CAK can be static or dynamic. Below is the method to use static pre-shared keys
root@mx480-a# show security
macsec {
connectivity-association test1 {
security-mode static-cak;
pre-shared-key {
ckn abcd1234abcd1234;
cak "$9$tB1vu0IhclMWx/Cev8LN-k.mPFn/9pOBEikfz3nCAvW87bsoJG"; ## SECRET-DATA
}
}
Bind the interface to use the static CAK
root@mx480-a# set security macsec interfaces et-5/1/0 connectivity-association test1
Change encryption algorithm
By default picked gcm-aes-128. For high speed links >25g it is advisable to pick xpn based. if a different config is added, then you might see the below error.
root@mx480-a# set security macsec connectivity-association test1 cipher-suite gcm-aes-256
Warning that you might see
[edit security macsec]
'connectivity-association test1'
warning: To maximize security, recommend configuring all 64 hexadecimal digits of pre-shared-key cak
[edit security macsec]
'interfaces et-5/1/0'
warning: Only XPN cipher suites should be configured on interface speed 25G and above
commit complete
Change to xpn based
root@mx480-a# set security macsec connectivity-association att-test1 cipher-suite gcm-aes-xpn-256
[edit security macsec]
'connectivity-association test1'
warning: To maximize security, recommend configuring all 64 hexadecimal digits of pre-shared-key ckn
[edit security macsec]
'connectivity-association test1'
warning: To maximize security, recommend configuring all 64 hexadecimal digits of pre-shared-key cak
commit complete
the above can be safely ignored. Since this is a test setup
Configure the interface
set interfaces et-5/1/0 description "test macsec"
set interfaces et-5/1/0 unit 0 family inet address 193.1.1.1/30
add similar configurations to the far end router on mx480-b
Verify
root@mx-480-a# run ping 193.1.1.2
PING 193.1.1.2 (193.1.1.2): 56 data bytes
64 bytes from 193.1.1.2: icmp_seq=0 ttl=64 time=1.043 ms
64 bytes from 193.1.1.2: icmp_seq=1 ttl=64 time=1.208 ms
64 bytes from 193.1.1.2: icmp_seq=2 ttl=64 time=0.877 ms
64 bytes from 193.1.1.2: icmp_seq=3 ttl=64 time=0.803 ms
64 bytes from 193.1.1.2: icmp_seq=4 ttl=64 time=0.950 ms
64 bytes from 193.1.1.2: icmp_seq=5 ttl=64 time=0.747 ms
64 bytes from 193.1.1.2: icmp_seq=6 ttl=64 time=0.744 ms
64 bytes from 193.1.1.2: icmp_seq=7 ttl=64 time=0.762 ms
64 bytes from 193.1.1.2: icmp_seq=8 ttl=64 time=1.088 ms
64 bytes from 193.1.1.2: icmp_seq=9 ttl=64 time=0.753 ms
64 bytes from 193.1.1.2: icmp_seq=10 ttl=64 time=1.008 ms
64 bytes from 193.1.1.2: icmp_seq=11 ttl=64 time=0.727 ms
64 bytes from 193.1.1.2: icmp_seq=12 ttl=64 time=1.156 ms
64 bytes from 193.1.1.2: icmp_seq=13 ttl=64 time=0.643 ms
64 bytes from 193.1.1.2: icmp_seq=14 ttl=64 time=1.056 ms
64 bytes from 193.1.1.2: icmp_seq=15 ttl=64 time=0.690 ms
64 bytes from 193.1.1.2: icmp_seq=16 ttl=64 time=1.011 ms
64 bytes from 193.1.1.2: icmp_seq=17 ttl=64 time=0.650 ms
64 bytes from 193.1.1.2: icmp_seq=18 ttl=64 time=0.751 ms
^C
--- 193.1.1.2 ping statistics ---
19 packets transmitted, 19 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.643/0.877/1.208/0.175 ms
verify connections and statistics
root@wf-mt-monroe# run show security macsec connections
Interface name: et-5/1/0
CA name: att-test1
Cipher suite: GCM-AES-XPN-256 Encryption: on
Key server offset: 0 Include SCI: no
Replay protect: off Replay window: 0
Outbound secure channels
SC Id: 88:E0:F3:C1:4E:E7/1
Outgoing packet number: 20
Secure associations
AN: 0 Status: inuse Create time: 00:03:12
Inbound secure channels
SC Id: 3C:61:04:CB:34:C1/1
Secure associations
AN: 0 Status: inuse Create time: 00:03:12
root@wf-mt-monroe# run show security macsec statistics
Interface name: et-5/1/0
Secure Channel transmitted
Encrypted packets: 19
Encrypted bytes: 1634
Protected packets: 0
Protected bytes: 0
Secure Association transmitted
Encrypted packets: 19
Protected packets: 0
Secure Channel received
Accepted packets: 19
Validated bytes: 0
Decrypted bytes: 1634
Secure Association received
Accepted packets: 19
Validated bytes: 0
Decrypted bytes: 0
Final configs
m480-a
set interfaces et-5/1/0 description "test macsec"
set interfaces et-5/1/0 unit 0 family inet address 193.1.1.1/30
set security macsec connectivity-association test1 cipher-suite gcm-aes-xpn-256
set security macsec connectivity-association test1 security-mode static-cak
set security macsec connectivity-association test1 pre-shared-key ckn abcd1234abcd1234
set security macsec connectivity-association test1 pre-shared-key cak "$9$tB1vu0IhclMWx/Cev8LN-k.mPFn/9pOBEikfz3nCAvW87bsoJG"
set security macsec interfaces et-5/1/0 connectivity-association test1
mx480-b
set interfaces et-3/2/0 unit 0 family inet address 193.1.1.2/30
set security macsec connectivity-association test1 cipher-suite gcm-aes-xpn-256
set security macsec connectivity-association test1 security-mode static-cak
set security macsec connectivity-association test1 pre-shared-key ckn abcd1234abcd1234
set security macsec connectivity-association test1 pre-shared-key cak "$9$tB1vu0IhclMWx/Cev8LN-k.mPFn/9pOBEikfz3nCAvW87bsoJG"
set security macsec interfaces et-3/2/0 connectivity-association test1
How about overlays and other protocols on top of the masec interface ?
Macsec is a L2 encryption method. So anything heading out of the physical interface will get encrypted. Any protocol packets or overlays configured would be encrypted.
For example, below FTI interface has been configured. These L3-VXLAN packets traverse the macsec interface and would get encrypted.
Configuration
root@mx480-a# show interfaces fti0 | display set
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe source address 193.1.1.1
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe destination address 193.1.1.2
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe tunnel-endpoint vxlan
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe destination-udp-port 4789
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe vni 300
set interfaces fti0 unit 1 family inet address 194.1.1.1/30
root@mx480-b# show interfaces fti0 | display set
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe source address 193.1.1.2
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe destination address 193.1.1.1
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe tunnel-endpoint vxlan
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe destination-udp-port 4789
set interfaces fti0 unit 1 tunnel encapsulation vxlan-gpe vni 300
set interfaces fti0 unit 1 family inet address 194.1.1.2/30
Verify
ping the FTI end point
root@mx480-a# run ping 194.1.1.2
PING 194.1.1.2 (194.1.1.2): 56 data bytes
64 bytes from 194.1.1.2: icmp_seq=0 ttl=64 time=0.951 ms
64 bytes from 194.1.1.2: icmp_seq=1 ttl=64 time=0.949 ms
64 bytes from 194.1.1.2: icmp_seq=2 ttl=64 time=1.060 ms
notice the packets count is now 24 from 21. The 3 packets were encrypted.
root@mx480-b# run show security macsec statistics
Interface name: et-3/2/0
Secure Channel transmitted
Encrypted packets: 24
Encrypted bytes: 2226
Protected packets: 0
Protected bytes: 0
Secure Association transmitted
Encrypted packets: 24
Protected packets: 0
Secure Channel received
Accepted packets: 24
Validated bytes: 0
Decrypted bytes: 2226
Secure Association received
Accepted packets: 24
Validated bytes: 0
Decrypted bytes: 0
How about encrypting the VXLAN tunnel itself instead of the underlying IFD ?
Lets say we want to encrypt the FTI tunnel interface itself ? This is not possible since we encrypt at IFD level.
root@mx480-a# set interfaces fti0 connectivity-association test1
error: name: 'fti0': Only ethernet interfaces allowed
error: statement creation failed: fti0
we will discuss in the later section an alternative approach for such a requirement. These are useful when we want to replace IPSEC for site to site use cases which needs high throughput.
Key chains
while rolling over keys in PSK method is not hitless. A way to solve this problem is use keychains and use hitless rollovers. Keychain is a set of keys (one or more) that can be added on the macsec devices. The important part here is the start-time knob which defines a future time at which the new key should be used. When the time reaches the defined value, the keys would be rolled over to the defined one. in this case, key1 would be used at time 2022-9-13.17:00:00. There can be many number of keys so this way it can rollover accordingly.
Configuration
[edit security]
+ authentication-key-chains {
+ key-chain chain1 {
+ key 1 {
+ secret "$9$JtUi.QF/0BEP5BEcyW8ZUjHP5z36AuO"; ## SECRET-DATA
+ key-name deadbeefcaf4;
+ start-time "2022-9-13.17:00:00 -0700";
+ }
+ }
+ }
The secret can be typed in plain text and it would salt it accordingly. Make sure you delete the knob pre-shared-key under set security macsec connectivity-association test1 pre-shared-key since you are using pre-shared-key-chain.
The secret should be a hex value.
refer the new key-chain chain1
set security macsec connectivity-association test1 pre-shared-key-chain chain1
add a future time on both sides. This way key management is easier and traffic is hitless when key roll overs happen.
Verify the connection
--(refreshed at 2022-09-13 14:05:59 PDT)---
Interface name: et-3/2/0
CA name: test1
Cipher suite: GCM-AES-XPN-256 Encryption: on
Key server offset: 0 Include SCI: no
Replay protect: off Replay window: 0
---(refreshed at 2022-09-13 14:06:04 PDT)---
Interface name: et-3/2/0
CA name: test1
Cipher suite: GCM-AES-XPN-256 Encryption: on
Key server offset: 0 Include SCI: no
Replay protect: off Replay window: 0
---(refreshed at 2022-09-13 14:06:09 PDT)---
Interface name: et-3/2/0
CA name: test1
Cipher suite: GCM-AES-XPN-256 Encryption: on
Key server offset: 0 Include SCI: no
Replay protect: off Replay window: 0
Outbound secure channels
SC Id: 3C:61:04:CB:34:C1/1
Outgoing packet number: 1
Secure associations
AN: 1 Status: inuse Create time: 00:00:02 <<< notice the key AN:1 . Also notice the time at which it got created. This would be the same time based on the configuration.
Inbound secure channels
SC Id: 88:E0:F3:C1:4E:E7/1
Secure associations
AN: 1 Status: inuse Create time: 00:00:02
Verify statistics
root@mx480-b# run show security macsec statistics
Interface name: et-3/2/0
Secure Channel transmitted
Encrypted packets: 1
Encrypted bytes: 48
Protected packets: 0
Protected bytes: 0
Secure Association transmitted
Encrypted packets: 1
Protected packets: 0
Secure Channel received
Accepted packets: 1
Validated bytes: 0
Decrypted bytes: 48
Secure Association received
Accepted packets: 1
Validated bytes: 0
Decrypted bytes: 0
root@mx480-b# run show security mka sessions
Interface name: et-3/2/0
Interface State: Secured - Primary
Member identifier: A22915FD8C07D0542B39270F
CAK name: DEAFBEEFCAF4 <<< indicates the correct CAK name
CAK type: primary
Security mode: static
MKA suspended: 0(s)
Transmit interval: 2000(ms)
SAK rekey interval: 0(s)
Preceding Key: enabled
Bounded Delay: disabled
Outbound SCI: 3C:61:04:CB:34:C1/1
Message number: 124 Key number: 1
Key server: yes Key server priority: 16
Latest SAK AN: 1 Latest SAK KI: A22915FD8C07D0542B39270F/1
Previous SAK AN: 0 Previous SAK KI: 000000000000000000000000/0
MKA Suspend For: disabled MKA Suspend On Request: disabled
Peer list
1. Member identifier: 41FE705CDC5A50F3D21B3F8E (live)
Message number: 124 Hold time: 5000 (ms)
SCI: 88:E0:F3:C1:4E:E7/1
Lowest acceptable PN: 1
Troubleshooting
set security macsec interfaces et-3/2/0 traceoptions file macsec-intf
set security macsec interfaces et-3/2/0 traceoptions file size 10m
set security macsec interfaces et-3/2/0 traceoptions flag keys
set security macsec interfaces et-3/2/0 traceoptions flag mka-packets
Macsec over GRE interface ?
As discussed on the earlier sections, you can only encrypt on the IFD level and you cannot encrypt a tunnel itself like gre and vxlan. Since these occur at L3 , typically one would use IPSEC VPN to encrypt site to site traffic at L3. how can we encrypt tunnels so that we can encrypt at line rate ? lets say 400G of tunnel traffic encrypted ? Below method suggests a way to achieve the same.
Topology
Create wan interface
root@mx480-a# show interfaces et-5/1/0
description "test macsec";
unit 0 {
family inet {
address 193.1.1.1/30;
}
}
Create L2 gre interface
root@mx480-a# show | compare
[edit interfaces]
+ gr-5/2/10 {
+ description "macsec gre";
+ unit 0 {
+ tunnel {
+ source 193.1.1.1;
+ destination 193.1.1.2;
+ }
+ family bridge {
+ core-facing;
+ interface-mode trunk;
+ vlan-id 10;
+ }
+ }
+ }
Loop the traffic to the L2 interface
root@mx480-a# show interfaces et-5/1/1
description "loopback 5/1/2";
vlan-tagging;
native-vlan-id 10;
unit 0 {
family bridge {
interface-mode trunk;
vlan-id 10;
}
}
Create a L2 domain using virtual switch
place the L2 gre interface and the L2 interface in the same domain
root@mx480-a# show routing-instances macsec
instance-type virtual-switch;
bridge-domains {
bd {
vlan-id 10;
no-arp-suppression;
}
}
interface et-5/1/1.0;
interface gr-5/2/10.0;
Create a L3 interface on the other end of the looped interface
root@mx480-a# show interfaces et-5/1/2
description "loopback 5/1/1";
unit 0 {
family inet {
address 194.1.1.2/30 {
}
}
}
Add the L3 interface into an L3 domain
root@mx480-a# show routing-instances macsec-l3
instance-type virtual-router;
interface et-5/1/2.0;
similarly, create on the other side. Finally ping E2E and notice traffic is encrypted.
Final configs
RTR1
set interfaces et-6/1/3 description "physical loopback to et-6/2/3"
set interfaces et-6/1/3 unit 0 family inet address 194.1.1.2/30
set interfaces gr-6/2/0 description "macsec l2 gre tunnel"
set interfaces gr-6/2/0 unit 0 tunnel source 193.1.1.2
set interfaces gr-6/2/0 unit 0 tunnel destination 193.1.1.1
set interfaces gr-6/2/0 unit 0 family bridge core-facing
set interfaces gr-6/2/0 unit 0 family bridge interface-mode trunk
set interfaces gr-6/2/0 unit 0 family bridge vlan-id 10
set interfaces et-6/2/1 description "macsec wan to alfa-0/0/3"
set interfaces et-6/2/1 unit 0 family inet address 193.1.1.2/30
set interfaces et-6/2/3 description "physical loopback to et-6/1/3"
set interfaces et-6/2/3 vlan-tagging
set interfaces et-6/2/3 native-vlan-id 10
set interfaces et-6/2/3 unit 0 family bridge interface-mode trunk
set interfaces et-6/2/3 unit 0 family bridge vlan-id 10
set routing-instances l3macsec01 instance-type virtual-router
set routing-instances l3macsec01 interface et-6/1/3.0
set routing-instances macsecwan1 instance-type virtual-switch
set routing-instances macsecwan1 description "L2 instance for L2 GRE and tunnel macsec packets"
set routing-instances macsecwan1 bridge-domains bd vlan-id 10
set routing-instances macsecwan1 bridge-domains bd no-arp-suppression
set routing-instances macsecwan1 interface gr-6/2/0.0
set routing-instances macsecwan1 interface et-6/2/3.0
set security macsec connectivity-association test1 cipher-suite gcm-aes-xpn-256
set security macsec connectivity-association test1 security-mode static-cak
set security macsec connectivity-association test1 mka eapol-address 00:c5:2c:0e:f3:af << destination unicast mac address. This belongs to alfa et-0/1/0
set security macsec connectivity-association test1 pre-shared-key ckn abcd1234abcd1234
set security macsec connectivity-association test1 pre-shared-key cak "$9$tB1vu0IhclMWx/Cev8LN-k.mPFn/9pOBEikfz3nCAvW87bsoJG"
set security macsec interfaces et-6/1/3 connectivity-association test1
RTR2
root@alfa# show interfaces | display set
set interfaces et-0/0/2 description "physical loopback to et-0/1/0"
set interfaces et-0/0/2 vlan-tagging
set interfaces et-0/0/2 native-vlan-id 10
set interfaces et-0/0/2 unit 0 family bridge interface-mode trunk
set interfaces et-0/0/2 unit 0 family bridge vlan-id 10
set interfaces et-0/0/3 description "macsec wan to ferarri01-1 et-6/2/1"
set interfaces et-0/0/3 unit 0 family inet address 193.1.1.1/30
set interfaces et-0/1/0 description "physical loop to et-0/0/2"
set interfaces et-0/1/0 unit 0 family inet address 194.1.1.1/30
set interfaces gr-0/2/0 description "macsec l2 gre tunnel"
set interfaces gr-0/2/0 unit 0 tunnel source 193.1.1.1
set interfaces gr-0/2/0 unit 0 tunnel destination 193.1.1.2
set interfaces gr-0/2/0 unit 0 family bridge core-facing
set interfaces gr-0/2/0 unit 0 family bridge interface-mode trunk
set interfaces gr-0/2/0 unit 0 family bridge vlan-id 10
set routing-instances l3macsec01 instance-type virtual-router
set routing-instances l3macsec01 interface et-0/1/0.0
set routing-instances macsecwan1 instance-type virtual-switch
set routing-instances macsecwan1 description "L2 instance of L2 gre macsec pkts"
set routing-instances macsecwan1 bridge-domains bd vlan-id 10
set routing-instances macsecwan1 bridge-domains bd no-arp-suppression
set routing-instances macsecwan1 interface et-0/0/2.0
set routing-instances macsecwan1 interface gr-0/2/0.0
set security macsec connectivity-association test1 cipher-suite gcm-aes-xpn-256
set security macsec connectivity-association test1 security-mode static-cak
set security macsec connectivity-association test1 mka eapol-address 44:aa:50:54:ac:2d
set security macsec connectivity-association test1 pre-shared-key ckn abcd1234abcd1234
set security macsec connectivity-association test1 pre-shared-key cak "$9$tB1vu0IhclMWx/Cev8LN-k.mPFn/9pOBEikfz3nCAvW87bsoJG"
set security macsec interfaces et-0/1/0 connectivity-association test1
Verify
root@alfa# run ping 194.1.1.2 routing-instance l3macsec01
PING 194.1.1.2 (194.1.1.2): 56 data bytes
64 bytes from 194.1.1.2: icmp_seq=0 ttl=64 time=1.030 ms
64 bytes from 194.1.1.2: icmp_seq=1 ttl=64 time=0.980 ms
64 bytes from 194.1.1.2: icmp_seq=2 ttl=64 time=1.245 ms
^C
root@alfa# run show security macsec connections
Interface name: et-0/1/0
CA name: test1
Cipher suite: GCM-AES-XPN-256 Encryption: on
Key server offset: 0 Include SCI: no
Replay protect: off Replay window: 0
Outbound secure channels
SC Id: 00:C5:2C:0E:F3:AF/1
Outgoing packet number: 37
Secure associations
AN: 0 Status: inuse Create time: 00:12:27
Inbound secure channels
SC Id: 44:AA:50:54:AC:2D/1
Secure associations
AN: 0 Status: inuse Create time: 00:12:27
root@alfa# run show security macsec statistics
Interface name: et-0/1/0
Secure Channel transmitted
Encrypted packets: 38
Encrypted bytes: 3268
Protected packets: 0
Protected bytes: 0
Secure Association transmitted
Encrypted packets: 38
Protected packets: 0
Secure Channel received
Accepted packets: 38
Validated bytes: 0
Decrypted bytes: 3268
Secure Association received
Accepted packets: 38
Validated bytes: 0
Decrypted bytes: 0
Performance
- noticed due to GRE overhead a drop of 17G. i.e. 83G encrypted on a 100G link
- packet size used is 400B fixed UDP traffic.
- Expecting IMIX to be lower.
- This also uses a physical loopback which is convoluted and hacky but works ;)
References
[junos mx