Skip to content

Gateway Peering ACLs

A Gateway Peering can carry an Access Control List (ACL): an ordered list of rules that decides, for each new flow or packet, whether the gateway admits it or drops it. ACLs are configured via the optional, peering-scoped spec.acl field.

ACLs are opt-in: a peering with no spec.acl behaves exactly as described in Gateway Peering.

The full list of fields is in the Fabric API reference.

Traffic filtering in Gateway Peerings

Several mechanisms restrict what traffic can flow between the two VPCs of a Gateway Peering. The first mechanism is the definition of the peering itself, via the prefixes in the expose blocks: if a packet does not match prefixes from exposed blocks in a peering, it is dropped.

Some stateful NAT modes, such as masquerade and port-forwarding, add a second restriction: a flow may only be initiated in one direction. When masquerade is set-up, a flow can only be initiated from the VPC with the masqueraded prefixes; a packet sent towards a masqueraded host will be dropped if the gateway has no associated stateful flow entry in its table, even if the address is otherwise valid with regards to the peering definition. Conversely, flows can only be initiated towards, and never from, an endpoint using port-forwarding.

ACL rules come as a third mechanism, and add explicit allow/deny decisions based on 5-tuples. They come on top of the restrictions implied by the Gateway Peering definition, so an ACL rule will never allow any flow that is not permitted by the prefixes exposed, and NAT restrictions, in the peering.

ACL configuration

To configure an ACL, use field spec.acl next to spec.peering. The ACL applies to the whole peering. The YAML definition for the expose blocks remains unchanged. Here is an example:

gw-acl-peer.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
apiVersion: gateway.githedgehog.com/v1alpha1
kind: GatewayPeering
metadata:
  name: vpc-app--vpc-db
  namespace: default
spec:
  acl:
    default: deny       # Default action when no rule matches; "deny-unless-exposed" if omitted
    rules:
      - name: app-to-db # Optional, used in logs and diagnostics
        from: vpc-app   # Side initiating the flow
        to: vpc-db      # Side receiving it; either "from" or "to" may be omitted (not both)
        action: allow   # "allow" or "deny" (default)
        match:
          src:          # From-side native (pre-NAT) addresses
            - cidr: 10.0.1.0/24
          dst:          # To-side advertised addresses, as dialed by the initiator
            - cidr: 10.0.2.0/24
              ports: ["5432"]
          proto: tcp    # Omit to match any protocol
        scope: packet   # "packet" or "flow" (default, rule applies to flow replies too)
        log: false      # true or false (default)
  peering:
    vpc-app:
      expose:
        - ips:
          - cidr: 10.0.1.0/24
    vpc-db:
      expose:
        - ips:
          - cidr: 10.0.2.0/24

This peering lets hosts in prefix 10.0.1.0/24 from vpc-app open TCP connections on port 5432 to hosts with prefix 10.0.2.0/24 from vpc-db, and nothing else (default: deny) in either direction.

Rule evaluation order

The gateway evaluates rules in spec.acl.rules top-down: the first rule whose from/to and match pattern match the flow decides the verdict, and evaluation stops immediately. If no rule matches, spec.acl.default applies. Conflicts between rules are resolved purely by order, so a broad rule placed early will shadow the more specific rules that follow it.

Rule direction: from and to

Every rule specifies a direction. Fields from and to must each name one of the peering's two sides, using the same names as the keys under spec.peering (externals included, prefixed with ext.). A rule applies to packets emitted by from towards to.

Because a peering always has exactly two sides, one of from or to may be omitted, and the other side is implied. Omitting both is not allowed.

Depending on the scope specified for the rule, rules may also apply to packets of an established flow's reverse direction. Refer to the section on rule scope for details.

Matching patterns

The match block defines what packets (or flows) an ACL rule applies to.

NAT considerations

The gateway evaluates ACL rules for a packet before any NAT is applied. This means that the patterns in the match block are compared with the real source address and the destination address the initiator dialed (typically, the address as exposed as a service).

For example, this allows writing rules to let endpoint 10.0.1.1 reach a server exposed with port-forwarding as 10.0.2.1, regardless of what address the server uses in its own VPC.

Match block

The match block is a 5-tuple, expressed from the initiator's point of view:

  • src contains the from side's native address prefixes and ports
  • dst contains the to side's advertised address prefixes and ports (from its expose's as, or ips where no NAT applies)
  • proto can be tcp, udp, or a quoted protocol number (for example: "132" for SCTP)

src and dst are lists of match endpoints. Each entry carries an IP prefix and an optional list of ports, where a port entry is either a single port such as "80" (do not omit the quotes) or an inclusive range such as "8000-8100". The prefix takes the same shape as an expose entry: either a literal cidr, or a vpcSubnet reference naming a VPC subnet. At most one of cidr and vpcSubnet can be set on a given entry.

Tip

Field proto supports values tcp, udp, or a quoted protocol number. An icmp keyword should be added in the future, with related matching options such as error types; until then, you can use protocol number "1" to match ICMP packets.

Optional match fields

Everything in match is optional, and omitting a field widens the rule:

Field If ommitted, then...
src, dst the rule matches any source or destination address and port within the peering configuration, respectively
src.{cidr,vpcSubnet}, dst.{cidr,vpcSubnet} the rule matches any source or destination IP address within the peering configuration (respectively), but can still be restricted to specified port ranges
src.ports, dst.ports the rule matches all source or destination ports within the peering configuration, respectively
proto the rule matches any protocol

If the match block is omitted altogether, the rule matches every flow in the specified direction: this is how to write a generic allow or deny for one direction.

Rule scope

The scope field in each rule selects whether the rule is stateful:

  • flow (the default) makes the rule bidirectional for established connections. Once a flow is admitted, its return traffic is matched by the flow table entry and allowed even if it does not match any other explicit allow rule. The connection can still only be initiated in the rule's direction.
  • packet makes the rule stateless. Return traffic is not implicitly allowed and must be permitted by a separate rule.

Note that under default: deny, a rule with scope: packet and no matching reverse rule admits the forward direction while dropping every reply. Add an explicit rule for the return direction whenever you use scope: packet and need bidirectional traffic.

Also, note that reply traffic for allow rules with scope: flow may still be dropped if it matches an explicit deny rule in the list: packets are evaluated as potential "return traffic" only after going through a first direct lookup against the ACL rules table.

The scope does not matter in the case of deny rules, given that no flow matching this rule can be initiated in the first place.

Warning

Due to internal limitations for the gateway, scope: flow is currently restricted to peerings for which all flows use masquerade or port-forwarding. Flows relying on static NAT, or no NAT at all, do not currently generate stateful entries in the gateway's table. This limitation will be lifted in a future release.

Default action

When no rule matches, the default verdict from spec.acl.default is applied. The field can take one of two values:

  • deny-unless-exposed (the default) allows packets to go through, even when no rule matched. The name of the option is a reminder that ACL rules apply on top of the peering configuration, and that packets only go through if they correspond to the peering setup, even if a wider ACL rule would otherwise authorize them.
  • deny means that packets not explicitly allowed by a rule (or corresponding to return traffic for an established flow allowed by a rule with scope: flow) are dropped.

Logging

Set the log field of a rule to true to log all packets matching that rule (as well as corresponding reply traffic, for rules with scope: flow).

Examples

This section contains some example use cases and configuration for the peering ACLs.

Restricting an exposed service

VPC vpc-01 exposes a prefix masqueraded behind 100.64.10.0/24; on the other side, vpc-02 publishes a backend as 100.64.20.10:443. Only vpc-01's workload subnet may reach it.

gw-acl-masquerade.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
spec:
  acl:
    default: deny
    rules:
      - name: vpc-01-workloads-to-web
        from: vpc-01
        to: vpc-02
        action: allow
        match:
          src:
            - vpcSubnet: vpc-01/subnet-1
          dst:
            - cidr: 100.64.2.10/32 # The virtual IP vpc-01 dials: vpc-02's "as"
              ports: ["443"]
          proto: tcp
        scope: flow
  peering:
    vpc-01:
      expose:
        - ips:
          - cidr: 10.0.1.0/24
          as:
          - cidr: 100.64.1.0/24
          nat:
            masquerade: {}
    vpc-02:
      expose:
        - ips:
          - cidr: 10.0.2.10/32
          as:
          - cidr: 100.64.2.10/32
          nat:
            portForward:
              ports:
                - proto: tcp
                  port: "8443"
                  as: "443"

A flow from 10.0.1.5 to 100.64.2.10:443 is evaluated before NAT, so the rule matches vpc-01's real source address even though the packets leave the gateway masqueraded behind 100.64.1.0/24. Based on the scope, reply traffic is automatically allowed.

Egress allow-list towards an external

A private subnet reaches the Internet through masquerade, with a strict list of allowed destination ports, and one carve-out rule placed ahead of it.

gw-acl-egress.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
spec:
  acl:
    default: deny
    rules:
      - name: block-blackholed-ranges
        from: vpc-internal  # "to" is omitted: the external side is implied
        action: deny
        match:
          dst:
            - cidr: 192.0.2.0/24
            - cidr: 198.51.100.66/32
        scope: flow
      - name: allow-web
        from: vpc-internal
        action: allow
        match:
          proto: tcp
          dst:
            - ports: ["80", "443"]
        scope: flow
      - name: allow-dns
        from: vpc-internal
        action: allow
        match:
          proto: udp
          dst:
            - ports: ["53"]
        scope: flow
  peering:
    vpc-internal:
      expose:
        - ips:
          - cidr: 10.0.1.0/16
          as:
          - cidr: 198.51.100.7/32
          nat:
            masquerade:
              idleTimeout: 5m0s
    ext.internet:
      expose:
      - default: true

Match field src is omitted throughout, so it matches any address vpc-internal exposes through the peering. The deny rule come first because the first match decides the verdict: moving allow-web above block-blackholed-ranges would allow HTTP/HTTPS towards the blackholed ranges.

Bidirectional traffic with static NAT

A static one-to-one peering where each side may open its own connections. Each side is named in the address space it dials: corp reaches the bench at its advertised 198.51.100.10, while the bench reaches corp at its real address, since corp applies no NAT.

In the following setup, vpc-01 exposes a prefix using static NAT, and uses an ACL list to restrict communications to a few hosts within the prefix. Rules applies per packet and not per flow (scope: packet), meaning that the return traffic has to be explicitly allowed.

gw-acl-bidir.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
spec:
  acl:
    default: deny
    rules:
      - from: vpc-01
        action: allow
        match:
          src:
            - cidr: 10.0.1.10/32
          dst:
            - cidr: 10.0.2.28/32
        scope: packet
      - from: vpc-02
        action: allow
        match:
          src:
            - cidr: 10.0.2.12/32
            - cidr: 10.0.2.28/32
          dst:
            - cidr: 192.168.0.10/32
        scope: packet
  peering:
    vpc-01:
      expose:
        - ips:
          - cidr: 10.0.1.10/32
          as:
          - cidr: 192.168.0.10/32
          nat:
            static: {}
    vpc-02:
      expose:
        - ips:
          - cidr: 10.0.2.0/24

Endpoint 10.0.2.28 from vpc-02 can communicate with 10.0.1.10 from vpc-01 both ways, thanks to the first rule allowing traffic from 10.0.1.10; however, endpoint 10.0.2.12 from vpc-02 can emit towards 10.0.1.10, but will not receive any return traffic.