Data Center Manager (DCM) System Configuration

Extreme Connect Modules for data center applications leverage ExtremeCloud IQ Site Engine end-system groups to create and manage virtual portgroups within 3rd party hypervisors.

DCM Fabric Manager

End-System Groups

Private VLANs

DCM Fabric Manager

After installing Extreme Connect, ExtremeCloud IQ Site Engine NAC Manager offers a new configuration menu at Tools > Management and Configuration > Data Center Fabric.

The Fabric Manager assists in the creation of new end-system groups and the corresponding description string that will be used by Extreme Connect to create portgroups on remote systems.

While the parameters could also be edited manually in the end-system group menu, it is strongly recommended to use the wizard to avoid accidental misconfiguration.

The individual configuration options are:

Configuration Options Description
approval=true|false If you set this value to “true”, end-system must be approved before it is added to this end-system group. Can be used for sensitive endsystem group like your DMZ group, for example, where you don’t want any VMs to be assigned to without proper approval. VMs which are allocated to such end-system groups/vSwitch but have not yet been approved manually by an administrator will temporarily be pushed to the default group “VM Pending Approval”.
sync=true|false Only if you set this value to ‘true’ a new portgroup (VMware) or network (XEN) will be created automatically with the same name by the Datacenter manager.
VLAN ID In order to define a VLAN ID for new VMware vSwitches/dvSwitches or XEN networks (this feature is not available for the Hyper-V module) you can use the following two formats:

vlan=#static_vlan_id#: Setting this value to ‘vlan=100’, for example, will create a new portgroup (for VMware vSwitches) or network (XEN) and assign the VLAN ID 100 to it. For proper configuration you would then need to create an Extreme Control NAC rule which would bind this endsystem group to a policy which also assigns (“Contain to”) the endsystem to VLAN 100 on the physical network. The VMware/XEN management will make sure that VMs within this portgroup/network will be tagged with VLAN ID 100.

vlan=#primary_vlan_id#:#secondary_vlan_id#:isolated_or_community: This format is exclusively used for VMware to create a new private VLAN and corresponding dvSwitch. The primary and secondary vlan IDs used must not be the same! The third parameter can only be “isolated” or “community”. VMs connected to isolated PVLANs are not able to communicate directly with each other – all communication will traverse the physical network. VMs connected to community PVLANs are able to communicate directly with each other through their dvSwitch. Example: “vlan=4000:4001:isolated”.
switchgroup=#name# This is a setting exclusively used for VMware. If you have ‘sync=true’ but don’t set this switchgroup value it will automatically create a new portgroup for this endsystem group on ALL vSwitches. If you have vSwitches should, for example, only be used for management purpose you might not want the Datacenter manager to create such portgroups on those vSwitches. You can use the following pre-defined values to adjust this settings as follows. In addition to these pre-defined values you can also use Regular Expression to granularly define the vSwitches where you want the new portgroups to be created.

vSwitchOnly: The new portgroup will only be created on all vSwitches, not on distributed virtual switches.

dvSwichtOnly: The new portgroup will only be created on all distributed vSwitches, not on the vSwitches.

includeAll: The new portgroup will be created on all vSwitches and distributed vSwitches.

excludeAll: There will be no new portgroup created.
nic=#list of NICs# This is a setting exclusively used for XEN. In order for the Datacenter manager to create a new network within XEN server it needs to know the physical interface used to attach this network to. This must be the name of the physical interface as seen by the operating system of the XEN servers. For both examples below, don’t forget to also use the settings “sync=true” and “vlan=XXXX” – this will create a so called external XEN network and setting both the vlan ID and the physical NIC is mandatory for external networks. Setting only one of these two values will result in the creation of an internal network that will not have a VLAN ID nor a connection to the physical network.

Example 1: If you use your first interface (eth0) for management of the XEN server and you want to create a new XEN network which connects to the second physical interface, use “nic=eth1” for the corresponding end-system configuration.

Example 2: If you want to create a bond instead of a simple network you will need to provide a list of NICs that should be attached to this bond. You could use the following syntax: “nic=eth1,eth2”

Verification

To verify the configuration:

  1. See which groups Extreme Connect is aware of at the “End-System Group” panel on the configuration page.

Note: The groups under “ExtremeCloud IQ Site Engine” lists the entire group inventory, while the list under “Extreme Connect” only lists those groups that are marked for synchronisation (sync=true).

  1. If synchronisation is not enabled for a group, Extreme Connect will act as if that group does not exists when creating external portgroups/networks.

End-System Groups

After initial installation the following groups should be present in IAM:

End-system group for Disconnected Devices Fusion Disconnected Systems
End-system group for Pending Approvals Fusion Pending Approval

We have shown the default names for each group. These names can be changed during installation or in the configuration page.

These groups provide the ability to configure access rules for end-systems that qualify for any of these. The approval pending group will contain end-systems which are connected to a portgroup with the “approval=true” flag being set, before they are approved by an administrator.

The disconnected devices group will create a portgroup on the hyper visor for case that an end-system group is deleted, the portgroup/network deletion feature is enabled and the to-be-deleted portgroup/network has still VMs attached. These VMs will be moved to the Disconnected Systems portgroup and consequently show up in the end-system group of the same name.

Private VLANs

Private VLANs (pVLANs) currently only exist within VMware. In a standard VMware setup, all VMs connected to the same distributed vSwitch (dvSwitch) are allowed to talk to each other. With pVLANs it is possible to isolate VMs connected to the same dvSwitch from each other. This ways they cannot directly communicate with each other. Any communication between those isolated VMs must be carried out outside of the VMware environment over the physical network. This is a great way to control the traffic/applications used by these VMs (using Extreme Policies) and also, if needed, screen that traffic using Netflow technology.

Requirements

VMware vCenter sufficient license to use distributed vSwitches.

At least one distributed virtual Switch (dvSwitch)

Useful Information on pVLANs

The vCenter Server will manage multiple ESX hosts. A dvSwitch is a virtual Switch which exists on all your ESX servers managed by a vCenter Server and is unique to all of them. You cannot use pVLANs on normal vSwitches.

Note: the following section is only informational. The described tasks are automated via Data Center Manager!

To create pVLANs:

  1. Create a new dvSwitch and navigate to its settings windows.
  2. Choose the “Private VLAN” tab.
  3. Create primary and secondary private VLANs. Every primary private VLAN ID must have one secondary VLAN ID with the same ID in promiscuous mode and then can have multiple other secondary VLAN IDs. The secondary VLANs can either be of type “isolated” or “community”. In isolated mode, the VMs connected to these secondary VLANs will not be able to communicate with other VMs on the same dvSwitch without being routed through the physical network. The community mode enables direct VM communication within the virtual network environment (dvSwitch). No secondary VLAN ID or static VLAN ID can be the same as any existing primary VLAN ID.
  4. When these VMs communicate on the physical network, you will see the secondary private VLAN ID, not the primary one. For additional information, we attach the knowledge base article directly from VMware regarding this topic:
    http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1010691

Promiscuous PVLANs have the same VLAN ID both for Primary and Secondary VLAN.

Community and Isolated PVLANs traffic travels tagged as the associated Secondary PVLAN.

Traffic inside PVLANs is not encapsulated (no Secondary PVLAN encapsulated inside a Primary PVLAN Packet).

Traffic between virtual machines on the same PVLAN but on different ESX hosts go through the Physical Switch. Therefore, the Physical Switch must be PVLAN aware and configured appropriately, to enable the secondary PVLANs to reach destination.

Switches discover MAC addresses per VLAN. This can be a problem for PVLANs because each virtual machine seems to be in more than one VLAN to the physical switch, or at least, it indicates that there is no reply to the request, because the reply travels back in a different VLAN. For this reason, it is a requirement that each physical switch, where ESX with PVLANs are connected, must be PVLAN aware.

In order to actually use these private VLANs you need to create a portgroup within the dvSwitch. Within the settings section of this portgroup you can configure the VLAN. Select “Private VLAN” as the type and then choose from those private VLANs you’ve configured before.

Note: If you configure a secondary private VLAN 201 and at the same time add the following string to an end-system group’s description field within Extreme Control NAC manager “vlan=200:201:isolated”, Datacenter manager will recognize this and create the appropriate config and add all VMs within this end-system group to that private VLAN dvSwitch.

Reference Setup

We’ve created a reference on how to deploy a pVLAN configuration. Goal of this setup was to create two VMs which are connected to the same dvSwitch within the same secondary isolated PVLAN which can only communicate with each other traversing the physical network. This has been extended to also traverse a routing instance. This way, one can create the same setup where the VMs are distributed over two physical ESX servers which are located in different routing networks.

Policy Domain Configuration

This section describes the setup of the different Policy Domains used for the different switching/routing layers. A short summary/overview before we dip into the details:

  1. 1. Dynamic role at the S-Series in Layer 2 Mode (switching) assigns all traffic based on the source MAC of the VMs into the following VLANs:
    1. All traffic to the router (VRRP) MAC address contained into VLAN 4000
    2. All ARP traffic contained into VLAN 4001
    3. You can add additional rules
  2. VLAN to Policy Map at the S-Series in Layer 3 Mode (routing) for PVLAN L3
    1. Is assigned to all traffic tagged with VLAN ID 4001 – no dynamic policy assignment based on the MAC addresses of the VMs
    2. Contains all ARP traffic to VLAN 4000 (all other traffic is already contained to 4000)
    3. Router interface within this VLAN 4000 is replying to the ARP requests with its own MAC address (local proxy ARP) and sends the reply in VLAN 4000
  3. The VLAN 4000 and 4001 must be statically configured on the uplinks/trunk in between the physical switches
Policy Domain Layer 2 – Role VM PVLAN Access

All traffic coming from the VM is tagged with VLAN ID 4001 (the secondary PVLAN ID for the dvSwitch where this VM is connected to). The following role configuration has been implemented:

  • Role level: VLAN 4000 tagged egress: to dynamically assign this VLAN to this port VLAN egress list so on the way back from the physical network to the VM the traffic will be tagged with VLAN 4000.
  • Role level: TCI overwrite enabled
  • Role level: Deny All traffic by default
  • Rule: Contain packets to the backbone router’s MAC address (00:00:5e:00:01:01) to VLAN 4000. This avoids inter-VM communication via broad- and multicasts
  • Rule: Value 0x806 (ARP) contain to VLAN 4001: Only ARP traffic is kept in VLAN 4001 to make sure it is only broadcasted to the upstream of the Layer 2 switch where the router is connected (this router replies to the ARP broadcasts)
Policy Domain Core – Policy VM PVLAN L3

The core router is S-Series switch configured as a router. It receives the IP traffic on VLAN 4000 and the ARP broadcasts on VLAN 4001 from the VMs. This router has “local proxy ARP” enabled to reply with its own MAC address when it receives any ARP broadcast for any VMs (even residing on the same local subnet) since all traffic from the secondary PVLAN 4001 should be routed through this router and not travel directly between the VMs. The following configuration has been implemented:

  • Role level: VLAN 4000 tagged egress: assign VLAN 4000 tagged egress for IP traffic back to the VMs
  • Role level: TCI override enabled
  • Role level: Role Mapping of VLAN ID 4001 to policy VM PVLAN L3
  • Rule: Value 0x806 (ARP) contain to VLAN 4000: This is where we re-map the ARP traffic from 4001 to 4000 to have this router’s interface within VLAN 4000 reply to that ARP broadcast with its own MAC address (local proxy ARP) – after this re-mapping is done, there is no more traffic on VLAN 4001