Internet Protocol security (IPSec)

Internet Protocol security (IPSec) is a framework of open standards for ensuring private, secure communications over Internet Protocol (IP) networks, through the use of cryptographic security services.

  • For tips about using IPSec, see Best practices.
  • For help with specific tasks, see How to.
  • For general background information, see Concepts.
  • For problem-solving instructions, see Troubleshooting.

Note

  • Portions of IPSec-related services were jointly developed by Microsoft® and Cisco Systems, Inc.


Best practices

  • Establish an IPSec deployment plan.

    • The deployment plan should address the following considerations: which deployment scenarios (such as server-to-server or remote access) require the use of IPSec, what level of security you require for each scenario, which types of data to secure, which computers to secure, which physical links to secure, who will manage IPSec policies, and how you will provide ongoing support and troubleshooting for end users after IPSec is deployed. The deployment plan should also address broader organizational security issues, such as the vulnerability of the network to specific types of attacks, the use of Active Directory, and how security is applied to Group Policy objects.

    For more information about IPSec security planning, see Establishing an IPSec security plan.

    For more information about Active Directory, see Active Directory overview.

  • Create and test IPSec policies for each deployment scenario.

    • Before deploying IPSec in a production environment, test the IPSec policies in a realistic lab environment. To obtain realistic performance data, run standard workloads on programs. During initial tests, view packet contents with Network Monitor, or use Authentication Header (AH) or Encapsulating Security Payload (ESP) with null encryption to view packet contents for test environments.

    For an example of how to set up an IPSec test lab, see Setting up an IPSec test lab.

  • Do not use preshared keys.

    • For enhanced security, the use of preshared key authentication is not recommended because it is a relatively weak authentication method. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment.

    For more information, see Preshared key authentication.

  • Do not use Diffie-Hellman Group 1 (low).

    • For enhanced security, do not use Diffie-Hellman Group 1, which provides 768 bits of keying strength. For maximum security, use Group 2048 (high), which provides 2,048 bits of keying strength. Use Group 2 (medium), which provides 1,024 bits of keying strength when required for interoperability with Windows 2000 and XOX. Strong Diffie-Hellman groups combined with longer key lengths increase the computational difficulty of determining a secret key.

    For more information, see Key exchange methods.

    Note

    • Diffie-Hellman Group 2048 is provided only with the Windows Server 2003 family.
  • Use the Triple Data Encryption Standard (3DES) algorithm for stronger encryption.

    • For enhanced security, when configuring key exchange security methods for IPSec policies, use 3DES, which is a stronger encryption algorithm than DES.

    For information about how to configure key exchange security methods, see To create key exchange security methods.

    Note

    • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.
  • Create and assign a persistent IPSec policy for failsafe security.

    • To enhance security, create and assign a persistent IPSec policy, so that computers can be secured if a local IPSec policy or an Active Directory-based IPSec policy cannot be applied. When you create and assign a persistent policy, it is applied before a local policy or an Active Directory-based policy, and it remains in effect regardless of whether the local policy or the Active Directory-based policy is applied (for example, an IPSec policy will not be applied if it is corrupted).

    Note

    • You cannot configure this feature in the IP Security Policy Management console. To configure this feature, you must use the Netsh IPSec command-line tool. For more information, see Netsh commands for Internet Protocol security.
  • For computers connected to the Internet, do not send the name of the certification authority (CA) with certificate requests.

    • When certificate authentication is used to establish trust between IPSec peers, each IPSec peer sends to the other peer a list of trusted root CAs from which it accepts a certificate for authentication. Each of these CA names is sent as a certificate request payload (CRP), and it must be sent before trust is established. Although transmitting this list aids in connectivity by facilitating the selection of a CA, it can expose sensitive information about the trust relationships of a computer, such as the name of the company that owns the computer and the domain membership of the computer (if an internal public key infrastructure is being used), to an attacker. Therefore, to secure computers that are connected to the Internet, enable the option to exclude the CA name from the certificate request.

  • For computers connected to the Internet, do not use Kerberos as an authentication method.

    • When Kerberos V5 authentication is used, during main mode negotiation, each IPSec peer sends its computer identity in unencrypted format to the other peer. The computer identity is unencrypted until encryption of the entire identity payload takes place during the authentication phase of the main mode negotiation. An attacker can send an Internet Key Exchange (IKE) packet that causes the responding IPSec peer to expose its computer identity and domain membership. To secure computers that are connected to the Internet, certificate authentication is recommended.

    For more information, see Authentication methods.

  • For computers that are connected to the Internet, do not allow unsecured communication.

    • If you configure a filter action to negotiate IPSec security, ensure that the following options are disabled in order to secure computers that are connected to the Internet:

    • Accept unsecured communication, but always respond using IPSec. This option allows initial incoming unsecured traffic (for example, TCP SYN packets) but requires outgoing traffic to be protected. This option should be disabled to prevent denial-of-service attacks.
    • Allow unsecured communication with non-IPSec-aware computers. This option allows unsecured communications with computers that cannot negotiate the use of IPSec or process IPSec-secured communications, and it is appropriate only in environments where IPSec-secured communication is not necessary.

    For more information, see Filter action.

  • Restrict the use of administrative credentials in your organization.

    • Members of the local Administrators group can view and modify the IPSec policy settings on their computers. For this reason, and as a general security best practice, ensure that end-users in your organization use the principle of least privilege.

    For information about general best practices for security, see Best practices for security.

  • When applying the same IPSec policy to computers running different versions of the Windows operating system, test the policy thoroughly.

    • Several features in the Windows Server 2003 family implementation of IPSec are not provided in Windows 2000 or in XOX. These new features include:

    • A stronger Diffie-Hellman group (2,048-bit Diffie-Hellman key exchange)
      • .
    • The ability for IPSec policy filters to allow logical addresses for local IP configuration
      • .
    • The removal of default traffic exemptions from IPSec filtering (only Internet Key Exchange traffic is now exempt by default)
      • .
    • The ability to exclude the name of the certification authority (CA) from a certificate request
      • .

    To ensure that the same IPSec policy functions as expected on computers running the Windows Server 2003 family and on computers running XOX or Windows 2000, test the policy thoroughly on all relevant operating systems before deployment.

    For more information about new features in IPSec, see New features for IPSec.

  • Use the Windows Server 2003 IP Security Policy Management console to manage the IPSec policies that use the new features that are available only in the Windows Server 2003 family implementation of IPSec.

    • If you plan to apply IPSec policies that use any of the new features that are available only in the Windows Server 2003 family implementation of IPSec, do not use the XOX or the Windows 2000 version of the IP Security Policy Management console to manage these policies. The settings in the earlier versions of the IP Security Policy Management console will override the settings in the Windows Server 2003 family IPSec policy, and the new features will not be functional.

  • Use Terminal Services to remotely manage and monitor IPSec on computers running different versions of the Windows operating system.

    • Remote management and monitoring of IPSec is supported only for computers running the same version of the Windows operating system. To remotely manage and monitor IPSec on a computer that is running a different version of Windows than the version of Windows that is running on your computer, use Terminal Services. For example, if your computer is running the Windows Server 2003 family, and you plan to remotely manage and monitor IPSec on computers running Windows 2000 or XOX, use Terminal Services to gain remote access to these computers.

    If your computer is running the Windows Server 2003 family, and you plan to manage and monitor IPSec on computers that are also running the Windows Server 2003 family, you can run the IPSec Policy Management console and IP Security Monitor, or you can use the Netsh command-line tool remotely.


How to...

  • Start the IP Security Policy Management snap-in
  • Define IPSec policies
  • Assign IPSec policy
  • Manage IPSec policy
  • Use Resultant Set of Policy (RSoP) to view IPSec policy assignments
  • Monitor IPSec activity


Start the IP Security Policy Management snap-in

Microsoft Windows graphic

To start the IP Security Policy Management snap-in

  • To start the IP Security Policy Management snap-in from the Microsoft Management Console:
    1. Click Start, click Run, type MMC, and then click OK.
    2. Click File, click Add/Remove Snap-in, and then click Add.
    3. Click IP Security Policy Management, and then click Add.
    4. Select the computer for which you want to manage IPSec policies:
      5. To | Do this
Manage only the computer on which this console is running | Click Local computer.
Manage IPSec policies for any domain members | Click The Active Directory domain of which this computer is a member.
Manage IPSec policies for a domain of which the computer that is running this console is not a member | Click Another Active Directory Domain.
Manage a remote computer | Click Another computer.
    5. Click Finish, click Close, and then click OK.
  • To access the IP Security Policy Management snap-in from Group Policy (Active Directory):
    1. Open Active Directory Users and Computers.
    2. In the console tree, right-click the domain or organizational unit for which you want to set Group Policy.

      Where?

      • Active Directory Users and Computers [DomainControllerName.DomainName]
      • Domain
      • OrganizationalUnit
      • ChildOrganizationalUnit...
    3. Click Properties, and then click the Group Policy tab.
    4. Click Edit to open the Group Policy object that you want to edit. Or, click New to create a new Group Policy object, and then click Edit.
    5. In the Group Policy console tree, click IP Security Policies on Active Directory.

      Where?

      • PolicyName [ComputerName] Policy
      • Computer Configuration
      • Windows Settings
      • Security Settings
      • IP Security Policies on Active Directory
  • To access the IP Security Policy Management snap-in from Local Computer Policy:
    1. Click Start, click Run, type MMC, and then click OK.
    2. Click File, click Add/Remove Snap-in, and then click Add.
    3. Click Group Policy Object Editor, and then click Add.
    4. Click Finish, click Close, and then click OK.
    5. In the Group Policy console tree, click IP Security Policies on Local Computer.

      Where?

      • Local Computer Policy
      • Computer Configuration
      • Windows Settings
      • Security Settings
      • IP Security Policies on Local Computer

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • To save console settings, on the File menu, click Save.
  • To view the saved console, see Related Topics.
  • You cannot administer Active Directory-based IPSec policy from a computer running XOX.
  • To manage policies in a remote domain, you must be using a computer that is a member of a domain that is trusted by the remote domain. You cannot configure policies in a remote domain from a computer that is a member of a workgroup (also known as a stand-alone computer).
  • To revise your initial choice, you can start the Microsoft Management Console, add this snap-in again, and save the console again. If you require multiple configurations, you can save the console with another name.

Related Topics


Define IPSec policies

  • Add, edit, or remove IPSec policies
  • Define IPSec rules
  • Define IPSec key exchange settings


Add, edit, or remove IPSec policies

Microsoft Windows graphic

To add, edit, or remove IPSec policies

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Choose whether to add a new policy, or edit or remove an existing one:
    3. To | Do this
Add a new policy | In the console tree, click IP Security Policies on Name. Next, on the Action menu, click Create IP Security Policy. Follow the instructions in the IP Security Policy Wizard until the Properties dialog box for your new policy appears.
Modify an existing policy | Double-click the policy that you want to modify.
Remove a policy | Click the policy that you want to remove, and on the Action menu, click Delete.
  3. If you are adding or modifying a policy, click the General tab, and then in Name, type a unique name.
  4. In Description, type a description of the security policy, such as which groups or domains it affects.
  5. To specify how often the IPSec Policy Agent checks for updates, type a value in Check for policy changes every number minute(s).
  6. If you have special requirements for the security of the key exchange, click Settings.
  7. Click the Rules tab, and create or modify any necessary rules for the policy.
  8. Activate or deactivate rules as needed.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • For information about defining IPSec key exchange settings, see Related Topics.
  • For information about configuring IPSec rules, see Related Topics.
  • For information about how to activate or deactivate rules, see Related Topics.

Related Topics


Define IPSec rules

  • Add, edit, or remove IPSec rules
  • Activate or deactivate IPSec rules
  • Define IPSec authentication methods
  • Specify IPSec connection types
  • Specify an IPSec tunnel
  • Define IP filter lists
  • Define filter actions


Add, edit, or remove IPSec rules

Microsoft Windows graphic

To add, edit, or remove IPSec rules

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. To add a rule, decide whether you want to use the Create IP Security Rule Wizard or add the rule manually:
    • To add a rule by using the Create IP Security Rule Wizard, confirm that the Use Add Wizard check box is selected, click Add, and then follow the instructions.
    • To add a rule manually, confirm that the Use Add Wizard check box is cleared, click Add, and then define settings on the IP Filter List, Filter Action, Authentication Methods, Tunnel Setting, and Connection Type tabs.
  4. To edit a rule, select the rule that you want to edit, click Edit, and then modify the rule properties as needed.
  5. To remove a rule, select the rule that you want to remove, and then click Remove.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • New rules are automatically applied to the policy being edited or created.
  • The rules for a policy are displayed in IP Security Policies in reverse alphabetical order, based on the name of the filter list selected for each rule. There is no method to specify an order in which to apply the rules in a policy. The IPSec driver automatically orders the rules based on the most specific filter list to the least specific filter list. For example, the IPSec driver places a rule containing a filter list that specifies individual IP addresses and TCP ports above a rule containing a filter list that specifies all addresses on a subnet.
  • The default response rule is automatically added to every new IPSec policy. If you do not want this rule to be part of your policy, you can deactivate it by clearing the check box next to <Dynamic>. The default response rule cannot be removed.

Related Topics


Activate or deactivate IPSec rules

Microsoft Windows graphic

To activate or deactivate IPSec rules

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. On the Rules tab, do one of the following:
    • To activate a rule, select the check box next to the rule that you want to activate.
    • To deactivate a rule, clear the check box next to the rule that you want to deactivate.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.

Related Topics


Define IPSec authentication methods

Microsoft Windows graphic

To define IPSec authentication methods

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify.
  4. On the Authentication Methods tab, click Add. Or, if you are reconfiguring an existing method, click the authentication method, and then click Edit.
  5. Select the authentication method that you want to add or modify:
    • To use the Kerberos V5 security protocol for authentication services, click Active Directory default (Kerberos V5 protocol) if this rule applies to computers that are validated by a Windows 2000 or Windows Server 2003 Active Directory domain or a trusted Active Directory domain.
    • To use a public key certificate for authentication services, click Use a certificate from this certification authority (CA), and then click Browse to select a root CA.
    • To prevent the name of the CA from being sent with the certificate request, select the Exclude the CA name from the certificate request check box.
    • To enable certificate to account mapping, select the Enable certificate to account mapping check box.
    • To specify your own key that will be used for authentication, click Use this string (preshared key).
  6. To delete an authentication method or change its preference order, do one of the following:
    • To delete the selected method, click Remove.
    • To move the selected method up one level, click Move up. Repeat until the method is at the required preference level.
    • To move the selected method down one level, click Move down. Repeat until the method is at the required preference level.

Important

  • The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • The Kerberos V5 protocol authentication method is not supported on computers running XOX.
  • If you choose to use a certificate for authentication, you must select a CA (most commonly the root CA for your installed computer certificate). You cannot leave this field blank.
  • For preshared key authentication, each IPSec peer must use the same preshared key, or authentication will fail.
  • If filters in more than one rule apply to traffic between the same two IP addresses, then the authentication methods list should be identical in both rules. Otherwise, security requests that are initiated by one rule might match the filter of the other rule, but have a different authentication method. In this case, the negotiation will fail. After a rule has initiated a successful main mode security association between two IP addresses, other rules that are filtering different traffic will not renegotiate the authentication and master key (the main mode security association). Instead, these other rules will use the same main mode security association to negotiate security (algorithms and session keys) for traffic that matches their particular filter.

Related Topics


Specify IPSec connection types

Microsoft Windows graphic

To specify IPSec connection types

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify.
  4. On the Connection Type tab, select the type of network connections to which this rule will apply:
    • To apply this rule to all network connections you have created on the computer, click All network connections.
    • To apply this rule to all LAN connections you have created on the computer, click Local area network (LAN).
    • To apply this rule to any remote VPN or dial-up connections you have created on the computer, click Remote access.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.

Related Topics


Specify an IPSec tunnel

Microsoft Windows graphic

To specify an IPSec tunnel

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify.
  4. On the Tunnel Setting tab, specify the computer that will be the tunnel endpoint:
    5. To | Do this
Disable tunneling for this rule | Click This rule does not specify an IPSec tunnel.
Use tunneled communications to a specific tunnel endpoint | Click The tunnel endpoint is specified by this IP address and type the IP address of the tunnel endpoint.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • Because you cannot mirror filters for tunneled traffic, you must configure two rules. One rule is used for the outbound traffic and the other is used for inbound traffic. For the outbound traffic rule, the tunnel endpoint is the IP address of the IPSec peer on the other end of the tunnel. For the inbound traffic rule, the tunnel endpoint is an IP address configured on the local computer.
  • IPSec tunnels are not supported for remote access VPN scenarios. L2TP/IPSec or PPTP should be used for remote access VPN connections. IPSec tunnels are used primarily for interoperability with other routers, gateways, or end-systems that do not support L2TP/IPSec or PPTP connections. IPSec tunnel mode is supported as an advanced feature, used only in gateway-to-gateway (also known as router-to-router) tunneling scenarios and for server-to-server or server-to-gateway configurations.

Related Topics


Define IP filter lists

  • Add, edit, or remove IP filter lists
  • Add, edit, or remove IPSec filters
  • Configure advanced IPSec filter settings
  • Set an active IP filter list for a rule
  • Manage multiple IP filter lists


Add, edit, or remove IP filter lists

Microsoft Windows graphic

To add, edit, or remove IP filter lists

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that contains the IP filter list that you want to modify.
  4. On the IP Filter List tab, do one of the following:
    • To add a new IP filter list, click Add.
    • To modify an existing IP filter list, click the filter list that you want to modify, and then click Edit.
    • To remove an IP filter list, click the filter list that you want to remove, and then click Remove.
  5. If you are adding or modifying an IP filter list, type a unique name for the list.
  6. Type a description to specify which computers and traffic types the IP filter list encompasses (for example, "Matches all TCP packets between this computer and any other computer").
  7. Create or modify the filters as needed.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • For procedures on creating filters and advanced filter settings, see Related Topics.

Related Topics


Add, edit, or remove IPSec filters

Microsoft Windows graphic

To add, edit, or remove IPSec filters

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that contains the IP filter list you want to modify.
  4. On the IP Filter List tab, double-click the IP filter list that contains the IPSec filter you want to modify.
  5. In the IP Filter List dialog box, do one of the following:
    • To add a filter, click Add.
    • To modify an existing filter, select the filter that you want to modify, and then click Edit.
    • To remove an existing filter, select the filter that you want to remove, and then click Remove.
  6. If you are adding or editing a filter, in IP Filter Properties, click the Addresses tab, and then select the Source Address:
    Select | To secure packets from
    My IP Address | All IP addresses on the computer for which you are configuring this filter.
    Any IP Address | Any computer.
    A Specific DNS Name | The Domain Name System (DNS) name that you specify in Host name. The DNS name is resolved to its IP addresses, and then filters are automatically created for the resolved IP addresses. This option is only available when creating new filters.
    A Specific IP Address | The IP address that you specify in IP Address.
    A Specific IP Subnet | The IP subnet, as defined by the IP address that you specify in IP Address and the subnet mask that you specify in Subnet Mask.
    DNS Servers <dynamic> | The DNS server(s) for the computer for which you are configuring this filter. Changes in DNS server addresses are automatically detected, and the filter is updated as needed.
    WINS Servers <dynamic> | The WINS server(s) for the computer for which you are configuring this filter. Changes in WINS server addresses are automatically detected, and the filter is updated as needed.
    DHCP Server <dynamic> | The DHCP server for the computer for which you are configuring this filter. Changes in DHCP server addresses are automatically detected, and the filter is updated as needed.
    Default Gateway <dynamic> | The default gateway for the computer for which you are configuring this filter. Changes in default gateway server addresses are automatically detected, and the filter is updated as needed.
  7. Click Destination Address, and repeat step 6 for the destination address.
  8. Select the appropriate Mirrored setting:
    To | Do this
    Automatically create two filters based on the filter settings, one for traffic to the destination and one for traffic from the destination | Select the Mirrored check box.
    Create a single filter based on the filter settings | Clear the Mirrored check box.
    Create a filter for an IPSec tunnel | Clear the Mirrored check box. For IPSec tunnels, you must create two filter lists: one list describes the traffic to be sent through the tunnel (outbound traffic) and another describes the traffic to be received through the tunnel (inbound). Then, create two rules that use the inbound and outbound filter lists in your policy.
  9. On the Description tab, in Description, type a description for this filter (for example, specify to what computers and traffic types it applies).
  10. If you require additional IP filtering by a specific protocol or port number, on the Protocol tab, configure advanced filter settings.

Caution

  • Filters are the most important part of IPSec policy for a computer which is protected by IPSec. If you do not specify the proper filters in either client or server policies, or if the IP addresses change before the policy's filters are updated, security might not be provided.
    • When adding a new static IP address to a protected computer:
      • Modify the IPSec policy filters on all clients and servers that make security requests to the protected computer. Ensure that those clients have updated their policy before adding the new address.
      • Inspect the policy being used on the protected computer. If the filters specify static IP addresses for local connections, after adding the new IP address to the interface, edit and save the new filter list to include the new static IP address. My IP Address filters will be automatically updated when the new static IP address is added.
    • If the protected computer is a Web server and your clients use a proxy server, make sure that communications over all network paths are secured by IPSec:
      • Between the Web server and all clients that directly connect
      • Between the Web server and the proxy server
      • Between the proxy server and all clients of the proxy server
    • The following filter is internally defined to permit (not secure) Internet Key Exchange (IKE) traffic:
      • Source Address = Any
      • Destination Address = Any
      • Protocol = UDP
      • Source port = 500
      • Destination port = 500

Important

  • In the Windows Server 2003 family, IPSec provides new source and destination address options for defining filters. In addition, the default behavior for traffic exemptions has changed. If you require previously exempted traffic types, you must create filters to permit this traffic. For instructions on how to configure permit filters for previously exempted traffic types and information about other IPSec special considerations, see Related Topics.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • To configure IP protocol, TCP port, or UDP port settings for a filter, see Related Topics.
  • For IPSec tunnel rules, only address-based filters are supported. Protocol-specific and port-specific filters are not supported. Tunnel filters should not be mirrored.
  • Filters are applied in the order of most-specific filters first. Filters are not applied in the order in which they appear in the list.
  • If an outbound packet does not match any filter, it is sent unsecured.
  • If an inbound packet does not match any filter, it is permitted.
  • If an IKE security request is received, the source IP address of the request is used to find a matching filter. The security action and tunnel setting that is associated with that filter determines the IKE response.
  • The A specific DNS Name option is used when creating filters to create IP address-based filters by resolving a DNS name to its IP addresses. The use of a computer name is for the one-time resolution of DNS names to IP addresses when creating the filter. The computer name is not used after the filters are created.
  • All filters used in tunnel rules are matched first, before end-to-end transport filters are matched.

Related Topics


Configure advanced IPSec filter settings

Microsoft Windows graphic

To configure advanced IPSec filter settings

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that contains the IP filter list you want to modify.
  4. On the IP Filter List tab, double-click the IP filter list that contains the IPSec filter you want to modify.
  5. In the IP Filter List dialog box, click the filter that you want to modify, and then click Edit.
  6. In IP Filter Properties, click the Protocol tab, and then specify settings for Select a protocol type:
    7. To filter | Do this
Any packets sent on any protocol | Click Any.
Traffic sent on a specific protocol | Click the protocol.
Packets sent on a custom protocol | Click Other, and then type the protocol number.
  7. If you chose TCP or UDP, you can also filter packets by the source port.
    • To filter packets sent on any port used by the protocol type that you selected, click From any port.
    • To filter packets sent on a specific port used by the protocol type that you selected, click From this port, and then type the port number.

    8. If you chose TCP or UDP, you can also filter packets by the destination port.

    • To filter packets received on any port used by the protocol type that you selected, click To any port.
    • To only filter packets received on the port number that you specify, click To this port, and then type the port number.

Caution

  • An erroneous port number might cause communication to be blocked or unsecured. If you are unsure of the correct value, see the Windows Resource Kits in Related Topics.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • For IPSec tunneling, only address-based filters are supported. Protocol-specific and port-specific filters are not supported.

Related Topics


Set an active IP filter list for a rule

Microsoft Windows graphic

To set an active IP filter list for a rule

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that contains the IP filter list you want to modify.
  4. On the IP Filter List tab, select the option that precedes the name of the IP filter list that you want to designate as the active filter list for this rule.

    5. The IP filter list you selected is now the active filter list for the rule.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • If the option preceding the name of an IP filter list is not selected, the IP filter list will not be set as active for the rule.

Related Topics


Manage multiple IP filter lists

Microsoft Windows graphic

To manage multiple IP filter lists

  1. In IP Security Policies, in the console tree, click IP Security Policies on Name.
  2. On the Action menu, click Manage IP filter lists and filter actions.
  3. On the Manage IP Filters Lists tab, click Add. Or, if you are modifying an existing list, double-click that list.
  4. Define the filters for the filter list.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.

Related Topics


Define filter actions

  • Add, edit, or remove filter actions
  • Add, edit, or remove IPSec security methods
  • Configure custom IPSec security methods
  • Select a filter action for a rule
  • Manage multiple filter actions


Add, edit, or remove filter actions

Microsoft Windows graphic

To add, edit, or remove filter actions

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify, and then click the Filter Action tab.
  4. To add a filter action, decide whether you want to use the IP Security Filter Action Wizard or add the filter action manually:
    • To add a filter action by using the IP Security Filter Action Wizard, confirm that the Use Add Wizard check box is selected, click Add, and then follow the instructions.
    • To add a filter action manually, confirm that the Use Add Wizard check box is cleared, click Add, and then define settings on the Security Methods and General tabs.
  5. To modify an existing filter action, select the filter action that you want to modify, and then click Edit.
  6. To remove a filter action, select the filter action that you want to remove, and then click Remove.
  7. If you are adding or modifying a filter action, choose a filter action type:
    • Click Permit to allow receiving or sending of packets in plaintext (unsecured traffic). Security will not be requested for these packets.
    • Click Block to discard packets. Security will not be requested for these packets.
    • Click Negotiate security to use the list of security methods in Security method preference order to provide security for packets that match this filter. Security requests will be accepted for these packets.
  8. If you chose Negotiate security, add new security methods or edit the existing ones for the filter action.
  9. If you do not want to block incoming, unsecured communications, but you want to ensure that all outgoing communications and subsequent two-way communications are secured, select the Accept unsecured communication, but always respond using IPSec check box.
  10. To enable communication with other computers that do not support IPSec, and ensure that communication continues if there is no response to a request for IPSec negotiation, select the Allow unsecured communication with non-IPSec-aware computers check box. After the initial IPSec negotiation has failed, IPSec negotiation will be retried at five minute intervals.
  11. To guarantee that no master keys or master keying material will be reused to generate the session key, select the Use session key perfect forward secrecy (PFS) check box.
  12. On the General tab, in Name, type a unique name.
  13. In Description, type a description. For example, you might type what security levels this filter action represents.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • The filter action types Permit and Block and the options to Allow unsecured communication with non-IPSec-aware computers and Accept unsecured communication, but always respond using IPSec are not available in the default response rule.
  • Session key perfect forward secrecy (PFS) renegotiates new master key keying material during every session rekey operation. This is the most secure setting. However, it adds overhead and increases the time required to complete a rekey operation. Both initiator and responder must be configured with the same setting for successful negotiation.

Related Topics


Add, edit, or remove IPSec security methods

Microsoft Windows graphic

To add, edit, or remove IPSec security methods

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify, and then click the Filter Action tab.
  4. Double-click the filter action that you want to modify.
  5. On the Security Methods tab, click Negotiate security, and then do one of the following:
    • To add a new security method, click Add.
    • To modify an existing security method, click the security method that you want to modify, and then click Edit.
    • To remove a security method, click the security method that you want to remove, and then click Remove.
  6. If you are adding or modifying a security method, on the Security Method tab, select the security level:
    • To use the ESP protocol to provide data confidentiality (encryption) with the triple Data Encryption Standard (3DES) algorithm, data integrity and authentication with the Secure Hash Algorithm 1 (SHA1) integrity algorithm, and default key lifetimes (100MB, 1 hour), click Integrity and Encryption.
    • To use the ESP protocol to provide data integrity and authentication with the SHA1 integrity algorithm and default key lifetimes (100MB, 1 hour), click Integrity only. ESP is not configured to provide data confidentiality (encryption).
    • Click Custom, and then click Settings to configure a custom security method or key lifetimes. For more information, see Related Topics.
  7. Repeat this procedure to configure additional security methods for the filter action.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 server operating system support 3DES and do not require installation of the High Encryption Pack.
  • Click Move up to move the selected security method up one level. Repeat until the security method is at the required preference level.
  • Click Move down to move the selected security method down one level. Repeat until the security method is at the required preference level.
  • For information about how key lifetimes affect security, see Filter action in Related Topics.
  • When negotiating security, the initiator proposes this list of security methods to the responder. The responder agrees to the first method (from the top down) that is found anywhere in its own security methods list. If none of the proposed security methods match an entry in the responder's list, then the responder fails the negotiation.

Related Topics


Configure custom IPSec security methods

Microsoft Windows graphic

To configure custom IPSec security methods

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify, and then click the Filter Action tab.
  4. Double-click the filter action that you want to modify.
  5. On the Security Methods tab, do one of the following:
    • To add a new security method, click Add.
    • To modify an existing security method, click the security method that you want to modify, and then click Edit.
    • To remove a security method, click the security method that you want to remove, and then click Remove.
  6. If you are adding or modifying a security method, on the Security Method tab, click Custom, and then click Settings.
  7. To provide integrity for both the packet's addressing information (IP header) and the data, select the Data and address integrity without encryption (AH) check box, and then, in Integrity algorithm, click one of the following:
    • MD5 to use the Message Digest 5 (MD5) integrity algorithm, which uses a 128-bit key.
    • SHA1 to use the Secure Hash Algorithm 1 (SHA1) integrity algorithm, which uses a 160-bit key.
  8. To provide both integrity and encryption (confidentiality) for the data, select the Data integrity and encryption (ESP) check box, and then, in Integrity algorithm, click one of the following:
    • <None> to use no data integrity.

      • If you have enabled AH, you can click None for the ESP integrity algorithm to improve performance.

    • MD5 to use the MD5 integrity algorithm, which uses a 128-bit key.
    • SHA1 to use the SHA1 integrity algorithm, which uses a 160-bit key.
  9. If you selected the Data integrity and encryption (ESP) check box, in Encryption algorithm, click one of the following:
    • <None> to use no encryption.
    • DES to use Data Encryption Standard (DES) with a 56-bit key.
    • 3DES to use the triple DES (3DES) with three 56-bit keys.
  10. To change the default session key lifetime settings, do one of the following:
    • To specify the number of kilobytes of data transferred before a new session key is generated, select the Generate a new key every check box, and then type a value in Kbytes.
    • To specify the number of seconds that elapse before a new session key is generated, select the Generate a new key every check box, and then type a value in seconds.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • You cannot choose <None> for both the ESP integrity and the encryption algorithm.
  • Use caution when specifying lifetimes. The default internal lifetimes should work in most environments and be interoperable with other products. If your security requirements demand shorter lifetime values, you can explicitly set appropriate lifetime values in each security method. The lowest values that are explicitly set on initiator and responder will be used.
  • Using shorter lifetime values does not increase the security with which data is protected. Short key lifetimes will only decrease the amount of data that is revealed if an attacker discovers one encryption key. Instead of adjusting the lifetime values lower, consider using a stronger encryption algorithm to protect data, such as 3DES.
  • Before changing the default lifetimes, you should understand how lifetimes and rekeying work in detail. For more information, see Filter action in Related Topics.
  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.

Related Topics


Select a filter action for a rule

Microsoft Windows graphic

To select a filter action for a rule

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Double-click the rule that you want to modify.
  4. On the Filter Action tab, select the option that precedes the name of the filter action that you want to designate as the active filter action for this rule.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • If the option preceding the name of a filter action is not selected, the filter action will not be set as active for the rule.
  • If you have not removed them, you can activate one of the three predefined filter actions: Permit, Request Security (Optional), or Require Security.

Related Topics


Manage multiple filter actions

Microsoft Windows graphic

To manage multiple filter actions

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. Click Action, and then click Manage IP filter lists and filter actions.
  4. On the Manage Filter Actions tab, do one of the following:
    • To add a new filter action, click Add.
    • To modify an existing filter action, click the filter action that you want to modify, and then click Edit.
    • To remove a filter action, click the filter action that you want to remove, and then click Remove.
  5. If you are adding or modifying a filter action, define settings on the Security Methods and General tabs as needed.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • For more information about defining settings for filter actions, see Related Topics.

Related Topics


Define IPSec key exchange settings

  • Configure key exchange settings
  • Create key exchange security methods


Configure key exchange settings

Microsoft Windows graphic

To configure key exchange settings

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Click the General tab, and then click Settings.
  4. To force reauthentication and the negotiation of new master key keying material each time a new session key is required, click Master key perfect forward secrecy (PFS).
  5. If you require a different setting, type a value in Authenticate and generate a new key after every number minutes, which will cause a reauthentication and new master key generation at that interval.
  6. If you require a different setting, type a value in Authenticate and generate a new key after every number sessions to set a maximum limit on the number of times a master key or its base keying material can be reused to generate the session key. This limit, when reached, will force a reauthentication and new master key generation. If you have enabled Master key perfect forward secrecy (PFS), the number of sessions is set to 1 and is not configurable.
  7. If you have special requirements for security methods on the master key exchange, click Methods.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • Portions of IPSec-related services were jointly developed by Microsoft and Cisco Systems, Inc.
  • Use Master key perfect forward secrecy (PFS) where it is required for interoperability. The default setting is disabled, which should work in most environments and be interoperable with most other products.
  • A session limit of zero (0) will cause rekeys to be determined only by the time setting.
  • Enabling master key perfect forward secrecy (PFS) might impact performance because each quick mode requires a new main mode negotiation.

Related Topics


Create key exchange security methods

Microsoft Windows graphic

To create key exchange security methods

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Double-click the policy that you want to modify.
  3. Click the General tab, click Settings, and then click Methods.
  4. In the Key Exchange Security Methods dialog box, do one of the following:
    • To add a new key exchange security method, click Add.
    • To modify an existing key exchange security method, click the security method that you want to modify, and then click Edit.
    • To remove a key exchange security method, click the security method that you want to remove, and then click Remove.
  5. If you are adding or modifying a key exchange security method, in the IKE Security Algorithms dialog box, select an Integrity algorithm:
    • Click MD5 to use a 128-bit key (faster).
    • Click SHA1 to use a 160-bit key (stronger).
  6. Select an Encryption algorithm:
    • Click 3DES to use the triple Data Encryption Standard (3DES) algorithm and three unique 56-bit keys.
    • Click DES to use the DES algorithm and a single 56-bit key. Use this option if you are required to connect to computers that do not have 3DES or if you do not need the higher security and overhead of 3DES. For more information, see Notes.
  7. Select a Diffie-Hellman group to set the length of base keying material used to generate the actual keys:
    • Click Low (1) to generate 768 bits of master key keying material.
    • Click Medium (2) to generate 1,024 bits of master key keying material (stronger).
    • Click High (2048) to generate 2,048 bits of master key keying material (strongest).

Important

  • For enhanced security, do not use Diffie-Hellman Group 1 (low). For maximum security, use Group 2048 whenever possible. Use Group 2 when required for interoperability with Windows 2000 and XOX. When you use a stronger group, the secret key that is derived from the Diffie-Hellman exchange has greater strength.
    • For more information about Diffie-Hellman groups and key exchange methods, see Related Topics.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • Portions of IPSec-related services were jointly developed by Microsoft and Cisco Systems, Inc.
  • A key exchange security method is a combination of three settings (integrity algorithm, encryption algorithm, and Diffie-Hellman group). The initiator and the responder must have a method in common (one that uses the same settings) for negotiations to succeed.
  • Diffie-Hellman Group 2048 is provided only with the Windows Server 2003 family.
  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.
  • Click Move up to move a selected security method up one level. Repeat until the security method is at the required preference level.
  • Click Move down to move a selected security method down one level. Repeat until the security method is at the required preference level.

Related Topics


Assign IPSec policy

  • Assign or unassign IPSec policy in Group Policy
  • Assign or unassign IPSec policy on a computer


Assign or unassign IPSec policy in Group Policy

Microsoft Windows graphic

To assign or unassign IPSec policy in Group Policy

  • For Active Directory-based Group Policy:
    1. Open Active Directory Users and Computers.
    2. In the console tree, right-click the domain or organizational unit for which you want to set Group Policy.

      Where?

      • Active Directory Users and Computers [DomainControllerName.DomainName]
      • Domain
      • OrganizationalUnit
      • ChildOrganizationalUnit...
    3. Click Properties, and then click the Group Policy tab.
    4. Click Edit to open the Group Policy object that you want to edit. Or, click New to create a new Group Policy object, and then click Edit.
    5. In the Group Policy console tree, click IP Security Policies on Active Directory.

      Where?

      • PolicyName [ComputerName] Policy
      • Computer Configuration
      • Windows Settings
      • Security Settings
      • IP Security Policies on Active Directory
    6. In the details pane, click the IPSec policy that you want to assign or unassign, and then do one of the following:
      • To assign the policy, click the Action menu, and then click Assign.
      • To unassign the policy, click the Action menu, and then click Unassign.
  • For local computer policy:
    1. Click Start, click Run, type MMC, and then click OK.
    2. Click File, click Add/Remove Snap-in, and then click Add.
    3. Click Group Policy Object Editor, and then click Add.
    4. Click Finish, click Close, and then click OK.
    5. In the Group Policy console tree, click IP Security Policies on Local Computer.

      Where?

      • Local Computer Policy
      • Computer Configuration
      • Windows Settings
      • Security Settings
      • IP Security Policies on Local Computer
    6. In the details pane, click the IPSec policy that you want to assign or unassign, and then do one of the following:
      • To assign the policy, click the Action menu, and then click Assign.
      • To unassign the policy, click the Action menu, and then click Un-assign.

Important

  • An IPSec policy might remain active even after the IPSec policy or Group Policy object to which it is assigned has been deleted. Therefore, you should unassign the IPSec policy before you delete either the policy or the Group Policy object. To prevent problems, use the following procedure:
    1. Unassign the IPSec policy in the Group Policy object.
    2. Wait 24 hours to ensure that the change is propagated.
    3. Delete the IPSec policy or Group Policy object.

    If you delete the IPSec policy or Group Policy object without following this procedure, computers in the Active Directory container to which the IPSec policy is assigned might treat the IPSec policy as if it cannot be located and continue to use a cached copy.

Notes

  • XOX
  • XOX
  • You cannot administer Active Directory-based IPSec policy from a computer running XOX.
  • The settings will take effect the next time Group Policy is refreshed.
  • If a policy is currently assigned and you assign a new policy, the currently assigned policy is automatically unassigned.
  • You cannot assign a policy from the IP Security Policies on Active Directory console; you can only configure policy. To assign policy for Active Directory containers, you must use IP Security Policies on Active Directory within the Group Policy console.

Related Topics


Assign or unassign IPSec policy on a computer

Microsoft Windows graphic

To assign or unassign IPSec policy on a computer

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. In the details pane, click the IPSec policy that you want to assign or unassign, and then do one of the following:
    • To assign the policy, click the Action menu, and then click Assign.
    • To unassign the policy, click the Action menu, and then click Un-assign.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • If a policy is currently assigned and you assign a new policy, the currently assigned policy is automatically unassigned.
  • If the computer is a member of a Windows 2000 or Windows Server 2003 domain and IPSec policy is assigned at the domain level, then the assigned local computer policy is not used.

Related Topics


Manage IPSec policy

  • Restart the IPSec service
  • Restore default IPSec policies
  • Export IPSec policies
  • Import IPSec policies
  • Refresh the list of IPSec policies
  • Delete an IPSec policy
  • Rename an IPSec policy
  • Export the list of IPSec policies


Restart the IPSec service

Microsoft Windows graphic

To restart the IPSec service

Using the Windows interface

  1. Open Services.
  2. Right-click IPSec services.
  3. Click Restart.

Important

  • Stopping and restarting the IPSec service can disconnect all computers that are using IPSec from the computer on which the IPSec service is stopped, and it can prevent further communication with that computer. For information about how to troubleshoot and resolve loss of connectivity in this situation, see Related Topics.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • You might need to restart the IPSec service in order to do any of the following:
    • Clear old security associations (SAs).
    • Clear soft SAs (unsecured negotiations).
    • Force an IPSec policy to download from Active Directory to domain clients.
  • If the IPSec service does not start, then use Event Viewer to determine possible causes of failure. For more information, see Related Topics.

Using a command line

  1. Open Command Prompt.
  2. Type:

    net stop policyagent & net start policyagent

Important

  • Stopping and restarting the IPSec service can disconnect all computers that are using IPSec from the computer on which the IPSec service is stopped, and it can prevent further communication with that computer. For information about how to troubleshoot and resolve loss of connectivity in this situation, see Related Topics.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • On a computer running the Routing and Remote Access service and either Windows 2000 Server or a Windows Server 2003 operating system, you must use the following sequence when restarting the IPSec service:
    1. Stop the Routing and Remote Access service using the net stop remoteaccess command.
    2. Stop and restart the IPSec service using the net stop policyagent & net start policyagent command.
    3. Start the Routing and Remote Access service using the net start remoteaccess command.
  • You might need to restart the IPSec service in order to do any of the following:
    • Clear old security associations (SAs).
    • Clear soft SAs (unsecured negotiations).
    • Force an IPSec policy to download from Active Directory to domain clients.
  • If the IPSec service does not start, then use Event Viewer to determine possible causes of failure. For more information, see Related Topics.
  • For more information about the net start and net stop commands, see Related Topics.

Related Topics


Restore default IPSec policies

Microsoft Windows graphic

To restore default IPSec policies

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. Click Action, point to All Tasks, and then click Restore Default Policies.
  4. When prompted, click Yes.

Important

  • Restoring default IPSec policies will overwrite any changes to the original default policies, even when the names of the default policies have been changed. If you have modified the default policies and you do not want those modifications to be overwritten, do not restore the default policies.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • The option to restore default IPSec policies is only available for computer-based IPSec policy. You cannot restore the default IPSec policies for Active Directory-based IPSec policy.

Related Topics


Export IPSec policies

Microsoft Windows graphic

To export IPSec policies

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. Click Action, point to All Tasks, and then click Export Policies.
  4. Type the path and the name of the file to which you want the policy information saved, and then click Save.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • To enhance interoperablity in a mixed environment with computers running Windows 2000, limit the name of the file to which you want the policy information saved to 60 characters.

Related Topics


Import IPSec policies

Microsoft Windows graphic

To import IPSec policies

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. Click Action, point to All Tasks, and then click Import Policies.
  4. Type the path and the name of the file from which you are importing the IPSec policy information, and then click Open.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • Importing IPSec policies replaces existing IPSec policies with the same name.

Related Topics


Refresh the list of IPSec policies

Microsoft Windows graphic

To refresh the list of IPSec policies

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. Click Action, and then click Refresh.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • This procedure is only required to update the list of policies for any additions or deletions made by using another management tool.

Related Topics


Delete an IPSec policy

Microsoft Windows graphic

To delete an IPSec policy

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Click the policy that you want to remove, click Action, and then click Delete.
  3. When prompted, click Yes.

Important

  • For Active Directory-based IPSec policy, an IPSec policy might remain active even after the IPSec policy has been deleted. Therefore, you should unassign the IPSec policy before you delete the policy. To prevent problems, use the following procedure:
    1. Unassign the IPSec policy in the Group Policy object.
    2. Wait 24 hours to ensure that the change is propagated.
    3. Delete the IPSec policy.

    If you delete the IPSec policy without following this procedure, computers in the Active Directory container to which the IPSec policy is assigned might treat the IPSec policy as if it cannot be located and continue to use a cached copy.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.

Related Topics


Rename an IPSec policy

Microsoft Windows graphic

To rename an IPSec policy

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. Click the policy that you want to rename, click Action, and then click Rename.
  3. Type a new name, and then press Enter.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.

Related Topics


Export the list of IPSec policies

Microsoft Windows graphic

To export the list of IPSec policies

  1. Create a console containing IP Security Policies. Or, open a saved console file containing IP Security Policies.
  2. In the console tree, click IP Security Policies on Name.
  3. Click Action, and then click Export List.
  4. Type the file name to which you want to save the selected IPSec policy or the displayed list of IPSec policies, and then click Save.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • To create a console containing IP Security Policies, start the IP Security Policies snap-in. To open a saved console file, open MMC. For more information, see Related Topics.
  • This procedure does not export the IPSec policies, only a text list of the policies displayed in the details pane.
  • To export IPSec policies, see Related Topics.

Related Topics


Use Resultant Set of Policy (RSoP) to view IPSec policy assignments

  • Use RSoP to view IPSec policy assignments for a computer
  • Use RSoP to view IPSec policy assignments for members of a Group Policy container


Use RSoP to view IPSec policy assignments for a computer

Microsoft Windows Graphic

To use RSoP to view IPSec policy assignments for a computer

  1. Open Resultant Set of Policy.

    2. An RSoP query runs in logging mode. For more information, see Related Topics.

  2. In the console tree, click IP Security Policies.

    Where?

    • Computer Configuration
    • Windows Settings
    • Security Settings
    • IP Security Policies

    3. The list of all IPSec policies is displayed in the details pane.

  3. To view detailed policy settings for the IPSec policy that is being applied, in the details pane, right-click the policy with the highest precedence, and then click Properties.

Notes

  • To open RSoP, click Start, click Run, type rsop.msc, and then click OK.
  • To run an RSoP query in logging mode, see Related Topics.
  • The RSoP console displays detailed IPSec policy settings for only the policy that is being applied. The detailed policy settings include the filter rules, filter actions, authentication methods, tunnel endpoints, and connection type.
  • The details pane in the RSoP console provides two ways to indicate which IPSec policy is being applied:
    • The lower the number in the Precedence column, the higher the precedence of the IPSec policy. The IPSec policy with a precedence number of 1 is the policy that is being applied.
    • In the Name column, the folder icon that precedes the name of the IPSec policy that is being applied contains a green dot.
  • The settings of the IPSec policy with the highest precedence apply in their entirety; they are not merged with the settings of IPSec policies that are assigned at higher levels of the Active Directory hierarchy. For more information about using RSoP to view IPSec policy assignments, see Related Topics.
  • If a local user overrides domain-level IPSec policies with a local IPSec policy, RSoP does not detect the application of the local IPSec policy. As a result, no IPSec policies will be displayed in the RSoP console when a logging mode query is run.
  • To hide or display columns in the RSoP console, see Related Topics.

Related Topics


Use RSoP to view IPSec policy assignments for members of a Group Policy container

Microsoft Windows Graphic

To use RSoP to view IPSec policy assignments for members of a Group Policy container

  1. On a domain controller, open Resultant Set of Policy (RSoP) as a Microsoft Management Console (MMC). For more information, see Related Topics.
  2. On the Action menu, click Generate RSoP Data.
  3. On the Welcome to the Resultant Set of Policy Wizard page, click Next.
  4. On the Mode Selection page, click Planning mode, and then follow the instructions in the wizard to run a planning mode query. For more information, see Related Topics.
  5. In the console tree, click IP Security Policies.

    Where?

    • Computer Configuration
    • Windows Settings
    • Security Settings
    • IP Security Policies

    6. The list of all IPSec policies is displayed in the details pane.

  6. To view detailed policy settings for the IPSec policy that is being applied, in the details pane, right-click the policy with the highest precedence, and then click Properties.

Notes

  • The RSoP console displays detailed IPSec policy settings for only the policy that is being applied. The detailed policy settings include the filter rules, filter actions, authentication methods, tunnel endpoints, and connection type.
  • The details pane in the RSoP console provides two ways to indicate which IPSec policy is being applied:
    • The lower the number in the Precedence column, the higher the precedence of the IPSec policy. The IPSec policy with a precedence number of 1 is the policy that is being applied.
    • In the Name column, the folder icon that precedes the name of the IPSec policy that is being applied contains a green dot.
  • The settings of the IPSec policy with the highest precedence apply in their entirety; they are not merged with the settings of IPSec policies that are assigned at higher levels of the Active Directory hierarchy. For more information about using RSoP to view IPSec policy assignments, see Related Topics.
  • You can run an RSoP planning mode query only on a domain controller (when you run a planning mode query, you must explicitly specify the domain controller name). However, you can specify any IPSec client as the target for the query, provided you have the appropriate permissions to do so.
  • To hide or display columns in the RSoP console, see Related Topics.

Related Topics


Monitor IPSec activity

  • Add the IP Security Monitor snap-in
  • Add a computer to the IP Security Monitor console
  • View details about active IPSec policies
  • View IP security statistics
  • View main mode details
  • View quick mode details
  • Find a matching specific filter
  • Change the automatic refresh interval
  • Enable DNS name resolution


Add the IP Security Monitor snap-in

Microsoft Windows graphic

To add the IP Security Monitor snap-in

  1. Click Start, click Run, type MMC, and then click OK.
  2. Click File, click Add/Remove Snap-in, and then click Add.
  3. Click IP Security Monitor, and then click Add.
  4. Click Close, and then click OK.

Notes

  • To save the console settings, on the File menu, click Save.
  • To view the saved console, see Related Topics.
  • To revise your initial choice, you can start the Microsoft Management Console, add this snap-in again, and save the console again. If you require multiple configurations, you can save the console with another name.

Related Topics


Add a computer to the IP Security Monitor console

Microsoft Windows graphic

To add a computer to the IP Security Monitor console

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. In the console tree, right-click IP Security Monitor, and then click Add Computer.
  3. In the Add Computer dialog box:
    • For the local computer, click This computer.
    • For a remote computer, click The following computer, and then type the name of the remote computer. Or, click Browse to find it on the network.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • If the IPSec service is not started on the computer that is being monitored, the server icon is displayed as a stopped service. To refresh IP Security Monitor after the IPSec service on that computer have been restarted, right-click the computer and then click Reconnect.
  • IP Security Monitor can be used to monitor IPSec only on computers running XOX or a Windows Server 2003 operating system. To monitor IPSec on a computer running Windows 2000, use the IPSecmon command at a Windows 2000 command prompt.

Related Topics


View details about active IPSec policies

Microsoft Windows Graphic

To view details about active IPSec policies

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. Double-click Active Policy.

    Where?

    • IP Security Monitor
    • ServerName
    • Active Policy

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • For information about the items displayed in Active Policy, see Related Topics.

Related Topics


View IP security statistics

Microsoft Windows graphic

To view IP security statistics

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. Click a server name.

    Where?

    • IP Security Monitor
    • ServerName
  3. Do one of the following:
    • To view main mode statistics, in the console tree, double-click Main Mode, and then double-click Statistics.
    • To view quick mode statistics, in the console tree, double-click Quick Mode, and then double-click Statistics.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • For more information about the items displayed in Statistics, see Related Topics.

Related Topics


View main mode details

Microsoft Windows graphic

To view main mode details

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. Double-click Main Mode.

    Where?

    • IP Security Monitor
    • ServerName
    • Main Mode
  3. To view main mode details, do one of the following:
    • For generic filter details, in the console tree, click Generic Filters, and then in the details pane, double-click a filter.
    • For specific filter details, in the console tree, click Specific Filters. To find a specific main mode filter, right-click Specific Filters, and then click Find matching filters.
    • For IKE policy details, in the console tree, click IKE Policies, and then in the details pane, double-click a policy.
    • For statistical details, in the console tree, click Statistics.
    • For security association details, in the console tree, click Security Associations, and then in the details pane, double-click a security association.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX

Related Topics


View quick mode details

Microsoft Windows graphic

To view quick mode details

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. Double-click Quick Mode.

    Where?

    • IP Security Monitor
    • ServerName
    • Quick Mode
  3. To view quick mode details, do one of the following:
    • For generic filter details, in the console tree, click Generic Filters, and then in the details pane, double-click a filter.
    • For specific filter details, in the console tree, click Specific Filters. To find a specific quick mode filter, right-click Specific Filters, and then click Find matching filters.
    • For negotiation policy details, in the console tree, click Negotiation Policies, and then in the details pane, double-click a policy.
    • For statistical details, in the console tree, click Statistics.
    • For security association details, in the console tree, click Security Associations, and then in the details pane, double-click a security association.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • Quick mode details do not display Block or Permit filters.
  • Although there are two separate quick mode SAs established (one for the inbound traffic and one for the outbound traffic), IP Security Monitor only displays a single quick mode SA.

Related Topics


Find a matching specific filter

Microsoft Windows graphic

To find a matching specific filter

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. For a specific main mode filter:
    1. Double-click Main Mode.

      Where?

      • IP Security Monitor
      • ServerName
      • Main Mode
    2. In the console tree, right-click Specific Filters, and then click Find matching filters.
    3. In the Find Matching Main Mode Filters dialog box, configure the filter settings, and then click Search.
  3. For a specific quick mode filter:
    1. Double-click Quick Mode.

      Where?

      • IP Security Monitor
      • ServerName
      • Quick Mode
    2. In the console tree, right-click Specific Filters, and then click Find matching filters.
    3. In the Find Matching Quick Mode Filters dialog box, configure the filter settings, and then click Search.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX

Related Topics


Change the automatic update refresh interval

Microsoft Windows graphic

To change the automatic update refresh interval

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. Right-click a server name.

    Where?

    • IP Security Monitor
    • ServerName
  3. Click Properties.
  4. Configure the automatic update refresh interval, and then click OK.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX

Related Topics


Enable DNS name resolution

Microsoft Windows graphic

To enable DNS name resolution

  1. Create a console containing IP Security Monitor. Or, open a saved console file containing IP Security Monitor.
  2. Right-click a server name.

    Where?

    • IP Security Monitor
    • ServerName
  3. Click Properties.
  4. Select the Enable DNS name resolution check box.

Notes

  • To perform this procedure, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • XOX
  • The Enable DNS name resolution check box specifies whether you want to use Domain Name System (DNS) reverse queries and NetBIOS Adapter Status messages to resolve the DNS or NetBIOS computer name that is associated with an IP address when displaying sources or destinations. To resolve the DNS computer name from its IP address, your DNS infrastructure must have the appropriate reverse domains and pointer records configured, either manually or with DNS dynamic update. To resolve the NetBIOS computer name from its IP address, NetBIOS over TCP/IP must be enabled on the computer.

Related Topics


Concepts

This section provides general background information about Internet Protocol security (IPSec):

  • New features for IPSec
  • Internet Protocol security overview
  • Understanding Internet Protocol security
  • Using Internet Protocol security
  • Resources


New features for IPSec

Internet Protocol security (IPSec) provides the following new features for enhanced security, scalability, and availability, and ease of deployment and administration.

Note

  • Because several new IPSec features are available only in the Windows Server 2003 family, as a best practice, if you plan to apply the same IPSec policy to computers running the Windows Server 2003 family and to computers running Windows 2000 or XOX, test the policy thoroughly on all relevant operating systems before deployment. For additional best practices for using IPSec, see Best practices.

IP Security Monitor

In Windows 2000, IP Security Monitor was implemented as an executable program (IPSecmon.exe). In XOX and the Windows Server 2003 family, IP Security Monitor is implemented as a Microsoft Management Console (MMC) console and includes enhancements that allow you to:

  • Monitor IPSec information for your local computer and for remote computers.
  • View details about active IPSec policies, including the name, description, date last modified, store, path, organizational unit, and Group Policy object name.
  • View main mode and quick mode generic filters and specific filters.
  • View main mode and quick mode statistics. For information about the statistics displayed in IP Security Monitor, see Viewing main mode and quick mode statistics in IP Security Monitor.
  • View main mode and quick mode security associations.
  • Customize refresh rates, and use DNS name resolution for filter and security association output.
  • Search for specific main mode or quick mode filters that match any source or destination IP address, a source or destination IP address on your local computer, or a specific source or destination IP address.

For information about adding the IP Security Monitor snap-in, see To add the IP Security Monitor snap-in. For information about additional IPSec troubleshooting tools, see Troubleshooting tools.

Stronger cryptographic master key (Diffie-Hellman)

For enhanced security, IPSec now supports the use of a 2048-bit Diffie-Hellman key exchange. With a stronger Diffie-Hellman group, the secret key that is derived from the Diffie-Hellman exchange has greater strength. Strong Diffie-Hellman groups combined with longer key lengths increase the computational difficulty of determining a secret key.

This feature is provided only with the Windows Server 2003 family.

For more information about Diffie-Hellman groups and key exchange, see Key exchange methods and Key management and protection.

Command-line management with Netsh

Using commands in the Netsh IPSec context, you can configure static or dynamic IPSec main mode settings, quick mode settings, rules, and configuration parameters. To enter the Netsh IPSec context, type netsh -c ipsec at a command prompt. The Netsh IPSec context replaces the Ipsecpol.exe tool, which is provided with the Windows 2000 Server Resource Kit. You can use this feature to script and automate IPSec configuration. For more information, see Troubleshooting tools.

This feature is provided only with the Windows Server 2003 family.

For more information about Netsh commands for IPSec, see Netsh commands for Internet Protocol security.

Computer startup security

For enhanced security, IPSec now provides stateful filtering of network traffic during computer startup. With stateful filtering, only the following traffic is permitted during computer startup: the outbound traffic that the computer initiates during startup, the inbound traffic that is sent in response to the outbound traffic, and DHCP traffic. As an alternative to stateful filtering, you can specify that all inbound and outbound traffic be blocked until an IPSec policy is applied. If you use stateful filtering, or if you specify that traffic be blocked during computer startup, you can also specify the traffic types that you want to exempt from IPSec filtering during computer startup.

Note

  • You cannot configure this feature in the IP Security Policy Management console. To configure this feature, you must use the Netsh IPSec command-line tool.

Persistent policy for enhanced security

You can now create and assign a persistent IPSec policy to secure a computer if a local IPSec policy or an Active Directory-based IPSec policy cannot be applied. When you create and assign a persistent policy, it is applied before the local policy or the Active Directory-based policy is applied, and it remains in effect regardless of whether the local policy or the Active Directory-based policy is applied (for example, an IPSec policy will not be applied if it is corrupted).

Note

  • You cannot configure this feature in the IP Security Policy Management console. To configure this feature, you must use the Netsh commands for IPSec.

Removed default traffic exemptions

In Windows 2000 and XOX, by default, all broadcast, multicast, Internet Key Exchange (IKE), Kerberos, and Resource Reservation Protocol (RSVP) traffic is exempt from IPSec filtering. To significantly improve security, in the Windows Server 2003 family, only IKE traffic (which is required for establishing IPSec-secured communication) is exempt from IPSec filtering. All other traffic types are now matched against IPSec filters, and you can configure, block, or permit filter actions specifically for multicast and broadcast traffic (IPSec does not negotiate security associations for multicast and broadcast traffic). For more information, see Special IPSec considerations.

IPSec certificate to account mapping for network access control

With the Windows Server 2003 family, if you use either Kerberos V5 or certificate authentication, you can set restrictions on which computers are allowed to connect. This functionality allows you to use IPSec to allow or deny any of the following access to a server running Windows Server 2003:

  • Computers that are members of a specific domain.
  • Computers that have a certificate from a specific issuing certification authority.
  • A specific group of computers.
  • A specific computer.

When you enable certificate to account mapping in IPSec, the IKE protocol associates (maps) a computer certificate to a computer account in an Active Directory domain or forest, and then retrieves an access token, which includes the list of the user rights that are assigned to the computer. You can restrict access by configuring Group Policy security settings and assigning either the Access this computer from the network user right or the Deny access to this computer from the network user right to individual or multiple computers, as needed.

For more information about certificate to account mapping for IPSec, see Authentication methods.

For information about configuring security settings in Group Policy, see Security Settings Overview.

Notes

  • Certificate to account mapping for IPSec is provided only with the Windows Server 2003 family.
  • Certificate mapping is not supported across forests.

Ability to exclude the name of the certification authority (CA) from certificate requests

For enhanced security, when you use certificate authentication to establish trust between IPSec peers, you can now exclude the name of the CA from the certificate request. When you exclude the name of the CA from the certificate request, you prevent the potential disclosure of sensitive information about the trust relationships of a computer, such as name of the company that owns the computer and the domain membership of the computer (if an internal public key infrastructure is being used), to an attacker.

IPSec policy filters allow logical addresses for local IP configuration

You can now use the IP Security Policy Management console to configure the source or the destination address fields that the local IPSec policy will interpret as the addresses for the DHCP server, the DNS servers, the WINS servers, and the default gateway. As a result, IPSec policies can now automatically accommodate changes in the IP configuration of the server, by using either DHCP or static IP configurations.

This feature is provided only with the Windows Server 2003 family.

IPSec functionality over network address translation (NAT)

IPSec Encapsulating Security Payload (ESP) packets can now pass through NATs that allow User Datagram Protocol (UDP) traffic. The IKE protocol automatically detects the presence of a NAT and uses UDP-ESP encapsulation to allow IPSec traffic to pass through the NAT. This functionality is an implementation of the Internet Engineering Task Force (IETF) IP Security Working Group standard for IPSec.

NATs are widely used for Internet Connection Sharing (ICS) and in locations that provide public Internet access (such as hotels and airports) and that are likely to be used by telecommuters. In addition, some Internet service providers (ISPs) use a centralized NAT to connect their clients to the Internet.

IPSec functionality over NAT enables IPSec-secured connections to be established in the following common deployment scenarios:

  • Layer Two Tunneling Protocol (L2TP)/IPSec virtual private network (VPN) clients that are behind NATs can establish IPSec-secured connections over the Internet to their corporate network, using IPSec ESP transport mode.
  • Servers running Routing and Remote Access can establish gateway-to-gateway IPSec tunnels when one of the servers running Routing and Remote Access is behind a NAT.
  • Clients and servers can send IPSec-secured TCP and UDP packets to other clients or servers using IPSec ESP transport mode, when one or both of the computers are behind a NAT. For example, a program running on a server on a perimeter network can be IPSec-secured when it is used to make connections to the corporate network.

For more information about VPNs and IPSec, see Virtual private networking with IPSec. For more information about IPSec transport mode and tunnel mode, see Transport mode and Tunnel mode.

Improved IPSec integration with Network Load Balancing

Improved IPSec integration with Network Load Balancing allows a Network Load Balancing group of servers to provide highly available IPSec-based VPN services. Network Load Balancing can accurately track IPSec-secured sessions, and the IPSec IKE protocol can detect when an IPSec-secured session is being established with a cluster server and quickly recover from a failover. Additionally, Network Load Balancing can now maintain IPSec-secured connections to the correct Network Load Balancing host, even when the number of hosts in the cluster (and the algorithm used to map clients to hosts) changes. Because the IKE protocol automatically detects the Network Load Balancing service, no additional configuration is required to use this feature.

This feature is provided only with XOX and XOX.

For more information about Network Load Balancing, see Introduction to Network Load Balancing and How Network Load Balancing works.

IPSec support for RSoP

To enhance IPSec deployment and troubleshooting, IPSec now provides an extension to the Resultant Set of Policy (RSoP) console. RSoP is an addition to Group Policy that you can use to view existing IPSec policy assignments for a computer or for members of a Group Policy container. To view IPSec policy assignments for a computer, run an RSoP logging mode query. To view IPSec policy assignments for members of a Group Policy container, run an RSoP planning mode query. For information about how to view IPSec policy assignments, see Use Resultant Set of Policy (RSoP) to view IPSec policy assignments.

After you run an RSoP logging mode query or an RSoP planning mode query, you can view detailed settings (the filter rules, filter actions, authentication methods, tunnel endpoints, and connection types that were specified when the IPSec policy was created) for the IPSec policy that is being applied.

For more information about using RSoP with IPSec, see Using Resultant Set of Policy to view IPSec policy assignments. For general information about RSoP, see RSoP overview.


Internet Protocol security overview

This section covers:

  • Security information for IPSec
  • Internet Protocol security defined
  • Introducing IPSec
  • Security issues with IP
  • Protection against attacks
  • Policy-based security
  • Simplified deployment


Security information for IPSec

Before deploying IPSec in your organization, consider the following security issues:

  • 3DES and computers running Microsoft® Windows® 2000
  • Authentication method
  • Firewall packet filtering
  • Protected traffic (using AH or ESP)
  • Unprotected traffic (using the default response rule)
  • Diffie-Hellman groups (using Diffie-Hellman Group 1, 2, or 2048)

3DES and computers running Windows 2000

IPSec policies allow the choice of a strong encryption algorithm, 3DES, which provides stronger encryption than DES for higher security. Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to perform the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, then the 3DES setting is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should use DES only as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX and the Windows Server 2003 family support 3DES and do not require installation of the High Encryption Pack.

Authentication method

If the computers in your organization are part of an Active Directory® domain, IPSec main mode authentication can be accomplished with the default authentication method (Kerberos V5). You do not have to deploy public key certificates for intranet communication. However, computers running XOX do not support the Kerberos V5 authentication method. Also, if you have computers connected to the Internet, it is recommended that you do not use Kerberos V5 as an authentication method. When Kerberos authentication is used, during main mode negotiation, each IPSec peer sends its computer identity in unencrypted format to the other peer. The computer identity is unencrypted until encryption of the entire identity payload takes place during the authentication phase of the main mode negotiation. An attacker can send an Internet Key Exchange (IKE) packet that causes the responding IPSec peer to expose its computer identity and domain membership. To secure computers that are connected to the Internet, certificate authentication is therefore recommended.

For enhanced security, the use of preshared key authentication is not recommended because it is a relatively weak authentication method. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing.

For more information, see Authentication methods and Preshared key authentication.

Firewall packet filtering

For a firewall, security gateway, router, or any other server or device that is connected to the Internet and providing packet filtering capabilities to computers on a perimeter network (also known as a demilitarized zone or DMZ), special filtering must be enabled on that computer to ensure that packets secured with IPSec are allowed to be forwarded to computers on the perimeter network. The firewall or other device should normally allow the following types of traffic to pass through it:

  • IP Protocol ID of 50 (0x32) for IPSec Encapsulating Security Payload (ESP) traffic
  • IP Protocol ID of 51 (0x33) for IPSec Authentication Header (AH) traffic
  • UDP port 500 (0x1F4) for Internet Key Exchange (IKE) negotiation traffic

Most packet filtering software allows you to be more specific. You can define separate packet filters for inbound traffic (inbound filters), for outbound traffic (outbound filters), and for each interface. Additionally, you can specify IP addresses for the IPSec computers on the perimeter network. The most restrictive packet filters for IPSec traffic exchanged with a single IPSec computer on the perimeter network are outlined in the following sections.

Filters on the Internet interface

Configure the following inbound packet filters on the Internet interface of the firewall to allow the specific types of traffic:

  • Destination IP address of the IPSec computer's perimeter network interface and UDP destination port of 500 (0x1F4)

    • This filter allows IKE traffic to be sent to the IPSec computer on the perimeter network.

  • Destination IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 50 (0x32)

    • This filter allows IPSec ESP traffic to be sent to the IPSec computer on the perimeter network.

  • Destination IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 51 (0x33)

    • This filter allows IPSec AH traffic to be sent to the IPSec computer on the perimeter network.

Configure the following outbound packet filters on the Internet interface of the firewall to allow the specific types of traffic:

  • Source IP address of the IPSec computer's perimeter network interface and UDP source port of 500 (0x1F4)

    • This filter allows IKE traffic to be sent from the IPSec computer on the perimeter network.

  • Source IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 50 (0x32)

    • This filter allows IPSec ESP traffic to be sent from the IPSec computer on the perimeter network.

  • Source IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 51 (0x33)

    • This filter allows IPSec AH traffic to be sent from the IPSec computer on the perimeter network.

Filters on the perimeter network interface

Configure the following inbound packet filters on the perimeter network interface of the firewall to allow the specific types of traffic:

  • Source IP address of the IPSec computer's perimeter network interface and UDP source port of 500 (0x1F4)

    • This filter allows IKE traffic to be sent from the IPSec computer on the perimeter network.

  • Source IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 50 (0x32)

    • This filter allows IPSec ESP traffic to be sent from the IPSec computer on the perimeter network.

  • Source IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 51 (0x33)

    • This filter allows IPSec AH traffic to be sent from the IPSec computer on the perimeter network.

Configure the following outbound packet filters on the perimeter network interface of the firewall to allow the specific types of traffic:

  • Destination IP address of the IPSec computer's perimeter network interface and UDP destination port of 500 (0x1F4)

    • This filter allows IKE traffic to be sent to the IPSec computer on the perimeter network.

  • Destination IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 50 (0x32)

    • This filter allows IPSec ESP traffic to be sent to the IPSec computer on the perimeter network.

  • Destination IP address of the IPSec computer's perimeter network interface and IP Protocol ID of 51 (0x33)

    • This filter allows IPSec AH traffic to be sent to the IPSec computer on the perimeter network.

Protected traffic (using AH or ESP)

When deciding whether to use AH or ESP to protect your IP traffic, consider the following:

  • AH provides both data authentication and integrity services through the computation and inclusion of a keyed hash for each packet. With AH, the hash calculation includes the entire IP packet and header. Some fields that are allowed to change in transit are excluded. Packet replay protection services are provided through the inclusion of a sequence number for each packet.
  • ESP provides data authentication and integrity services through the computation and inclusion of a keyed hash for each packet. With ESP, the hash calculation only includes the ESP header, trailer, and payload. The IP header is not protected with the hash. ESP provides data confidentiality services by encrypting the ESP payload with the DES (Data Encryption Standard) or 3DES (triple DES) encryption algorithms. Packet replay protection services are provided through the inclusion of a sequence number for each packet.

It is less CPU-intensive to verify the hash for each packet than it is to encrypt and decrypt each packet. If performance is a significant consideration, you can use AH to protect most of your traffic. When confidentiality is required, you can use ESP instead. For example, you can use AH to protect traffic on your intranet and ESP for traffic that is sent over the Internet.

Note

  • This performance consideration assumes that you are not using IPSec offload network adapters in your organization. Offload network adapters perform cryptographic calculations, such as the calculation and verification of the hash and the encryption and decryption services, on the adapter itself, enhancing performance for IPSec-secured traffic.

Unprotected traffic (using the default response rule)

You can easily implement an IPSec deployment scenario that secures traffic from a specific set of servers, as follows:

  1. Place the servers in an organizational unit and assign an IPSec policy to a Group Policy object of that organizational unit with a rule that requires secured traffic between the servers and any other computer. Within the filter action for the rule, enable the secure servers to accept unsecured communication but always respond using IPSec.
  2. Place the client computers in an organizational unit and assign an IPSec policy to a Group Policy object of that organizational unit that uses only the default response rule. For more information, see IPSec policy rules.

In this configuration, the following traffic between the client computers and the secure servers is unsecured:

  • The initial request (for example, a TCP SYN packet) sent by the client computer to establish communication is unsecured because the client does not have a rule in its policy to initiate secured communication with the secure server.
  • Although ongoing communications are secured after secured communication is negotiated between the client and secure server computers, the quick mode security association (SA) that provides security settings for traffic can eventually time out and be removed from both the secure server and the client computer. This occurs when a TCP connection exists and no traffic is sent between the client computer and the secure server for an extended period of time. The dynamic filter created on the client computer by the default response rule is also removed after the dynamic filter has been timed out.

    • If the secure server sends new data on the existing connection, it renegotiates the quick mode SA before sending this data to the client computer because a rule exists on the secure server to secure traffic between itself and all other computers. However, if the client computer sends new data after the quick mode SAs and dynamic filter time out, the new data is sent unsecured because the client does not have a rule to initiate secure communications with the secure server. For TCP connections, one or multiple TCP segments can be sent unsecured by the client computer. To prevent client computers from sending unsecured data to secure servers, you must configure your client computer IPSec policy with additional rules that initiate secured communications to secure servers.

Diffie-Hellman groups (using Diffie-Hellman group 1, 2, or 2048)

Diffie-Hellman groups are used to determine the length of the base prime numbers that are used during the key exchange process. Group 2048 (high) is stronger (more secure) than Group 2 (medium), which is stronger than Group 1 (low). Group 1 provides 768 bits of keying strength, Group 2 provides 1,024 bits, and Group 2048 provides 2,048 bits.

Strong Diffie-Hellman groups combined with longer key lengths increase the computational difficulty of determining a secret key.

Important

  • For enhanced security, do not use Diffie-Hellman Group 1. For maximum security, use Group 2048 whenever possible. Use Group 2 when required for interoperability with Windows 2000 and XOX.

Note

  • Diffie-Hellman Group 2048 is provided only with the Windows Server 2003 family.

For more information about Diffie-Hellman groups and key exchange, see Key exchange methods and Key management and protection.


Internet Protocol security defined

Internet Protocol security (IPSec) is a framework of open standards for ensuring private, secure communications over Internet Protocol (IP) networks, through the use of cryptographic security services. The Microsoft® Windows® 2000, XOX, and the Windows Server 2003 family implementations of IPSec are based on standards developed by the Internet Engineering Task Force (IETF) IPSec working group.

IPSec is the long-term direction for secure networking. It provides aggressive protection against private network and Internet attacks through end-to-end security. The only computers that must know about IPSec protection are the sender and receiver in the communication. In XOX and the Windows Server 2003 family, IPSec provides the ability to protect communication between workgroups, local area network computers, domain clients and servers, branch offices (which might be physically remote), extranets, and roving clients.


Introducing IPSec

IPSec is the long-term direction for secure networking. It provides a key line of defense against private network and Internet attacks.

IPSec has two goals:

  1. To protect the contents of IP packets.
  2. To provide a defense against network attacks through packet filtering and the enforcement of trusted communication.

Both goals are met through the use of cryptography-based protection services, security protocols, and dynamic key management. This foundation provides both the strength and flexibility to protect communications between private network computers, domains, sites, remote sites, extranets, and dial-up clients. It can even be used to block receipt or transmission of specific traffic types.

IPSec is based on an end-to-end security model, establishing trust and security from a source IP to a destination IP address. The IP address itself does not necessarily have to be considered an identity, rather the system behind the IP address has an identity that is validated through an authentication process. The only computers that must know about the traffic being secured are the sending and receiving computers. Each computer handles security at its respective end, with the assumption that the medium over which the communication takes place is not secure. Any computers that only route data from source to destination are not required to support IPSec, unless firewall-type packet filtering or network address translation is being done between the two computers. This model allows IPSec to be successfully deployed for the following enterprise scenarios:

  • Local area network (LAN): client/server and peer-to-peer
  • Wide area network (WAN): router-to-router and gateway-to-gateway
  • Remote access: dial-up clients and Internet access from private networks

Typically both sides require IPSec configuration (called an IPSec policy), to set options and security settings that will allow two systems to agree on how to secure traffic between them. The Microsoft® Windows® 2000, XOX, and the Windows Server 2003 family implementations of IPSec is based on industry standards developed by the Internet Engineering Task Force (IETF) IPSec working group. Portions of IPSec-related services were jointly developed by Microsoft and Cisco Systems, Inc.


Security issues with IP

Without security, both public and private networks are susceptible to unauthorized monitoring and access. Internal attacks might be a result of minimal or nonexistent intranet security. Risks from outside the private network originate from connections to the Internet and extranets. Password-based user access controls alone do not protect data transmitted across a network.

Common types of network attacks

Without security measures and controls in place, your data might be subjected to an attack. Some attacks are passive in that information is only monitored. Other attacks are active and information is altered with intent to corrupt or destroy the data or the network itself. Your networks and data are vulnerable to any of the following types of attacks if you do not have a security plan in place.

Eavesdropping

In general, the majority of network communications occur in a plaintext (unencrypted) format, which allows an attacker who has gained access to data paths in a network to monitor and interpret (read) the traffic. When an attacker is eavesdropping on communications, it is referred to as sniffing or snooping. The ability of an eavesdropper to monitor the network is generally the biggest security problem that administrators face in an enterprise. Without strong encryption services that are based on cryptography, data can be read by others as it traverses the network.

Data modification

After an attacker has read data, the next logical step is often to modify it. An attacker can modify the data in the packet without the knowledge of the sender or receiver. Even if you do not require confidentiality for all communications, you do not want any of your messages to be modified in transit. For example, if you are exchanging purchase requisitions, you do not want the items, amounts, or billing information to be changed.

Identity spoofing (IP address spoofing)

Most networks and operating systems use the IP address to identify a computer as being valid on a network. In some cases, it is possible for an IP address to be falsely used. This is known as identity spoofing. An attacker might use special programs to construct IP packets that appear to originate from valid addresses inside an organization intranet.

After gaining access to the network with a valid IP address, the attacker can modify, reroute, or delete data. The attacker can also conduct other types of attacks, as described in the following sections.

Password-based attacks

A commonality among most operating systems and network security plans is password-based access control. Access to both a computer and network resources are determined by a user name and password.

Historically, many versions of operating system components have not always protected identity information as it was passed through the network for validation. This might allow an eavesdropper to determine a valid user name and password and use it to gain access to the network by posing as a valid user.

When an attacker finds and accesses a valid user account, the attacker has the same rights as the actual user. For example, if the user has administrative rights, the attacker can create additional accounts for access at a later time.

After gaining access to a network with a valid account, an attacker can do any of the following:

  • Obtain lists of valid user and computer names and network information.
  • Modify server and network configurations, including access controls and routing tables.
  • Modify, reroute, or delete data.

Denial-of-service attack

Unlike a password-based attack, the denial-of-service attack prevents normal use of a computer or network by valid users.

After gaining access to a network, an attacker can do any of the following:

  • Distract information systems staff so that they do not immediately detect the intrusion. This gives an attacker the opportunity to make additional attacks.
  • Send invalid data to applications or network services, causing applications or services to close or operate abnormally.
  • Send a flood of traffic until a computer or an entire network is shut down.
  • Block traffic, which results in a loss of access to network resources by authorized users.

Man-in-the-middle attack

As the name indicates, a man-in-the-middle attack occurs when someone between two users who are communicating is actively monitoring, capturing, and controlling the communication without the knowledge of the users. For example, an attacker can negotiate encryption keys with both users. Each user then sends encrypted data to the attacker, who can decrypt the data. When computers are communicating at low levels of the network layer, the computers might not be able to determine with which computers they are exchanging data.

Compromised-key attack

A key is a secret code or number required to encrypt, decrypt, or validate secured information. Although determining a key is a difficult and resource-intensive process for an attacker, it is possible. After an attacker determines a key, that key is referred to as a compromised key.

An attacker uses the compromised key to gain access to a secured communication without the sender or receiver being aware of the attack. With the compromised key, the attacker can decrypt or modify data. The attacker can also attempt to use the compromised key to compute additional keys, which might allow access to other secured communications.

Sniffer attack

A sniffer is an application or device that can read, monitor, and capture network data exchanges and packets. If the packets are not encrypted, a sniffer provides a full view of the data that is inside of the packet. Even encapsulated (tunneled) packets can be opened and read if they are not encrypted.

Using a sniffer, an attacker can do the following:

  • Analyze a network and access information, eventually causing the network to stop responding or become corrupted.
  • Read private communications.

Application-layer attack

An application-layer attack targets application servers by causing a fault in a server's operating system or applications. This results in the attacker gaining the ability to bypass normal access controls. The attacker takes advantage of this situation, gaining control of an application, system, or network, and can do any of the following:

  • Read, add, delete, or modify data or an operating system.
  • Introduce a virus that uses computers and software applications to copy viruses throughout the network.
  • Introduce a sniffer program to analyze the network and gain information that can eventually be used to cause the network to stop responding or become corrupted.
  • Abnormally close data applications or operating systems.
  • Disable other security controls to enable future attacks.


Protection against attacks

IPSec protects data so that an attacker finds it extremely difficult or impossible to interpret it. The level of protection provided is determined by the strength of the security levels specified in your IPSec policy structure.

IPSec has a number of features that significantly reduce or prevent the following attacks:

  • Sniffers (lack of confidentiality)

    • The Encapsulating Security Payload (ESP) protocol in IPSec provides data confidentiality by encrypting the payload of IP packets.

  • Data modification

    • IPSec uses cryptography-based keys, shared only by the sending and receiving computers, to create a cryptographic checksum for each IP packet. Any modification to the packet data alters the checksum, which indicates to the receiving computer that the packet was modified in transit.

  • Identity spoofing, password-based, and application-layer attacks

    • IPSec allows the exchange and verification of identities without exposing that information to interpretation by an attacker. Mutual verification (authentication) is used to establish trust between the communicating systems and only trusted systems can communicate with each other. After identities are established, IPSec uses cryptography-based keys, shared only by the sending and receiving computers, to create a cryptographic checksum for each IP packet. The cryptographic checksum ensures that only the computers that have knowledge of the keys could have sent each packet.

  • Man-in-the-middle attacks

    • IPSec combines mutual authentication with shared, cryptography-based keys.

  • Denial-of-service attacks

    • IPSec uses IP packet filtering methodology as the basis for determining whether communication is allowed, secured, or blocked, according to the IP address ranges, IP protocols, or even specific TCP and UDP ports.


Policy-based security

Although stronger security methods based on cryptography have become necessary to fully protect communication, they can often greatly increase administrative overhead. To reduce overhead, IPSec uses policy-based administration.

IPSec policies, rather than application programming interfaces (APIs), are used to configure IPSec security services. The policies provide variable levels of protection for most traffic types in most existing networks.

You can configure IPSec policies to meet the security requirements of a computer, application, organizational unit, domain, site, or global corporation. You can use the IP Security Policy Management console provided in XOX and the Windows Server 2003 family to define IPSec policies for computers through Active Directory® (for domain members) or on the local computer (for computers that do not belong to domains).


Simplified deployment

To achieve secure communications with a low cost of ownership, XOX and the Windows Server 2003 family simplify the deployment of IPSec with the following features:

  • Integration with the security framework in Windows 2000 and the Windows Server 2003 family.

    • IPSec uses the secure domain in Windows 2000 and the Windows Server 2003 family as a trust model. By default, IPSec policies use the Active Directory® default authentication method (Kerberos V5 authentication) to identify and trust communicating computers. Computers that are members of a Windows 2000 or a Windows Server 2003 domain and are in trusted domains can easily establish IPSec-secured communications.

  • Centralized IPSec policy administration through Active Directory

    • IPSec policies can be assigned through Group Policy configuration of Active Directory domains and organizational units. This allows the IPSec policy to be assigned at the domain, site, or organizational unit level, eliminating the administrative overhead of configuring each computer separately.

  • Transparency of IPSec to users and applications

    • Integration at the IP layer (layer 3) provides security for any protocol in the TCP/IP suite of protocols. You do not need separate security for each protocol in the TCP/IP suite of protocols, because applications using TCP/IP pass the data to the IP protocol layer, where it is secured.

  • Flexible security configuration

    • The security services within each policy can be customized to meet the majority of security requirements for the network and data traffic.

  • Automatic key management

    • Internet Key Exchange (IKE) services dynamically exchange and manage cryptographic keys between communicating computers.

  • Automatic security negotiation

    • Internet Key Exchange (IKE) services dynamically negotiate a common set of security settings between communicating computers, eliminating the need for both computers to have identically configured policies.

  • Public key infrastructure support

    • Using public key certificates for authentication is supported. This allows trust and secure communication for computers that do not belong to a trusted Windows 2000 or Windows Server 2003 domain, non-Microsoft operating systems, computers that have membership in untrusted domains, and instances in which computer access must be restricted to a smaller group than domain authentication allows.

  • Preshared key support

    • If authentication using the Kerberos V5 protocol or public key certificates is not possible, a preshared authentication key can be configured.

Important

  • The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment.

Note

  • The integration with the security framework in Windows 2000 or the Windows Server 2003 family, centralized IPSec policy administration through Active Directory, and the use of the Kerberos V5 protocol as described here do not apply to computers running XOX. You cannot administer Active Directory-based IPSec policy from computers running XOX because these computers cannot join Active Directory domains.


Understanding Internet Protocol security

This section introduces the key concepts and terminology you should review before you deploy IPSec. This section covers:

  • The IPSec process
  • Virtual private networking with IPSec
  • IPSec security services
  • IPSec protocol types
  • IPSec components
  • Description of an IPSec policy


The IPSec process

This topic provides an overview of IPSec concepts that are central to understanding the IPSec process, including IPSec policy configuration and the Internet Key Exchange (IKE) protocol. In addition, this topic describes how IPSec network traffic processing works, using two intranet computers as an example.

IPSec Policy Configuration

In Windows 2000, XOX, and the Windows Server 2003 family, IPSec is implemented primarily as an administrative tool that you can use to enforce security policies on IP network traffic. A security policy is a set of packet filters that define network traffic as it is recognized at the IP layer. A filter action defines the security requirements for the network traffic. A filter action can be configured to: Permit, Block, or Negotiate security (negotiate IPSec).

IPSec filters are inserted into the IP layer of the computer TCP/IP networking protocol stack so that they can examine (filter) all inbound or outbound IP packets. Except for a brief delay required to negotiate a security relationship between two computers, IPSec is transparent to end-user applications and operating system services.

A collective set of IPSec security settings is known as an IPSec policy. Windows 2000, XOX, and the Windows Server 2003 family provide a graphical user interface and several command-line tools that you can use to configure an IPSec policy, and then assign it to a computer.

To ensure that IPSec communication is successful and that IPSec meets the security requirements of your organization, you must carefully design, configure, coordinate, and manage IPSec policies. In many organizations, one administrator might be responsible for configuring and managing IPSec policies for many, if not all, computers.

Internet Key Exchange (IKE) security associations

The IKE protocol is designed to securely establish a trust relationship between each computer, to negotiate security options, and dynamically generate shared, secret cryptographic keying material. The agreement of security settings associated with keying material is called a security association, also known as an SA. These keys will provide authenticity, integrity, and optionally, encryption of IP packets that are sent using the security association. IKE negotiates two types of security associations:

  • A main mode security association (the IKE security association that is used to protect the IKE negotiation itself).
  • IPSec security associations (the security associations that are used to protect application traffic).

You can configure IPSec policy settings for both types of security associations.

The IPSec service interprets an IPSec policy, expanding it into the components that it needs to control the IKE negotiation. The IPSec policy contains one definition of a packet filter. The packet filter is interpreted in two ways: one uses only the address and identity information to allow IKE to establish a main mode SA (the IKE security association); the other allows IKE to establish the IPSec security associations (also known as quick mode security associations).

IPSec network traffic processing

The following illustration shows how IPSec works in terms of the IPSec components for two intranet computers.

The IPSec process

For simplicity, this example is of an intranet in which two computers have an active IPSec policy.

  1. Alice, using a data application on Computer A, sends an application IP packet to Bob on Computer B.
  2. The IPSec driver on Computer A checks its outbound IP filter lists and determines that the packets should be secured.
  3. The action is to negotiate security, so the IPSec driver notifies IKE to begin negotiations.
  4. The IKE service on Computer A completes a policy lookup, using its own IP address as the source and the IP address of Computer B as the destination. The main mode filter match determines the main mode settings that Computer A proposes to Computer B. Computer A sends the first IKE message in main mode, using UDP source port 500, destination port 500. IKE packets receive special processing by the IPSec driver to bypass filters.
  5. Computer B receives an IKE main mode message requesting secure negotiation. It uses the source IP address and the destination IP address of the UDP packet to perform a main mode policy lookup, to determine which security settings to agree to. Computer B has a main mode file that matches, and so replies to begin negotiation of the main mode SA.
  6. Computer A and Computer B now negotiate options, exchange identities, verify trust in those identities (authentication), and generate a shared master key. They have now established an IKE main mode SA. Computer A and Computer B must mutually trust each other.
  7. Computer A then performs an IKE quick mode policy lookup, using the full filter to which the IPSec driver matched the outbound packet. Computer A selects the quick mode security settings and proposes them, and the quick mode filter, to Computer B.

    8. Computer B also performs an IKE quick mode policy lookup, using the filter description offered by Computer A. Computer B selects the security settings required by its policy and compares those settings to those offered by computer A. Computer B accepts one set of options and completes the remainder of the IKE quick mode negotiation to create a pair of IPSec security associations.

  8. One IPSec SA is inbound and one IPSec SA is outbound. The IPSec SAs are identified by a Security Parameter Index (SPI), which is inserted into the IPSec header of each packet sent.
  9. The IPSec driver on Computer A uses the outbound SA to sign and, if required, encrypt the packets. If the network adapter can perform hardware offload of IPSec cryptographic functions, the IPSec driver formats the packets, but does not perform the IPSec cryptographic functions.
  10. The IPSec driver passes the packets to the network adapter driver, indicating whether the adapter must perform the IPSec cryptographic functions. The network adapter transmits the packets into the network.
  11. The network adapter driver at Computer B receives the encrypted packets from the network. The SPI is used by the receiver of an IPSec packet to find the corresponding IPSec security association, with the cryptographic keys required to verify and decrypt the packets. If the network adapter can decrypt the packets in hardware, it verifies whether it can recognize the SPI. If it cannot decrypt the packets in hardware, or if it cannot recognize the SPI, it passes the packets up to the IPSec driver.
  12. The IPSec driver on Computer B uses the inbound SA SPI to retrieve the keys required to validate authentication and integrity and, if required, to decrypt the packets.
  13. The IPSec driver converts the packets from IPSec format back to standard IP packet format. It passes the validated and decrypted IP packets to the TCP/IP driver, which passes them to the receiving application on Computer B.
  14. The IPSec SAs continue to provide very strong, transparent protection for application data traffic. The IPSec SAs are automatically refreshed by an IKE quick mode negotiation for as long as the application sends and receives data. When the application stops sending and receiving data, the IPSec SAs become idle and are deleted.
  15. Typically, the IKE main mode SA is not deleted. By default, the main mode SA has a lifetime of 8 hours. You can configure the main mode SA lifetime to as short as 5 minutes to a maximum of 48 hours. Whenever more traffic is sent, a new quick mode is negotiated automatically to create two new IPSec SAs to protect application traffic. This process is rapid, because the main mode SA already exists. If a main mode SA expires, it is automatically renegotiated as needed.

Any routers or switches in the path between the communicating computers simply forward the encrypted IP packets to their destination. However, if there is a firewall, security router, or proxy server in the path, IPSec and IKE traffic might not be forwarded. These devices must be configured to allow IPSec and IKE protocol packets through. If the IPSec packets are not encrypted, the firewall or security router can still inspect the TCP or UDP ports or other content in the packets. If the contents of these packets are modified after they are sent, the receiving IPSec computer detects the modification and discards the packets.

However, the Windows Server 2003 family implementation of IPSec provides support for a new Internet specification that allows IPSec packets to be modified by a network address translator (NAT). IPSec Encapsulating Security Payload (ESP) packets can pass through NATs that allow UDP traffic. The IKE protocol automatically detects the presence of a NAT and uses User Datagram Protocol-ESP (UDP-ESP) encapsulation to allow IPSec traffic to pass through the NAT.


Virtual private networking with IPSec

Tunneling is the entire process of encapsulation, routing, and decapsulation. Tunneling wraps, or encapsulates, the original packet inside a new packet. This new packet might have new addressing and routing information, which enables it to travel through a network. When tunneling is combined with data confidentiality, the original packet data (as well as the original source and destination) is not revealed to those listening to traffic on the network. After the encapsulated packets reach their destination, the encapsulation is removed, and the original packet header is used to route the packet to its final destination.

The tunnel itself is the logical data path through which the encapsulated packets travel. To the original source and destination peer, the tunnel is usually transparent and appears as just another point-to-point connection in the network path. The peers are unaware of any routers, switches, proxy servers, or other security gateways between the tunnel’s beginning point and the tunnel’s end point. When tunneling is combined with data confidentiality, it can be used to provide a virtual private network (VPN).

The encapsulated packets travel through the network inside the tunnel. In this example, the network is the Internet. The gateway might be an edge gateway that stands between the outside Internet and the private network. The edge gateway can be a router, firewall, proxy server, or other security gateway. Also, two gateways can be used inside the private network to protect traffic across untrusted parts of the network.

Tunnel types that use IPSec

In XOX and the Windows Server 2003 family, two types of tunneling are provided that use IPSec:

  • The combination of the Layer Two Tunneling Protocol (L2TP) and IPSec (L2TP/IPSec)

    • L2TP is used to tunnel the data across a shared or public network such as the Internet, and IPSec Encapsulating Security Payload (ESP) is used to encrypt the data. L2TP/IPSec can be used to tunnel IP or Internetwork Packet Exchange (IPX) traffic.

  • IPSec in tunnel mode

    • When IPSec is used in tunnel mode, IPSec itself provides encapsulation for IP traffic only. The primary reason for using IPSec tunnel mode is interoperability with other routers, gateways, or end-systems that do not support L2TP/IPSec or PPTP VPN tunneling. IPSec tunnel mode is supported only in gateway-to-gateway tunneling scenarios and for certain server-to-server or server-to-gateway configurations as an advanced feature. IPSec tunnel mode is not supported for remote access VPN scenarios. L2TP/IPSec or PPTP should be used for remote access VPN connections.

    For information about IPSec tunnel mode and configuring IPSec tunnels, see Tunnel mode.

How L2TP and IPSec work

L2TP and IPSec are combined to provide both tunneling and security for IP, IPX, and other protocol packets across any IP network. IPSec can also perform tunneling without L2TP, but it is only recommended for interoperability, when one of the gateways does not support L2TP or PPTP.

L2TP encapsulates original packets first inside a PPP frame (performing compression when possible) and then inside a UDP message using port 1701. Because the UDP message is an IP payload, L2TP uses IPSec transport mode to secure the tunnel. This design supports RFC 3193 ("Securing L2TP Using IPSec"), and it is compatible with XOX and Windows 2000. The IPSec Internet Key Exchange (IKE) protocol negotiates security for the L2TP tunnel using certificate-based or preshared key authentication. If IPSec main mode and quick mode security associations are successfully established, L2TP negotiates the tunnel, including compression and user authentication options, and performs PPP-based user authentication.

Tunnel packet using L2TP and IPSec

The original packet, shown here as the PPP payload, includes the original and ultimate source and destination addresses used on the private network. The outer IP header, shown as IP header, contains the source and destination addresses of the VPN client and server on the public network. The L2TP header includes L2TP tunnel control information. The PPP header identifies the protocol of the original packet (for example, IP or IPX).

Note

  • XOX

Common L2TP/IPSec scenarios

Two common scenarios for L2TP/IPSec are securing communications between remote access clients and the corporate network across the Internet and securing communications between branch offices.

Remote access clients with L2TP/IPSec

A common requirement is securing communications between remote access clients and the corporate network across the Internet. Such a client might be a sales consultant who spends most of the time traveling, or an employee working from a home office.

Remote access clients using L2TP and IPSec

In this figure, the remote gateway is a server that provides edge security for the corporate intranet. The remote client represents a roaming user who requires regular access to network resources and information. An ISP is used as an example to demonstrate the path of communication when the client uses an ISP to access the Internet. L2TP is combined with IPSec to provide a simple, efficient way to build the tunnel and protect the data across the Internet.

For more information about configuring L2TP-based client remote access VPN connections, see Setting up remote access VPNs, To make a virtual private network (VPN) connection, and Virtual private network (VPN) connections.

Connecting branch offices with L2TP/IPSec

A large corporation often has multiple sites that require communication—for example, a corporate office in New York and a sales office in Washington. In this case, L2TP is combined with IPSec to provide the VPN connection and protect the data between the sites.

L2TP/IPSec Router-to-Router VPN Connection

In this figure, the router running the Windows Server 2003 family provides edge security. The routers might have a leased line, dial-up, or other type of Internet connection. The IPSec SA and L2TP tunnel runs between the routers only and provides protected communication through the Internet.

For more information about configuring L2TP-based router-to-router VPN connections, see Setting up router-to-router VPNs.


IPSec security services

IPSec provides a high level of security by using cryptography-based services. Cryptography allows information to be transmitted securely by hashing (to provide integrity and authentication) and encrypting (to provide confidentiality) the information. IPSec provides or uses the following security services:

  • Security properties
  • Public key certificate-based authentication
  • Preshared key authentication
  • Data integrity with hash functions
  • Data confidentiality with encryption
  • Key management and protection


Security properties

IPSec provides the following properties for secured communications:

  • Anti-replay

    • Ensures the uniqueness of each IP packet. Anti-replay is also called replay prevention. Anti-replay ensures that data captured by an attacker cannot be reused or replayed to establish a session or gain information illegally. This protects against attempts to intercept a message and then use the identical message to illegally gain access to resources, possibly months later.

  • Integrity

    • Protects data from unauthorized modification in transit, ensuring that the data received is exactly the same as the data sent. Hash functions sign each packet with a cryptographic checksum using a shared, secret key. Only the sender and receiver have the key that is used to calculate the checksum. If the packet contents have changed, the checksum verification fails and the packet is discarded.

  • Confidentiality (encryption)

    • Ensures that data is only disclosed to intended recipients. This is achieved by encrypting the data before transmission, ensuring that the data cannot be read during transmission, even if the packet is monitored or intercepted. Only the communicating computers with the shared, secret key are able to read the data after it has been decrypted. This property is optional and is dependent on IPSec policy settings.

  • Authentication

    • Verifies that a message could only have been sent from a computer that has knowledge of the shared, secret key. The sender includes (with the message) a message authentication code with a calculation that includes the shared, secret key. The receiver performs the same calculation and, if the receiver’s calculation does not match the message authentication code that is included in the message, the message is discarded. The message authentication code is the same as the cryptographic checksum that is used for integrity.


Public key certificate-based authentication

An effectively implemented public key infrastructure, in which security credentials can be presented without compromising them in the process, resolves many security problems. IPSec works with your public key infrastructure to enable certificate-based authentication of computers.

A public key certificate is one type of authentication that reliably provides identity verification. Public key certificates are like digital passports. They are used to verify the identities of non-Microsoft operating system computers, stand-alone computers, clients that are not members of a trusted domain, or computers that are not running the Kerberos V5 authentication protocol (the default authentication method in XOX and Windows Server 2003 family) and Routing and Remote Access.


Preshared key authentication

IPSec can use preshared keys for authentication. Preshared means that the parties agree on a shared, secret key that is used for authentication in an IPSec policy.

During security negotiation, information is encrypted before transmission by using a session key. The session key is created by using a Diffie-Hellman calculation and the shared, secret key. Information is decrypted on the receiving end using the same key. One IPSec peer authenticates the other peer's packet by decryption and verification of the hash inside the packet (the hash inside the packet is a hash of the preshared key). If authentication fails, the packet is discarded.

The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext in the registry. In Active Directory, preshared keys are stored in readable hexadecimal format.

Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment. For more information about IPSec authentication methods, see Authentication methods and Public key certificate-based authentication.


Data integrity with hash functions

Hash message authentication codes (HMAC) sign packets to verify that the information received is exactly the same as the information sent. This is called integrity. HMACs provide integrity through a keyed hash, the result of a mathematical calculation on a message using a hash function (algorithm) combined with a shared, secret key. A hash is commonly described as a signature on the packet. However, a hash differs from a digital signature. A hash uses a secret, shared key, and a digital signature uses public key technology and the private key of the sending computer. A digital signature provides nonrepudiation, and a hash does not. Nonrepudiation ensures that a communication can be proven to have originated from a specific person whose identity can be verified. It also ensures that the communication actually occurred.

Hash functions are also called one-way functions because it is easy to determine the hash from the message but mathematically infeasible to determine the message from the hash. Conversely, in two-way functions, the original message can be determined from its converted form. Encryption and decryption schemes are examples of two-way functions.

The hash is a cryptographic checksum or message integrity code (MIC) that each party must compute to verify the message. For example, the sending computer uses a hash function and shared key to compute the checksum for the message, including it with the packet. The receiving computer must perform the same hash function on the received message and shared key and compare it to the original (included in the packet from the sender). If the message has changed in transit, the hash values are different and the packet is rejected.

For integrity, you can choose between two hash functions when setting policy:

  • MD5

    • Message Digest 5 (MD5) is based on RFC 1321. MD5 completes four passes over the data blocks, using a different numeric constant for each word in the message on each pass. The number of 32-bit constants used during the MD5 computation ultimately produces a 128-bit hash that is used for the integrity check.

  • SHA1

    • Secure Hash Algorithm 1 (SHA1) was developed by the National Institute of Standards and Technology as described in Federal Information Processing Standard (FIPS) PUB 180-1. The SHA process is closely modeled after MD5. The SHA1 computation results in a 160-bit hash that is used for the integrity check. Because longer hash lengths provide greater security, SHA is stronger than MD5.


Data confidentiality with encryption

IPSec in XOX and the Windows Server 2003 family uses the US Data Encryption Standard (DES) to provide confidentiality (data encryption). The DES algorithm was published in 1977 by the US National Bureau of Standards.

DES is a block cipher that uses a 56-bit key. A block cipher is an encryption algorithm that operates on a fixed size block of data. DES encrypts data in 64-bit blocks using a 64-bit key. The key appears to be a 64-bit key, but one bit in each of the 8 bytes is used for error checking, resulting in 56 bits of usable key.

Cipher block chaining (CBC) is also used to hide patterns of identical blocks of data within a packet. An initialization vector (an initial random number) is used as the first random block to encrypt and decrypt a block of data. Different random blocks are used in conjunction with the secret key to encrypt each successive block. This ensures that identical sets of unsecured data (plaintext) result in unique, encrypted data blocks.

The XOX and the Windows Server 2003 family implementation of IPSec supports the use of:

  • DES

    • Used when the high security and overhead of 3DES are not necessary.

  • 3DES

    • Used when high security is required. 3DES processes each block three times, using a unique 56-bit key each time:

    1. Encryption on the block with key 1
    2. Decryption on the block with key 2
    3. Encryption on the block with key 3

    This process is reversed if the computer is decrypting a packet.

Note

  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.


Key management and protection

A key is a secret code or number that is required to read, modify, or verify secured data. Keys are used in conjunction with algorithms (a mathematical process) to secure data.

Key management

XOX and the Windows Server 2003 family automatically handle key generation and implement the following keying properties:

  • Variable key lengths

    • With IPSec, you can select integrity and encryption algorithms that provide a range of key lengths. Every time the length of a key is increased by one bit, the number of possible keys doubles, making it exponentially more difficult to determine the key. Shorter key lengths are processed more quickly than longer key lengths. Longer key lengths provide a higher level of security, but are more resource-intensive to process.

  • Key material generation: The Diffie-Hellman algorithm

    • To enable secure communication, two computers must be able to gain the same shared key (session key), without sending the key across a network and compromising the secret.

    The Diffie-Hellman algorithm predates Rivest-Shamir-Adleman (RSA) cryptographic algorithms and offers better performance. It is one of the oldest and most secure algorithms used for key exchange. The two peers publicly exchange keying information, which XOX and the Windows Server 2003 family additionally protect with a hash function signature. Neither peer ever exchanges the actual key; however, after their exchange of keying material, each can generate the identical shared key.

    The Diffie-Hellman keying material that is exchanged by the two parties can be based on 768, 1,024, or 2,048 bits of keying material, known as Diffie-Hellman groups 1, 2, and 2048, respectively. The strength of the Diffie-Hellman group is proportional to the strength of the key that is computed from the Diffie-Hellman exchange. Strong Diffie-Hellman groups combined with longer key lengths increase the computational difficulty of determining a secret key.

    IPSec uses the Diffie-Hellman algorithm to provide the keying material for all other encryption keys. Diffie-Hellman does not provide authentication. In the XOX and the Windows Server 2003 family implementation of IPSec, identities are authenticated after the Diffie-Hellman exchange takes place, providing protection against man-in-the-middle attacks.

  • Dynamic rekeying

    • IPSec policy controls how often a new key is generated during communication, by using a method called dynamic rekeying. The communication is sent in blocks, and each block of data is secured with a different key. This prevents an attacker who has obtained part of a communication, and the corresponding session keys, from obtaining the remainder of the communication. This on-demand security negotiation and automatic key management service is provided using IKE, as defined in RFC 2409, "The Internet Key Exchange (IKE)."

    IPSec policy allows you to control how often a new key is generated. If no values are configured, keys are regenerated automatically at default intervals.

Key protection

The following features enhance the base prime numbers (keying material) and the strength of the keys for the master and session keys.

  • Key lifetimes

    • Key lifetimes determine when a new key is generated, rather than how it is generated. A key lifetime allows you to force an automatic key regeneration after a specific interval of either kilobytes (KB) or seconds, whichever occurs first. This ensures that even if an attacker is able to decipher part of a communication protected by one key, new keys protect the remainder of the communication. Whenever a key lifetime is reached, the security association (SA) is also renegotiated and the key is refreshed or regenerated. For this reason, key lifetimes are also referred to as SA lifetimes. You can specify key lifetimes for the master key and for session keys.

    The master key lifetime corresponds to the IKE SA lifetime created by the IKE main mode (phase I) negotiation. It is configured in terms of time and number of quick mode negotiations, and it applies to all security rules in the IPSec policy. The IKE SA and master key typically have a long lifetime (the default value is eight hours). Master keys are much more resource-intensive to generate than session keys, due to the fact that reauthentication and additional Diffie-Hellman exchanges are required.

    The session keys correspond to the IPSec SAs that are used to protect program traffic. The session keys and IPSec SAs are quickly derived from the master key by the IKE quick mode (phase II) negotiation. Thus the session keys are used to protect the data, and they have lifetimes based on the amount of data sent and the amount of time elapsed since the key started being used. Typically, session keys have shorter lifetimes than master keys. The default values are 100,000 KB (approximately 100 megabytes) or one hour. When the session keys are refreshed, new IPSec SAs replace the old ones. The IPSec SA session keys must be refreshed before either the data or time lifetime expires, or traffic is discarded. A session key will be deleted if the IPSec SAs become idle (by default, SAs become idle after five minutes). Because the master key lives longer than the session key, it allows new IPSec SAs and session keys to be established quickly. To prevent data loss, the quick mode SA session key is generated shortly before the main mode SA expires. The IKE SA and corresponding master key are not deleted when session keys and IPSec SAs are deleted, unless the specified number of quick mode SA negotiations has elapsed.

    For example, if a communication takes one hour (3,600 seconds), and if you specify the minimum session key lifetime of five minutes (300 seconds), more than 12 keys are generated to complete the communication. If a 100 megabytes (MB) file is transferred over a fast corporate LAN using an IPSec security association with a 100 MB and one hour lifetime, at least one, if not two, session rekeys occur.

    Notes

    • To provide maximum protection against cryptographic attacks, the amount of data encrypted using Data Encryption Standard (DES) or Triple DES (3DES) by a single session key should not exceed 100 MB. In a high-security environment, consider using session key lifetimes for MD5 or Secure Hash Algorithm (SHA-1) that are shorter then the default values, to provide stronger protection against future cryptographic attacks.
    • In certain scenarios, there might be practical operational limits on key lifetimes. The shorter the key lifetimes, the more CPU cycles are spent refreshing keys. This might limit either the number of active IPSec SAs that a computer can maintain or the throughput performance of IPSec-secured traffic.
    • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.
  • Session key refresh limit

    • Repeated rekeying off of a session key can compromise the Diffie-Hellman shared secret. Because of this, you might want to set a limit to the number of quick mode session keys that can be derived from a main mode negotiation.

    If you have enabled master key perfect forward secrecy (PFS), the session key limit is not used. Setting a session key limit to 1 is identical to enabling master key PFS. If both a master key lifetime and a session limit are specified, whichever limit is reached first causes a new main mode negotiation. By default, IPSec policy does not specify a session limit.

  • Diffie-Hellman groups

    • Diffie-Hellman groups are used to determine the length of the base prime numbers (key material) for the Diffie-Hellman exchange. The cryptographic strength of any key derived from a Diffie-Hellman exchange depends, in part, on the strength of the Diffie-Hellman group on which the prime numbers are based.

    Diffie-Hellman Group 2048 (high) is stronger than Group 2 (medium), which is stronger than Group 1 (low). When a stronger group is used, the key that is derived from a Diffie-Hellman exchange is stronger and more difficult for an attacker to break.

    IKE negotiates which specific group to use, ensuring that there are not any negotiation failures that result from a mismatched Diffie-Hellman group between the two peers.

    If session key PFS is enabled, a new Diffie-Hellman key is negotiated during the first quick mode SA negotiation. This new key removes the dependency of the session key on the Diffie-Hellman exchange that is performed for the master key.

    Both the initiator and responder must have session key PFS enabled, or negotiation fails.

    The Diffie-Hellman group is the same for both the main mode and quick mode SA negotiations. When session key PFS is enabled, even though the Diffie-Hellman group is set as part of the main mode SA negotiation, it affects any rekeys during session key establishment.

    Important

    • For enhanced security, do not use Diffie-Hellman Group 1. For maximum security, use Group 2048 whenever possible. Use Group 2 when required for interoperability with Windows 2000 and XOX.

      • For more information about Diffie-Hellman groups and key exchange, see Key exchange methods.

    Note

    • Diffie-Hellman Group 2048 is provided only with the Windows Server 2003 family.
  • Perfect forward secrecy

    • Unlike key lifetimes, PFS determines how a new key is generated, rather than when it is generated. Specifically, PFS ensures that the compromise of a single key permits access only to data that is protected by it, not necessarily to the entire communication. To achieve this, PFS ensures that a key used to protect a transmission cannot be used to generate additional keys. In addition, if the key that was used was derived from specific keying material, that material cannot be used to generate other keys.

    Master key PFS requires a reauthentication and is resource intensive. When it is enabled, IKE must reauthenticate identities, increasing overhead for domain controllers when Kerberos V5 is used for authentication. It requires a new main mode negotiation for every quick mode negotiation that occurs.

    Session key PFS can be used without a reauthentication and is less resource intensive. Session key PFS results in a Diffie-Hellman exchange to generate new keying material. It requires only four messages and no authentication.

    Unlike session key PFS, master key PFS does not have to be enabled on both peers. This is because master key PFS is not part of the SA negotiation. If the responder requires PFS, and the quick mode SA of the initiator expires, the responder rejects the message from the initiator and requires a new main mode negotiation. The initiator expires the main mode SA and renegotiates. PFS can be individually set for both the master and session keys.


IPSec protocol types

IPSec protocols provide data and identity protection for each IP packet by adding their own security protocol header to each packet.

For information about AH and ESP when used in transport and tunnel mode, see the following topics:

  • Transport mode
  • Tunnel mode


Transport mode

Transport mode is the default mode for IPSec, and it is used for end-to-end communications (for example, for communications between a client and a server). When transport mode is used, IPSec encrypts only the IP payload. Transport mode provides the protection of an IP payload through an AH or ESP header. Typical IP payloads are TCP segments (containing a TCP header and TCP segment data), a UDP message (containing a UDP header and UDP message data), and an ICMP message (containing an ICMP header and ICMP message data).

Authentication Header transport mode

Authentication Header (AH) provides authentication, integrity, and anti-replay protection for the entire packet (both the IP header and the data payload carried in the packet). It does not provide confidentiality, which means that it does not encrypt the data. The data is readable, but protected from modification. AH uses keyed hash algorithms to sign the packet for integrity. For more information, see Data integrity with hash functions.

For example, Alice on Computer A sends data to Bob on Computer B. The IP header, the AH header, and the data are protected with integrity. This means Bob can be certain it was really Alice who sent the data and that the data was unmodified.

Integrity and authentication are provided by the placement of the AH header between the IP header and the IP payload, as shown in the following illustration.

Authentication Header

AH is identified in the IP header with an IP protocol ID of 51. AH can be used alone or combined with the Encapsulating Security Payload (ESP) protocol.

The AH header contains the following fields:

  • Next Header

    • Identifies the IP payload by using the IP protocol ID. For example, a value of 6 represents TCP.

  • Length

    • Indicates the length of the AH header.

  • Security Parameters Index (SPI)

    • Used in combination with the destination address and the security protocol (AH or ESP) to identify the correct security association for the communication. The receiver uses this value to determine with which security association the packet is identified.

  • Sequence Number

    • Provides anti-replay protection for the packet. The sequence number is a 32-bit, incrementally increasing number (starting from 1) that indicates the packet number sent over the security association for the communication. The sequence number cannot repeat for the life of the quick mode security association. The receiver checks this field to verify that a packet for a security association with this number has not already been received. If one has been received, the packet is rejected.

  • Authentication Data

    • Contains the integrity check value (ICV), also known as the message authentication code, which is used to verify both message authentication and integrity. The receiver calculates the ICV value and checks it against this value (which is calculated by the sender) to verify integrity. The ICV is calculated over the IP header, the AH header, and the IP payload.

Packet signature with the AH header

AH signs the entire packet for integrity, with the exception of some fields in the IP header which might change in transit (for example, the Time to Live and Type of Service fields). If another IPSec header is being used in addition to AH, the AH header is inserted before any other IPSec headers. The AH packet signature is shown in the following illustration.

Packet signature with the AH header

Encapsulating Security Payload transport mode

Encapsulating Security Payload (ESP) provides confidentiality (in addition to authentication, integrity, and anti-replay protection) for the IP payload. ESP in transport mode does not sign the entire packet. Only the IP payload (not the IP header) is protected. ESP can be used alone or in combination with AH.

For example, Alice on Computer A sends data to Bob on Computer B. The IP payload is encrypted and signed for integrity. Upon receipt, after the integrity verification process is complete, the data payload in the packet is decrypted. Bob can be certain that it was Alice who sent the data, the data is unmodified, and no one else was able to read it.

ESP is identified in the IP header with the IP protocol ID of 50. As shown in the following illustration, the ESP header is placed before the IP payload, and an ESP trailer and ESP authentication trailer is placed after the IP payload.

ESP Header

The ESP header contains the following fields:

  • Security Parameters Index

    • Identifies the correct security association for the communication when used in combination with the destination address and the security protocol (AH or ESP). The receiver uses this value to determine the security association with which this packet should be identified.

  • Sequence Number

    • Provides anti-replay protection for the packet. The sequence number is a 32-bit, incrementally increasing number (starting from 1) that indicates the packet number sent over the quick mode security association for the communication. The sequence number cannot repeat for the life of the quick mode security association. The receiver checks this field to verify that a packet for a security association with this number has not already been received. If one has been received, the packet is rejected.

The ESP trailer contains the following fields:

  • Padding

    • Padding of 0 to 255 bytes is used to ensure that the encrypted payload with the padding bytes are on byte boundaries required by encryption algorithms.

  • Padding Length

    • Indicates the length of the Padding field in bytes. The receiver uses this field to remove padding bytes after the encrypted payload with the padding bytes has been decrypted.

  • Next Header

    • Identifies the type of data in the payload, such as TCP or UDP.

The ESP authentication trailer contains the following field:

  • Authentication Data

    • Contains the integrity check value (ICV), also known as the message authentication code, which is used to verify both message authentication and integrity. The receiver calculates the ICV value and checks it against this value (which is calculated by the sender) to verify integrity. The ICV is calculated over the ESP header, the payload data, and the ESP trailer.

Packet signature and encryption

As shown in the following illustration, ESP provides protection for IP payloads. The signed portion of the packet indicates where the packet has been signed for integrity and authentication. The encrypted portion of the packet indicates what information is protected with confidentiality.

Encapsulating Security Payload

The IP header is not signed and is not necessarily protected from modification. To provide data integrity and authentication for the IP header, use ESP and AH.


Tunnel mode

When IPSec tunnel mode is used, IPSec encrypts the IP header and the payload, whereas transport mode only encrypts the IP payload. Tunnel mode provides the protection of an entire IP packet by treating it as an AH or ESP payload. With tunnel mode, an entire IP packet is encapsulated with an AH or ESP header and an additional IP header. The IP addresses of the outer IP header are the tunnel endpoints, and the IP addresses of the encapsulated IP header are the ultimate source and destination addresses.

IPSec tunnel mode is useful for protecting traffic between different networks, when traffic must pass through an intermediate, untrusted network. Tunnel mode is primarily used for interoperability with gateways, or end-systems that do not support L2TP/IPSec or PPTP connections. You can use tunnel mode in the following configurations:

  • Gateway-to-gateway
  • Server-to-gateway
  • Server-to-server

AH tunnel mode

As shown in the following illustration, AH tunnel mode encapsulates an IP packet with an AH and IP header and signs the entire packet for integrity and authentication.

AH tunnel mode

ESP tunnel mode

As shown in the following illustration, ESP tunnel mode encapsulates an IP packet with both an ESP and IP header and an ESP authentication trailer.

ESP tunnel mode

The signed portion of the packet indicates where the packet has been signed for integrity and authentication. The encrypted portion of the packet indicates what information is protected with confidentiality.

Because a new header for tunneling is added to the packet, everything that comes after the ESP header is signed (except for the ESP authentication trailer) because it is now encapsulated in the tunneled packet. The original header is placed after the ESP header. The entire packet is appended with an ESP trailer before encryption occurs. Everything that follows the ESP header, except for the ESP authentication trailer, is encrypted. This includes the original header which is now considered to be part of the data portion of the packet.

The entire ESP payload is then encapsulated within the new tunnel header, which is not encrypted. The information in the new tunnel header is used only to route the packet from origin to tunnel endpoint.

If the packet is being sent across a public network, it is routed to the IP address of the gateway for the receiving intranet. The gateway decrypts the packet, discards the ESP header, and uses the original IP header to route the packet to the intranet computer.

ESP and AH can be combined when tunneling, providing both confidentiality for the tunneled IP packet and integrity and authentication for the entire packet.

Using IPSec tunnels

IPSec tunnels provide security for IP traffic only. The tunnel is configured to protect traffic between either two IP addresses or two IP subnets. If the tunnel is used between two computers instead of two gateways, the IP address outside the AH or ESP payload is the same as the IP address inside the AH or ESP payload. In XOX and the Windows Server 2003 family, IPSec does not support protocol-specific or port-specific tunnels. You can configure tunnels by using the IP Security Policy Management and Group Policy consoles to configure and enable two rules:

  1. A rule for the outbound traffic for the tunnel.

    2. The rule for the outbound traffic is configured with a filter list that describes the traffic to be sent through the tunnel and a tunnel endpoint of an IP address configured on the IPSec tunnel peer (the computer or router on the other side of the tunnel).

  2. A rule for the inbound traffic for the tunnel.

    3. The rule for the inbound traffic is configured with a filter list that describes the traffic to be received through the tunnel and a tunnel endpoint of a local IP address (the computer or router on the local side of the tunnel).

Additionally, filter actions, authentication methods, and other settings need to be specified for each rule.

For conceptual information about IPSec policy tunnel settings, see Tunnel endpoint. For information about configuring an IPSec tunnel, see To specify an IPSec tunnel. For information about how tunneling is used for virtual private networking, see Virtual private networking with IPSec.


IPSec components

The following topics describe IPSec components that are installed with XOX and the Windows Server 2003 family:

  • IPSec Policy Agent service
  • Internet Key Exchange
  • IPSec driver


IPSec Policy Agent service

The purpose of the IPSec Policy Agent is to retrieve policy information and pass it to other IPSec components that require this information to perform security services, as shown in the following illustration.

IPSec Policy Agent

The IPSec Policy Agent is a service that resides on each computer running a Windows Server 2003 operating system, appearing as IPSec services in the list of system services in the Services console. The IPSec Policy Agent:

  • Retrieves the appropriate IPSec policy (if one has been assigned) from Active Directory, if the computer is a domain member, or from the local registry, if the computer is not a member of a domain.
  • Polls for changes in policy configuration.
  • Sends the assigned IPSec policy information to the IPSec driver.

If the computer is a member of a domain, policy retrieval occurs when the system starts, at the interval specified in the IPSec policy, and at the default Winlogon polling interval. You can also manually poll Active Directory for policy using the gpupdate /target:computer command.

The following are additional aspects of IPSec policy behavior for a computer that is a member of a domain:

  • If IPSec policy information is centrally configured for computers that are domain members, the IPSec policy information is stored in Active Directory and cached in the local registry of the computer to which it applies.
  • If the computer is temporarily not connected to the domain and policy is cached, new policy information for that computer replaces old, cached information when the computer reconnects to the domain.
  • If the computer is a stand-alone computer or a member of a domain that is not using Active Directory for policy storage, IPSec policy is stored in the local registry.

The IPSec Policy Agent starts automatically at system start time. If there are no IPSec policies in Active Directory or the registry, or if the IPSec Policy Agent cannot connect to Active Directory, the IPSec Policy Agent waits for policy to be assigned or activated.


Internet Key Exchange

Before secured data can be exchanged, a security agreement between the two computers must be established. In this security agreement, called a security association (SA), both agree on how to exchange and protect information, as shown in the following illustration.

ISAKMP/Oakley Service

To build this agreement between the two computers, the IETF has established a standard method of security association and key exchange resolution named Internet Key Exchange (IKE) which:

  • Centralizes security association management, reducing connection time.
  • Generates and manages shared, secret keys that are used to secure the information.

This process not only protects communication between computers, it also protects remote computers that request secure access to a corporate network. In addition, this process works whenever the negotiation for the final destination computer (endpoint) is performed by a security gateway.

Security association (SA) defined

A security association (SA) is the combination of a negotiated key, security protocol, and security parameters index (SPI), which together define the security used to protect the communication from sender to receiver. The SPI is a unique, identifying value in the SA that is used to distinguish among multiple security associations that exist at the receiving computer. For example, multiple associations might exist if a computer is securely communicating with multiple computers at the same time. This is a common occurrence when the computer is a file server or a remote access server that serves multiple clients. In these situations, the receiving computer uses the SPI to determine which SA is used to process the incoming packets.

Phase I or main mode SA

In order to ensure successful and secure communication, IKE performs a two-phase operation. Confidentiality and authentication are ensured during each phase by the use of encryption and authentication algorithms that are agreed upon by the two computers during security negotiations. With the duties split between two phases, key creation can be rapidly accomplished.

During the first phase, the two computers establish a secure, authenticated channel. This is called the phase I or main mode SA. IKE automatically provides necessary identity protection during this exchange.

Phase I or main mode negotiation

The following are the steps that comprise a main mode negotiation.

  1. Policy negotiation

    2. The following four mandatory parameters are negotiated as part of the main mode SA:

    • The encryption algorithm (DES or 3DES)
    • The integrity algorithm (MD5 or SHA1)
    • The Diffie-Hellman group to be used for the base keying material: Group 1 (768 bits of keying material) Group 2 (1,024 bits), or Group 2048 (2,048 bits)
    • The authentication method (Kerberos V5, certificate, or preshared key authentication)

    If certificates or preshared keys are used for authentication, the computer identity is protected. If Kerberos V5 authentication is used, the computer identity is unencrypted until encryption of the entire identity payload takes place during authentication.

    Important

    • For enhanced security, do not use Diffie-Hellman Group 1. For maximum security, use Group 2048 whenever possible. Use Group 2 when required for interoperability with Windows 2000 and XOX.

      • For more information about Diffie-Hellman groups, see Key exchange methods. For more information about preshared key authentication, see Preshared key authentication.

  2. Diffie-Hellman exchange (of public values)

    3. At no time are actual keys exchanged. Only the base information required by the Diffie-Hellman key determination algorithm to generate the shared, secret key is exchanged. After this exchange, the IKE service on each computer generates the master key that is used to protect authentication.

  3. Authentication

    4. To prevent a successful man-in-the-middle attack, the computers attempt to authenticate the Diffie-Hellman key exchange. Without successful authentication, communication will not proceed. The master key is used, in conjunction with the negotiation algorithms and methods, to authenticate identities. The entire identity payload is hashed and encrypted using the keys generated from the Diffie-Hellman exchange in the second step. The payload includes the identity type (for authentication), port, and protocol. IPSec uses the following identity types for authentication: For certificate authentication, the certificate distinguished name and general name; for Kerberos V5 and preshared key authentication, IPv4 addresses, the fully qualified domain name (FQDN) of the computer, and FQDN of the user. The identity payload, regardless of which authentication method is used, is protected from both modification and interpretation.

The sender presents an offer for a potential security association to the receiver. The responder cannot modify the offer. Should the offer be modified, the initiator rejects the responder's message. The responder sends either a reply accepting the offer or a reply with alternatives.

Messages sent during this phase have an automatic retry cycle that is repeated five times. If a response is received before the retry cycle ends, standard SA negotiation begins. If allowed by IPSec policy, unsecured communications will begin after a brief interval. If unsecured communications begin, after five minutes of idle time (during which no messages are sent), secured communication negotiation is attempted the next time messages are sent. If messages are sent continuously, the communication remains unsecured during the lifetime set for the main mode policy. After the policy time has elapsed, a new secured communication negotiation attempt is made.

There is no preset limit to the number of exchanges that can take place. The number of SAs established is only limited by system resources. When estimating the number of SAs that can be established without significantly degrading computer performance, consider the CPU processing strength and RAM of the computer, the lifetime of the SA, and how much traffic is being sent over the SAs.

For more information about unsecured communications, see Filter action.

Phase II or quick mode SA

In this phase, SAs are negotiated on behalf of the IPSec driver.

Phase II or quick mode negotiation

The following are the steps that comprise a quick mode negotiation.

  1. Policy negotiation occurs.

    2. The IPSec computers exchange the following requirements for securing the data transfer:

    • The IPSec protocol (AH or ESP)
    • The hash algorithm for integrity and authentication (MD5 or SHA1)
    • The algorithm for encryption, if requested (3DES or DES)

    A common agreement is reached, and two SAs are established. One SA is for inbound communication and the other is for outbound communication.

  2. Session key material is refreshed or exchanged.

    3. IKE refreshes the keying material and new shared keys are generated for packet integrity, authentication, and encryption (if negotiated). If rekeying is required, either a second Diffie-Hellman exchange (as described in main mode negotiation) occurs, or a refresh of the original Diffie-Hellman key is used.

  3. The SAs and keys, along with the SPI, are passed to the IPSec driver.

The second negotiation of security settings and keying material (for the purpose of securing data) is protected by the main mode SA. As the first phase provided identity protection, the second phase provides protection by refreshing the keying material prior to sending data. IKE can accommodate a key exchange payload for an additional Diffie-Hellman exchange if a rekey is necessary—that is, master key perfect forward secrecy (PFS) is enabled. Otherwise, IKE refreshes the keying material from the Diffie-Hellman exchange completed in main mode.

Quick mode results in a pair of security associations, each with its own SPI and key. One SA is used for inbound communication, and the other for outbound communication.

Notes

  • Although there are two separate quick mode SAs established, IP Security Monitor only displays a single quick mode SA.
  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.

The retry algorithm for a message is similar to the process described in main mode negotiation. However, if this process times out for any reason during the second or higher negotiation off of the same main mode SA, a renegotiation of the main mode SA is attempted. If a message for this phase is received without an established main mode SA, it is rejected.

Using a single main mode SA for multiple quick mode SA negotiations increases the speed of the process. As long as the main mode SA does not expire, renegotiation and reauthentication are not necessary. The number of quick mode SA negotiations that can be performed is determined by IPSec policy settings.

Note

  • Excessive rekeying off of the same main mode SA might make the shared, secret key vulnerable to a known plaintext attack. A known plaintext attack is a sniffer attack in which the attacker attempts to determine the encryption key from encrypted data based on known plaintext.

SA lifetimes

The main mode SA is cached to allow multiple quick mode SA negotiations (unless master key PFS is enabled). When a key lifetime is reached for the master or session key, the SA is renegotiated. In addition, the key is refreshed or regenerated.

When the default time-out period elapses for the main mode SA, or the master or session key lifetime is reached, a delete message is sent to the responder. The IKE delete message tells the responder to expire the main mode SA. This prevents additional new quick mode SAs from being created from the expired main mode SA. IKE does not expire the quick mode SA, because only the IPSec driver contains the number of seconds or bytes that have passed to reach the key lifetime.

Use caution when setting very different key lifetimes for master and session keys. For example, setting a master key lifetime of eight hours and a session key lifetime of two hours might leave a quick mode SA in place for almost two hours after the main mode SA has expired. This occurs when the quick mode SA is generated shortly before main mode SA expiration.

It is generally recommended that all of the IKE settings (for example, master key PFS and key lifetime) and security methods remain at their defaults to avoid unnecessary administrative overhead. This provides a standard (medium) level of security. If your security plan calls for a high level of security, you should consider modifying the default security methods.


IPSec driver

The IPSec driver receives the active IP filter list from the IPSec Policy Agent, as shown in the following illustration, and then attempts to match every inbound and outbound packet against the filters in the list.

IPSec driver

When a packet matches a filter, it applies the filter action. When a packet does not match any filters, the packet is passed back without modification to the TCP/IP driver to be received or transmitted.

If the filter action permits transmission, the packet is received or sent with no modifications. If the action blocks transmission, the packet is discarded. If the action requires the negotiation of security, main mode and quick mode SAs are negotiated.

The negotiated quick mode security association (SA) and keys are used with both outbound and inbound processing. The IPSec driver stores all current quick mode SAs in a database. The IPSec driver uses the Security Parameters Index (SPI) field to match the correct SA with the correct packet.

When an outbound IP packet matches the IP filter list with an action to negotiate security, the IPSec driver queues the packet and then notifies Internet Key Exchange (IKE), which begins security negotiations with the destination IP address of that packet. If several outbound packets are going to the same destination and match the same filter before IKE has finished the negotiation, then only the last packet sent is saved.

After a successful negotiation is completed, the IPSec driver on the sending computer:

  1. Receives the SA containing the session key from IKE.
  2. Locates the outbound SA in its database, and inserts the SPI from the SA into the AH or ESP header.
  3. Signs the packets (and encrypts them if confidentiality is required).
  4. Sends the packets to the IP layer to be forwarded to the destination computer.

If the negotiation failed, the IPSec driver discards the packet.

When an IPSec-secured inbound packet matches a filter in the IP filter list, the IPSec driver:

  1. Receives the session key, SA, and SPI from IKE.
  2. Locates the inbound SA in its database by destination address and SPI.
  3. Checks the signature and, if required, decrypts the packets.
  4. Matches the IP packet searches for a matching filter in the filter list to ensure that no traffic, other than what was agreed upon during the negotiation, is received.
  5. Sends packets to the TCP/IP driver to pass to the receiving application.

When an unsecured IP packet is received, the IPSec driver searches for a matching filter in the filter list. If a match occurs and the filter action for that filter either requires IP security or blocks the packet, then the packet is discarded.

Note

  • The IPSec driver matches all inbound unsecured packets with the list of filters that specify IPSec tunnels first, and then matches the packet with all filters that specify end-to-end (transport) filters. The IPSec driver does not filter certain types of IP packets. For more information, see To add, edit, or remove IPSec filters.


Description of an IPSec policy

An IPSec policy consists of:

  • General IPSec policy settings

    • Settings that apply regardless of which rules are configured. These settings determine the name of the policy, its description for administrative purposes, key exchange settings, and key exchange methods.

  • IPSec policy rules

    • One or more IPSec rules that determine which types of traffic IPSec must examine, how traffic is treated, how to authenticate an IPSec peer, and other settings.

After the policies are created, they can be assigned and applied at the domain, site, organizational unit, and local level. For more information, see Creating, modifying, and assigning IPSec policies.


General IPSec policy settings

The general settings for an IPSec policy consist of the following types:

  • Name

    • The name for the policy.

  • Description

    • Optional text that describes the purpose of the IPSec policy. It is recommended that you complete and update this description to provide a summary of the settings and rules for the policy.

  • Policy change poll interval

    • The number of minutes between consecutive polls for changes in Active Directory-based IPSec policies. This polling does not detect a change in domain or organizational unit membership, or the assigning or unassigning of a new policy. These events are detected when the Winlogon service polls for changes in Group Policy, which occurs by default every 90 minutes.

  • Key exchange settings

    • The way in which new keys are derived and how often they are renewed.

  • Key exchange methods

    • The ways in which identities are protected during the key exchange.

The default key exchange settings and methods are configured to work for most IPSec deployments. Unless there are special security requirements, default settings should not have to be changed.


Key exchange settings

The strength of the keys which protect the key exchange phase is enhanced by the following features:

Key lifetimes

Lifetime settings determine when a new key is generated. Any time a key lifetime is reached, the associated SA is also renegotiated. The process of generating new keys at intervals is called dynamic rekeying or key regeneration. Lifetimes allow you to force the generation of a new key after a specific interval. For example, if the communication takes 100 minutes and you specify the key lifetime as 10 minutes, 10 keys will be generated (one every 10 minutes) during the exchange. Using multiple keys ensures that if an attacker manages to gain the key to one part of a communication, the entire communication is not compromised. Automatic key regeneration is provided by default. You can override the defaults and either specify a master key lifetime in minutes or by the number of session keys, or enable master key perfect forward secrecy (PFS).

Caution should be taken when setting very different key lifetimes, since they also determine the lifetime of the SA. For example, setting a master key lifetime of 8 hours (480 minutes) and a session key lifetime (set within a filter action) of 2 hours might result in a quick mode SA that is in place for almost 2 hours after the main mode SA has expired. This has the potential to occur if the new quick mode SA is generated just before the main mode SA expires.

  • Session key limit

    • Repeated rekeying off of the same master key might eventually compromise the key. For example, if Alice on Computer A sends a message to Bob on Computer B, and then a few minutes later sends another message to Bob, the same master key material can be reused since an SA was recently established with that computer. If you want to limit the number of times this reuse occurs, you can specify a session key limit.

    If you decide to enable master key perfect forward secrecy (PFS), the session key limit will be set to 1. Master key PFS forces key regeneration each time. If you specify both a master key lifetime in minutes, and a session key limit, whichever interval is reached first will start a new key.

  • Master key perfect forward secrecy (PFS)

    • Master key PFS determines how a new session key is generated. Enabling master key PFS ensures that the master key keying material cannot be used to generate more than one session key.

    Master key PFS should be used with caution because it requires reauthentication and might impact performance. It is not required to be enabled on both peers.

For information about how to configure key exchange settings and methods, see To configure key exchange and To create key exchange security methods.


Key exchange methods

In addition to key properties, you can set security methods for main mode IKE negotiation. For example, you can specify which algorithms are used for integrity and confidentiality. The same algorithms that are available for quick mode security methods are available for main mode IKE negotiation: MD5 and SHA1 for integrity; DES and 3DES for confidentiality.

Diffie-Hellman groups

In addition to setting integrity and confidentiality algorithms, you can specify which Diffie-Hellman groups to use. Diffie-Hellman groups are used to determine the length of the base prime numbers used during the key exchange process. The cryptographic strength of any key derived depends, in part, on the strength of the Diffie-Hellman group upon which the prime numbers are based.

Group 2048 (high) is stronger (more secure) than Group 2 (medium), which is stronger than Group 1 (low). Group 1 provides 768 bits of keying strength, Group 2 provides 1024 bits, and Group 2048 provides 2048 bits. If mismatched groups are specified on each peer, negotiation fails. The group cannot be switched during the negotiation.

The Diffie-Hellman group is configured as part of the main mode key exchange settings. New session keys generated during quick mode are derived from the Diffie-Hellman main mode master key material, unless master key or session key perfect forward secrecy (PFS) is enabled. If either master key or session key PFS is enabled, a new Diffie-Hellman exchange is performed to obtain new master key keying material for each new session key that is required. The difference between master key PFS and session key PFS is that master key PFS requires a reauthentication of the main mode SA in addition to the Diffie-Hellman exchange.

Important

  • For enhanced security, do not use Diffie-Hellman Group 1. For maximum security, use Group 2048 whenever possible. Use Group 2 when required for interoperability with Windows 2000 and XOX.

Notes

  • Diffie-Hellman Group 2048 is provided only with the Windows Server 2003 family.
  • For a standard level of security, it is generally recommended that all of the IKE settings (for example, master key PFS and key lifetime) and security methods remain at their defaults. For more information about IKE settings, see Internet Key Exchange.
  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.

For information about how to configure key exchange settings and methods, see To configure key exchange and To create key exchange security methods.


IPSec policy rules

An IPSec policy consists of one or more rules that determine IPSec behavior. IPSec rules are configured on the Rules tab in the properties of an IPSec policy. Each IPSec rule contains the following configuration items:

  • Filter list

    • A single filter list is selected that contains one or more predefined packet filters that describe the types of traffic to which the configured filter action for this rule is applied. The filter list is configured on the IP Filter List tab in the properties of an IPSec rule within an IPSec policy.

  • Filter action

    • A single filter action is selected that includes the type of action required (Permit, Block, or Negotiate Security) for packets that match the filter list. For the Negotiate Security filter action, the negotiation data contains one or more security methods that are used (in order of preference) during IKE negotiations and other IPSec settings. Each security method determines the security protocol (such as AH or ESP), the specific cryptographic and hashing algorithms, and session key regeneration settings used. The filter action is configured on the Filter Action tab in the properties of an IPSec rule within an IPSec policy.

  • Authentication methods

    • One or more authentication methods are configured (in order of preference) and used for authentication of IPSec peers during main mode negotiations. The available authentication methods are the Kerberos V5 protocol, use of a certificate issued from a specified certification authority, or a preshared key. The negotiation data is configured on the Authentication Methods tab in the properties of an IPSec rule within an IPSec policy.

    Important

    • The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment.
  • Tunnel endpoint

    • Specifies whether the traffic is tunneled and, if it is, the IP address of the tunnel endpoint. For outbound traffic, the tunnel endpoint is the IP address of the IPSec tunnel peer. For inbound traffic, the tunnel endpoint is a local IP address. The tunnel endpoint is configured on the Tunnel Setting tab in the properties of an IPSec rule within an IPSec policy. For more information, see Tunnel mode.

  • Connection type

    • Specifies whether the rule applies to local area network (LAN) connections, dial-up connections, or both. The connection type is configured on the Connection Type tab in the properties of an IPSec rule within an IPSec policy.

The rules for a policy are displayed in IP Security Policies in reverse alphabetical order based on the name of the filter list selected for each rule. There is no method for specifying an order in which to apply the rules in a policy. The IPSec driver automatically orders the rules based on the most specific to the least specific filter list. For example, the IPSec driver would apply a rule containing a filter list that specified individual IP addresses and TCP ports before a rule containing a filter list that specified all addresses on a subnet.

Default response rule

The default response rule, which can be used for all policies, has the IP filter list of <Dynamic> and the filter action of Default Response when the list of rules is viewed with the IP Security Policy Management console. The default response rule cannot be deleted, but it can be deactivated. It is activated by default for all policies.

The default response rule is used to ensure that the computer responds to requests for secure communication. If an active policy does not have a rule defined for a computer that is requesting secure communication, then the default response rule is applied and security is negotiated. For example, when Computer A communicates securely with Computer B, and Computer B does not have an inbound filter defined for Computer A, the default response rule is used.

Security methods and authentication methods can be configured for the default response rule. The filter list of <Dynamic> indicates that the filter list is not configured, but that filters are created automatically based on the receipt of IKE negotiation packets. The filter action of Default Response indicates that the action of the filter (Permit, Block, or Negotiate security) cannot be configured. Negotiate Security will be used. However, you can configure:

  • The security methods and their preference order on the Security Methods tab.
  • The authentication methods and their preference order on the Authentication Methods tab.


Filter list

When traffic matches a source, destination, and type of IP traffic in a filter, security negotiations are initiated. This type of IP packet filtering enables a network administrator to precisely define what IP traffic is secured. Each IP filter list contains one or multiple filters, which define IP addresses and traffic types. One IP filter list can be used for multiple communication scenarios.

IPSec requires both an inbound and outbound filter between the computers specified in the filter list. Inbound filters apply to incoming traffic. Outbound filters apply to traffic leaving a computer towards a destination. For example, if Computer A wants to securely exchange data with Computer B:

  • The active IPSec policy on Computer A must have a filter for any outbound packets to Computer B. Source=A and Destination=B.
  • The active IPSec policy on Computer A must have a filter for any inbound packets from Computer B. Source=B and Destination=A.

Each peer must also have the reverse filter:

  • The active IPSec policy on Computer B must have a filter for any inbound packets from Computer A. Source=A and Destination=B.
  • The active IPSec policy on Computer B must have a filter for any outbound packets to Computer A. Source=B and Destination=A.

Filter settings

Each filter defines a particular subset of inbound or outbound network traffic that should be secured. You must have a filter to cover any traffic to which the associated rule applies. A filter contains the following settings:

  • The source and destination address of the IP packet. You can specify any IP address assigned to the IPSec peer, a single IP address, IP addresses by DNS name, or groups of addresses to specify IP subnets.
  • The protocol over which the packet is being transferred. By default, all protocols in the TCP/IP protocol suite are selected. However, you can specify an individual protocol for this filter to meet special requirements, including custom protocols.
  • The source and destination port for TCP and UDP. By default, all TCP and UDP ports are selected, but you can select a specific TCP or UDP port.

For information about how to configure IP filter lists, see To add, edit, or remove IP filter lists.

Special filters might be required for certain types of traffic, such as SNMP traffic or security gateways. For more information, see Special IPSec considerations.


Filter action

A filter action defines the security requirements for the data transmission. A filter action can be configured to:

  • Permit traffic (Permit)

    • IPSec passes this traffic without modification or the requirement for security. This is appropriate for traffic from specific computers that are not IPSec-capable. Be sure to limit the IP filter list to a minimal scope when using this type of filter action, so you do not let traffic through which should be secured.

  • Block traffic (Block)

    • IPSec silently discards this traffic. Be sure to use an IP filter list that appropriately defines the scope of traffic when using a blocking filter action so that you do not block valid computers from communicating.

  • Negotiate IPSec (Negotiate security)

    • IPSec requires the negotiation of security associations and the sending or receiving of IPSec-secured traffic. After you choose to negotiate IPSec, you can also configure the following:

    • Security methods and their order

      • For more information, see IPSec security methods.

    • Allowance of initial incoming unsecured traffic (Accept unsecured communication, but always respond using IPSec)

      • IPSec allows an incoming packet that matches the configured filter list to be unsecured (not protected by IPSec). However, the outgoing response to the incoming packet must be protected. This setting is useful when you are using the default response rule for clients. When a group of servers are configured with a rule that secures communications with any IP address and accepts unsecured communication, responding with only secured communications, enabling the default response rule on client computers ensures that the clients will respond to the server request to negotiate security. This option should be disabled for secure computers connected to the Internet to prevent denial of service attacks.

    • Enabling of communication with non-IPSec-enabled computers (Allow unsecured communication with non-IPSec-aware computer)

      • IPSec falls back to unsecured communication, if necessary. Again, you might need to limit the IP filter list to a minimal scope. Otherwise, if negotiation fails for any reason, any communications affected by the rule in which this filter action resides could result in data being sent without security. If you are concerned about unsecured communication, you might consider disabling this setting. However, communication with computers that cannot initiate IPSec, such as legacy systems, might be blocked. This option should be disabled for secure computers connected to the Internet.

    • Generation of session keys from new keying material (Use session key perfect forward secrecy (PFS))

      • This determines whether existing master key keying material can be used to derive a new session key. Enabling session key perfect forward secrecy (PFS) ensures that master key keying material cannot be used to derive more than one session key. When session key PFS is enabled, a new Diffie-Hellman key exchange is performed to generate new master key keying material before the new session key is created. Session key PFS does not require main mode reauthentication and uses less resources than master key PFS.

For information about how to configure filter actions, see To add, edit, or remove filter actions.

IPSec security methods

Each security method defines the security requirements of any communications to which the associated rule applies. Creating multiple security methods increases the chance that a common method can be found between two computers. The IKE component reads the list of security methods in descending order and sends a list of allowed security methods to the other peer. The first method in common is selected. Typically, the most secure methods are placed at the top of the list and the least secure methods are placed at the bottom of the list.

Predefined security methods

The following security methods are predefined:

  • Encryption and Integrity

    • Uses the ESP protocol to provide data confidentiality (encryption) with the triple Data Encryption Standard (3DES) algorithm, data integrity and authentication with the Secure Hash Algorithm 1 (SHA1) integrity algorithm, and default key lifetimes (100MB, 1 hour). If you require both data and addressing (IP header) protection, you can create a custom security method. If you do not require encryption, use Integrity only.

  • Integrity only

    • Uses the ESP protocol to provide data integrity and authentication with the SHA1 integrity algorithm and default key lifetimes (100MB, 1 hour). In this configuration, ESP does not provide data confidentiality (encryption). This is appropriate when your security plan calls for standard levels of security.

Custom security methods

If the predefined Encryption and Integrity or Integrity only settings do not meet your security requirements, you can specify custom security methods. For example, you can use custom methods when encryption and address integrity, stronger algorithms, or key lifetimes need to be specified. When configuring a custom security method, you can configure the following:

  • Security protocols

    • Both AH (Data and address integrity without encryption) and ESP (Data integrity and encryption) can be enabled in a custom security method when you require IP header integrity and data encryption. If you chose to enable both, you do not need to specify an integrity algorithm for ESP. The algorithm that you select for AH provides integrity.

  • Integrity algorithm
    • Message Digest 5 (MD5), which uses a 128-bit key.
    • Secure Hash Algorithm 1 (SHA1), which uses a 160-bit key.
  • Encryption algorithm
    • 3DES is the most secure of the DES combinations, and somewhat slower in performance. 3DES processes each block three times, using three unique 56-bit keys.
    • DES is used when the higher security and overhead of 3DES is not necessary. DES uses a single 56-bit key.
  • Session key settings

    • Session key settings determine when a new key is generated, rather than how it is generated. You can specify a lifetime in kilobytes, seconds, or both. For example, if the communication takes 10,000 seconds and you specify the key lifetime as 1000 seconds, 10 keys will be generated to complete the transfer. This ensures that, even if an attacker manages to determine one session key and decipher part of a communication, deciphering the entire communication is not possible. By default, new session keys are generated for every 100 MB of data or every hour. Note that any time a key lifetime is reached, the quick mode SA is also renegotiated in addition to the key refresh or regeneration.

Note

  • When IPSec provides data confidentiality between a XOX or Windows Server 2003 family-based computer (or a Windows 2000-based computer that has the High Encryption Pack or Service Pack 2 or higher installed) and another computer that does not use 3DES, your list of security methods must include DES. Otherwise, the computer that you are trying to communicate with might not be able to reach a common security agreement with your computer. If confidentiality of the data is not required, you can select ESP format with an integrity algorithm only and set the encryption to <None>. Alternately, you can select an AH format with an integrity algorithm.

For information about how to configure security methods, see To add, edit, or remove IPSec security methods.


Authentication methods

Each rule defines a list of authentication methods. Each authentication method defines the requirements for how identities are verified in communications to which the associated rule applies. The two peers must have at least one common authentication method or communication will fail. Creating multiple authentication methods increases the chance that a common method between two computers can be found.

Only one authentication method can be used between a pair of computers, regardless of how many are configured. If you have multiple rules that apply to the same pair of computers, you must configure the authentication methods list in those rules to enable the pair to use the same method. For example, if a rule between a pair of computers specifies only Kerberos for authentication and filters only TCP data and, in another rule, specifies only certificates for authentication and filters only UDP data, authentication will fail.

Overview of authentication methods

For authentication, IPSec allows you to use the Kerberos V5 protocol, certificate-based authentication, or preshared key authentication.

  • The Kerberos V5 security protocol is the default authentication technology. This method can be used for any clients running the Kerberos V5 protocol (whether or not the clients are running Windows 2000, XOX, or a Windows Server 2003 operating system) that are members of the same or trusted domains.
  • A public key certificate should be used in situations that include Internet access, remote access to corporate resources, external business partner communications, or computers that do not run the Kerberos V5 security protocol. This requires that at least one trusted certification authority (CA) and associated certificate have been configured. Computers running Windows 2000, XOX, or a Windows Server 2003 operating system support X.509 Version 3 certificates, including computer certificates generated by commercial CAs.
  • A preshared secret key can be specified. It is simple to use and does not require the client to run the Kerberos V5 protocol or have a public key certificate. Both parties must manually configure IPSec to use this preshared key.

    • Important

    • The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment.

Notes

  • The ability to authenticate using Kerberos V5 is not supported on computers running XOX or computers running any other version of Windows 2000, XOX, or a Windows Server 2003 operating system that are not a member of an Active Directory domain.
  • For information about how to configure authentication methods, see To define IPSec authentication methods.

IPSec certificate to account mapping

When you use either Kerberos V5 or certificate authentication, you can set restrictions on which computers are allowed to connect. When you enable IPSec certificate to account mapping, the IKE protocol associates (maps) a computer certificate to a computer account in an Active Directory domain or forest, and then retrieves an access token, which includes the list of user rights assigned to the computer. You can restrict access by configuring Group Policy security settings and assigning either the Access this computer from the network user right or the Deny access to this computer from the network user right to individual or multiple computers as needed.

When a Windows 2000 or a Windows Server 2003 family certification authority is used for computer certificate autoenrollment, the certificates are automatically stored in Active Directory as a property of the computer account. Alternatively, you can use a non-Microsoft certification authority in which to enroll computers. If you do not use computer certificate autoenrollment, you must manage certificate to account mapping manually.

Notes

  • If you use the Kerberos V5 protocol for authentication, the IKE protocol verifies access controls, but only if IKE is a responder. Therefore, for access controls to be applied, you must configure the IPSec policy on the client computer to initiate IKE to the server. If you use certificate authentication, access controls are applied if either the server or the client computer initiates IKE to the server.
  • For information about configuring security settings in Group Policy, see Security Settings Overview.


Tunnel endpoint

As discussed in Virtual private networking with IPSec, IPSec can perform layer 3 tunneling for scenarios in which Layer Two Tunneling Protocol (L2TP) cannot be used. If you are using L2TP for remote communications, no tunnel configuration is needed because the client and server VPN components of XOX and the Windows Server 2003 family automatically create the appropriate rules to secure L2TP traffic.

To create a layer 3 tunnel using IPSec, use the IP Security Policy Management or Group Policy consoles to configure and enable the following two rules for the appropriate policy:

  1. A rule for the outbound traffic for the tunnel.

    2. The rule for the outbound traffic is configured with both a filter list that describes the traffic to be sent across the tunnel and a tunnel endpoint of an IP address configured on the IPSec tunnel peer (the computer or router on the other side of the tunnel).

  2. A rule for the inbound traffic for the tunnel.

    3. The rule for the inbound traffic is configured with both a filter list that describes the traffic to be received across the tunnel and a tunnel endpoint of a local IP address (the computer or router on this side of the tunnel).

For each rule, you must also specify filter actions, authentication methods, and other settings.

For information about how to configure IPSec tunnel endpoints, see To specify an IPSec tunnel. For more information about IPSec tunnel mode, see Tunnel mode.

For advanced information about configuring IPSec tunnels, see article Q252735, "How to Configure IPSec Tunneling in Windows 2000," in the Microsoft Knowledge Base.(http://search.support.microsoft.com/kb/)


Connection type

For each IPSec rule, you can define which connection types on your computer are affected. The connection types apply to all connections in Network Connections on the computer for which you are configuring IPSec policy.

Each rule has one connection type setting:

  • All Network Connections. The rule applies to communications sent through any of the network connections that you have configured on the computer.
  • Local Area Network (LAN). The rule only applies to communications sent through LAN connections that you have configured on the computer.
  • Remote Access. The rule only applies to communications sent through any virtual private network (VPN) or dial-up connections that you have configured on the computer.

For information about how to configure connection types, see To specify IPSec connection types.

For information about configuring network connections on a computer, see Configuring a network or dial-up connection.


Using Internet Protocol security

This section covers:

  • Managing IPSec from the command line
  • Designing and planning IPSec policies
  • Setting up an IPSec test lab
  • Viewing IPSec statistics and details about active IPSec policies


Managing IPSec from the command line

The Netsh commands for Internet Protocol security (IPSec) provide a fully equivalent alternative to the console-based management and diagnostic capabilities provided by the IP Security Policy Management and IP Security Monitor consoles. You can use Netsh commands for IPSec to script IPSec policy creation, display details about IPSec policies, and change the IPSec configuration for troubleshooting. In addition, administering IPSec from the command line is useful when you want to extend the security and manageability of IPSec. For example, you can use Netsh commands for IPSec to enable IPSec driver event logging, set default traffic exemptions, and configure computer startup security.

For a comprehensive netsh ipsec command reference (including syntax and parameters), see Netsh commands for Internet Protocol security.

Notes

  • For information about how to use Netsh commands for IPSec to change the IPSec configuration for troubleshooting, see Using Netsh to change the IPSec configuration on computers running the Windows Server 2003 family.
  • You can use the Netsh commands for IPSec to configure IPSec policies only on computers running members of the Windows Server 2003 family. To use the command line to configure IPSec policies on computers running XOX, use Ipseccmd.exe. To use the command line to configure IPSec policies on computers running Windows 2000, use Ipsecpol.exe, which is provided with the Windows 2000 Server Resource Kit. For more information, see Troubleshooting tools.


Designing and planning IPSec policies

This section provides information about planning and deploying IPSec in your organization. You should also review Best practices for general information about IPSec deployment.

For information on specific topics, see:

  • Establishing an IPSec security plan
  • Default IPSec policies
  • Creating, modifying, and assigning IPSec policies
  • Special IPSec considerations
  • Troubleshooting tools


Establishing an IPSec security plan

Whether for a large domain or a small workgroup, implementing IPSec means finding a balance between making information easily available to the largest number of users, and protecting sensitive information from unauthorized access.

Finding the proper balance requires:

  • Assessing the risk and determining the appropriate level of security for your organization.
  • Identifying valuable information.
  • Defining security policies that use your risk management criteria and protect the identified information.
  • Determining how the policies can best be implemented within the existing organization.
  • Ensuring that management and technology requirements are in place.
  • Providing all users with both secure and efficient access to the appropriate resources, according to their needs.

Security considerations are also influenced by the way the computer will be used. For example, the required security might differ, depending on whether the computer is a domain controller, Web server, remote access server, file server, database server, or intranet or remote client. The security framework in Windows 2000, XOX, and Windows Server 2003 family is designed to fulfill the most stringent security requirements. However, software alone might be less effective without careful planning and assessment, effective security guidelines, enforcement, auditing, and sensible security policy design and assignment.

There is no exact definition of the measures that define standard security. These can vary widely, depending on an organization's policies and infrastructures. The following security levels can be considered a general basis for planning your IPSec deployment:

  • Minimal security

    • Computers do not exchange sensitive data. IPSec is not active by default. No administrative action to disable IPSec is required.

  • Standard security

    • Computers, especially file servers, are used to store valuable data. Security must be balanced so it does not become a barrier to users trying to perform their tasks. XOX and the Windows Server 2003 family provide example IPSec policies that secure data, but do not necessarily require the highest level of security: Client (Respond Only) and Server (Request Security). Use these as a basis for your IPSec policies. These, or similar custom policies, optimize efficiency without compromising security.

  • High security

    • Computers that contain highly sensitive data are at risk for data theft, accidental or malicious disruption of the system (especially in remote dial-up scenarios), or any public network communications. Secure Server (Require Security), a default policy, requires IPSec protection for all traffic being sent or received (except initial inbound communication) with stronger security methods. Unsecured communication with a non-IPSec-aware computer is not allowed.

From planning to policy configuration

After you have identified the types of traffic that need to be secured and the level of security required, you can begin policy configuration. XOX and Windows Server 2003 IPSec policy configuration is the translation of your security requirements to one or more IPSec policies, only one of which can be assigned at each of the following levels: domain, site, organizational unit, or local levels. Each IPSec policy consists of one or more IPSec rules. Each IPSec rule consists of:

  • A selected filter list.
  • A selected filter action.
  • Selected authentication methods.
  • A selected connection type.
  • A selected tunnel setting.

You can configure IPSec policy in the following ways:

  • Create a new policy and define the set of rules for the policy, adding filter lists and filter actions as required.

    • In this method, an IPSec policy is created first and then rules are added and configured. Filter lists (specifying traffic types) and filter actions (specifying how the traffic is treated) are added during rule creation.

    For more information, see To add, edit, or remove IPSec policies and To add, edit, or remove IPSec rules.

  • Create the set of filter lists and filter actions, and then create the policies and add rules that combine the filter lists with filter actions.

    • In this method, the filter lists and the filter actions are configured first. Next, IPSec policies are created and rules are added that combine the appropriate filter list with the appropriate filter action. Additionally, authentication methods, connection types, and tunnel settings are specified.

    For information about how to individually configure IP filter lists and filter actions, see To manage multiple IP filter lists and To manage multiple filter actions.

For either configuration method, after the IPSec policies have been created, they must be assigned. For more information, see Creating, modifying, and assigning IPSec policies.

Notes

  • To manage Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory, or you must have been delegated the appropriate authority. To manage local or remote IPSec policies for a computer, you must be a member of the Administrators group on the local or remote computer. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure.
  • The integration of IPSec policies with Active Directory described here does not apply to computers running XOX. You cannot administer Active Directory-based IPSec policy from a computer running XOX because these computers cannot join Active Directory domains.


Default IPSec policies

XOX and the Windows Server 2003 family provide a set of predefined IPSec filter lists and filter actions and default policies. This is only intended to provide an example. It is not designed for operational use without modification. A knowledgeable IPSec network administrator or advanced user should design new, custom polices for operational use.

Computers that are connected to the Internet must be carefully protected from network attacks with custom policies. The default policies are defined as examples for intranets because they allow a computer to receive unsecured traffic. The default policies are designed for computers that are members of an Active Directory domain.

Predefined filter lists

The following predefined filter lists are provided as examples for use in the default policies:

  • All ICMP Traffic

    • A filter list for all ICMP traffic (IP protocol 1) sent and received between this computer and all other computers.

  • All IP Traffic

    • A filter list for all IP traffic sent and received between this computer and all other computers. By default, all Internet Key Exchange (IKE) traffic is excluded. All other traffic types are matched against IPSec filters, however, and you can configure, block, or permit filter actions specifically for multicast and broadcast traffic (IPSec does not negotiate security associations for multicast and broadcast traffic).

Predefined filter actions

The following predefined filter actions are provided as examples for use in the default policies:

  • Permit

    • Traffic is permitted (the security method is set to Permit).

  • Request Security (Optional)

    • Traffic is secured (the security method is set to Negotiate security), however, the following exceptions apply:

    • Initial incoming unsecured communication is allowed. This behavior is known as inbound passthrough.
    • If security negotiation fails after 3 seconds, unsecured communication to a non-IPSec-aware computer is allowed. This filter action secures traffic to IPSec-aware computers, but allows unsecured communication if it is required. This behavior is known as fallback to clear.

    In order to avoid the potential for denial-of-service attacks, this filter action must not be used on the Internet.

    The following table lists the security methods (in preference order) for the Request Security (Optional) predefined filter action.

    Type | AH integrity | ESP confidentiality | ESP integrity | Key lifetimes (KB/sec)
Custom | None | 3DES | SHA1 | 100000/900
Custom | None | DES | SHA1 | 100000/900
Custom | SHA1 | None | None | 100000/300
Custom | MD5 | None | None | 100000/300

    The security negotiation for traffic using this filter action attempts ESP encryption (first 3DES, and then DES) with SHA1 integrity. If those security methods cannot be negotiated with the peer, AH integrity (first SHA1, and then MD5) only is attempted.

  • Require Security

    • Traffic is secured (the security method is set to Negotiate security), however, initial incoming unsecured communication is allowed (inbound passthrough is enabled). Unsecured communication with non-IPSec-aware computers is not allowed (fallback to clear is disabled). For example, an incoming connection request packet is allowed. The computer's reply to the connection request matches the outbound filter, causing a request for security to be sent to the sender of the unsecured traffic. If security negotiation is successful, traffic in both directions is required to be secure. If security negotiation is not successful, outgoing traffic is not allowed. The computer might continue to receive incoming unsecured traffic and will continue to attempt to negotiate security.

    In order to avoid the potential for denial-of-service attacks, this filter action must not be used on the Internet.

    The following table lists the security methods (in preference order) for the Require Security predefined filter action.

    Type | AH integrity | ESP confidentiality | ESP integrity | Key lifetimes (KB/sec)
Custom | None | 3DES | SHA1 | 100000/900
Custom | None | 3DES | MD5 | 100000/900
Custom | None | DES | SHA1 | 100000/900
Custom | None | DES | MD5 | 100000/900

    The security negotiation for traffic using this filter action will only attempt to negotiate ESP encryption (first 3DES, and then DES) with differing ESP authentication methods (first SHA1, and then MD5).

The most secure filter action is one that negotiates security and neither receives unsecured traffic (inbound passthrough is disabled) nor allows communication with non-IPSec-aware computers (fallback to clear is disabled). This is known as a lockdown filter action. When a server is using this filter action, clients must initiate secured communication before sending any IP traffic. The IPSec policy for client computers must be configured with additional rules that force the client to negotiate security with the server. A client configured with policy that consists of only the default response rule is not able to communicate with a locked down server.

Predefined rules

The only predefined rule that is available on all policies is the default response rule. The default response rule is automatically created in a new policy and cannot be created manually. It is used to respond to security requests when no other rule applies. For example, a client policy that enables a client to respond to security requests but does not secure any traffic by default, would have only the default response rule active. In some cases, it is enabled by default. You can disable the default response rule on the Rules tab in properties for a policy. You can also modify the authentication method and security methods of the default response rule for specific needs. For more information, see Default response rule.

Default policies

The following are provided as examples for use as default policies:

  • Client (Respond Only)

    • This is an example policy for computers that secure communication on request. For example, intranet clients might not require IPSec except when requested by another computer. This policy enables the computer on which it is active to respond to requests for secured communications. This policy contains the default response rule, which creates dynamic IPSec filters for inbound and outbound traffic based on the requested protocol and port traffic for the communication that is being secured.

    This policy has the following settings:

    • First rule (default response rule)
      • IP Filter List: <Dynamic>
      • Filter Action: Default Response
      • Authentication: Kerberos
      • Tunnel Setting: None
      • Connection Type: All
  • Server (Request Security)

    • This is an example policy for computers that should secure communications in most cases, allowing unsecured communication with non-IPSec-aware computers. With this policy, the computer accepts unsecured traffic, but always attempts to secure additional communications by requesting security from the original sender. This policy allows the entire communication to be unsecured if the other computer is not IPSec-enabled. For example, communication to specific servers can be secure while allowing the server to communicate in an unsecured manner to accommodate a mixture of clients (some of which support IPSec and some of which do not).

    This policy has a rule to request security for all IP traffic, a rule to permit ICMP traffic, and the default response rule to respond to requests for security from other computers.

    This policy has the following settings:

    • First rule
      • IP Filter List: All IP Traffic
      • Filter Action: Request Security (Optional)
      • Authentication: Kerberos
      • Tunnel Setting: None
      • Connection Type: All
    • Second rule
      • IP Filter List: All ICMP Traffic
      • Filter Action: Permit
      • Authentication: N/A
      • Tunnel Setting: None
      • Connection Type: All
    • Third rule (default response rule)
      • IP Filter List: <Dynamic>
      • Filter Action: Default Response
      • Authentication: Kerberos
      • Tunnel Setting: None
      • Connection Type: All
  • Secure Server (Require Security)

    • This is an example policy for computers on an intranet that require secure communications, such as a server that transmits highly sensitive data. Administrators can use this IPSec policy as an example to create their own custom IPSec policy for production use. The filters used in this policy require all outbound communication to be secured, allowing only the initial inbound communication request to be unsecured.

    This policy has a rule to require security for all IP traffic, a rule to permit ICMP traffic, and the default response rule to respond to requests for security from other computers.

    • First rule
      • IP Filter List: All IP Traffic
      • Filter Action: Require Security
      • Authentication: N/A
      • Tunnel Setting: None
      • Connection Type: All
    • Second rule
      • IP Filter List: All ICMP Traffic
      • Filter Action: Permit
      • Authentication: Kerberos
      • Tunnel Setting: None
      • Connection Type: All
    • Third rule (default response rule)
      • IP Filter List: <Dynamic>
      • Filter Action: Default Response
      • Authentication: Kerberos
      • Tunnel Setting: None
      • Connection Type: All

Notes

  • In the Windows Server 2003 family, IPSec provides new source and destination address options for defining filters. In addition, the default behavior for traffic exemptions has changed. If you require previously exempted traffic types, you must create filters to permit this traffic. For more information about the new source and destination options provided by IPSec and for instructions on how to configure permit filters for several previously exempted traffic types, see Special IPSec considerations.
  • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.


Creating, modifying, and assigning IPSec policies

IPSec policies can be applied to local computers, domains, sites, organizational units, or any Group Policy object in Active Directory. You can create, modify, and assign IPSec policies by using the IP Security Policy Management console that is available in Microsoft Management Console (MMC) or by using the Netsh commands for IPSec that are available at the command line.

Your organization's IPSec policies should be based on your written guidelines for secure operations. Policies can store multiple rules, so that one policy can be used to affect multiple types of traffic.

The Windows 2000, XOX, and Windows Server 2003 family include example IPSec policies. These examples are provided only to show how policies can be designed. They are not designed as default policies for securing a client or a server in a production environment. Instead, you should design and create custom IPSec policies that apply to the specific scenarios in your security plan.

Notes

  • For information about how to use the IP Security Policy Management console, see Define IPSec policies.
  • For information about how to use Netsh, see Netsh commands for Internet Protocol security.
  • Although you can assign local IPSec policies for a computer by using the IP Security Policy Management console, you must use Group Policy to assign IPSec policies for groups of computers in Active Directory domains. For more information, see To assign or unassign IPSec policy in Group Policy.

IPSec driver modes

To understand how IPSec policies are processed and applied, it is important to understand the modes in which the IPSec driver operates. The IPSec driver operates in computer startup mode and operational mode and, optionally, in a diagnostics mode for troubleshooting.

Computer startup mode

During computer startup, the Windows operating system loads the IPSec driver to perform in computer startup mode. Computer startup mode is used until the IPSec Policy Agent sets the IPSec driver into an operational mode. The IPSec driver can perform in any one of the following computer startup modes:

  • Permit. When the IPSec driver operates in permit mode, it does not process any IP packets, and, therefore, no IPSec security is provided. If an IPSec policy has never been assigned to a computer, permit mode is the default mode for the IPSec driver.
  • Stateful. When the IPSec driver operates in stateful mode, it allows all outbound traffic, and it automatically creates inbound permit filters in response to the outbound traffic. All other inbound unicast, broadcast, and multicast packets are dropped. If an IPSec policy has been assigned to a computer, the IPSec Policy Agent sets the stateful mode for the IPSec driver by default.
  • Block. When the IPSec driver operates in block mode, it discards all inbound and outbound IP packets, except for those that match specific filters that you can configure to be used during block mode. By default, all inbound and outbound DHCP traffic is permitted so that an IP address can be obtained. You can configure filters for block mode by using the netsh ipsec dynamic set config bootexemptions command.

To change the default mode for the IPSec driver during computer startup, use the netsh ipsec static set config bootmode command.

Operational mode

After the IPSec service starts, the IPSec Policy Agent can set the IPSec driver to any one of the following operational modes. The operational mode that is chosen does not change the mode that the IPSec driver uses during computer startup.

  • Secure. When the IPSec driver operates in secure mode, IPSec policy filters are enforced for standard IPSec operation. The IPSec Policy Agent sets the driver into secure mode after it applies persistent policy (if a persistent policy has been configured) and before it applies the Active Directory-based policy or the local policy. If no persistent policy has been configured and assigned, secure mode does not provide IPSec protection until either the assigned Active Directory-based policy or the local policy is applied. When the Active Directory-based policy or the local policy changes, the IPSec Policy Agent updates the IPSec filter list in the IPSec driver. If no IPSec policy is assigned, the IPSec Policy Agent deletes the filters from the IPSec driver and no IPSec protection is provided in this mode.
  • Permit. When the IPSec driver operates in permit mode, it does not process any IP packets, and, therefore, no IPSec protection is provided. Permit mode is used when the IPSec service is manually stopped.
  • Block. When the IPSec driver operates in this mode, the exemptions that might apply during computer startup are not applied. Instead, all inbound and outbound traffic is blocked. This mode is used to enhance security, in the event that the IPSec Policy Agent fails to apply a persistent policy, if a persistent policy has been configured.

The operational mode can be configured only by the IPSec Policy Agent. You cannot use the Netsh commands for IPSec to configure this mode.

Diagnostic mode

To record all inbound and outbound dropped packets and other packet processing errors in the Event Viewer System log for troubleshooting, you can enable IPSec driver event logging and specify the level of logging to provide. The types of packet processing errors that the IPSec driver records in the System log depend on the level of logging that is provided. IPSec driver logs can record inbound and outbound per-packet drop events during computer startup mode and operational mode.

IPSec driver event logging is disabled by default, and it should not be used for extended periods. Depending on the logging level that you set, many events might be generated that fill the System log very quickly. For more information, see Using Netsh to change the IPSec configuration on computers running the Windows Server 2003 family.

Application methods for IPSec policy

IPSec policies are provided in six ways, in the order described in the following sections.

IPSec startup policy

When a computer loads the Windows operating system during startup, the IPSec driver is loaded at the same time as the TCP/IP driver. The following table describes the behavior of the IPSec driver based on the startup type of the IPSec service.

IPSec service startup type | IPSec driver behavior
Disabled | The IPSec driver loads in permit mode, meaning that no IPSec security is applied. In this mode, the IPSec driver does not filter any traffic.
Manual | The IPSec driver loads in permit mode, meaning that no IPSec security is applied. In this mode, the IPSec driver does not filter any traffic.
Automatic

The IPSec driver loads in a startup mode configured by the IPSec Policy Agent. If no IPSec policy is assigned, by default the IPSec driver loads in permit mode. If IPSec policy is assigned, the IPSec Policy Agent configures the IPSec driver to load in stateful mode.

After the IPSec service starts, persistent policy (if it has been configured) is applied, and the IPSec driver is set to secure mode (normal operation).

When the IPSec driver is loaded, it reads the registry to determine the actions that it must perform during computer startup. These actions include:

  • Whether to apply stateful filtering, permit all traffic, or block all traffic.
  • Whether to exempt specific traffic types from filtering during computer startup.
  • Whether to enable IPSec diagnostic logging and, if so, which level of logging to provide.
  • Whether to modify the default IPSec traffic exemption. By default, in the Windows Server 2003 family, RSVP, Kerberos, multicast, and broadcast traffic are not exempt from IPSec filtering. Only IKE traffic is exempt.

The IPSec driver remains in a startup mode until the IPSec Policy Agent sets it to an operational mode. After the IPSec driver is set to secure operation mode, the startup stateful mode settings are discarded. You can use the Netsh commands for IPSec to configure IPSec driver settings. For more information, see the netsh ipsec dynamic set config section in Netsh commands for Internet Protocol security.

Persistent policy

In the Windows Server 2003 family, you can use the Netsh commands for IPSec to add, modify, and maintain IPSec policy settings permanently, after you run the appropriate commands. This type of policy is known as a persistent policy. You can configure persistent policies to extend existing Active Directory-based or local IPSec policies, override Active Directory-based or local IPSec policies, and enhance security during computer startup. Persistent policies enhance security by providing a secure transition from computer startup to Active Directory-based IPsec policy enforcement.

Persistent policies provide the following two key benefits:

  • You can configure a persistent policy as your most restrictive IPSec policy. For example, the persistent policy can block all inbound traffic to the Internet IP address of a computer, except for ICMP traffic and communication with a particular management station. To complement the persistent policy, you can design an Active Directory-based policy or a local policy with additional rules that include more specific filters to allow additional communication types.
  • You can selectively enforce security rules that are required only for specific servers, due to unique network adapter configurations or program needs. For example, you can enforce security rules specifying that:
    • A specific port always be blocked.
    • Traffic that uses a specific network adapter be permitted without IPSec security, or only with IPSec security.
    • Unsecured traffic to a specific monitoring station be permitted.

Persistent policies, if configured, are stored in the local registry. The persistent store is loaded by the IPSec Policy Agent during service startup. After a persistent policy is successfully applied, the IPSec Policy Agent sets the IPSec driver to secure mode. If an error occurs when a persistent policy is applied, the IPSec Policy Agent removes the persistent policy settings from the IPSec driver and sets the IPSec driver to block operation mode for security. You can update a persistent policy at any time, as long as the IPSec service is running. However, changes in persistent policy are not active immediately. You must restart the IPSec service to load the new persistent policy settings.

If you have configured Active Directory-based policies, you can use a persistent policy as a tool to require that traffic to Active Directory always be secured by IPSec, including the retrieval of Active Directory-based IPSec policies. When an Active Directory-based or local policy is applied, those policy settings are added to the persistent policy settings.

To configure persistent policies, you must use the netsh ipsec static set store location=persistent command. To view the IPSec policy that is currently being applied, you can use either IP Security Monitor or the netsh ipsec dynamic show commands. For information about how to use Netsh, see Netsh commands for Internet Protocol security.

Important

  • To provide maximum protection against attacks during computer startup, it is highly recommended that you configure and assign a persistent policy. If you do not configure a persistent policy, no policy can be enforced by the IPSec driver until the IPSec Policy Agent retrieves and applies the assigned Active Directory-based or local IPSec policy.

Active Directory-based policy

A Group Policy object defines access, configuration, and usage settings for accounts and resources. IPSec policies can be assigned to the Group Policy object of a site, domain, or organizational unit. When the IPSec policy is applied to one of the Group Policy objects for the Active Directory object, the IPSec policy is propagated to all of the computer accounts that are affected by that Group Policy object.

You can manage Active Directory-based policy by using the IP Security Policy Management console or by using the netsh ipsec static set store location=domain command. IPSec policies that are configured and assigned for the domain take precedence over the local, active IPSec policy on a computer, when that computer is a member of the domain. Active Directory-based policy overrides any local IPSec policy that is assigned, and it adds to the persistent IPSec policy that has already been applied by the IPSec Policy Agent, if a persistent policy has been configured.

Important

  • Several features in the Windows Server 2003 family implementation of IPSec are not provided in Windows 2000 or in XOX. To ensure that the same IPSec policy functions as expected on computers running the Windows Server 2003 family and on computers running Windows 2000 or XOX, test the policy thoroughly on all relevant operating systems before deployment. For more information about new features in IPSec, see New features for IPSec.
  • If you plan to apply IPSec policies that use the new features that are available only in the Windows Server 2003 family implementation of IPSec, do not use the Windows 2000 or the XOX version of the IP Security Policy Management console to manage these policies. The settings in the earlier versions of the IP Security Policy Management console will override the settings in the Windows Server 2003 family IPSec policy, and the new features will not be functional.

When assigning an IPSec policy in Active Directory, consider the following:

  • The list of all IPSec policies is available to assign at any level in the Active Directory hierarchy. However, only a single IPSec policy can be assigned at a specific level in Active Directory.
  • An IPSec policy that is assigned to an organizational unit in Active Directory takes precedence over a domain-level policy for members of that organizational unit.
  • An IPSec policy that is assigned to the lowest-level organizational unit in the domain hierarchy overrides an IPSec policy that is assigned to a higher-level organizational unit, for member computers of that organizational unit.
  • An organizational unit inherits the policy of its parent organizational unit unless either policy inheritance is explicitly blocked or policy is explicitly assigned.
  • IPSec policies from different organizational units are never merged.
  • The highest possible level of the Active Directory hierarchy should be used to assign policies to reduce the amount of configuration and administration required.
  • An IPSec policy might remain active even after the Group Policy object to which it is assigned has been deleted. Because of this, you should unassign the IPSec policy before you delete the policy object. To prevent problems, use the following procedure:
    1. Unassign the IPSec policy in the Group Policy object.
    2. Wait 24 hours to ensure that the change is propagated.
    3. Delete the Group Policy object.

    If you delete the Group Policy object without following this procedure, computers in the Active Directory container to which the IPSec policy is assigned treat the IPSec policy as if it cannot be located and continue to use a cached copy.

  • You can use Group Policy backup and restore tools to store information about which policy is assigned to a Group Policy object. To ensure consistency, when you use these tools, make sure that you include IPSec policies. Because these tools do not store the IPSec policies themselves, however, you cannot use them to back up and restore the IPSec policies themselves. You can only back up and restore information about whether the IPSec policies are assigned to Group Policy objects and, if so, to which Group Policy objects. To back up and restore all of the IPSec policies in the IPSec container in Active Directory, use the Export Policies and Import Policies commands in the IP Security Policy Management console. For more information, see Export IPSec policies and Import IPSec policies.
  • Before assigning an IPSec policy to a Group Policy object, verify the Group Policy settings that are required for the IPSec policy. For example, if an IPSec policy requires certificate authentication, assign the Group Policy settings that allow computers to enroll for certificates (usually one or two days before you assign the IPSec policy that requires use of the computer certificate). In addition, you should test the certificate enrollment process and resolve any errors before assigning the IPSec policy.

The IPSec Policy Agent on a computer running XOX or a Windows Server 2003 operating system polls Active Directory for updates to the assigned IPSec policy. This polling does not detect a change in domain or organizational unit membership or the assigning or unassigning of a new IPSec policy. These events are detected when the Winlogon service polls for changes in Group Policy, which occurs by default every 90 minutes. The Winlogon service discovers these changes, notifies the IPSec Policy Agent, and the IPSec policy changes are applied.

For more information about Group Policy, see Group Policy overview.

Notes

  • The integration of IPSec policies with Active Directory described here does not apply to computers running XOX because you cannot administer Active Directory-based IPSec policy from a computer running XOX.
  • Careful planning is required when you design Active Directory-based IPSec policies for computers that are used the internal network and on other public or partner networks.
  • By default, IPSec policies are read from Active Directory when the connection to Active Directory is interpreted by Group Policy to be a slow link.

Directory cache policy

If you assign an Active Directory-based IPSec policy, a copy of the current policy is maintained in a cache in the local registry. If the computer to which the policy is assigned cannot connect to the domain, the cached copy of the Active Directory-based policy is applied. This cached policy adds to the settings of the persistent policy, if a persistent policy has been configured. Note that you cannot configure or manage the cached copy of an Active Directory-based IPsec policy.

Local computer policy

Each computer running XOX or a Windows Server 2003 operating system has exactly one local Group Policy object, often called the local computer policy. In using this local Group Policy object, Group Policy settings can be stored on individual computers regardless of whether they are members of an Active Directory domain. The local Group Policy object can be overwritten by Group Policy objects associated with sites, domains, or organizational units in an Active Directory environment. On a network without an Active Directory domain (a network that lacks a Windows 2000 or a Windows Server 2003 domain controller), the local Group Policy object settings determine IPSec behavior because they are not overwritten by other Group Policy objects.

You can manage local IPSec policies by using the IP Security Policy Management console or by using the netsh ipsec static set store location=local command. The settings of a local IPSec policy are added to the persistent policy, if a persistent policy has been configured. If an Active Directory-based IPSec policy is assigned, and if the computer is connected to an Active Directory domain, the settings of the Active Directory-based policy are applied instead.

For information about how to assign IPSec policy locally, see To assign or unassign IPSec policy on a computer.

Netsh dynamic mode policy

You can use the netsh ipsec dynamic commands to create, modify, and assign IPSec rules that take effect immediately and are not stored. If the IPSec service is stopped, either administratively or when the computer is restarted, these rules are lost. However, if you use the netsh ipsec dynamic set config commands to change the IPSec configuration, those settings are not lost.


Special IPSec considerations

The following special IPSec considerations can help simplify the administration of IPSec policies.

Recommended IPSec scenarios and IPSec scenarios that are not recommended

Following are recommended scenarios and scenarios that are not recommended for the Windows Server 2003 family implementation of IPSec.

Recommended IPSec scenarios

The Windows Server 2003 family implementation of IPSec is recommended for use as follows:

  • Packet filtering. IPSec provides limited firewall capabilities for end-systems. You can also use IPSec with Internet Connection Firewall and Routing and Remote Access to permit or block inbound or outbound traffic.
  • Securing host-to-host traffic on specific paths. You can use IPSec to provide mutual authentication and cryptographic protection for traffic between servers or other static IP addresses or subnets. For example, IPSec can secure traffic between domain controller sites or forests, or between Web servers and database servers.
  • Securing traffic to servers. You can use IPSec to require mutual authentication for all client computers that access a server. In addition, you can set restrictions on which computers are allowed to connect to a server running Windows Server 2003 family.
  • Using L2TP/IPSec tunneling for VPN connections. You can use the combination of the Layer Two Tunneling Protocol (L2TP) and IPSec (L2TP/IPSec) for all virtual private network VPN scenarios.
  • Using IPSec in tunnel mode for gateway-to-gateway tunnels. For interoperability with other routers, gateways, or end-systems that do not support L2TP/IPSec or PPTP VPN tunneling, you can use IPSec in tunnel mode for gateway-to-gateway tunnels.

IPSec scenarios that are not recommended

When using IPSec authentication and data protection features, the policy management model in Windows 2000, XOX, and Windows Server 2003 family lends itself best to server-to-server and client-to-server scenarios where one end-point has a static address. In large network deployments, when policy involves dynamic addresses at both end–systems, and in some mobility cases, policy management complexity can make IPSec deployment difficult. Because of this, IPSec is not recommended for use as follows:

  • Securing communication between domain members and their domain controllers. Because of the complexity of the IPSec policy configuration and management required for this scenario, using IPSec to secure communication between domain members and their domain controllers is not recommended.
  • Securing all traffic in a network. Configuring IPSec communications between all client computers and all servers in order to secure many or all computers in a network is difficult. IPSec cannot negotiate security for multicast and broadcast traffic. Traffic from real-time communications, applications that require ICMP, and peer-to-peer applications might be incompatible with IPSec. For these reasons, using IPSec to secure all traffic in a network is not recommended.
Additionally, the IPSec protocol and implementation has characteristics that require special consideration in other scenarios:
  • Securing traffic over wireless 802.11 networks. IPSec policies are not optimized for configuration for mobile clients. Because of this, IPSec is not recommended as the only method to secure traffic sent over wireless 802.11 networks. Instead, it is recommended that you use IEEE 802.1X authentication. 802.1X enhances security and ease of deployment by providing support for centralized user identification, authentication, dynamic key management, and accounting. In cases where clients roam between access points on the same network, IPSec can be used in combination with 802.11 and 802.1x. In cases where roaming causes the client IP address to change, IPSec security associations become invalid and new security associations are renegotiated.
  • Using IPSec in tunnel mode for remote access VPN connections. Using IPSec in tunnel mode is not recommended for remote access VPN scenarios. Instead, use L2TP/IPSec or PPTP for remote access VPN connections.

Managing policies that use new features available only in the Windows Server 2003 family

The following features are available only in the Windows Server 2003 family implementation of IPSec:

  • Support for new source and destination address options for defining filters. For example, you can now create:
    • Filters with a source address and destination address of Any IP Address.
    • Filters that use a non-standard subnet class as a source address or a destination address.
    • Filters that use a multicast or a broadcast address as a destination address.
    • Filters with a source address or a destination address that corresponds to the local IP configuration for the DNS servers, WINS servers, and the DHCP server and default gateway for the computer to which the filter applies.
  • The ability to exclude the name of the certification authority (CA) from certificate requests.
  • Certificate-to-account mapping for network access control.
  • Support for root CA names that are represented in extended ASCII character sets.
  • A stronger Diffie-Hellman group (2048-bit Diffie-Hellman key exchange).

Computers running XOX and Windows 2000 cannot use these new policy settings. If you plan to apply Active Directory-based IPSec policies that use features that are only available in the Windows Server 2003 family, use the Windows Server 2003 family version of the IP Security Policy Management console to manage them. If you use the XOX or Windows 2000 version of the IP Security Policy Management console to manage these policies, the earlier versions of the IP Security Policy Management console will delete the new feature settings.

In addition, if you want to apply the same IPSec policy to computers running different versions of Windows, test the policy to ensure that it functions as expected on both servers running Windows Server 2003 and computers running XOX or Windows 2000.

For information about new features available with the Windows Server 2003 family implementation of IPSec, see New features for IPSec. For more information about best practices for using IPSec, see Best Practices.

IP filters

Consider the following when configuring IP filters:

  • Use general filters when you want a single filter to apply to a group of computers. For example, when configuring the filter, use My IP address, Any IP Address, or a subnet address instead of specifying a specific computer's source and destination IP addresses.
  • Define filters that allow you to group and secure traffic from logically associated segments or subnets within your network.
  • Be aware that when you are viewing the IPSec policy, the order in which the filters apply is not in the order displayed. When the IPSec Policy Agent reads an IPSec policy, filters are processed into one ordered list, sorted from most to least specific. You can use the IP Security Monitor console to view filters sorted by weight. Filters that have the same weight might appear in any order relative to each other.

    • During policy updates, the IPSec Policy Agent recalculates the sorted order and updates the IPSec driver with the changes only. If a filter is deleted, or if the filter action is changed for a filter, all security associations related to that filter are deleted. As a result, some packets are lost when a new security association is negotiated. TCP connections, however, should immediately recover by retransmitting the lost packets.

  • If you are configuring an IPSec policy to secure Kerberos traffic, IPSec can only negotiate security associations if your authentication method does not use Kerberos. If Kerberos is required for authentication between domain members, special consideration must be given to the traffic required by Kerberos between domain members and domain controllers, and the following requirements must be met:
    • If a domain member and a domain controller are members of different domains, a two-way trust must exist between the domains or Internet Key Exchange (IKE) rekeys will fail.
    • If a client computer or a server in the domain uses Kerberos authentication to negotiate IPSec with all peers, you must create mirrored permit filters for all Kerberos-dependent traffic to all domain controllers. It is recommended that you create permit filters to exempt all traffic to each domain controller IP address in the domain member's domain. You do not need to create permit filters to exempt all traffic to all trusted domain controller IP addresses. Instead, create permit filters to exempt only Kerberos traffic and domain controller site location (LDAP) queries to all domain controller IP addresses in the trust path between the computers. If DNS service is not provided by the domain controller (for which you already permit all traffic), you might also need to exempt DNS traffic to the DNS server IP addresses so that DNS can be used for domain controller discovery.

      • To exempt these traffic types in the IPSec policy, create the following filters, each with a permit filter action:

      • All traffic to my domain controllers
        • Mirrored
        • Source Address = My IP Address
        • Destination Address = A specific IP Address (where A specific IP Address is the static IP address of each domain controller in the domain)
        • Protocol = Any

          • This filter permits LDAP site location queries and ICMP, DNS, and Kerberos traffic. In addition, it permits the remote procedure call (RPC) traffic and Group Policy downloads required to manage Active Directory-based IPSec policies.

      • Kerberos
        • Mirrored
        • Source Address = My IP Address
        • Destination Address = A specific IP Address (where A specific IP Address is the static IP address of each domain controller in trusted domains)
        • Protocol = UDP
        • Source port = Any
        • Destination port = 88

          • After you create this filter, create another filter with the same settings, but specify TCP for the protocol.

      • Domain controller site location (LDAP) queries
        • Mirrored
        • Source Address = My IP Address
        • Destination Address = A specific IP Address (where A specific IP Address is the static IP address of each domain controller in the domain)
        • Protocol = UDP
        • Source port = Any
        • Destination port = 389
      • DNS client queries
        • Mirrored
        • Source Address = My IP Address
        • Destination Address = DNS Servers <dynamic> (or A specific IP Address, where the IP address is the static IP address of each DNS server in the network)
        • Protocol = UDP
        • Source port = Any
        • Destination port = 53

          • After you create this filter, create another filter with the same settings, but specify TCP for the protocol.

        Caution

        • To minimize the vulnerability of your network to attacks, you should allow these exemptions only for the specific IP addresses of the domain controllers and the DNS servers.
    • When designing an IPSec policy to secure or block ICMP traffic, ensure that you plan carefully and test the policy before implementing it in a production environment. Be aware that ICMP is required for the following scenarios:
      • Outbound TCP connections that traverse networks with different link frame sizes require path maximum transmission unit (PMTU) discovery. For PMTU discovery to function correctly, inbound ICMP traffic from routers and other gateways must be received.
      • The tracert command requires ICMP traffic to determine the path taken to a destination. If IPSec protects ICMP traffic, however, tracert output shows only the destination computer--not the path of intermediate routers.
      • File sharing and other applications use ICMP traffic to determine whether a destination IP address can be reached.
      • Network logon and Group Policy measure ICMP response times to each domain controller IP address. This measurement is performed to both aid in the selection of a domain controller to service a network logon and determine whether to apply Group Policy settings over slow links.

      In some cases, even the three-second delay before a soft SA (an unsecured connection) is established might impact upper-layer communication that requires fast ICMP responses.

    • In Windows 2000 and XOX, by default, broadcast, multicast, Kerberos, Internet Key Exchange (IKE), and Resource Reservation Protocol (RSVP) traffic is exempt from filter matches. In the Windows Server 2003 family, broadcast, multicast, Kerberos, and RSVP traffic is not exempt from filter matches by default (only IKE traffic is exempt). Broadcast and multicast packets will be dropped if they match a filter with a filter action to negotiate security. By default, the Windows Server 2003 family provides limited support for filtering broadcast and multicast traffic. A filter with a source address of Any IP Address will match multicast and broadcast addresses. A filter with a source address of Any IP Address and a destination address of Any IP Address will match inbound and outbound multicast addresses. You can use such a filter to block all traffic. One-way filters that would be used to block or permit specific multicast or broadcast traffic, however, are not supported.

      • As a result of the change in default exemption behavior for the Windows Server 2003 family implementation of IPSec, you should verify the behavior of IPSec policies designed for Windows 2000 or XOX, and determine whether to configure explicit permit filters to permit specific traffic types. To restore the default Windows 2000 and XOX behavior for IPSec policies, you can use the netsh ipsec dynamic set config command, or you can modify the registry. For more information, see Using Netsh to change the IPSec configuration on computers running the Windows Server 2003 family.

      Caution

      • The Windows 2000 and XOX default exemption settings for IPSec are designed for corporate LAN environments with a low risk of attack. For this reason, you should only use the Windows 2000 and XOX default exemption settings when necessary for troubleshooting, in low-risk environments, or when you cannot solve program compatibility issues by configuring explicit filters in IPSec policies.

    Safe Mode with Networking

    If you start a computer in Safe Mode with Networking, consider the following:

    • The IPSec service cannot apply filters from an IPSec policy or negotiate IP security when the computer is in Safe Mode with Networking. As a result, when Directory Services Restore Mode operations are in progress, communication does not occur with remote domain controllers or network backup devices that enforce an IPSec policy to require IPSec.
    • When a computer loads the Windows operating system during startup, the IPSec driver is loaded at the same time as the TCP/IP driver. If an IPSec policy is assigned to the computer and the IPSec service is configured for automatic startup, by default, the IPSec driver startup mode is changed to perform stateful filtering. When stateful filtering is performed, all outbound traffic is permitted, but all inbound traffic is blocked, except for DHCP traffic and traffic that is sent in response to the outbound traffic initiated during computer startup. (The IPSec driver automatically creates inbound filters to permit responses to outbound traffic.) The IPSec service itself, however, does not start. As a result, no persistent, local, or Active Directory-based IPSec policies can be applied.
    • When the computer is in Safe Mode with Networking, you can allow the IPSec driver to continue to provide stateful filtering protection by configuring inbound exemptions for specific traffic. Or, you can force the IPSec driver to permit all traffic.
    • To configure inbound exemptions for specific traffic, restart the computer normally (not in Safe Mode), use the netsh ipsec dynamic set config bootexemptions command to specify the traffic types to exempt during computer startup, and then restart the computer. The new exemptions will take effect when the computer is restarted and will remain in effect until they are deleted or changed.
    • If you allow the IPSec driver to configure inbound exemptions for specific traffic, keep in mind that unless you configure the appropriate exemptions, the following will occur:
      • Inbound connections will be blocked to network services, such as Remote Desktop Connection and Remote Assistance, which open ports for receiving connections. To allow clients to use Remote Desktop Connection or Remote Assistance, you must configure an exemption to allow inbound traffic to TCP port 3389. To configure this exemption, use the following Netsh command:

        • netsh ipsec dynamic set config bootexemptions TCP:any:3389:inbound

      • Some outbound TCP connections, such as those requiring PMTU discovery, might not work. To enable TCP PMTU discovery, you must configure an exemption to allow inbound ICMP traffic. To configure this exemption, use the following Netsh command:

        • netsh ipsec dynamic set config bootexemptions ICMP:inbound

      Important

      • Modifying IPSec traffic exemptions from startup security (that is, modifying the bootexemptions= parameter) will overwrite all previous exemptions from startup security.

      For more information about how to specify traffic types to exempt during computer startup, see Netsh commands for Internet Protocol security.

      To force the IPsec driver to permit all inbound traffic while the computer is in Safe Mode with Networking, you must either configure the IPSec service for manual startup or disable it, and then restart the computer in Safe Mode with Networking.

    Filter actions

    Consider the following special requirements when configuring IP filter actions:

    • If there is specific traffic that you always want to block, such as traffic to and from either invalid addresses or specific TCP ports, create a blocking filter action with the Security Method set to Block.
    • If there is specific traffic that you always want to be unsecured, create a permit filter action with the Security Method set to Permit.
    • When you are configuring custom security methods for encrypted traffic, you should only set the ESP confidentiality selection to <None> when a higher layer protocol provides data encryption.
    • When you are planning Internet scenarios, such as IPSec tunneling between branch offices, consider using a list of security methods that specifies high levels of security (for example, 3DES only), short key lifetimes (less than 50 MB), and perfect forward secrecy (PFS) for the session keys. This helps protect against known key attacks.

    Note

    • Computers running Windows 2000 must have the High Encryption Pack or Service Pack 2 (or later) installed in order to use the 3DES algorithm. If a computer running Windows 2000 receives a 3DES setting, but does not have the High Encryption Pack or Service Pack 2 (or later) installed, the 3DES setting in the security method is set to the weaker DES, to provide some level of confidentiality for communication, rather than blocking all communication. However, you should only use DES as a fallback option if not all computers in your environment support the use of 3DES. Computers running XOX or a Windows Server 2003 operating system support 3DES and do not require installation of the High Encryption Pack.

    VPN connections using L2TP

    When you are using Layer Two Tunneling Protocol (L2TP) for your remote access or router-to-router VPN connections, certificates are the default authentication method for main mode security associations. Computer certificates must be installed either on both the VPN server and remote access client (for remote access connections), or on both the calling router and answering router (for router-to-router VPN connections). The simplest method for installing computer certificates is to configure your Windows 2000 and Windows Server 2003 domains to auto-enroll computer certificates for domain members. For more information, see Computer certificates for L2TP/IPSec VPN connections.

    When you are using the certificate authentication method for L2TP connections, the list of certification authorities (CAs) is not configurable. Instead, each computer in the L2TP connection sends a list of root CAs to its IPSec peer from which it accepts a certificate for authentication. The root CAs in this list correspond to the root CAs that issued computer certificates to the computer. For example, if Computer A was issued computer certificates by root CAs CertAuth1 and CertAuth2, it notifies its IPSec peer during main mode negotiation that it will accept certificates for authentication from only CertAuth1 and CertAuth2. If the IPSec peer, Computer B, does not have a valid computer certificate issued from either CertAuth1 or CertAuth2, IPSec main mode negotiation fails. Therefore, ensure one of the following before attempting an L2TP connection:

    • Both the VPN client and VPN server were issued computer certificates from the same CA.
    • Both the VPN client and VPN server were issued computer certificates from CAs that follow a certificate chain up to the same root CA.

    In general, the VPN client must have a valid computer certificate installed that was issued by a CA that follows a valid certificate chain from the issuing CA up to a root CA that the VPN server trusts. Additionally, the VPN server must have a valid computer certificate installed that was issued by a CA that follows a valid certificate chain from the issuing CA up to a root CA that the VPN client trusts.

    For example, if the VPN client was issued computer certificates by root CAs CertAuth1 and CertAuth2, it notifies the VPN server during main mode negotiation that it will accept certificates for authentication from only CertAuth1 and CertAuth2. If the VPN server does not have a valid computer certificate issued from a CA that follows a certificate chain to either CertAuth1 or CertAuth2, IPSec main mode negotiation fails.

    A single CA commonly issues computer certificates to all computers in an organization. Because of this, all computers within the organization both have computer certificates from a single CA and request certificates for authentication from the same single CA.

    XOX and Windows Server 2003 family VPN clients also support the use of preshared key authentication to create L2TP/IPSec connections.

    Important

    • The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing or as an interim measure in a production environment, when your public key infrastructure (PKI) is being deployed to obtain computer certificates, or when preshared key authentication is required by your VPN server. For long-term use in production environment, it is recommended instead that you use certificates or Kerberos V5.

    For a remote access VPN client running XOX or a Windows Server 2003 operating system, you can enable preshared key authentication and configure a preshared key in IPSec settings on the Security tab in the properties of a VPN connection in Network Connections.

    Domain controllers and DHCP, DNS, or WINS servers

    Before enabling IPSec for a computer that functions as a domain controller or DHCP, DNS, or WINS server, determine whether all of the clients are IPSec-enabled. If the IPSec policy assigned to the domain controller or DHCP, DNS, or WINS server is not configured to allow fall back to clear, which permits unsecured communication with non-IPSec-capable clients, secure negotiation will fail and access to these network services will be blocked.

    SNMP

    If you are using Simple Network Management Protocol (SNMP) and your SNMP management systems or agents include computers running Windows 2000, XOX, or a Windows Server 2003 operating system that are configured with IPSec policy, you can add a rule to the appropriate IPSec policy to prevent SNMP messages from being secured. The filter list must specify the source and destination addresses of the SNMP management systems and agents for all UDP traffic that uses ports 161 and 162. The filter action should be set to Permit, which passes through any traffic that matches the IP filter list.


Troubleshooting tools

This topic provides information about the following IPSec troubleshooting tasks and the troubleshooting tools that you can use to perform each of these tasks:

  • Viewing IPSec policy assignment information
  • Viewing details about the active IPSec policy and IPSec statistics
  • Viewing details about the active IPSec policy and IPSec statistics in IP Security Monitor
  • Verifying that security auditing is enabled
  • Viewing IPSec-related events in Event Viewer
  • Enabling detailed tracing for Internet Key Exchange (IKE) negotiations
  • Viewing IPSec and other network communication with Network Monitor
  • Using Netsh to change the IPSec configuration on computers running the Windows Server 2003 family
  • Using Ipseccmd.exe to manage and monitor IPSec on computers running XOX
  • Using Netdiag.exe to display IPSec information and to test and view network configuration
  • Disabling TCP/IP and IPSec hardware acceleration

Notes

  • For general recommended practices that can help you enhance security and minimize the potential for problems when deploying IPSec, see Best practices.
  • For the Windows Server 2003 family, the Netsh commands for IPSec replace and enhance the functionality provided by Netdiag.exe and Ipseccmd.exe. For information about how to use Netsh to script IPSec policy creation and to monitor IPSec activity, see Netsh commands for Internet Protocol security.
  • Ipseccmd.exe is a support tool that is provided only on the XOX CD. To install this tool, run the Setup.exe file from the Support\Tools folder. You can use this tool to manage and monitor IPSec policies only on computers running XOX. For more information about Ipseccmd.exe, see the Microsoft TechNet Web site. (http://www.microsoft.com/)
  • Netdiag.exe is available in the Windows Server 2003 family, XOX, and in Windows 2000, as follows:
    • In the Windows Server 2003 family, run the Suptools.msi file from the \Support\Tools folder on the Windows Server 2003 family CD (select the Complete setup option). Although you can still use this version of Netdiag.exe to obtain basic networking information that is not IPSec-specific, the Netsh commands for IPSec replace all IPSec-specific functionality.
    • In XOX, run the Setup.exe file from the \Support\Tools folder on the XOX CD (select the Complete setup option).
    • For Windows 2000, an updated version of Netdiag.exe is available for download from the Web. For more information, see "Windows 2000 Resource Kits," at the Microsoft Windows Resource Kits Web site. (http://www.microsoft.com/)

    For more information about Netdiag.exe, see Using Netdiag.exe to display IPSec information and to test and view network configuration.

Viewing IPSec policy assignment information

Viewing the name of an active IPSec policy and the name of the Group Policy object to which the active IPSec policy is assigned can be useful for troubleshooting policy precedence issues. The following table summarizes the tools that you can use to view this information for the Windows Server 2003 family, XOX, and Windows 2000.

On computers running Use these tools to view the name of the active IPSec policy Use these tools to view the name of the Group Policy object to which the active IPSec policy is assigned
Windows Server 2003 family
  • IP Security Monitor console.

    • For more information, see To view details about active IPSec policies.

  • Netsh, netsh ipsec static show gpoassignedpolicy command.

    • For more information, see Netsh commands for Internet Protocol security.
  • Resultant Set of Policy (RSoP) console.

    • For more information, see Use Resultant Set of Policy (RSoP) to view IPSec policy assignments.

  • Netsh, netsh ipsec static show gpoassignedpolicy command
    • .
XOX
  • IP Security Policy Management console (for local policies only).
  • Network Connectivity Tester (Netdiag.exe), netdiag /test:ipsec command.

    • For more information, see Using Netdiag.exe to display IPSec information and to test and view network configuration.
  • Netdiag.exe, netdiag /test:ipsec command.
Windows 2000
  • Netdiag.exe, netdiag /test:ipsec command.
  • In the properties for the relevant network connection, TCP/IP Properties/Advanced/Options/IPSec.

    • The assigned IPSec policy that is displayed in TCP/IP Properties is global. It is not specific to the connection.
  • Netdiag.exe, netdiag /test:ipsec command.
  • Gpresult.exe (Group Policy Results).
  • Gpotool.exe (Group Policy Verification Tool).

    • Gpresult.exe and Gpotool.exe are both available for download from the Web. For more information, see "Windows 2000 Resource Kits," at the Microsoft Windows Resource Kits Web site (http://www.microsoft.com/).

Notes

  • To view all IPSec policies that are available (but not necessarily assigned or applied) to computers, use the IP Security Policy Management console. For information about policy precedence and IPSec policy behavior in an Active Directory environment, see Active Directory-based policy.
  • For the Windows Server 2003 family, to determine which IPSec policies are assigned but are not being applied to IPSec clients, use the RSoP console.

    • The XOX implementation of the RSoP console does not support the display of IPSec policies. In addition, the gpresult /scope computer command does not display the Group Policy object that contains an IPSec policy assignment. For these reasons, you should use Netdiag.exe to view IPSec policy assignment information on computers running Windows 2000 or XOX. The netdiag /test:ipsec command displays the Group Policy object that contains the IPSec policy assignment, and the organizational unit to which the Group Policy object is assigned.

  • Gpotool.exe allows you to monitor the health of Group Policy objects on domain controllers. You can use this tool to check the consistency and replication of Group Policy objects and to display Group Policy object properties. Gpotool.exe is available only for computers running Windows 2000.

Viewing details about the active IPSec policy and IPSec statistics

The IP Security Policy Management console, Netdiag.exe, Gpresult.exe, and Gpotool.exe allow you to determine which IPSec policy has been assigned through Group Policy. After you verify this information, you might need to view details about the assigned IPSec policy and IPSec statistics (for example, filters, filter actions, and active security associations). The following table summarizes the tools that you can use to view an active IPSec policy and IPSec statistics for the Windows Server 2003 family,XOX, and Windows 2000.

On computers running Use these tools to display the active IPSec policy, IPSec statistics, or both Notes
Windows Server 2003 family
  • IP Security Monitor console.

    • For more information, see To view IP security statistics

  • Netsh, netsh ipsec dynamic show all command.
  • To view details about the active IPSec policy and IPSec statistics on a local or remote computer, you must be a member of the Administrators group on that computer.
XOX
  • IP Security Monitor console.
  • IPseccmd.exe, ipseccmd show all command.
  • To view details about the active IPSec policy and IPSec statistics for a local or remote computer, you must be a member of the Administrators group on that computer.
Windows 2000
  • Netdiag.exe, netdiag /test:ipsec /v /debug command.
  • Ipsecmon.exe.
  • To use the /debug option, you must be logged on as a member of the Administrators group on that computer. In addition, to view details about Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory.
  • Ipsecmon.exe only displays active outbound quick mode security associations (SAs).

Note

  • To remotely monitor IPSec on a computer that is running a different version of Windows than your computer, use Remote Desktop Connection. For example, if your computer is running the Windows Server 2003 family and you plan to remotely monitor IPSec on computers running Windows 2000 or XOX, use Remote Desktop Connection to gain remote access to these computers. For information about Remote Desktop Connection, see Remote Desktop Connection.

Viewing details about the active IPSec policy and IPSec statistics in IP Security Monitor

In XOX and the Windows Server 2003 family, IP Security Monitor is implemented as a Microsoft Management Console (MMC) snap-in, and it includes enhancements that allow you to view details about an active IPSec policy that is applied by the domain or locally, as well as quick mode and main mode statistics, and active IPSec SAs. IP Security Monitor also enables you to search for specific main mode or quick mode filters. To troubleshoot complex IPSec policy designs, you can use IP Security Monitor to search for all matches for filters of a specific traffic type.

  • For procedures on how to use IP Security Monitor, see Monitor IPSec activity.
  • For detailed information about IP security statistics, see Viewing main mode and quick mode statistics in IP Security Monitor.

Notes

  • You can use IP Security Monitor to monitor only computers that are running XOX or the Windows Server 2003 family. In addition, the computer that is being monitored must run the same version of the Windows operating system as the computer on which IP Security Monitor is running. To monitor IPSec on a computer that is running Windows 2000, use the ipsecmon command at the Windows 2000 command prompt on the computer that is being monitored.
  • For remote monitoring, you can use IP Security Monitor only to monitor computers that are running the same version of the Windows operating system. To remotely monitor IPSec on a computer that is running a different version of Windows than your computer, use Remote Desktop Connection. For information about Remote Desktop Connection, see Remote Desktop Connection.
  • If your computer is running the Windows Server 2003 family, and you plan to monitor IPSec on computers that are also running the Windows Server 2003 family, you can run IP Security Monitor or use the Netsh command-line tool remotely.

Verifying that security auditing is enabled

You can use Local Security Policy settings (for a local computer) or Group Policy Object Editor (for a domain) to verify that security auditing is enabled, so you can ensure that the success and failure of IKE negotiations is recorded. Auditing for IKE is supported in Windows 2000, XOX, and the Windows Server 2003 family (IKE uses the Logon Events category). In the Windows Server 2003 family, you can also enable auditing for the security policy database (SPD). SPD uses the Policy Change category. For more information, see To define or modify auditing policy settings for an event category.

Viewing IPSec-related events in Event Viewer

You can use Event Viewer to view the following IPSec-related events:

  • IKE events (negotiation success and failure) in the security log.

    • To view these events, enable success or failure auditing for the Audit logon events audit policy for your domain or local computer. The IKE event category is also used for auditing user logon events in services other than IPSec. For more information, see To define or modify auditing policy settings for an event category.

    When you enable success or failure auditing for the Audit logon events audit policy, IPSec records the success or failure of each main mode and quick mode negotiation and the establishment and termination of each negotiation as separate events. However, enabling this type of auditing can cause the security log to fill with IKE events. For example, for servers that are connected to the Internet, attacks on the IKE protocol can cause the security log to fill with IKE events. IKE events can also fill the security log for servers that use IPSec to secure traffic to many clients. To avoid this, you can disable auditing for IKE events in the security log by modifying the registry.

    • To disable auditing of IKE events in the security log, do the following:
      1. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Audit\DisableIKEAudits registry setting to a value of 1.

        2. The DisableIKEAudits key does not exist by default and must be created.

        Caution

        • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
      2. Do one of the following:
        • Restart the computer
        • Stop, and then restart the IPSec service by running the net stop policyagent and net start policyagent commands at the command prompt.

          • Note

          • Stopping and restarting the IPSec service can disconnect all of the computers that are using IPSec from the computer on which the IPSec service is stopped, and it can prevent further communication with that computer. If you restart the IPSec service immediately, TCP-based communication might resume, due to the retransmit behavior of TCP, after new IKE and IPSec SAs are established. For more information, see "Stopping and restarting the IPSec service," later in this topic.
  • IPSec policy change events in the Security log.

    • To view these events, enable success or failure auditing for the Audit policy change audit policy for your domain or local computer. For more information, see To define or modify auditing policy settings for an event category.

  • IPSec driver per-packet drop events in the System log.

    • In Windows 2000, XOX, and the Windows Server 2003 family, you can enable packet event logging for the IPSec driver by modifying the registry. The IPSec driver reads the registry during computer startup. In the Windows Server 2003 family, you can enable packet event logging for the IPSec driver by using the Netsh ipsec command-line tool. To enable logging of dropped inbound and outbound packets, specify a value of 7. For information about the other levels of IPSec driver event logging, see Notes.

    • To enable IPSec driver logging of dropped inbound and outbound packets by modifying the registry, do the following:
      1. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSec\EnableDiagnostics DWORD registry setting to a value of 7.

        2. Caution

        • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
      2. Restart the computer.
    • To enable IPSec driver logging of dropped inbound and outbound packets by using the Netsh IPSec command-line tool in the Windows Server 2003 family, do the following:
      1. At the command prompt, type:
        2. netsh ipsec dynamic set config ipsecdiagnostics 7
      2. Restart the computer.

    You can also change the interval for writing IPSec driver packet events to the System log. By default, the IPSec driver writes events to the System log once an hour or after a threshold for the number of events has been reached. For troubleshooting, you should set this interval to the minimum value, 60 seconds.

    • To change the interval for writing IPSec driver packet events by modifying the registry, do the following:
      1. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSec\LogInterval DWORD registry setting to 60 decimal.

        2. Caution

        • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
      2. Restart the computer.
    • To change the interval for writing IPSec driver packet events by using the Netsh ipsec command-line tool in the Windows Server 2003 family, do the following:
      1. At the command prompt, type netsh ipsec dynamic set config ipsecloginterval 60
      2. Restart the computer.

    The IPSec driver reads the registry during computer startup.

    Notes

    • By default, packet event logging for the IPSec driver is disabled (that is, the value is set to 0). When you enable packet event logging for the IPSec driver, you can specify any of the following values, to enable different levels of logging:
      • When 1 is specified, bad SPI packets (the total number of packets for which the Security Parameters Index or SPI was incorrect), IKE negotiation failures, IPSec processing failures, packets received with invalid packet syntax, and other errors are recorded in the System log. Unauthenticated hashes (with the exception of the "Clear text received when should have been secured" event) are logged as well.
      • When 2 is specified, inbound per-packet drop events are recorded in the System log.
      • When 3 is specified, level 1 and level 2 logging are performed. In addition, unexpected clear text events (packets that are sent or received in plaintext) are also recorded.
      • When 4 is specified, outbound per-packet drop events are recorded in the System log.
      • When 5 is specified, level 1 and level 4 logging are performed.
      • When 6 is specified, level 2 and level 4 logging are performed.
      • When 7 specified, all levels of logging are performed.

      For more information about IPSec driver event logging, see the Microsoft Windows Resource Kits Web site. (http://www.microsoft.com/).

    • You cannot configure audit policies on a computer running XOX. However, success and failure auditing for the Audit logon events and Audit policy change audit policies for the local computer are enabled by default.
    • Enabling auditing in the security log or IPSec driver diagnostics in the system log can cause these logs to fill with events quickly. Before you perform either of these tasks, you should do the following:
      • Ensure that the size of the log is at least 10 megabytes (MB).
      • Save the existing log to a file.
      • Clear all events in the log so that the log is empty.
      • Consider disabling auditing of IKE events in the security log by modifying the registry, if your computer hosts many simultaneous network connections.

Enabling detailed tracing for Internet Key Exchange (IKE) negotiations

Enabling audit logging for IKE events and viewing the events in Event Viewer is the fastest and simplest way to troubleshoot failed main mode or quick mode negotiations. However, some scenarios might require a more detailed analysis of the IKE main mode negotiation and quick mode negotiations for troubleshooting. You can enable tracing for IKE negotiations if the audit failure events do not provide enough information. The IKE tracing log is a very detailed log intended for troubleshooting IKE interoperability under controlled circumstances. Expert knowledge of the ISAKMP RFC 2408 and IKE RFC 2409 is required to interpret this log.

The IKE tracing log appears as the systemroot\Debug\Oakley.log file. The log has a fixed size of 50,000 lines and will overwrite as necessary. A new Oakley.log file is created each time the IPSec service is started and the previous version of the Oakley.log file is saved as Oakley.log.sav. When the Oakley.log file becomes full, the current file is saved as Oakley.log.bak, and a new Oakley.log file is created.

Because many IKE negotiations can occur simultaneously, you should minimize the number of negotiations and log for as short a period of time as possible to capture a more easily interpreted log.

Enabling and disabling the IKE tracing log in the Windows Server 2003 family

In the Windows Server 2003 family, you can enable or disable the IKE tracing log dynamically while the IPSec service is running by doing the following:

  • To enable the IKE tracing log, type the following at the command prompt:

    • netsh ipsec dynamic set config ikelogging 1

    This command creates the IKE tracing log file if it does not exist. If the file does exist, it appends logging information to the existing file.

  • To disable the IKE tracing log, type the following at the command prompt:

    • netsh ipsec dynamic set config ikelogging 0

Enabling the IKE tracing log in Windows 2000 and XOX

In Windows 2000 and XOX, you must enable IKE tracing by modifying the registry. For the changes to take effect, you must also stop and restart the IPSec service:

  • To enable the IKE tracing log in XOX and Windows 2000, do the following:
    1. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PolicyAgent\Oakley\EnableLogging DWORD registry setting to a value of 1.

      2. The Oakley key does not exist by default and must be created.

      Caution

      • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
    2. Stop and start the IPSec service by running the net stop policyagent and net start policyagent commands at the command prompt.
Stopping and restarting the IPSec service

Stopping and restarting the IPSec service can disconnect all of the computers that are using IPSec from the computer on which the IPSec service is stopped, and it can prevent further communication with that computer. However, if you restart the IPSec service immediately, TCP-based communication might resume, due to the retransmit behavior of TCP, after new IKE and IPSec SAs are established. If you leave the IPSec service stopped, any clients that used the default response rule to establish security will be unable to communicate with this computer for two hours.

To avoid losing Terminal Services connectivity for computers that are using IPSec over a Terminal Server session, you must stop and restart the IPSec service by using a single command line:

  • To stop and restart the IPSec service for computers over a Terminal Server session
    • At the command prompt, type the following:

      • net stop policyagent & net start policyagent

If you are restarting the IPSec service while an L2TP/IPSec VPN tunnel is connected, the tunnel will lose connectivity and must be reconnected when the IPSec service is restarted. A PPTP tunnel, however, does not use IPSec and therefore will stay connected if you stop the IPSec service.

If you are restarting the IPSec service on a computer that is running the Windows Server 2003 family or the Windows Server 2003 family and that is also running the Routing and Remote Access service, any IPSec configuration for L2TP will be lost and the L2TP tunnels will be disconnected. Therefore, you must stop and restart the Routing and Remote Access service, as well as the IPSec service.

  • To stop and restart the IPSec service and the Routing and Remote Access service, do the following:
    1. Stop the Routing and Remote Access service using the net stop remoteaccess command.
    2. Stop the IPSec service by using the net stop policyagent command.
    3. Start the IPSec service by using the net start policyagent command.
    4. Start the Routing and Remote Access service using the net start remoteaccess command.

Viewing IPSec and other network communication with Network Monitor

You can install and use Network Monitor to view IPSec and other network communication. Note that the version of Network Monitor that is provided with the Windows Server 2003 family can be used only to view the network traffic that is sent to or from the computer on which it is installed. To view network traffic that is sent to or from another computer and is routed through your computer (using the Routing and Remote Access service), you must use the Network Monitor component that is provided with Microsoft Systems Management Server.

The Network Monitor component that is provided with the Windows Server 2003 family includes parsers for the ISAKMP (IKE), AH, and ESP protocols. The Network Monitor parsers for ESP can parse inside the ESP packet only if null-encryption is being used and the full ESP packet is captured. Network Monitor cannot parse the encrypted portions of IPSec-secured ESP traffic when encryption is performed in software. However, if encryption is being performed by an IPSec hardware offload network adapter, the ESP packets are decrypted when Network Monitor captures them and as a result, can be parsed and interpreted into the upper-layer protocols. If you need to diagnose ESP software-encrypted communication, you must disable ESP encryption and use ESP-null encryption by changing the IPSec policy on both computers.

Notes

  • For information about how to install the version of Network Monitor that is provided with the Windows Server 2003 family, see To install Network Monitor.
  • For information about Systems Management Server, go to the Microsoft Web site.(http://www.microsoft.com/smsmgmt)

Using Netsh to change the IPSec configuration on computers running the Windows Server 2003 family

For computers running the Windows Server 2003 family, you can use the Netsh commands for IPSec to script IPSec policy creation, display details about IPSec policies, and change the IPSec configuration for troubleshooting.

For troubleshooting, you can use the Netsh commands for IPSec to change the following settings:

Disable Internet Key Exchange (IKE) certificate revocation list (CRL) checking

By default, in Windows 2000 CRLs are not checked during IKE certificate authentication. In XOX and the Windows Server 2003 family, CRLs are checked during IKE certificate authentication, but a fully successful check is not required for the certificate to be accepted. In some cases, failures during CRL processing might cause IKE to not accept the certificate. Or, the delay required for CRL checking might delay IKE negotiation enough to cause the connection attempt to time-out. To determine whether certificate authentication will be successful without CRL checking, you can disable IKE CRL checking. To do this, type the following at the command prompt:

netsh ipsec dynamic set config strongcrlcheck 0

This setting takes effect immediately. It does not require a restart.

Enable IPSec driver event logging

To record all inbound and outbound dropped packets and other packet processing errors in the Event Viewer System log, you can set the IPSec driver event logging level to 7. To do this, type the following at the command prompt:

netsh ipsec dynamic set config ipsecdiagnostics 7

This setting will not take effect until you restart your computer.

Notes

  • If you configured an IPSec filter action on a computer to block traffic, setting the IPSec driver event logging level to 7 on that computer might generate many events that fill the System log very quickly. Accordingly, you should set the IPSec driver event logging level to 7 only for testing.
  • All packets that are dropped by IPSec contribute to the Datagrams Received Discarded or Datagrams Outbound Discarded System Monitor counters for the IP object. IPSec itself does not provide a counter object. You can monitor IPSec performance counters manually by using IP Security Monitor.

Permit inbound and outbound traffic during computer startup

To permit inbound and outbound traffic during computer startup (before the IPSec service starts), you can use the following Netsh command:

netsh ipsec dynamic set config bootmode permit

This setting will not take effect until you restart your computer.

Note

  • If you start the computer in Safe Mode with Networking, the IPSec service will not start and cannot retrieve IPSec policy settings from Active Directory or the local registry, and persistent policy cannot be applied. As a result, inbound and outbound traffic will continue to be permitted until the computer is no longer in Safe Mode.

Exempt all broadcast, multicast, IKE, Kerberos, and RSVP traffic from IPSec filtering

In Windows 2000 and XOX, by default, the IPSec driver exempts all broadcast, multicast, IKE, Kerberos, and RSVP traffic from IPSec filtering. In the Windows Server 2003 family, only IKE traffic is exempt from IPSec filtering, and you can configure, block, or permit filter actions specifically for multicast and broadcast traffic (IPSec does not negotiate SAs for multicast and broadcast traffic). To restore the IPSec driver to the default Windows 2000 and XOX filtering behavior, you can use the following Netsh command:

netsh ipsec dynamic set config ipsecexempt 0

This setting will not take effect until you restart your computer.

Caution

  • The Windows 2000 and XOX default exemption settings for IPSec are designed for corporate LAN environments with a low risk of attack. For this reason, you should use only the Windows 2000 and XOX default exemption settings when necessary for troubleshooting, in low-risk environments, or when you cannot solve program compatibility issues by configuring explicit filters in IPSec policies.

Note

  • You can also exempt all broadcast, multicast, IKE, Kerberos, and RSVP traffic from IPSec filtering by modifying the registry as follows:
    1. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSEC\NoDefaultExempt DWORD registry setting to a value of 0.

      2. The NoDefaultExempt key does not exist by default and must be created.

      Caution

      • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
    2. Restart the computer.

Using Ipseccmd.exe to manage and monitor IPSec on computers running XOX

For computers running XOX, you can use the Ipseccmd.exe command-line tool to script IPSec policy creation and to display IPSec policy assignments, active SAs, and detailed IPSec policy settings, including filters, filter actions, and authentication methods. Ipseccmd.exe replaces and enhances the functionality provided by the Ipsecpol.exe (used in Windows 2000 for scripting IPSec policy creation) and the Netdiag.exe /test:ipsec /v /debug command (used in Windows 2000 to display configuration information and SAs for an active IPSec policy).

To install Ipseccmd.exe on a computer running XOX, run the Setup.exe file from the \Support\Tools folder on the XOX CD.

Notes

  • Ipseccmd.exe is provided only with XOX, and you can use this tool to manage and monitor IPSec policies only on computers running XOX.
  • To display all IPSec policy settings and statistics for diagnostics, use the ipseccmd show all command.

Using Netdiag.exe to display IPSec information and to test and view network configuration

You can use the Netdiag.exe command-line tool to display information about IPSec policies and statistics, report network configuration, and test basic networking capabilities and domain-based functionality. The following table describes how to install Netdiag.exe and provides information about functional changes relevant to IPSec for each version of this tool.

On computers running Use this method to install Notes
Windows Server 2003 family For the Windows Server 2003 family, Netdiag.exe is available for installation from the Windows Server 2003 family CD (run the Suptools.msi file from the \Support\Tools folder, and choose the Complete setup option.
  • Although you can still use Netdiag.exe to obtain basic networking information that is not IPSec-specific, for the Windows Server 2003 family, the netdiag /test:ipsec option has been removed, and the Netsh commands for IPSec replace all IPSec-specific functionality. To view details about IPSec policies, use either the netsh ipsec static show command or the netsh ipsec dynamic show command.
XOX For XOX, Netdiag.exe is available for installation from the XOX CD (run the Setup.exe file from the \Support\Tools folder, and choose the Complete setup option).
  • Displays information about IPSec policy assignments, including the name of the active IPSec policy, the name of the Group Policy object that assigned the policy, and the policy path.
  • The display of detailed IPSec policy information and IPSec statistics (provided by the /debug and /v options in Windows 2000) is not supported in XOX. Instead, to view details about IPSec policies, use the Ipseccmd.exe show command.
Windows 2000 For Windows 2000, an updated version of Netdiag.exe is available for download from the Web. For information, see "Windows 2000 Resource Kits," at the Microsoft Windows Resource Kits Web site (http://www.microsoft.com/).
  • Netdiag.exe was enhanced in Windows 2000 Service Pack 1 to display the number of bytes offloaded by the IPSec driver to a network adapter that is capable of IPSec hardware offload. The hardware offload statistics are Offloaded Bytes Sent and Offloaded Bytes Received.
  • To use the /debug option to view details about Active Directory-based IPSec policies, you must be a member of the Domain Admins group in Active Directory.

You can use the netdiag /v /l command to obtain networking information that is not IPSec-specific, for any platform. Typical uses of these options include:

  • Reporting the IP configuration and routing configuration for a computer with one command.
  • Testing WINS and DNS name resolution and consistency.
  • Reporting the build version of a computer and the hotfixes that are installed on that computer.
  • Testing the validity of domain membership, whether domain members can successfully contact domain controllers, and trust relationships.

The /l option generates a Netdiag.log file. This file is written to the folder in which Netdiag.exe is run.

Notes

  • To run all Netdiag.exe commands, you must be a member of the Administrators group on the local computer.
  • Each version of Netdiag.exe is customized to run on a different version of the Windows operating system (that is, Windows 2000, XOX, or the Windows Server 2003 family). You can only run Netdiag.exe on a computer that is running the Windows operating system for which Netdiag.exe has been specifically designed.
  • If you are running a version of Netdiag.exe that is designed for Windows 2000 or XOX in a Windows Server 2003 domain, Netdiag.exe might not report the correct operating system version of the domain.
  • If you run Netdiag.exe on a computer that is assigned an IPSec policy that affects most inbound and outbound IP traffic on that computer, Netdiag.exe might not report results correctly the first time it is run. This is because it might take a few seconds for IPSec to establish SAs to the many remote computers for which Netdiag.exe tests connectivity. If this occurs, wait five seconds, and then run Netdiag.exe again.

Disabling TCP/IP and IPSec hardware acceleration

Network adapters can accelerate IPSec processing by performing hardware offload of IPSec cryptographic functions (IPSec offload) and the calculation of TCP checksums (checksum offload). Such adapters might also process large TCP segments for very fast transmission (large-send offload). By default, when a network adapter driver that can perform hardware offload is enabled during the Plug and Play initialization process, the driver advertises this capability to TCP/IP and IPSec. TCP/IP and IPSec then offload tasks to the network adapter driver as appropriate.

If you use network adapters to perform hardware offload of IPSec cryptographic functions, you might need to verify that hardware acceleration is not causing problems with the packet processing that is performed by the network adapter. To do so, you can:

  • Disable hardware offload functions in the network adapter driver (if this function is supported by the driver). This task does not require you to restart the computer.
  • Disable TCP/IP and IPSec hardware acceleration. This task requires you to restart the computer.
  • Disable only IPSec hardware acceleration. This task requires you to restart the computer.
Disabling hardware offload functions in the network adapter driver

To determine whether you can disable hardware offload functions in the network adapter driver, do one of the following:

  • Open Device Manager, click Network Adapters, right-click the network adapter whose properties you want to view, and then click Properties. For more information, see To change network adapter settings
  • Open Network Connections, right-click the network adapter that you want to view, click Properties, and then, on the General tab, click Configure.
Disabling TCP/IP and IPSec hardware acceleration

To disable both TCP/IP and IPSec hardware acceleration by modifying the registry, do the following:

  1. Set the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\DisableTaskOffload DWORD registry setting to a value of 1.

    2. Caution

    • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
  2. Restart the computer.

If the DisableTaskOffload DWORD registry setting is set to a value of 0, and if the network adapter advertises hardware offload capabilities, IPSec and TCP/IP will attempt to offload the appropriate functions to the network adapter.

Disable only IPSec hardware acceleration

To disable only IPSec hardware acceleration by modifying the registry, do the following:

  1. Set the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\IPSEC\EnableOffload DWORD registry setting to a value of 0.

    2. Caution

    • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.
  2. Restart the computer.

Note

  • To help minimize the potential for hardware acceleration to cause problems with packet processing performed by the network adapter, make sure that you use the latest network adapter driver available from the vendor.


Setting up an IPSec test lab

This section provides information about how you can use six computers to create a test lab to configure and test the IPSec features in XOX and the Windows Server 2003 family. These instructions are designed to take you through a set of tasks, exposing you to IPSec in a variety of configurations. Beyond the set of tasks, you can use these instructions to create a functioning IPSec configuration. You can then use this configuration to experiment with IPSec features and functionality in XOX and the Windows Server 2003 family prior to deployment on your production network.

This section covers:

  • Setting up the infrastructure
  • IPSec test lab tasks

Note

  • XOX


Setting up the infrastructure

The infrastructure for the IPSec test lab network consists of six computers performing the following services:

  • A computer running a Windows Server 2003 operating system that is used as a domain controller, a Domain Name System (DNS) server, and a certification authority (CA). This computer is named DC1.
  • A computer running a Windows Server 2003 operating system that is used as a Web server. This computer is named IIS1.
  • A computer running a Windows Server 2003 operating system that is used as a file sharing server. This computer is named FS1.
  • A computer running a Windows Server 2003 operating system that is used as a VPN server. This computer, named VPN1, has two network adapters installed.
  • A computer running XOX that is used as a client. This computer is named CLIENT1.
  • Another computer running XOX that is used as a client. This computer is named CLIENT2.

The following illustration shows the configuration of the IPSec test lab.

IPSec test lab configuration

There are two network segments. One segment represents a corporate intranet and the other represents the Internet. All computers on the corporate intranet are connected to a common hub or Layer 2 switch. All computers on the Internet are connected to another, separate common hub or Layer 2 switch. Private IP addresses are used throughout the test lab configuration. The private network ID 172.16.0.0 is used for the intranet. The private network ID 10.0.0.0 is used for the simulated Internet.

Each computer is manually configured with an IP address, subnet mask, and DNS server IP address. Dynamic Host Configuration Protocol (DHCP) and Windows Internet Name Service (WINS) servers are not used.

The following sections describe the configuration for each of the computers in the test lab. To reconstruct this test lab, configure the computers in the order presented.

Note

  • XOX

DC1

DC1 is a computer running a Windows Server 2003 operating system that is providing several different services. DC1 is functioning as:

  • The domain controller for the testlab.microsoft.com domain.
  • The DNS server for the testlab.microsoft.com DNS domain.
  • The enterprise root CA for the testlab.microsoft.com domain.

Note

  • The domain name testlab.microsoft.com is used here for example purposes only. You can use any domain name in your test lab configuration.

To configure DC1 for these services, perform the following steps.

  1. Install a Windows Server 2003 operating system as a stand-alone server.
  2. Configure the TCP/IP protocol with the IP address of 172.16.0.1 and the subnet mask of 255.255.255.0. For more information, see To configure TCP/IP for static addressing.
  3. At the command prompt, run dcpromo for a new domain named testlab.microsoft.com in a new forest. Install the DNS service when prompted.
  4. Install the Certificate Services component as an enterprise root certification authority. For more information, see To install an enterprise root certification authority.
  5. Configure the testlab.microsoft.com domain for automatic enrollment of computer certificates. For more information, see To configure automatic certificate allocation from an enterprise CA.

IIS1

IIS1 is a computer running a Windows Server 2003 operating system and Internet Information Services (IIS). It is providing Web server services for intranet clients. To configure IIS1 as a Web server, perform the following steps:

  1. On DC1, add a computer account for the IIS1 computer. For more information, see To create a new computer account.
  2. Install a Windows Server 2003 operating system as a stand-alone server.
  3. Configure the TCP/IP protocol with the IP address of 172.16.0.2, the subnet mask of 255.255.255.0, and the DNS server IP address of 172.16.0.1. For more information, see To configure TCP/IP for static addressing and To configure TCP/IP to use DNS.
  4. Join IIS1 to the testlab.microsoft.com domain. For more information, see To join a domain.
  5. Install Internet Information Services (IIS).
  6. To determine whether the Web server is working correctly, open Internet Explorer on DC1. When prompted by the Internet Connection Wizard, configure the wizard for a LAN connection. In Internet Explorer, in Address, type http://IIS1.testlab.microsoft.com/win2000.gif. You should see a Windows 2000 graphic.

FS1

FS1 is a computer running a Windows Server 2003 operating system that is providing file and print sharing services.

To configure FS1 as a file server, perform the following steps:

  1. On DC1, add a computer account for the FS1 computer. For more information, see To create a new computer account.
  2. Install a Windows Server 2003 operating system as a stand-alone server.
  3. Configure the TCP/IP protocol with the IP address of 172.16.0.3, the subnet mask of 255.255.255.0, and the DNS server IP address of 172.16.0.1. For more information, see To configure TCP/IP for static addressing and To configure TCP/IP to use DNS.
  4. Join FS1 to the testlab.microsoft.com domain. For more information, see To join a domain.
  5. In Windows Explorer, share the root directory of Local Disk (C:) using the share name ROOT to the group Everyone with full access. For more information, see To share a folder or drive.

VPN1

VPN1 is a computer running a Windows Server 2003 operating system that is providing VPN server services for Internet-based VPN clients. To configure VPN1 as a VPN server, perform the following steps:

  1. On DC1, add a computer account for VPN1. For more information, see To create a new computer account.
  2. Install a Windows Server 2003 operating system as a stand-alone server.
  3. For the intranet local area connection, configure the TCP/IP protocol with the IP address of 172.16.0.4, the subnet mask of 255.255.255.0, and the DNS server IP address of 172.16.0.1. For more information, see To configure TCP/IP for static addressing and To configure TCP/IP to use DNS.
  4. For the Internet local area connection, configure the TCP/IP protocol with the IP address of 10.0.0.2 and the subnet mask of 255.255.255.0. For more information, see To configure TCP/IP for static addressing.
  5. Join VPN1 to the testlab.microsoft.com domain. For more information, see To join a domain.
  6. Configure and enable the Routing and Remote Access service. For more information, see To enable the Routing and Remote Access service. In the Routing and Remote Access Server Setup Wizard, select Virtual private network (VPN) server from the list of common configurations. When prompted for IP address assignment, select From a specified range of addresses and configure the range 172.16.0.248 to 172.16.0.255. Do not configure RADIUS authentication.

CLIENT1

CLIENT1 is a computer running XOX that is acting as an intranet client and member of the testlab.microsoft.com domain. CLIENT1 is initially configured and connected to the simulated intranet. Later, the CLIENT1 computer is moved to the simulated Internet. To configure CLIENT1 as an intranet client, perform the following steps:

  1. On DC1, add a computer account for CLIENT1. For more information, see To create a new computer account.
  2. On CLIENT1, install XOX as a workgroup computer.
  3. Configure the TCP/IP protocol with the IP address of 172.16.0.5, the subnet mask of 255.255.255.0, and the DNS server IP address of 172.16.0.1. For more information, see To configure TCP/IP for static addressing and To configure TCP/IP to use DNS.
  4. Join CLIENT1 to the testlab.microsoft.com domain. For more information, see To join a domain.

CLIENT2

CLIENT2 is a computer running XOX that is acting as an intranet client but is not a member of the testlab.microsoft.com domain and does not have local IPSec policy. In this configuration, CLIENT2 is acting as a computer running Windows 95 or Windows 98 (a non-IPSec-aware client). To configure CLIENT2 as an intranet client, perform the following steps:

  1. Connect CLIENT2 to the intranet network segment.
  2. On CLIENT2, install XOX as a workgroup computer.
  3. Configure the TCP/IP protocol with the IP address of 172.16.0.6, the subnet mask of 255.255.255.0, and the DNS server IP address of 172.16.0.1. For more information, see To configure TCP/IP for static addressing and To configure TCP/IP to use DNS.

Domain configuration

On DC 1, perform the following steps using Active Directory Users and Computers:

  1. Create a Lockdown organizational unit within the testlab.microsoft.com domain. For more information, see To create a new organizational unit.
  2. Move the FS1 computer account to the Lockdown organizational unit. For more information, see To move a computer account.
  3. Create a Secure organizational unit within the testlab.microsoft.com domain. For more information, see To create a new organizational unit.
  4. Move the IIS1 computer account to the Secure organizational unit. For more information, see To move a computer account.
  5. Create a Clients organizational unit within the testlab.microsoft.com domain. For more information, see To create a new organizational unit.
  6. Move the CLIENT1 computer account to the Clients organizational unit. For more information, see To move a computer account.
  7. Enable success and failure auditing for the Audit logon events and Audit policy change audit policies for the testlab.microsoft.com domain. For more information, see To define or modify auditing policy settings for an event category.

IPSec policy configuration

On DC1, perform the following steps using Group Policy:

  • Configure filter list
  • Configure filter actions
  • Configure IPSec policy
  • Assign IPSec policy
  • Propagate IPSec policy to domain members

For more information, see To start the IP Security Policy Management snap-in.

Configure filter list

  1. Create an IP filter list with the following settings:
    • Filter list name: All IP Traffic
    • Filter 1
      • Addressing: My IP address to Any IP address, mirrored
      • Protocol: Any IP protocol

For more information, see Filter list and To manage multiple IP filter lists.

Configure filter actions

  1. Create a filter action with the following settings:
    • Filter action name: Lockdown
    • Security methods
      • Negotiate security
      • Security method preference order:
        • Encryption and Integrity
      • Accept unsecured communication, but respond with IPSec - disabled
      • Allow unsecured communication with non-IPSec-aware computer - disabled
      • Session key perfect forward secrecy (PFS) - disabled

    For more information, see Filter action and To manage multiple filter actions.

  2. Create a filter action with the following settings:
    • Filter action name: Secure
    • Security methods
      • Negotiate security
      • Security method preference order:
        • Encryption and Integrity
      • Accept unsecured communication, but respond with IPSec - enabled
      • Allow unsecured communication with non-IPSec-aware computer - disabled
      • Session key perfect forward secrecy (PFS) - disabled

    For more information, see Filter action and To manage multiple filter actions.

  3. Create a filter action with the following settings:
    • Filter action name: Fallback
    • Security methods
      • Negotiate security
      • Security method preference order:
        • Encryption and Integrity
      • Accept unsecured communication, but respond with IPSec - enabled
      • Allow unsecured communication with non-IPSec-aware computer - enabled
      • Session key perfect forward secrecy (PFS) - disabled

    For more information, see Filter action and To manage multiple filter actions.

Configure IPSec policy

  1. Create an IPSec policy with the following settings:
    • Policy name: Lockdown (for Lockdown organizational unit)
      • Rule 1
        • IP Filter List: All IP Traffic
        • Filter Action: Lockdown
        • Authentication methods: Kerberos
        • Tunnel setting: This rule does not specify an IPSec tunnel.
        • Connection type: All network connections
      • Default response rule: disabled

    For more information, see Description of an IPSec policy and To add, edit, or remove IPSec policies.

  2. Create an IPSec policy with the following settings:
    • Policy name: Secure (for Secure organizational unit)
      • Rule 1
        • IP Filter List: All IP Traffic
        • Filter Action: Secure
        • Authentication methods: Kerberos
        • Tunnel setting: This rule does not specify an IPSec tunnel.
        • Connection type: All network connections
      • Default response rule: disabled

    For more information, see Description of an IPSec policy and To add, edit, or remove IPSec policies.

  3. Create an IPSec policy with the following settings:
    • Policy name: Fallback (for Fallback organizational unit)
      • Rule 1
        • IP Filter List: All IP Traffic
        • Filter Action: Fallback
        • Authentication methods: Kerberos
        • Tunnel setting: This rule does not specify an IPSec tunnel.
        • Connection type: All network connections
      • Default response rule: disabled

    For more information, see Description of an IPSec policy and To add, edit, or remove IPSec policies.

  4. Create an IPSec policy with the following settings:
    • Policy name: Default Response
      • Default response rule: enabled

    For more information, see Description of an IPSec policy and To add, edit, or remove IPSec policies.

Assign IPSec policy

Perform the following configuration from DC1 using Group Policy:

  1. For the Lockdown organizational unit, assign the Lockdown IPSec policy.
  2. For the Secure organizational unit, assign the Secure IPSec policy.
  3. For the Fallback organizational unit, assign the Fallback IPSec policy.
  4. For the Clients organizational unit, assign the Default Response IPSec policy.
  5. For the testlab.microsoft.com domain, assign the Default Response IPSec policy.

    6. The Default Response policy is assigned to the testlab.microsoft.com domain so that the VPN1 computer can respond to requests to negotiate IPSec communications if required.

  6. For the Domain Controllers organizational unit, assign the Default Response IPSec policy.

    7. The Default Response policy is assigned to the Domain Controllers organizational unit so that the DC1 computer can respond to requests to negotiate IPSec communications if required.

For more information, see To start the IP Security Policy Management snap-in and To assign or unassign IPSec policy in Group Policy.

Propagate IPSec policy to domain members

To immediately update changes in IPSec policy settings, rather than waiting for the next polling period, run the gpupdate /target:computer command at the Command Prompt on the following computers:

  • DC1
  • IIS1
  • FS1
  • CLIENT1
  • VPN1


IPSec test lab tasks

The following tasks are designed to take you through the most common elements of setting up IPSec support with XOX and the Windows Server 2003 family using the test lab infrastructure configured in Setting up the infrastructure.

  • Secured ping
  • Secure Web traffic
  • Communication with a lockdown server
  • Communication with a fallback server
  • L2TP/IPSec connection

Note

  • XOX

Secured ping

  1. From CLIENT1, ping CLIENT2. For more information, see To test a TCP/IP configuration by using the ping command.

    2. The ping traffic is not secured with IPSec because CLIENT1 and CLIENT2 do not have rules that require security.

  2. From CLIENT1, ping IIS1. For more information, see To test a TCP/IP configuration by using the ping command.

    3. Ping traffic is secured because IIS1 is in the Secure organizational unit, which uses the Secure IPSec policy, and CLIENT1 is in the Clients organizational unit, which uses the Default Response policy (where only the default response rule is enabled).

Secure Web traffic

  1. On CLIENT1, open Internet Explorer.
  2. When prompted, use the Internet Connection Wizard to configure CLIENT1 for a LAN connection. In Internet Explorer, in Address, type http://IIS1.testlab.microsoft.com/win2000.gif. You should see a Windows 2000 graphic.

    3. Web traffic is secured because IIS1 is in the Secure organizational unit, which uses the Secure IPSec policy, and CLIENT1 is in the Clients organizational unit, which uses the Default Response policy (where only the default response rule is enabled).

  3. Use Event Viewer on both IIS1 and CLIENT1 to view IPSec entries in the security log.
  4. Use IP Security Monitor on both IIS1 and CLIENT1 to view IPSec statistics and main mode and quick mode details. For more information, see To add the IP Security Monitor snap-in and Monitor IPSec activity.

Communication with a lockdown server

  1. On CLIENT1, at a Command Prompt, type dir \\FS1\ROOT.

    2. Communication fails because FS1 accepts only secured communication and CLIENT1, which is using the default response rule, does not initiate secured communication.

  2. On DC1, add a new filter list with the following settings:
    • IP Filter list: CLIENT1 to FS1
    • 172.16.0.5 to 172.16.0.3, mirrored
    • Any IP protocol

    3. This filter list specifies all traffic between CLIENT1 and FS1. For more information, see Filter list and To manage multiple IP filter lists.

  3. On DC1, add a rule to the Default Response policy:
    • IP Filter List: CLIENT1 to FS1
    • Filter Action: Lockdown
    • Authentication methods: Kerberos
    • Tunnel setting: This rule does not specify an IPSec tunnel.
    • Connection type: All network connections

    4. This rule ensures that all traffic between CLIENT1 and FS1 uses the Lockdown filter action. For more information, see Description of an IPSec policy and To add, edit, or remove IPSec policies.

  4. On CLIENT1, use the gpupdate /target:computer command to refresh IPSec policy.
  5. On CLIENT1, at a Command Prompt, type dir \\FS1\ROOT.

    6. Communication succeeds because CLIENT1 is now initiating secure communication.

Communication with a fallback server

  1. On CLIENT2, open Internet Explorer.
  2. When prompted, use the Internet Connection Wizard to configure CLIENT1 for a LAN connection. In Internet Explorer, in Address, type http://IIS1.testlab.microsoft.com/win2000.gif.

    3. Communication fails because IIS1 requires secured communication and CLIENT2 is simulating a non-IPSec-aware computer.

  3. On DC1, move the IIS1 computer account from the Secure organizational unit to the Fallback organizational unit. For more information, see To move a computer account.
  4. On IIS1, at a Command Prompt, use the gpupdate /target:computer command to refresh IPSec policy.
  5. On CLIENT2, open Internet Explorer. In Internet Explorer, in Address, type http://IIS1.testlab.microsoft.com/win2000.gif. You should see a Windows 2000 graphic.

    6. Communication succeeds because IIS1 now allows unsecured communication with non-IPSec-aware computers.

L2TP/IPSec connection

To create an L2TP-based remote access VPN connection between CLIENT1 and VPN1 and test the availability of intranet resources, perform the following steps:

1. Move CLIENT1 from the intranet to the Internet

    1. Configure the TCP/IP protocol with the IP address of 10.0.0.1, the subnet mask of 255.255.255.0, and no DNS server IP address. For more information, see To configure TCP/IP for static addressing and To configure TCP/IP to use DNS.
    2. Shut down the CLIENT1 computer.
    3. Disconnect CLIENT1 from the intranet network segment and connect it to the simulated Internet network segment.
    4. Restart the CLIENT1 computer and log on using the cached credentials of the testlab.microsoft.com administrator account.

2. Create a user account

    1. On DC1, use Active Directory Users and Computers to create a user account named L2TPUser and set the password. For more information, see To create a new user account.
    2. Set the remote access permission on the Dial-in tab to Allow access. For more information, see To grant remote access permission for a user.

3. Create the L2TP connection

    1. On CLIENT1, use the Make New Connection Wizard to create a new L2TP-based connection named L2TPtoCorpnet, using the IP address of 10.0.0.2. For more information, see To make a virtual private network (VPN) connection.
    2. Obtain properties on the newly created L2TPtoCorpnet connection, and then click the Networking tab.
    3. Under Type of VPN server I am calling, select Layer-2 Tunneling Protocol (L2TP), and then click OK.

4. Make the L2TP connection

    1. On CLIENT1, double-click the L2TPtoCorpnet connection.
    2. When prompted for credentials, type L2TPUser as the user name, testlab.microsoft.com as the domain name, and the password.

5. Access intranet resources

    1. From CLIENT1, ping CLIENT2. For more information, see To test a TCP/IP configuration by using the ping command.

      2. This should be successful.

    2. On CLIENT1, open Internet Explorer.
    3. In Internet Explorer, in Address, type http://IIS1.testlab.microsoft.com/win2000.gif.

      4. You should see a Windows 2000 graphic. This traffic is secured end-to-end from CLIENT1 to IIS1.

    4. On CLIENT1, at a Command Prompt, type dir \\FS1\ROOT.

      5. This should fail because FS1 is using a lockdown policy and the additional rule configured for CLIENT1's intranet IP address does not include the IP address that is assigned to CLIENT1 as a remote access client.

6. Use IP Security Monitor

    • On CLIENT1, open IP Security Monitor and view IPSec statistics, and main mode and quick mode details. For more information, see To add the IP Security Monitor snap-in.

7. Disconnect the L2TP connection

    • On CLIENT1, right-click the L2TPtoCorpnet connection, and then click Disconnect.


Viewing IPSec statistics and details about active IPSec policies

This section covers:

  • Viewing main mode and quick mode statistics in IP Security Monitor
  • Viewing details about active IPSec policies in IP Security Monitor
  • Using Resultant Set of Policy to view IPSec policy assignments


Viewing main mode and quick mode statistics in IP Security Monitor

Main Mode (IKE) Statistics

The following table describes the items in the list of main mode (IKE) statistics in IP Security Monitor.

For information about how to view these statistics, see To view IP security statistics.

Main Mode (IKE) Statistic | Description
Active Acquire | The number of pending requests to initiate an Internet Key Exchange (IKE) negotiation in order to establish a security association (SA) between IPSec peers. The Active Acquire statistic includes the outstanding request and the number of any queued requests. Under a heavy load, the number of active acquires is 1 plus the number of requests that are queued by IKE for processing.
Active Receive | The number of IKE messages received that are queued for processing.
Acquire Failures | The total number of acquire outbound requests that have failed since the IPSec service was last started. Acquires are requests to establish SAs between IPSec peers.
Receive Failures | The total number of errors that have occurred during the process of receiving IKE messages since the IPSec service was last started.
Send Failures | The total number of errors that have occurred during the process of sending IKE messages since the IPSec service was last started. The number of Send Failures typically increases for computers that establish SAs over temporary network connections, such as dial-up connections, virtual private network tunnels, and wireless connections.
Acquire Heap Size | The number of entries in the acquire heap. The acquire heap stores successful acquires. Acquires are outbound requests to establish SAs between IPSec peers.
Receive Heap Size | The number of entries in the IKE receive buffers. The receive buffers store incoming IKE messages.
Authentication Failures | The total number of identity authentication (Kerberos, certificate, and preshared key) failures that have occurred during main mode negotiation since the IPSec service was last started. If you are having difficulty communicating securely, attempt the communication and determine whether the number of Authentication Failures increases. If it does, check your authentication settings for either an unmatched authentication method or an incorrect authentication method configuration (for example, the use of preshared keys that do not match).
Negotiation Failures | The total number of negotiation failures that have occurred during main mode or quick mode negotiation since the IPSec service was last started. If you are having difficulty communicating securely, attempt the communication and determine whether the number of Negotiation Failures increases. If it does, check your authentication and security method settings for an unmatched authentication method, an incorrect authentication method configuration (for example, the use of preshared keys that do not match), or unmatched security methods or settings.
Invalid Cookies Received | The total number of cookies that could not be matched with an active main mode SA since the IPSec service was last started. A cookie is a value contained in a received IKE message that is used to help identify the corresponding main mode SA.
Total Acquire | The total number of requests that have been submitted to IKE since the IPSec service was last started to establish an SA. This number includes acquires that result in soft SAs.
Total Get SPI | The total number of requests that have been submitted by IKE to the IPSec driver to obtain a unique Security Parameters Index (SPI) since the IPSec service was last started. The SPI matches inbound packets with SAs.
Key Additions | The total number of outbound quick mode SAs that have been added by IKE to the IPSec driver since the IPSec service was last started.
Key Updates | The total number of inbound quick mode SAs that have been added by IKE to the IPSec driver since the IPSec service was last started.
Get SPI Failures | The total number of failed requests that have been submitted by IKE to the IPSec driver to obtain a unique SPI since the IPSec service was last started.
Key Addition Failures | The total number of failed outbound quick mode SA addition requests that have been submitted by IKE to the IPSec driver since the IPSec service was last started.
Key Update Failures | The total number of failed inbound quick mode SA addition requests that have been submitted by IKE to the IPSec driver since the IPSec service was last started.
ISADB List Size | The number of main mode state entries. This number includes successfully negotiated main modes, main mode negotiations in progress, and main mode negotiations that failed or expired and have not yet been deleted.
Connection List Size | The number of quick mode negotiations that are in progress.
IKE Main Mode | The total number of successful SAs that have been created during main mode negotiations since the IPSec service was last started.
IKE Quick Mode | The total number of successful SAs that have been created during quick mode negotiations since the IPSec service was last started.
Soft Associations | The total number of SAs formed with computers that have not responded to main mode negotiation attempts since the IPSec service was last started. Although these computers did not respond to main mode negotiation attempts, IPSec policy allowed communications with the computers. Soft SAs are not secured by IPSec.
Invalid Packets Received | The total number of invalid IKE messages that have been received since the IPSec service was last started. This number includes IKE messages with invalid header fields, incorrect payload lengths, and incorrect values for the responder cookie. Invalid IKE messages are commonly caused by retransmitted IKE messages or an unmatched preshared key between the IPSec peers.

Quick Mode (IPSec) Statistics

The following table describes the items in the list of quick mode (IPSec) statistics in IP Security Monitor.

Quick Mode (IPSec) Statistic | Description
Active Security Associations | The number of active quick mode SAs.
Offloaded Security Associations | The number of active quick mode SAs offloaded to hardware. Certain network adapters can accelerate IPSec processing by performing hardware offload of IPSec cryptographic functions.
Pending Key Operations | The number of IPSec key exchange operations that are in progress but are not yet completed.
Key Additions | The total number of keys for quick mode SA negotiations that have been successfully added since the computer was last started.
Key Deletions | The total number of keys for quick mode SAs that have been successfully deleted since the computer was last started.
Rekeys | The total number of successful rekey operations for quick mode SAs since the computer was last started.
Active Tunnels | The number of active IPSec tunnels.
Bad SPI Packets | The total number of packets for which the SPI has been incorrect since the computer was last started. If the SPI is incorrect, it might mean that the inbound SA has expired and a packet using the old SPI has recently arrived. This number is likely to increase if rekey intervals are short and there are a large number of SAs. A large number of packets with bad SPIs that are received within a short amount of time might indicate a packet spoofing attack.
Packets Not Decrypted | The total number of packets that could not be decrypted since the computer was last started. A packet might not be decrypted if it fails a validation check.
Packets Not Authenticated | The total number of packets for which data could not be verified (for which the integrity hash verification failed) since the computer was last started. Increases in this number might indicate an IPSec packet spoofing or modification attack or packet corruption by network devices.
Packets With Replay Detection | The total number of packets that have contained an invalid sequence number since the computer was last started. Increases in this number might indicate a network problem or replay attack.
Confidential Bytes Sent | The total number of bytes that have been sent using the Encapsulating Security Payload (ESP) protocol (excluding non-encrypted ESP) since the computer was last started.
Confidential Bytes Received | The total number of bytes that have been received using the ESP protocol (excluding non-encrypted ESP) since the computer was last started.
Authenticated Bytes Sent | The total number of authenticated bytes that have been sent using the Authentication Header (AH) protocol or the ESP protocol since the computer was last started.
Authenticated Bytes Received | The total number of authenticated bytes that have been received using the AH protocol or the ESP protocol since the computer was last started.
Transport Bytes Sent | The total number of bytes that have been sent using IPSec transport mode since the computer was last started.
Transport Bytes Received | The total number of bytes that have been received using IPSec transport mode since the computer was last started.
Bytes Sent in Tunnels | The total number of bytes that have been sent using IPSec tunnel mode since the computer was last started.
Bytes Received in Tunnels | The total number of bytes that have been received using IPSec tunnel mode since the computer was last started.
Offloaded Bytes Sent | The total number of bytes that have been sent using IPSec hardware offload since the computer was last started.
Offloaded Bytes Received | The total number of bytes that have been received using IPSec hardware offload since the computer was last started.


Viewing details about active IPSec policies in IP Security Monitor

IP Security Monitor allows you to view details about the active IPSec policies that are applied to the domain, the local computer, or a remote computer. This topic describes the type of details that IP Security Monitor displays.

For information about how to view details about active IPSec policies, see To view details about active IPSec policies. For information about creating, assigning, and assigning IPSec policies, see Creating, modifying, and assigning IPSec policies.

Active IPSec policies that are applied by the domain

The following table describes the items that are displayed when you use IP Security Monitor to view details about an active IPSec policy that is applied by the domain.

Active IPSec policy item | Description
Policy Name | Specifies the name of the active IPSec policy.
Description | Optional text that describes the purpose of the active IPSec policy.
Policy Last Modified | Not applicable (this information is displayed only for active IPSec policies that are applied to a computer).
Policy Store | Specifies the storage location for the active IPSec policy. For active IPSec policies that are applied by the domain, this location appears as "Domain Store."
Policy Path | Specifies the LDAP path to the active IPSec policy.
Organizational Unit | Specifies the organizational unit to which the Group Policy object is applied.
Group Policy Object Name | Specifies the Group Policy object to which the active IPSec policy is applied.

Active IPSec policies that are applied to a computer

The following table describes the items that are displayed when you use IP Security Monitor to view details about active IPSec policies that are applied to a computer.

Active IPSec policy item | Description
Policy Name | Specifies the name of the active IPSec policy.
Description | Optional text that describes the purpose of the active IPSec policy.
Policy Last Modified | Specifies the date and the time that the active IPSec policy was last modified.
Policy Store | Specifies the storage location of the active IPSec policy. For active IPSec policies that are applied to a computer, this location appears as "Local Store."
Policy Path | Not applicable (this information is only displayed for active IPSec policies that are applied by the domain).
Organizational Unit | Not applicable (this information is only displayed for active IPSec policies that are applied by the domain).
Group Policy Object Name | Not applicable (this information is only displayed for active IPSec policies that are applied to the domain).

Note

  • You cannot use IP Security Monitor to add, edit, or remove IPSec policies. For information about how to add, edit, or remove IPSec policies, see To add, edit, or remove IPSec policies.


Using Resultant Set of Policy to view IPSec policy assignments

Resultant Set of Policy (RSoP) is an addition to Group Policy that you can use to view IPSec policy assignments for a computer or for members of a Group Policy container. This information can help you troubleshoot policy precedence issues and plan your deployment.

To view IPSec policy assignments in RSoP, you must first open the RSoP MMC console, and then run a query. RSoP provides two types of queries: logging mode queries (for viewing IPSec policy assignments for a computer) and planning mode queries (for viewing IPSec policy assignments for members of a Group Policy container).

Logging mode queries

You can run an RSoP logging mode query to view all of the IPSec policies that are assigned to an IPSec client. The query results display the precedence of each IPSec policy assignment, so that you can quickly determine which IPSec policies are assigned but are not being applied and which IPSec policy is being applied. The RSoP console also displays detailed settings (that is, the filter rules, filter actions, authentication methods, tunnel endpoints, and connection type) for the IPSec policy that is being applied.

When you run a logging mode query, RSoP retrieves policy information from the Windows Management Instrumentation (WMI) repository on the target computer, and then displays this information in the RSoP console. In this way, RSoP provides a view of the policy settings that are being applied to a computer at a given time.

For information about how to run RSoP logging mode queries for IPSec policies, see To use RSoP to view IPSec policy assignments for a computer. For general information about RSoP logging mode queries, see RSoP logging mode. For general information about RSoP, see RSoP overview.

Planning mode queries

You can run an RSoP planning mode query to view all of the IPSec policies that are assigned to members of a Group Policy container. For example, a planning mode query can be useful if you are planning a company reorganization and you want to move computers from one organizational unit to a new organizational unit. By supplying the appropriate information and then running a planning mode query, you can determine which IPSec policies are assigned but are not being applied to the new organizational unit and which IPSec policy is being applied. In this way, you can identify which policy would be applied if you were to move the computers to the new organizational unit. As with logging mode queries, when you run a planning mode query, the RSoP console displays detailed policy settings for the IPSec policy that is being applied.

When you run a planning mode query, RSoP retrieves the names of the target user, computer, and domain controller from the WMI repository on the domain controller. WMI then uses the Group Policy Data Access Service (GPDAS) to create the policy settings that would be applied to the target computer, based on the RSoP query settings that you entered. RSoP reads the policy settings from the WMI repository on the domain controller, and then displays this information in the RSoP console user interface.

For information about how to run RSoP planning mode queries for IPSec policies, see To use RSoP to view IPSec policy assignments for members of a Group Policy container. For general information about RSoP planning mode queries, see RSoP planning mode.

Note

  • You can run an RSoP planning mode query only on a domain controller (when you run a planning mode query, you must explicitly specify the domain controller name). However, you can specify any IPSec client as the target for the query, provided you have the appropriate permissions to do so.

Assigning and processing IPSec policy in Group Policy

IPSec policies can be assigned from and stored in Active Directory, as part of Group Policy, or they can be assigned and stored locally, on a computer. When a computer is joined to an Active Directory domain, the domain-level IPSec policy applies. If a computer is not joined to an Active Directory domain, the local IPSec policy applies.

Group Policy settings are contained in Group Policy objects, which are linked with specific Active Directory objects (sites, domains, and organizational units). When an IPSec policy is assigned to a Group Policy object for an Active Directory object (such as an organizational unit), that IPSec policy is propagated to any computer accounts that are affected by the Group Policy object.

Multiple Group Policy objects, each of which can contain an IPSec policy, can be assigned to a computer account. When multiple IPSec policies are assigned, the last policy that is processed is the policy that is applied (that is, the last policy takes the highest precedence and overrides the settings of any IPSec policy assignments that were processed earlier).

Policy precedence is based on the Group Policy inheritance model. The policy used is the policy assigned at the lowest level of the domain hierarchy for the domain container of which the computer is a member. For example, if there are IPSec policies that are configured for both the domain and for an organizational unit within the domain, the computers that are members of the domain use the domain IPSec policies. The computers that are members of the organizational unit within the domain use the organizational unit IPSec policies. If there are multiple organizational units, members of each organizational unit use the IPSec policy assigned to the organizational unit that is closest in level to their container in the Active Directory hierarchy. If no IPSec policies are configured for Active Directory, or if a computer is not connected to an Active Directory domain, the local IPSec policy is used.

IPSec policy information displayed in the RSoP console

The RSoP console simplifies the task of determining which IPSec policy is being applied by displaying the following information for each Group Policy object that contains an IPSec policy assignment: the name of the IPSec policy, the name of the Group Policy object that the IPSec policy is assigned to, the IPSec policy precedence (the lower the number, the higher the precedence), and the name of the site, domain, and organizational unit to which the Group Policy object containing the IPSec policy applies (that is, the scope of management for the Group Policy object).

For information about policy precedence and IPSec policy behavior in an Active Directory environment, see Active Directory-based policy. For general information about Group Policy, see Group Policy overview.

Note

  • The settings of the IPSec policy with the highest precedence apply in their entirety; they are not merged with the settings of IPSec policies that are applied at higher levels of the Active Directory hierarchy.


Resources

Windows Resource Kits

The Windows Resource Kits contain extensive technical information on TCP/IP and IPSec. For more information, see Windows Deployment and Resource Kits.

IPSec-related Web sites

The following Web sites contain links that relate to IPSec:

  • The IPSec Working Group page contains links to RFCs, Internet Drafts, and other IPSec-related links at the Internet Engineering Task Force Web site.
  • For information and the latest recommendations on security, see Microsoft Security at the Microsoft Web site.(http://www.microsoft.com/)
  • For information about the standards and technology for IPSec, as well as other computer security topics, see the National Institute of Standards and Technology Web site.

Note

  • XOX


Troubleshooting

What problem are you having?

Traffic is not being secured.

Cause: An IPSec security association (SA) that secures traffic (also known as a hard SA) is not being established.

Solution: This might be caused by either filter list misconfiguration or the enabling of unsecured communication with non-IPSec-aware computers. Examine the policies that are assigned to both IPSec peers and determine, based on the rules, filter lists, and filter action settings, whether IPSec requires traffic to be secured. Check to determine whether the Allow unsecured communication with non-IPSec-aware computer option on the Security Methods tab of the appropriate filter action is enabled. Use audit logging to determine the cause of the failed negotiation.

See also: Troubleshooting tools; To add, edit, or remove IPSec security methods

IPSec SA negotiation is failing.

Cause: An IPSec security association (SA) that secures traffic (also known as a hard SA) is not being established. If unsecured communication with non-IPSec-aware computers is enabled, the negotiation failure is expected.

Negotiation failure between two IPSec-aware computers commonly occurs when the default policies are used on two IPSec peers and one or both are not members of an Active Directory domain. Computers running XOX do not support Active Directory domain membership or the Kerberos V5 authentication method. Default policies use the Kerberos V5 authentication method, which does not work for computers that are not Active Directory domain members.

Solution: A negotiation failure between two IPSec-aware computers is most often caused by a mismatched key exchange method, authentication method, or security method. Examine the policies that are assigned to both IPSec peers and determine, based on the advanced settings, rules, filter lists, and filter action settings, whether IPSec should be securing the traffic. If it should, determine whether there is a mismatched key exchange, authentication method, or security method that is preventing a successful negotiation of a hard SA. Use audit logging to determine the cause of the failed negotiation.

See also: Troubleshooting tools; To add, edit, or remove IPSec security methods; To define IPSec authentication methods; To create key exchange security methods

IPSec SA negotiation is failing when using certificate authentication.

Cause: The correct computer certificates are not installed on the IPSec peers.

Solution: Install the correct computer certificates on the IPSec peers.

Cause: An incorrect certification authority (CA) is selected in the authentication method.

Solution: For the rules of each IPSec peer, select the CA that issued the computer certificate that is installed on the other IPSec peer.

Cause: The certificate chain from the issuing CA to the root CA is broken.

Solution: Install a computer certificate on each IPSec peer that follows a valid certificate chain up to a trusted root CA.

Cause: The computer certificate on one or both of the IPSec peers has expired.

Solution: Install a current computer certificate on each IPSec peer.

See also: Troubleshooting tools; Authentication methods; To define IPSec authentication methods; Special IPSec considerations; Using certificates

Some computers are unable to communicate with secure computers.

Cause: Secure computers are using a lockdown policy that requires client computers to initiate secure communications. The computers that are unable to communicate are not configured with a rule to initiate secure communication.

Solution: Either allow unsecured incoming connection requests on the secure servers or configure additional rules for the computers that are unable to communicate so that they can initiate secure communication with the secure computers.

Cause: Client computers are not IPSec-capable and secure computers are not allowing unsecured communications.

Solution: Either install client computers that are IPSec-capable or reconfigure the secure computers to allow communication with computers that are not IPSec-capable.

Cause: Some computers are configured with a policy that has a key exchange, authentication method, or security method that is mismatched with the policy configured for the secure computers.

Solution: Examine the policies that are assigned to both IPSec peers and determine, based on the advanced settings, rules, filter lists, and filter action settings, whether IPSec should be securing the traffic. If it should, determine whether there is a mismatched key exchange, authentication method, or security method that is preventing a successful SA negotiation. Use audit logging to determine the cause of the failed negotiation.

See also: To add, edit, or remove filter actions; Troubleshooting tools; To add, edit, or remove IPSec security methods; To define IPSec authentication methods; To create key exchange security methods

Local computer IPSec policy is not being used.

Cause: The computer is a member of an Active Directory domain and is receiving Active Directory-based IPSec policy.

Solution: All domain members receive Active Directory-based IPSec policy, which overrides local IPSec policy settings. To prevent this, remove the computer from the domain.

Cause: The IPSec service is not started.

Solution: Start or restart the IPSec service.

See also: To restart the IPSec service

Unable to assign or unassign an IP Security policy.

Cause: You cannot assign or unassign policies when IP Security Policies are used for Active Directory.

Solution: To assign Active Directory-based policies, use IP Security Policies within Group Policy.

See also: To assign or unassign IPSec policy in Group Policy

DES is being used when 3DES is needed.

Cause: The IPSec peer is a computer running Windows 2000 without either the Windows 2000 High Encryption Pack or Windows 2000 Service Pack 2 (or later) installed.

Solution: Install either the Windows 2000 High Encryption Pack or Windows 2000 Service Pack 2 (or later) on the computer running Windows 2000.

Unsecured traffic is being sent to a secure server.

Cause: The client computer is using the default response rule.

Solution: When the client computer is configured with the default response rule, it sends unsecured packets when it either initiates communication or restarts communication after a delay long enough for the quick mode SA and dynamic filter list to time out. To prevent this, configure additional rules for client computers that always initiate secure communication to secure servers.

See also: Security information for IPSec

Event Viewer is displaying error messages related to IPSec.

Cause: There might be a problem with either the IPSec Policy Agent or the Oakley service.

Solution: View the system and security event logs for messages that might help in determining the origin of the problem.

A policy location error message is displayed.

Cause: This error might occur if the IPSec Policy Agent is attempting to retrieve policy from an incorrect storage location.

Solution: Examine both the IPSec Policy Agent events in Event Viewer and the IPSec Policy Agent log for messages that indicate from what location policy is being retrieved (the local registry or the domain). If HKLM\SOFTWARE\Policies\Microsoft\Windows\IPSec\GPTIPSecPolicy\DSIPSecPolicyFlags is set to 1, then IPSec policy is obtained from Active Directory. Otherwise, it is obtained from the registry. The local computer warns a user that domain policy is in effect when it starts. If the policy is supposed to be Active Directory-based, but cannot be obtained, you can delete HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Group Policy\History and then restart the computer.

Caution

  • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.

See also: View event logs.

Event Viewer is displaying "Bad SPI" error message.

Cause: This error might occur if a key lifetime value is set too low, or if the SA has expired but the sender continues to transmit data to the receiver.

Solution: To determine and correct the problem, use IP Security Monitor to examine the number of rekeys. If the number of rekeys is very large compared to the amount of time the connections have been active, set the key lifetimes in the policy to be longer. Good values for high-traffic Ethernet connections are greater than both 50 megabytes and five minutes. This event is usually logged when an SA has been deleted on one IPSec peer, and the other IPSec peer sends secure traffic on the SA. This is usually a temporary situation that occurs on systems that are busy and fixes itself.

IP Security Monitor displays the error message "The IPSec server is unavailable or incompatible with the IPSec monitor."

Cause: You are using the IP Security Monitor console to monitor IPSec on a computer running Windows 2000.

Solution: The IP Security Monitor console can only be used to monitor IPSec on computers running XOX or a Windows Server 2003 operating system. To monitor IPSec on a computer running Windows 2000, use the IPSecmon command at the Windows 2000 command prompt on the computer that is being monitored.

See also: Troubleshooting tools