4.6. Configuring and Using the Auto Enrollment Proxy

Red Hat Certificate System's Auto Enrollment Proxy for Windows allows users and computers in a Microsoft Windows domain to enroll for certificates automatically, and have them automatically issued from Red Hat Certificate System.
Certificate System's auto enrollment proxy is designed to integrate seamlessly with an existing Windows infrastructure and to minimize the amount of administration for requesting, issuing, and managing certificates on a Windows domain:
  • Users and computers registered in a Windows domain can automatically discover the location of the proxy on their network.
  • Computers in a domain can automatically generate a certificate request and submit it to a Certificate System CA through the proxy.
  • Certificate requests are authenticated, and therefore can be approved, through the Kerberos authentication mechanism built into Windows.
When the Certificate System CA issues a certificate, the new certificate is then automatically installed into the requesting application.
The Certificate System auto enrollment proxy can issue certificates for domain controllers, web servers, computers, and users by default, and can have custom profiles mapped to other certificate profiles recognized by the Windows domain.

4.6.1. About Auto Enrollment

Windows servers for Server 2000, XP, 2003, and Vista all have a feature for automatic certificate enrollment which allows Windows systems within a domain to contact a domain controller, find available certificate services, and request and receive those services based on their domain credentials.
That is a technical way of saying that whenever a new identity joins a domain — a server, a user, or an administrator — then simply being in the domain allows the entity to be automatically given a certificate because the process of requesting, authenticating, and installing the certificate is automatically handled by the domain.
Certificate System's auto enrollment proxy plugs a Certificate System CA into the Windows domain, so that the Certificate System CA is part of the domain's PKI and can be used for issuing certificates used in and by the domain. Looking at How the Auto Enrollment Proxy Works on Windows

Auto enrollment within a Windows domain simplifies creating and managing secure connections within the domain and establishing a PKI for servers and users on the systems. Users can be issued a certificate when they first log into the domain or servers can be issued certificates when they are first set up.
Most of the time, the certificates issued within the domain are issued by a certificate service on the domain controller. However, administrators may want to use a third-party CA, such as a Certificate Manager, to issue certificates. For that, Microsoft allows an enrollment proxy which essentially imitates a Windows certificate service on the domain.
Microsoft certificate services use a special request interface (ICertRequestD) to manage requests within the domain.
ICertRequestD is a DCOM object. Windows uses components to manage APIs as if they are objects. The intent of the component object model, or COM, is to enable processes to communicate with each other and to generate objects dynamically. Each object is identified in the system registry, and each component exposes itself through some kind of interface. It is possible for COM interfaces to be shared over a network connection, rather than being on the same machine; these networked objects are called distributed component object model, or DCOM, objects.
Every DCOM and COM interface is defined in the registry with a interface identifier (IID) and globally unique identifier (GUID). For example, the IID for the COM object which handles certificate enrollment (ICertEnroll) is 43F8F288-7A20-11D0-8F06-00C04FC295E1, so its registry entry is as follows:
For Microsoft's auto enrollment process, an application (like a web server, a domain server, or the management console) calls a control like ICertEnroll, and then the enroll object manages the entire issuance process, from creating keys to generating and submitting the certificate request.
In a Windows domain, servers and applications poll Active Directory to get the list of available certificate services. When the Auto Enrollment Proxy, is configured, its information is added to Active Directory as one of the available certificate services. Then, when an enrollee (like a server) first asks the domain controller for available services, Certificate System is included. The enrollee process then sends certificate request, through the DCOM objects, to the proxy, which then forwards the request to the Certificate System CA.
Using DCOM Objects for Enrollment

Figure 4.1. Using DCOM Objects for Enrollment

The Auto Enrollment Proxy is another Windows service running within the domain, and it has registry entries which match the DCOM ID for the ICertRequestD object. The RPC service (RPCSS) on the machine will perform necessary authorization checks to verify that the enrollee can access the proxy.
Any type of user of the domain can access the process: a person running the Microsoft Management Console, a user running certreq, or a server or web service which initiates an automatic enrollment. Regardless of the method of accessing the proxy, Microsoft's enrollment object will run through a series of checks to authorize the request:
  • That the requested certificate profile is supported. The PKCS#10 request contains an extension which identifies the type of certificate being requested; the template can be mapped to a Certificate System profile.
  • That the Certificate Authority is trusted.
  • That the requester has permission to access the proxy.
If the process meets those requirements, the Windows server generates a PKCS#10 certificate request and submits it to the proxy. The proxy parses the request, pulls out required information and may derive or add other required information, and re-formats the request to meet the requirements for the Red Hat Certificate System CA. The proxy then sends the reformatted request to the Certificate Manager, which automatically issues the certificate (since the request was authenticated in the Windows domain) and sends it to the proxy. The proxy then presents the certificate to the enrolling application.
The Auto Enrollment Process

Figure 4.2. The Auto Enrollment Process

At several points in the process, the DCOM objects pull information about the proxy service from the registry settings or from the entry in Active Directory:
  1. The server runs an LDAP search on the root DSE to find the configuration naming context.
  2. Then, it runs an LDAP search under the CN=Enrollment Services, CN=Public Key Services, CN=Services branch of the configuration naming context, which lists every configured enrollment service, including the Auto Enrollment Proxy.
  3. The enrollment process then locates and uses any enrollment service which matches the required criteria:
    • Has a certificateTemplates attribute which matches the requested certificate type.
      Certificate System 8.0 has two certificate profiles supported by default, for domain controller certificates (caDomainController) and web server certificates (caAgentServerCert).
    • Has a trusted CA that is listed in the CN=Certification Authorities subtree.
    • The last CA in the trust chain must be a self-signed root CA. Planning the Auto Enrollment Configuration

It only requires a single Auto Enrollment Proxy for an entire domain, but it depends on the domain configuration where and what machine the Auto Enrollment Proxy should be installed on. The proxy has to be able to authenticate the requester against information in the domain, so the requester must be in the same forest as the proxy.
Additionally, for security, the proxy should be run on a dedicated machine in a secure environment with access limited to trusted administrators.
The simplest configuration is to install the proxy as the same machine as the domain controller. This limits the field of the proxy to that single domain.
Having the Proxy on the Domain Controller

Figure 4.3. Having the Proxy on the Domain Controller

Because of traffic or security, it may be better to install the proxy on a dedicated machine within the domain, but not a domain controller. Depending on the domain configuration, this can also limit the proxy to entities within the single domain.
Having the Proxy within the Domain

Figure 4.4. Having the Proxy within the Domain

The most complex arrangement has the proxy installed within a single domain but accessible to multiple domains within a Windows forest. For this configuration, see the Windows server and Active Directory documentation to explain how to configure the domain properly.
Using a Single Proxy within a Forest

Figure 4.5. Using a Single Proxy within a Forest

4.6.2. Installing and Setting up the Auto Enrollment Proxy

The Auto Enrollment Proxy must be installed and configured on a Windows server within a Windows domain. The latest releases of the Certificate System Auto Enrollment Proxy are available at http://directory.fedoraproject.org/wiki/Windows_Certificate_Auto_Enrollment. Requirements for the Windows Environment

The Windows environment must be configured and running before you can install the Auto Enrollment Proxy.
  • Active Directory must be installed, and if there are multiple domains in the forest, then they must be able to cross-trust each other.
  • Audit logging should be enabled for the group policy.
  • DNS must be properly configured; the DNS settings can be verified using dcdiag.
  • All Windows servers should access the same NTP server so that their dates and times are in sync.
  • The Microsoft Management Console must be configured, as described in Section, “Configuring the Microsoft Management Console to Use with the Auto Enrollment Proxy”. Configuring the Microsoft Management Console to Use with the Auto Enrollment Proxy

The Microsoft Management Console must have the appropriate snap-ins added for the Auto Enrollment Proxy to function.
  1. Open the Microsoft Management Console. The easiest way to do this is to open the Start menu, select Run, and type in mmc.
  2. To add the required snap-ins, open the File, and select Add/Remove Snap-ins.
  3. Select the profile to which to add the snap-ins. (It may be beneficial to have a separate profile for the proxy.) Then click Add.
  4. Select and go through the configuration each required snap-in.
    • Services
    • Certificate Templates
    • Certificates (with the My user account option, to create a snap-in for Certificates -Current User)
    • Certificates (with the Computer account option, to create a snap-in for Certificates [Local Computer)
    • Active Directory users and computers
    • Active Directory domains and trusts
    • DNS
    • Component Services
  5. Save the Microsoft Management Console configuration to the desktop; this ensures that it is easy to access.
  6. Verify that the console is properly configured by re-opening it and double-clicking Certificate Templates folder. Setting up the Auto Enrollment Proxy

  1. Open the Microsoft Management Console. If a profile was saved to the desktop, then launch it from there; otherwise, open the Start menu, select Run, and type in mmc.
  2. Set up the CA's agent certificate. The proxy authenticates to the CA as an agent so that it can immediately approve any certificate requests submitted by the proxy. To do this, the proxy must have the agent certificate.
    1. Retrieve the CA agent certificate from the CA's end entities pages, and save it to a PKCS#12 file. Copy the PKCS#12 to the Windows machine.
    2. In the Microsoft Management Console, open the Certificates - Current User snap-in.
    3. Right-click on Personal, select All tasks, then select Import, to import the saved agent certificate.
  3. Add the CA certificate into the domain group policy. The CA must be trusted in order to issue certificates, meaning the CA certificate has to be loaded.
    1. Use IE and connect to the CA's agent page. No errors/warning should be displayed. If they appear, make sure they don't appear the next time.
    2. Retrieve the CA certificate chain, in binary form, from the CA's end entities pages. Save the certificate chain to the desktop with a name like cacert.cer.
    3. In the Microsoft Management Console, open the Active Directory Users and Computers snap-in.
    4. Right-click Domain in the left menu, and select Properties.
    5. Open the Group Policy tab, then select the Default Domain policy, and press Edit.
    6. In the edit window, open a series of menus: in Computer Configuration, then Windows Settings and Security Settings, and finally open Public Key Policies and Trusted Root Certification Authorities.
    7. Right-click on the right panel, Select 'Import...'. Open the 'cacert.cer' file you saved earlier. Right-click on Personal, select All tasks, then select Import, to import the saved CA certificate chain file.
  4. Install the Auto Enrollment Proxy packages.
    1. Double-click the .exe, and go through the installer.
  5. Configure the Auto Enrollment Proxy by importing the CA certificate, setting the CAs to use, and setting the Auto Enrollment Proxy settings.
    1. Open the Start menu, and select Red Hat Auto Enrollment Proxy.
    2. Open the CA Certificate tab.
      Click Load from File, and import that CA certificate chain from the file. Then click Set to apply the certificate.
    3. Next, click the Active Directory tab.
      Click the Populate AD button to create the Active Directory entry for the proxy service.
    4. Add the connection information for each Certificate Manager which will be used by the proxy. Click Add to add each CA.
      The CA connection requires the following information:
      • The fully-qualified domain name of the Certificate Manager
      • The port number of the Certificate Manager
      • The Certificate System version number of the Certificate Manager
      • The certificate to use to authenticate to the Certificate Manager
    5. In the Logging tab, set any log levels to use for the service.
    6. When all of the configuration settings have been made, click Apply to save the settings.
  6. The last configuration area is setting up the DCOM service.
    1. In the Microsoft Management Console, select the DCOM - Components snap-in.
    2. Select or expand the Computers folder, then My Computer, and DCOM Config.
    3. Right-click Red Hat Auto Enrollment Proxy, and select Properties
    4. In the Security tab, click the Customize radio button under Launch and Activation Permissions, and click the Edit button. Make sure that the administrator and that Everyone is selected.
      Then, click the Customize radio button under Access Permissions, and click the Edit button. Make sure that the administrator and that Everyone is selected.


      The user that launches the proxy and the computer account for the proxy host must be members of the Distributed COM Built-in Principals Group.
    5. In the Identity tab, select the This user radio button. It is better to browser for the user, even if you know the username for the administrator, because the domain name may have to be prepended in order for the user to log into the domain.
    6. Save the changes to the DCOM snap-in.
  7. In Administrative Tools, open Services, and manually start the Auto Enrollment Proxy service. This should then be listed in the Task Manager as rhcsproxy.exe. Troubleshooting and Diagnostic Tips

Microsoft supplies several tools that are beneficial for diagnosing and troubleshooting problems with auto enrollment or the Auto Enrollment Proxy. A list of Windows Server 2003 Support tools is at http://support.microsoft.com/kb/892777, which includes LDP (an LDAP browser) and DCDIAG (used for diagnosing domain controller, and DNS problems). Two monitoring tools, filemon and regmon, are available at http://www.microsoft.com/technet/sysinternals/ProcessesAndThreads/Regmon.ms and are extremely useful for monitoring and troubleshooting registry and file issues, which can help with analyzing the auto enrollment process.
Kerberos Authentication
Auto enrollment uses Microsoft's Kerberos mechanisms to authenticate users, servers, and requests. Make sure that Kerberos, not NTLM, is being used to authenticate entities (this is visible in the Event Viewer security logs). The kerbtray.exe tool can be used to purge Kerberos tickets if NTLM is being used instead of Kerberos.
dcdiag and Replication
If dcdiag shows that there are replication problems, create new replication agreements.
The certificate request failed...
One common error message when generating a certificate using the Microsoft Management Console is The certificate request failed because of one of the following conditions...
If this error occurs when the Microsoft Management Console request wizard is first opened, then it means that the console could not connect to the enrollment service, and there are a couple of different possible reasons:
  • The hostname in enrollment services is incorrect. Use LDP to view the enrollment service in Active Directory for the proxy, and verify the dNSHostName attribute. This value is automatically populated when the proxy is first configured.
  • The proxy host is unreachable. Try to ping the above hostname to make sure DNS resolves the hostname to an IP address correctly, and that the host is online.
  • The proxy service is not running. The proxy is a registered service. It is stopped by default, but Windows should automatically start the service when a certificate request is entered. Check Task Manager to see if the rhcsproxy.exe process is running or check the Services page to restart the service.
  • If the proxy is installed on another machine or another domain, then the DCOM or domain configuration may not be allowing the host to contact the proxy service.
If the request failed error occurs after going through the request wizard, then the console was able to connect to the proxy and generate a request, but the proxy did not return a certificate. Check the Event Viewer application log on the system that is running the proxy to see what errors were recorded.

4.6.3. Managing Auto Enrollment Proxy Settings

The Auto Enrollment proxy is configured in a central Active Directory entry and in the Windows registry of the host where it is installed. These configuration entries can be viewed and, to some extent, edited to redefine the proxy service. Auto Enrollment Proxy Registry Settings

The Auto Enrollment Proxy stores its configuration settings in the Windows registry, underneath the following key:
This entry defines the basic behaviors of the proxy service.

Table 4.3. Auto Enrollment Proxy Registry Settings

Name Description Example
RequestType The type of certificate request to send to the CA. The only supported value is PKCS10. PKCS10
LogOptions An decimal integer representing a bitmask of all the selected log options. 503
AuthenticationCertificate A hash of the chosen certificate to use for SSL client authentication to the CA.
CACertificate A binary value, for the DER encoded binary CA certificate.
RetryInterval The number of seconds to wait before trying to use a CA which was previously failing. Listing and Adding CAs in the Windows Domain

All of the CAs configured for enrollment services for a domain are listed in Active directory in the CN=Enrollment Services,CN=Public Key Services subtree. This subtree can be queried to show what Certificate Managers are configured for the proxy and what certificate templates and other settings they have available. For example:
dsquery * "CN=Example RHCS CA,CN=Enrollment Services,CN=Public Key Services,CN=Services,CN=Configuration,DC=server,DC=example,DC=com" -scope base -attr *
The actual configuration for the Certificate Manager is defined in the registry entries for the proxy service. All proxy CAs are listed in the registry under the following key:
HKEY_LOCAL_MACHINE\SOFTWARE\Red Hat\RHCSProxy\Config\CertificateAuthorities
Each configured CA is then a subkey under the main entry.
[HKEY_LOCAL_MACHINE\SOFTWARE\Red Hat\RHCSProxy\Config\CertificateAuthorities\1] "hostname"="ca.example.com" "port"="9444" "catype"="3"
New CAs can be added by directly editing the registry entry and adding a new CA or by opening the proxy configuration console and adding a new entry there. Mapping Certificate System Enrollment Profiles to Windows Profiles

The proxy maintains a list of available certificate profiles, by default allowing domain controller certificates (caDomainController) and web server certificates (caAgentServerCert). These profiles are mapped to corresponding Windows certificate templates, maintained in the registry under the following key:
To add additional certificate profiles to the proxy service, add a subkey under the ProfileMap folder which maps a Windows template to the Certificate System profile. The Windows template is identified in the key name; the corresponding Certificate System profile is set in a CAProfileName parameter. For example:
[HKEY_LOCAL_MACHINE\SOFTWARE\Red Hat\RHCSProxy\Config\ProfileMap\WebServer] "CAProfileName"="caServerCert"


Profile maps can only be done by editing the registry keys; they cannot be set in the Auto Enrollment Proxy configuration console.

4.6.4. Manually Requesting Domain Certificates

The auto enrollment proxy, naturally, automatically enrolls servers, hardware, and even users as soon as the entity is added to the Windows domain. However, once the auto enrollment proxy for Red Hat Certificate System is configured, it is also possible to request and receive certificates manually on a Windows domain through a Certificate System CA.
Certificates can be requested using the Microsoft Management Console (MMC) or using the Windows certreq tool. Requesting Certificates through MMC

  1. Open MMC
  2. Open Certificates (Local Computer) -> Persona
  3. Right click the right panel, and select Request New Certificate. This opens a certificate request wizard.
  4. The available types of certificates that can be requested are listed. Select the type of certificate to request.
  5. Fill in the information to use to configure the certificate, such as a name or description.
  6. Click Finish to submit the certificate request to the auto enrollment proxy. Requesting Certificates Using certreq

  1. Create an .inf file to use to supply values for the certificate request. This file defines the settings to use for the certificate request, such as the certificate profile for the Windows domain, the key settings, and any extensions. For example:
     Signature="$Windows NT$"
     Subject = "CN=domain.example.com"
     KeySpec = 1
     KeyLength = 1024
     Exportable = TRUE
     MachineKeySet = TRUE
     SMIME = False
     PrivateKeyArchive = FALSE
     UserProtected = FALSE
     UseExistingKeySet = FALSE
     ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
     ProviderType = 12
     RequestType = PKCS10
     KeyUsage = 0xa0
     CertificateTemplate = DomainController
    More information on using certreq and the format of the request file is available at http://technet.microsoft.com/en-us/library/cc736326(WS.10).aspx.
  2. On the Windows machine, run the certreq command to generate the request, specifying the request .inf file and an output file for the certificate request. For example:
    certreq -new request.inf dc-cert-request.req
  3. Submit the certificate request to the CA, and set the name of the final output certificate file.
    certreq -submit dc-cert-request.req dc-cert.cer
  4. Install the new certificate in the server's database.
    certreq -accept dc-cert.cer
  5. To verify that the certificate is available by checking the certificate list in the management console.


It is also possible to request and submit a certificate in a single pass. For example:
certreq -f -v -config "domain.example.com\Certificate Authority - SUBCA - server.example.com" -submit dc-cert-request.req dc-cert.cer