Workflow Manager Configuration Error

I recently attempted to configure Workflow Manager 1.0 and Service Bus 1.0 for use by SharePoint 2016, using a certificate issued by our domain CA instead of self-generated certificates. I ran into the following error though.

System.Management.Automation.CmdletInvocationException: Could not successfully send message to scope ‘/WF_Management’ despite multiple retires over a timespan of 00:02:07.8300000.. The exception of the last retry is: A recoverable error occurred while interacting with Service Bus. Recreate the communication objects and retry the operation. For more details, see the inner exception..  —> System.TimeoutException: Could not successfully send message to scope ‘/WF_Management’ despite multiple retries over a timespan of 00:02:07.8300000.. The exception of the last retry is: A recoverable error occurred while interacting with Service Bus. Recreate the communication objects and retry the operation. For more details, see the inner exception..  —> System.OperationCanceledException: A recoverable error occurred while interacting with Service Bus. Recreate the communication objects and retry the operation. For more details, see the inner exception. —> Microsoft.ServiceBus.Messaging.MessagingCommunicationException: Identity check failed for outgoing message. The expected DNS identity of the remote endpoint was ‘WFMServer1.contoso.com’ but the remote endpoint provided DNS claim ‘WFMServer3.contoso.com’. If this is a legitimate remote endpoint, you can fix the problem by explicitly specifying DNS identity ‘WFMServer3.contoso.com’ as the Identity property of EndpointAddress when creating channel proxy.  —> System.ServiceModel.Security.MessageSecurityException: Identity check failed for outgoing message. The expected DNS identity of the remote endpoint was ‘WFMServer1.contoso.com’ but the remote endpoint provided DNS claim ‘WFMServer3.contoso.com’. If this is a legitimate remote endpoint, you can fix the problem by explicitly specifying DNS identity ‘WFMServer3.contoso.com’ as the Identity property of EndpointAddress when creating channel proxy.

To cut a long troubleshooting story short, the problem was with the certificate I had requested from the CA. More specifically the DNS extension. I had the following DNS entries in the certificate.

  1. *.contoso.com
  2. WFMServer1.contoso.com
  3. WFMServer2.contoso.com
  4. WFMServer3.contoso.com

For what ever reason, WFM only seems to look at the final DNS entry when trying to add the host to the WFM farm. To confirm this, I tried the installation from all three hosts and it worked fine on WFMServer3.contoso.com, but not the other 2.

I’m still not entirely sure if it was WFM or SB that was causing this issue, but I fixed it by simply revoking the certificate on our CA and re-installing SB and WFM using a certificate with wfm.contoso.com as the Common Name and DNS entries in the following order:

  1. WFMServer1.contoso.com
  2. WFMServer2.contoso.com
  3. WFMServer3.contoso.com
  4. *.contoso.com

 

Making SharePoint Trust Enterprise CAs

If you have an enterprise PKI, or even just a single CA, that issues certificates to services such as ADFS or Workflow Manager that you intend to use with your SharePoint Farm, it might be worth while importing you Root and Intermediate CA Certificates into SharePoint to make it trust the other services.

Currently we have a PKI setup similar to the following:

-Enterprise Root CA (Offline)
—Enterprise Intermediate CA (Offline)
——Web Services CA (Online)
——Authentication CA (Online)

The Root and the Intermediate CA are offline and secured except for planned maintenance, such as publishing new certificate revocation lists. The Web Services CA is the CA that issues the certificates we use for web services like Workflow Manager, and the Authentication CA issues the ADFS signing Certificate, which also needs to be trusted by SharePoint.

First of all, you need to grab a copy of the certificate of each CA in the Chain. If you have your PKI configured correctly, they should be readily available so I won’t go into that process. If you are exporting them from the CA servers, DO NOT export private keys, SharePoint doesn’t need them and that would be a major security concern. Once you have them, copy them to a directory on your SharePoint server(s).

Then, make sure the certificates are in the trusted root certificate authorities on every server in your farm. the easiest was to do this is to publish the certificates using group policy, but you can do this manually if you like. Simply follow these steps on each server in the farm:

  1. Double click the first certificate.
    cert_overview.jpeg
  2. Click Install.
  3. Select “Local Machine” and click Next.
  4. Click Yes on AUC prompt.
  5. Select “Place all certificates in the following store”.
  6. Click Browse and select “Trusted Root Certificate Authorities” or “Intermediate Certificate Authorities” depending on if the certificate is at the root of the chain.
  7. Click Next.
  8. Click Finish.
  9. Repeat for each certificate in the chain.

On your SharePoint server execute the following powershell commands for each certificate, changing “C:\RootCA.cer” to the path of each certificate, and “Enterprise Root CA” to a friendly name for each CA.


$path = “C:\RootCA.cer”
$root = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($path)
New-SPTrustedRootAuthority -Name "Enterprise Root CA" -Certificate $cert

If you don’t like powershell or are unable to access powershell for whatever reason, you can also install the certificates from Central Admin.:

  1. Navigate to Central Admin -> Security -> Manage Trust.
  2. Click “New”.sharepoint_ca_add_trust.jpeg
  3. Add a friendly Name for the certificate.
  4. Click Browse and select your certificate.
  5. Click OK.
  6. Repeat for each certificate in your PKI chain.

 

SharePoint 2013 Product Configuration Wizard hangs on step 9 of 10

I was patching a SharePoint 2013 farm recently and all was going great while installing the CU’s on the servers. Once all of the patches were installed, I started to run the SharePoint 2013 Products Configuration Wizard on the first WFE server, which went without an issue. The second WFE server seemed to be running fine too, getting to step 9 out of 10. It was on this step for about an hour when I went home for the evening (this is a test environment). The next morning, it was still at the same stage.

Initially I reboot the server and tried again, but got the same results. To remedy this issue, I performed the following actions.

  1. Open services.msc
  2. Stop the SharePoint Timer Service
  3. Browse to C:ProgramDataMicrosoftSharePointConfig
  4. Open the folder with the newest date modified on it.
  5. Delete all files from the folder except for cache.ini
  6. Open cache.ini and change the number within to any random number ensuring it has the same number of digits as the original. I went with 12345.
  7. Run the following command from an elevated command prompt – Psconfig.exe -cmd upgrade -inplace b2b -wait -force

After that, the server was updated and ready to serve content again.

SharePoint 2013 webapps don’t work on some sites

I ran in to an issue today that had my head battered for a good half an hour before turning to Google.

The issue involved opening documents, which happened to be in my SkyDrive Pro library, with Office Web Apps. The apps threw different errors, listed below;

Word Document : “Sorry, there was a problem and we can’t open this document. If this happens again, try opening the document in Microsoft Word.”
Excel Document: “We couldn’t find the file you wanted. It’s possible the file was renamed, moved or deleted”
PowerPoint: “Sorry, we ran into a problem. Please try again”
One Note:”Sorry, you don’t have permission to edit this notebook.”

After an hour of Googling and trawling through false positive results (thanks to Microsoft’s genius idea of keeping the “SkyDrive Pro” brand the same in hosted and on-premises SharePoint farms) I found a blog post (linked below) that shone some light on my issue. It also made me realize that the issue wasn’t local to SkyDrive Pro as well.

I turns out that the issue was down to the fact that I provisioned Web Apps using SSL, and the SkyDrive Pro library wasn’t using SSL. This caused OAuth not to function on the http site. Web Apps, being https, didn’t much like this and shit itself. Oh well.

The resolution is either to enable SSL on all SharePoint sites that Web Apps should be accesible in, or simply allow OAuth over http (detailed below.)

Run the following PowerShell script in SharePoint Management Shell on your SharePoint box(es.)

$config = (Get-SPSecurityTokenServiceConfig)
$config.AllowOAuthOverHttp = $true
$config.Update()

Credit for this solution goes to ms_dynamics.

References: Office Web Apps 2013 and SharePoint Issue relating to SSL

SharePoint 2010 User Profile Sync Observations

The past few days I have had the pleasure of working with User Profile Sync in SharePoint 2010 and my observations follow.

1. It’s shit! – Although Microsoft have improved this feature of SharePoint with a series of CU’s, it is the most complicated part of SharePoint 2010 to setup, or at least the trickiest to troubleshoot.

2. It doesn’t like proxy servers – I struggled to get the UPS to start successfully for a full day. The log in ULS viewer was getting to “ILM Configuration: The ExportMiisEncryptionKey process completed successfully” before restarting the provision process. It turned out it was our automatic proxy configuration script that was causing the problems. To fix this I followed the steps below.

1. Hold Shift and Right-click Internet Explorer
2. Click Run as different user
3. Enter the credentials for your Farm account
4. G to Tools > Internet Options > Connections > LAN Settings and either disable the “Automatically Detect Settings” check box or add an exception for your SharePoint server
5. Reboot Server

3. User Profile Picture Import could be more seamless – OOTB SharePoint will import user profile pictures from AD. Unfortunately not many of the guides on the internet mention the fact that the import doesn’t create the required thumbnails automatically. This is a problem as the images in AD are stored as a blob and the “Picture” field in UPA is expecting a URL to the image. To fix this, run the following PS command (I had to login as the farm admin).

Update-SPProfilePhotoStore -CreateThumbnailsForImportedPhotos 1 -MySiteHostLocation http://mysitesurl

If you get a Null Reference exception when attempting to run this command, then you don’t have the permission to do so. This Technet article explains the permissions requirements for updating user profiles, along with a few other things.

All in all, this is a great product until it goes wrong, then it is an absolute beast!

I know all of this information is already available online, but the more times it is published the less time it will take for people to find it in Google 🙂