Friday, February 26, 2016

PowerShell PSRemoting error on SharePoint servers

Last week I run into very strange issue, which took me days to figure out and resolve it. It is not directly involved with SharePoint, but it can give you a lot of headache.

As a part of much greater project, one of my tasks was to enable PSRemoting on SharePoint server. The problem was that our clients have SharePoint 2010 on Windows 2008R2 servers, on which PSRemoting is not enabled automatically. So we had to enable PSRemoting on all of these servers. In order to save time, we implemented group policy to enable PSRemoting on all servers. Once implemented, I tried to connect to one of SharePoint 2010 servers, let's call it SPTest, and received following message:

Enter-PSSession -ComputerName SPTest

Enter-PSSession : Connecting to remote server SPTest failed with the following error message : WinRM cannot process the request. The following error with errorcode 0x80090311 occurred while using Kerberos authentication: There are currently no logon servers available to service the logon request.
Possible causes are:
-The user name or password specified are invalid.
-Kerberos is used when no authentication method and no user name are specified.
-Kerberos accepts domain user names, but not local user names.
-The Service Principal Name (SPN) for the remote computer name and port does not exist.
-The client and remote computers are in different domains and there is no trust between the two domains.
After checking for the above issues, try the following:
-Check the Event Viewer for events related to authentication.
-Change the authentication method; add the destination computer to the WinRM TrustedHosts configuration setting or use HTTPS transport. Note that computers in the TrustedHosts list might not be authenticated.
-For more information about WinRM configuration, run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic. At line:1 char:1 + Enter-PSSession -ComputerName SPTest

My first thought was that something went wrong with configuring WinRM service on server, because I was able to connect to other, non-SharePoint servers. After manually enabling PSRemoting nothing changed, same error message appeared. I searched Event log on SharePoint server and there was nothing which would indicate what went wrong. Also when I tried Test-WSMan cmdlet it stated that everything is properly configured. Very strange.

After looking in Event Viewer on my client machine, I found following error message:

The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server SPTest$. The target name used was HTTP/SPTest. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (ADTest.COM) is different from the client domain (ADTest.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.

Ok, this problem has something to do with Kerberos. But what? SPTest machine is joined into domain, and everything is configured properly. How can the ticket be the problem? After consulting Google found this blog: http://ift.tt/1LOq8mf by Damien Caro, where he explained  that, if you have Application pools running under a domain service account, you may run into same issue, since WinRM seems to check for a HTTP SPN set on the computer object (HTTP\name-of-the-server), it will always fail because it will find the HTTP SPN but not set on the computer object. Thus, it cannot authenticate using Kerberos and just fails. It turned out that this is what was causing the problem from the beginning.

I found two solutions for this issue.

First one is quick and dirty, you just have to add one more DNS A record for you machine, or add record to your host file, adding different name for your server with same IP address. For instance, my server name is SPTest, I added in my client machine host file SPTest_PSRemoting with same IP address. After that I tried Enter-PSSession -ComputerName SPTest_PSRemoting and it worked! However, this solution is applicable just for testing purposes, you don't want to add additional DNS record for all of your servers, or hassle with host files.

The other solution is to configure your Service account to utilize SPN  just for ports used by Web applications. This includes typing few commands in command prompt, but be careful, you first need to make a list of all the ports your IIS server is using for Web applications.

First step was to query SPN, I opened command prompt and typed:   

C:\Windows\system32>setspn -Q http/SPTest
Checking domain DC=ADTest,DC=com
CN=SPS_WebApp,OU=Users,DC=ADTest,DC=com
HTTP/SPTEST.ADTest.com
HTTP/SPTEST
HTTP/report
HTTP/reportstaging
HTTP/reporttest
HTTP/reportdev

Existing SPN found!

You can see that Service account SPS_WebApp has SPN HTTP/SPTEST, and since WinRM seems to check for a HTTP SPN set on the computer object (HTTP\SPTEST), it will always fail because it will find the HTTP SPN but not set on the computer object, but on Service account. Thus, it cannot authenticate using Kerberos and just fails.

To get the WinRM Windows Service remote connection to work, narrow the scope of the SPN to the port number for the website. Rather than adding:

HTTP/SPTest.ADTest.com
use
HTTP/SPTest.ADTest.com:80

In this case the services are hosted in the web site running on port 80. The WinRM service runs on port 5195 by default and therefore does not match on the port specific SPN. Rather than specifying port, it is also possible to use the NETBIOS name for the network host if the fully qualified domain name is used for the SPN.

For instance IIS server on my machine uses 3 ports (80,81 and 85) for different Web Applications.

First I had to add SPN which included the ports:

Setspn -S http/SPTest:80 ADTest\SPS_WebApp
Setspn -S http/SPTest.ADtest,com:80 ADTest\SPS_WebApp
Setspn -S http/SPTest:81 ADTest\SPS_WebApp
Setspn -S http/SPTest.ADtest,com:81 ADTest\SPS_WebApp
Setspn -S http/SPTest:85 ADTest\SPS_WebApp
Setspn -S http/SPTest.ADtest,com:85 ADTest\SPS_WebApp

After that, had to delete SPN:

Setspn -D http/SPTest ADTest\SPS_WebApp
Setspn -D http/SPTest.ADtest,com ADTest\SPS_WebApp

And that's it! Once I checked SPN for http/SPtest and received message that no such SPN was found:

C:\Windows\system32>setspn -Q http/SPTest
Checking domain DC=ADTest,DC=com
No such SPN found.

But after querying SPN with explicit port got the following:

C:\Windows\system32>setspn -Q http/SPTest:80
Checking domain DC=ADTest,DC=com
CN=SPS_WebApp,OU=Users,DC=ADTest,DC=com
HTTP/SPTEST.ADTest.com:80
HTTP/SPTEST:80
HTTP/SPTEST.ADTest.com:81
HTTP/SPTEST:81
HTTP/SPTEST.ADTest.com:85
HTTP/SPTEST:85
HTTP/report
HTTP/reportstaging
HTTP/reporttest
HTTP/reportdev

After this intervention I had no more problems with connecting to SharePoint servers.


by Krsto Savic via Everyone's Blog Posts - SharePoint Community

No comments:

Post a Comment