We recently had a problem where Citrix Servers 2008 R2 in a farm would randomly hang when random users logged off. The users would get “Waiting for User Profile Service” when trying to RDP in. When you went onto the console of the server and tried a Ctrl , Alt and Delete it would display the plain windows wallpaper but nothing else ( no username or password prompt ). The only way to get the server back would be to reset the machine. However users logged into the server would be fine until they tried to log off!
When the machine restarted you could see WinLogon.exe event log issues which proved the problem above was related
I wrote the following powershell script to query the servers everyday so we could restart them in the evening until we found the fix
UPDATE*****
The below script used application names instead of manually entering in server names
$PSEmailServer = "email server"
Add-PSSnapin Citrix*
function QuerySessions($computer, $timeout)
{
#Ping the server to make sure online
If (Test-Connection $computer -quiet) {
#Check RDP is Alive
$job = Start-Job -ScriptBlock {param($computer) query session /server:$computer } -ArgumentList $computer
Wait-Job $job -Timeout $timeout
Stop-Job $job
$result = Receive-Job $job
$processresult = query process /server:$computer
if($result -eq $null -or $processjob -eq "Error enumerating processes" )
{Send-MailMessage -From "[email protected]" -to "[email protected]" -Subject "Citrix Server Hung" -Body $computer}
Remove-Job $job
}
#Else {Write 'Tango Down'}
}
$servers = Get-XAServer -BrowserName 'CitrixAppName'| select ServerName
foreach ($server in $servers) {
QuerySessions -computer $server -timeout "5"
}
Old Script to manually check specific servers
$PSEmailServer = "emailserver"
function QuerySessions($computer, $timeout)
{
#Ping the server to make sure online
If (Test-Connection $computer -quiet) {
$job = Start-Job -ScriptBlock {param($computer) query session /server:$computer } -ArgumentList $computer
Wait-Job $job -Timeout $timeout
Stop-Job $job
$result = Receive-Job $job
if($result -eq $null)
{Send-MailMessage -From address@from.com -to address@to.com -Subject "Citrix Server Hung" -Body $computer}
Remove-Job $job
}
#Else {Write 'Tango Down'}
}
QuerySessions -computer "citrixserver1" -timeout "5"
QuerySessions -computer "citrixserver2" -timeout "5"
Looking through some Citrix forum posts , users had suggested the following hot fixes
KB2525332 – Do not automatically make redirected folders available offline
Citrix Forum Said none of these fix the issue of Logging off
- KB2581608
- KB2561285
Other Policy/Profile related hotfixes I found useful
- KB2590550
- KB2065362
- KB2510636
- KB2383928 – Logging Off Probs +++ ( Not Applyable Already Applied to SP1 )
- KB2575077
- KB2265716 + ( Not Applyable )
- KB2617858
- KB2775511
- KB2661001 +
- KB2465772 + ( Not Applyable )
- KB2379016
- KB2004121
- CTX133595
Other Reasons
- User Laptop Going to Sleep?
- System\%registry quota in use maxing out?
- Stop NLASVC
- Run Logon Scripts Syncronosly
- WMI filtering
- Check Shortcut’s Group Policys uses Asyncrhonous Logins
- Mutliple NICS on DC?
- OFF print driver isolation via group policy on the Citrix servers
- HP Universal Printer Driver version 5.2.6 was the problem. We updated it to the newest available version (5.3.1)
- Better: Enable GPSVC debug logging and check the log file with Sysprosoft Policy Reporter for time lags… http://sysprosoft.com/policyreporter.shtml (Policy Reporter allows you to enable debug logging, too ;-))
The Fix
We enabled print driver isolation which seems to make things better but still happened randomly
After disabling the Citrix Print Manager Service the problems stopped straight away , Great! However we needed this for users with local printers mapping to Citrix. After sending Citrix and Microsoft dumps of the crashed servers , they confirmed this and Citrix got me to delete and readded the Citrix Universal Print Driver via http://support.citrix.com/article/CTX125780