I have come across a few issues where I have seen the above error. Below are two scenarios of the issue and the symptoms that I've noticed during that time.
· Domain Workstations going into a state where they are unable to access resources over the network.
· Member Servers unable to access network resources and stop allowing domain users to login.
**In both the above scenarios, the issue seemed to get fixed for some time if the computer was Rebooted. Further investigations show a common set of errors being reported on these problem computers.
5719 NETLOGONThis computer was not able to set up a secure session with a domain controller in domain %Domainname% due to the following: Not enough storage is available to process this command. This may lead to authentication problems. Make sure that this computer is connected to the network.
40960 LSASRVThe Security System detected an authentication error for the server cifs/hostname.domain,com. The failure code from authentication protocol Kerberos was "There are currently no logon servers available to service the logon request. (0xc000005e)".
1053 UserenvWindows cannot determine the user or computer name. (Not enough storage is available to complete this operation. ). Group Policy processing aborted.
Though this errors clearly says "Not enough storage.." you may still see enough available RAM and good amount of free space on the hard drive, on that problem computer. So why does it still say "Not enough storage is available to complete this operation"?
What I understood from this error was that a RESTRICTED amount of MEMORY allocated for particular type of OS RESOURCE, has got EXHAUSTED.
In both the cases above, the resource type that was getting exhausted or causing other resources to be exhausted was a HANDLE. A Handle is an OS object used to get a reference of another object (like File, registry, port etc.). It’s using these handles that a process will work with open files, read registry keys or work with a network port.
In both the above scenarios, we saw a process (Non-Microsoft) consuming large amount of Handles. You can view the handles owned by different processes through "Task Manager" in Windows. In the "Task Manager", under the Process tab you can add an additional column to display Handles information. Stopping or killing this process that was using large amount of handles restored this computer back from the problem state and every thing started working fine.
Though the problem process/application has to be debugged by its developers but what can help them further is to know what type of handles did the application have the most and to which objects. This can be found easily by using a tool called Handle.exe (http://technet.microsoft.com/en-us/sysinternals/bb896655.aspx). "Handle -a > handle_output.txt" would dump information about all open handles on the computer, sorted by processes into the text file handle_output.txt. You can review the handles under the problem process to check, what type of handle this process has the most.
One of the most common questions being asked here is "how does one conclude which process's handle usage is abnormal?”
The best way to find that out is to check the handles owned by different process from Task Manager and find the process that owns the maximum number of handles. Then check what the process does and if it is normal for that process to be holding references to those many objects via handles. For example:
· On a Domain controller you would see LSASS using most amount of handles, this may be normal as LSASS is the main process on a DC which is responsible for most of the work done by that server.
· On a SQL server you would see the SQL server process having a lot of handles, this may again be quite normal depending on the size of the database the SQL server has and the amount of queries/request coming to it.
· BUT if an application or a management agent process is using large number of handles like 30,000 or 140,000, it is something that needs to be investigated.
**Another thing to note is, if the handle usage goes down when this process is not actively working or does the handle count keeps increasing. If you see the handle count keeps increasing and never coming down, this may indicate a handle leak, which means the process is requesting and acquiring a handle but not releasing it when its work is done.
There could be other reasons behind the "Not enough storage is available to complete this operation" error too, so just don’t restrict you troubleshooting to the above steps.
An additional link that will help you get more help to troubleshoot this error:
Good post, but if you can't log on the server when the issue occurs, how could you check which process has the handle leak.
When I had this issue on one of my servers, I was able to log on with the local Administrator account. HTH
I faced the Same issue and the reason from my problem was the over handeling of resource by Sharepoint Servers got my server down for 30 minutes. I rebooted the server and i asked our Sharepoint team to look into the Handler issue seriously.
Yes, you can only log in locally when this happen.
Another point I've seen that all the network port where exausted. I can say that because a RDP connection was attempted on the port 65536.
Beautifully written.. Thank you so much..
Good stuff. Much appreciated. Mine issue is with a Citrix server. This will certainly help in troubleshooting. Thanks!
Its a good article, well explained one. I was able to fix my issue, but in my case the handle count was 12,000 - 14,000. And the handles were consumed by ccmexec.exe. So when I killed the corresponding service SMS Agent host on the member server, the users were able to logon to domain successfully. I have tried resetting the SMS Agent host, so the handle count will drop down to 300 - 500 range, then also the users were successfully logged into domain. Thanks once again for the article.
It worked for me, there was a process called winscp.exe which opened 38K handles. I'd to log off the disconnected session of user under which this process was running and issue got fixed right away.
Tanks for your suggestions.