Fides Tamen Quin

Trust, but Verify...and all the other things learned while troubleshooting.

Windows 2012 servers that use HpCISSs2.sys become unresponsive typically every 3-7 days

Windows 2012 servers that use HpCISSs2.sys become unresponsive typically every 3-7 days

  • Comments 11
  • Likes

(updated 11/4/2013)

Just a quick note out there on a specific issue we’ve been seeing lately. It is the really the only issue with Windows 2012 server hanging or becoming unresponsive that we’ve had up almost a year into the release of this OS.

On Windows 2012 running on HP servers that use the October 15th 2012 version of HpCISSs2.sys, those servers will run into an issue were we have an IO stall accessing the disk.  This stall happens when we hit the HpCISSs2.sys and will never recover.  The first time it happens the servers don’t actually hang but that is when the traffic jam starts to backup.  Meaning – someone can have a car accident block the highway 5 miles down the road. You wouldn’t stop moving right away but eventually you’re going to be stopped in the resulting line of traffic.

Some of the symptoms you may see is that the server will show you a grey screen instead of the logon option. You may not be able to RDP to the server but if you test ports such as 3389 135 445 with port query they will be open.  Remote WMI queries may fail. Ping will work. File shares will not be accessible.

For those familiar with the Windows debugger the driver version info is:

14: kd> lmvm
hpcisss2

start             end                 module name

    Loaded symbol image file: HpCISSs2.sys

    Image path:
\SystemRoot\System32\drivers\HpCISSs2.sys

    Image name: HpCISSs2.sys

    Timestamp:        Mon Oct 15 14:09:41 2012 (507C5165)

    CheckSum:         0002A400

    ImageSize:        00028000

    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

From Microsoft diagnostics reports (MPS or MSDT) we see the file as:

STORAGE (SCSI) HPCISSS2.SYS       Hewlett-Packard Company      62.26.0.64          10/15/2012       153 KB    (156,992 bytes) Smart Array P410i Controller

Although MSinfo32 may show a different date, the version number in these cases has been 62.26.0.64:

Driver c:\windows\system32\drivers\hpcisss2.sys (62.26.0.64, 153.31 KB (156,992 bytes), 7/2/2013 12:28 AM)

 

Customers with this issue have  followed what HP has documented here for a separate issue involving HpCISSs2.sys on 2012.

If you have that version of the driver, are running Windows 2012 and are running into this behavior please contact HP and open a ticket with them to investigate this further.

To get conclusive data that confirms this is the issue, a Kernel Memory dump is the best route. Since Servers are typically setup for a Kernel dump by default the only change that would be required is to setup the system to generate that dump on an NMI signal:

How to generate a complete crash dump file or a kernel crash dump file by using an NMI on a Windows-based system

http://support.microsoft.com/kb/927069

Not having to get a Full memory dump and only needing a Kernel dump greatly reduces the resulting size of the dump file, which is becoming problematic when generating dumps on systems with a large amount of RAM.

Comments
  • This has been an issue with HP Servers since Server 2003 with that hpciss2 driver. That and storport don't play well together unless they are at specific versions.

  • There have been issues in the past with that driver, yes.  I cannot tell from the data that we have if the cause here is due to similar code issues with that driver or if we're getting stuck down another code path.  Unfortunately, once we hit that driver the debugging goes into a black box.

  • Hello,

    I believe there might be a small hickup in the article.

    We had the hang problem on our HP Windows Server 2012 hosts with upgraded firmware/driver versions and version 62.26.0.64 of the Smart Array driver (February 2013 version in SPP-2013.02).

    In fact we had to downgrade to the October 2012 version 62.24.2.64 which SOLVES the problem. More recent versions have the hang problem as documented in this HP article.

    h20000.www2.hp.com/.../Document.jsp

    Kind regards

  • All the Server 2012 hang cases we have seen have been with the October 15th 2012 version of that file.

  • we are experiencing the exact issue on our HP Blade server all of sudden.  hopefully the downgrade of the driver will fix this problem.  Thanks for the article!

  • Hi Guys,

    I face some similar issue to this . See below

    I got HP server 2012 storage box (HP Store Easy 1430) running which is member sever in domain.

    This server hanged two times last two months and when this happens AD users can't log-in there computes either, Not sure this because of storage box.

    I can't shutdown or restart the server2012 storage box ((HP Store Easy 1430)and only option it to power off.

    After powering on back the server2012 storage box everything comes back to normal.

    You guys reckon that I should downgrade  the   hpciss2 driver in HP Store Easy 1430?

    Cheers

    PrazAU

  • Worked for me. All systems stable after downgrade! thanks!!

  • HpCISSs2.sys new version - 62.28.0.64 with SPP2014020 was released on February 18th 2014.

  • DL360p Gen8 here
    Symptoms: Disk Management - Connecting to Virtual Disk Service

    as per this and following topic
    http://social.technet.microsoft.com/Forums/en-US/47156c37-9cde-49c2-b97a-a668d71ef663/disk-management-hangs-cant-connect-to-virtual-disk-management-service?forum=winserveressentials

    i upgraded HpCISSs2.sys to 62.28.0.64

    After restart still Connecting to Virtual Disk Service problem

  • Solution:
    F9 in BIOS then Advanced - Disable virtual install disk

    All is ok now

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment